GNU bug report logs - #23064
PostgreSQL logs to stdout/stderrr, which is lost

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: bug#23064: closed (Re: bug#23064: herd has incomplete status
 reporting, not so helpful)
Date: Tue, 22 Mar 2016 23:25:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#23064: PostgreSQL logs to stdout/stderrr, which is lost

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 23064 <at> debbugs.gnu.org.

-- 
23064: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23064
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 23064-done <at> debbugs.gnu.org
Subject: Re: bug#23064: herd has incomplete status reporting, not so helpful
Date: Wed, 23 Mar 2016 00:23:55 +0100

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

>> Could you try this and report back?
>
> $ patch -p1 < ...
> $ sudo ./pre-inst-env guix system reconfigure /etc/config.scm
> $ sudo herd stop postgres
> $ sudo herd start postgres
> Service postgres has been started.
> $ tail -f /var/log/messages 
> Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was created, adding watch
> Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was written to
> Mar 22 20:20:40 localhost postgres[2115]: [1-1] FATAL:  database files are incompatible with server
> Mar 22 20:20:40 localhost postgres[2115]: [1-2] DETAIL:  The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.5.1.

Great, thanks for testing!  Pushed as
9b1cee97a5a75766cb52553111794c758c4f1651.

> Although it would be nice to have shepherd have a field in "herd status" which tells you when it respawned the service in < 1 s intervals (or maybe if it respawned it < 1 s after the initial start) - or maybe just how long ago it was (last) started and when the actual herd start command was :-)

In trying to implement that, I realized there’s a bug in respawn delay
handling.  More on that later.

Ludo’.

[Message part 3 (message/rfc822, inline)]

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: bug-guix <at> gnu.org
Subject: herd has incomplete status reporting, not so helpful
Date: Sat, 19 Mar 2016 16:13:37 +0100

So there was a postgresql update where the data format was incompatible.

That happens regularily and is ... acceptable, I guess (even more acceptable in guix because I can easily switch back to the old release).

However, in order to find that out, I had to:

$ psql
... something about a socket not being reachable

$ sudo herd status postgres
Status of postgres:
  It is started.
  Running value is 18438.
  It is enabled.
  Provides (postgres).
  Requires (user-processes loopback).
  Conflicts with ().
  Will be respawned.

So it claimed that everything was just peachy.

$ sudo herd stop postgres
$ sudo herd start postgres

... which reported nothing out of the ordinary either.

$ ps -ef |grep postgres
[xxxx] <defunct>

Aha! finally an indication that something is amiss.

So I tried to find the log entries, to no avail so far.
(I found: /gnu/store/b073csgl56g2wnq7azj4wrswb6azsvc0-postgresql-9.5.1/bin/pg_ctl -D /var/lib/postgresql/data -l logfile start, not sure how that helps)
$ man pg_ctl
No manual entry for pg_ctl

In the mean time, I decided I can just start up the postgres server process manually and it will tell me.

$ postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data
FATAL:  data directory "/var/lib/postgresql/data" has wrong ownership
HINT:  The server must be started by the user that owns the data directory.

$ sudo -i
# su - postgres
Password:

errr... what password? 

# passwd postgres
...
# su - postgres
Password:
su: Authentication failure

# passwd -u postgres
# su - postgres
Password:
This account is currently not available.

Hmmm...

$ sudo -u postgres postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data
FATAL:  database files are incompatible with server
DETAIL:  The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.5.1.
$ 

Aha! It says so right there in the standard output (or error?). Where did that message go in the shepherd case?

$ cd /var/log
$ grep -r incompatible .
$ 

???

Anyway, it works again now but we shouldn't have to do this kind of digging.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.