From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 19 Mar 2016 15:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 23064@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.145840043323015 (code B ref -1); Sat, 19 Mar 2016 15:14:02 +0000 Received: (at submit) by debbugs.gnu.org; 19 Mar 2016 15:13:53 +0000 Received: from localhost ([127.0.0.1]:53817 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIZc-0005z9-TR for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIZb-0005yw-Ne for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahIZV-0005XE-OH for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:46 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_50, FILL_THIS_FORM_FRAUD_PHISH,T_FILL_THIS_FORM_SHORT autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:42371) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZV-0005X9-MN for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37757) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZU-0004jx-Jb for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahIZR-0005VO-AQ for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:44 -0400 Received: from dd1012.kasserver.com ([85.13.128.8]:37648) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZR-0005VH-3c for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:41 -0400 Received: from localhost (91.141.2.247.wireless.dyn.drei.com [91.141.2.247]) by dd1012.kasserver.com (Postfix) with ESMTPSA id 1077D1CA0551 for ; Sat, 19 Mar 2016 16:13:39 +0100 (CET) Date: Sat, 19 Mar 2016 16:13:37 +0100 From: Danny Milosavljevic Message-ID: <20160319161337.609ada55@scratchpost.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.6 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) So there was a postgresql update where the data format was incompatible. That happens regularily and is ... acceptable, I guess (even more acceptable in guix because I can easily switch back to the old release). However, in order to find that out, I had to: $ psql ... something about a socket not being reachable $ sudo herd status postgres Status of postgres: It is started. Running value is 18438. It is enabled. Provides (postgres). Requires (user-processes loopback). Conflicts with (). Will be respawned. So it claimed that everything was just peachy. $ sudo herd stop postgres $ sudo herd start postgres ... which reported nothing out of the ordinary either. $ ps -ef |grep postgres [xxxx] Aha! finally an indication that something is amiss. So I tried to find the log entries, to no avail so far. (I found: /gnu/store/b073csgl56g2wnq7azj4wrswb6azsvc0-postgresql-9.5.1/bin/pg_ctl -D /var/lib/postgresql/data -l logfile start, not sure how that helps) $ man pg_ctl No manual entry for pg_ctl In the mean time, I decided I can just start up the postgres server process manually and it will tell me. $ postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data FATAL: data directory "/var/lib/postgresql/data" has wrong ownership HINT: The server must be started by the user that owns the data directory. $ sudo -i # su - postgres Password: errr... what password? # passwd postgres ... # su - postgres Password: su: Authentication failure # passwd -u postgres # su - postgres Password: This account is currently not available. Hmmm... $ sudo -u postgres postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data FATAL: database files are incompatible with server DETAIL: The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.5.1. $ Aha! It says so right there in the standard output (or error?). Where did that message go in the shepherd case? $ cd /var/log $ grep -r incompatible . $ ??? Anyway, it works again now but we shouldn't have to do this kind of digging. From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 19 Mar 2016 15:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.145840113524490 (code B ref 23064); Sat, 19 Mar 2016 15:26:01 +0000 Received: (at 23064) by debbugs.gnu.org; 19 Mar 2016 15:25:35 +0000 Received: from localhost ([127.0.0.1]:53822 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIkx-0006Mw-0d for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:25:35 -0400 Received: from dd1012.kasserver.com ([85.13.128.8]:33126) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIkv-0006Mm-3g for 23064@debbugs.gnu.org; Sat, 19 Mar 2016 11:25:33 -0400 Received: from localhost (91.141.2.247.wireless.dyn.drei.com [91.141.2.247]) by dd1012.kasserver.com (Postfix) with ESMTPSA id 59DE91CA0551 for <23064@debbugs.gnu.org>; Sat, 19 Mar 2016 16:25:31 +0100 (CET) Date: Sat, 19 Mar 2016 16:25:29 +0100 From: Danny Milosavljevic Message-ID: <20160319162529.5bfd95ca@scratchpost.org> In-Reply-To: <20160319161337.609ada55@scratchpost.org> References: <20160319161337.609ada55@scratchpost.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) > Aha! It says so right there in the standard output (or error?). Where did that message go in the shepherd case? It's on standard error. The exit code is 1. From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 19 Mar 2016 21:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Danny Milosavljevic Cc: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.14584221163973 (code B ref 23064); Sat, 19 Mar 2016 21:16:02 +0000 Received: (at 23064) by debbugs.gnu.org; 19 Mar 2016 21:15:16 +0000 Received: from localhost ([127.0.0.1]:53969 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahODL-00011k-Vx for submit@debbugs.gnu.org; Sat, 19 Mar 2016 17:15:16 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39038) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahODK-0000vq-F0 for 23064@debbugs.gnu.org; Sat, 19 Mar 2016 17:15:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahODB-0006hA-7f for 23064@debbugs.gnu.org; Sat, 19 Mar 2016 17:15:09 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36298) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahODB-0006h4-4Y; Sat, 19 Mar 2016 17:15:05 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:57044 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ahODA-0003uv-H6; Sat, 19 Mar 2016 17:15:04 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <20160319161337.609ada55@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 =?UTF-8?Q?Vent=C3=B4se?= an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Sat, 19 Mar 2016 22:15:01 +0100 In-Reply-To: <20160319161337.609ada55@scratchpost.org> (Danny Milosavljevic's message of "Sat, 19 Mar 2016 16:13:37 +0100") Message-ID: <87lh5ef9uy.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Danny Milosavljevic skribis: > $ sudo herd status postgres > Status of postgres: > It is started. > Running value is 18438. > It is enabled. > Provides (postgres). > Requires (user-processes loopback). > Conflicts with (). > Will be respawned. > > So it claimed that everything was just peachy. > > $ sudo herd stop postgres > $ sudo herd start postgres > > ... which reported nothing out of the ordinary either. > > $ ps -ef |grep postgres > [xxxx] Sounds like postgresql died and shepherd did not notice? Or maybe it keeps trying to respawn it? What did /var/log/shepherd.log say? > $ sudo -u postgres postgres --config-file=3D/etc/postgresql.conf -D /var/= lib/postgresql/data > FATAL: database files are incompatible with server > DETAIL: The data directory was initialized by PostgreSQL version 9.3, wh= ich is not compatible with this version 9.5.1. > $=20 > > Aha! It says so right there in the standard output (or error?). Where did= that message go in the shepherd case? > > $ cd /var/log > $ grep -r incompatible . > $=20 Currently the Shepherd does not do log stdout/stderr of its child processes, which sucks somewhat. However, daemons can usually be told to write to syslog, which is more appropriate than writing things to stdout/stderr anyway. What=E2=80=99s the right command-line/configuration option to have postgresql use syslog? Thanks, Ludo=E2=80=99. From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 19 Mar 2016 21:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Cc: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.145842356611569 (code B ref 23064); Sat, 19 Mar 2016 21:40:01 +0000 Received: (at 23064) by debbugs.gnu.org; 19 Mar 2016 21:39:26 +0000 Received: from localhost ([127.0.0.1]:53974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahOaj-00030X-Vd for submit@debbugs.gnu.org; Sat, 19 Mar 2016 17:39:26 -0400 Received: from dd1012.kasserver.com ([85.13.128.8]:50085) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahOai-00030P-6x for 23064@debbugs.gnu.org; Sat, 19 Mar 2016 17:39:24 -0400 Received: from localhost (77.119.130.254.wireless.dyn.drei.com [77.119.130.254]) by dd1012.kasserver.com (Postfix) with ESMTPSA id 86C931CA0551; Sat, 19 Mar 2016 22:39:22 +0100 (CET) Date: Sat, 19 Mar 2016 22:39:21 +0100 From: Danny Milosavljevic Message-ID: <20160319223921.107f27bc@scratchpost.org> In-Reply-To: <87lh5ef9uy.fsf@gnu.org> References: <20160319161337.609ada55@scratchpost.org> <87lh5ef9uy.fsf@gnu.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) > Sounds like postgresql died and shepherd did not notice? Or maybe it > keeps trying to respawn it? What did /var/log/shepherd.log say? 2016-03-19 10:34:48 Service postgres has been started. 2016-03-19 10:34:49 Respawning postgres. 2016-03-19 10:34:49 Service postgres has been started. 2016-03-19 10:34:50 Respawning postgres. 2016-03-19 10:34:50 Service postgres has been started. 2016-03-19 10:34:51 Respawning postgres. 2016-03-19 10:34:51 Service postgres has been started. 2016-03-19 10:34:52 Respawning postgres. 2016-03-19 10:34:52 Service postgres has been started. 2016-03-19 10:34:53 Respawning postgres. 2016-03-19 10:34:53 Service postgres has been started. 2016-03-19 10:34:54 Respawning postgres. 2016-03-19 10:34:54 Service postgres has been started. > Currently the Shepherd does not do log stdout/stderr of its child > processes, which sucks somewhat. Yeah, that's not good. > However, daemons can usually be told to write to syslog, which is more > appropriate than writing things to stdout/stderr anyway. =20 It can always be unable to open syslog for some reason.=20 If shepherd can't/doesn't redirect stderr on its own, it would be nice to h= ave init write it somewhere and everyone else inherit it as default. Just t= hrowing it away is not nice.=20 But I agree, the chance of being able to write it to syslog is high. Btw: H= ow does guixsd know to start the syslog service before the postgres service? >What=E2=80=99s the right command-line/configuration option to have postgre= sql use syslog? The option is log_destination =3D 'syslog' in postgresql.conf which is generated in gnu/services/databases.scm (%default-postgres-config). Does shepherd back off from respawing it eventually (if it respawns too fas= t) or will it log the same messages into syslog once every 0.1 s until my d= isk is full? :-> From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 21 Mar 2016 08:36:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Danny Milosavljevic Cc: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.145854935510604 (code B ref 23064); Mon, 21 Mar 2016 08:36:01 +0000 Received: (at 23064) by debbugs.gnu.org; 21 Mar 2016 08:35:55 +0000 Received: from localhost ([127.0.0.1]:56187 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahvJa-0002ky-Qe for submit@debbugs.gnu.org; Mon, 21 Mar 2016 04:35:55 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54403) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahvJY-0002kj-RM for 23064@debbugs.gnu.org; Mon, 21 Mar 2016 04:35:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahvJQ-0003UD-K3 for 23064@debbugs.gnu.org; Mon, 21 Mar 2016 04:35:47 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50833) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahvJQ-0003U9-GQ; Mon, 21 Mar 2016 04:35:44 -0400 Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:46262 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ahvJP-0002R0-UR; Mon, 21 Mar 2016 04:35:44 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <20160319161337.609ada55@scratchpost.org> <87lh5ef9uy.fsf@gnu.org> <20160319223921.107f27bc@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 2 Germinal an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Mon, 21 Mar 2016 09:35:41 +0100 In-Reply-To: <20160319223921.107f27bc@scratchpost.org> (Danny Milosavljevic's message of "Sat, 19 Mar 2016 22:39:21 +0100") Message-ID: <87shzkcjoi.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Danny Milosavljevic skribis: >> Sounds like postgresql died and shepherd did not notice? Or maybe it >> keeps trying to respawn it? What did /var/log/shepherd.log say? > > 2016-03-19 10:34:48 Service postgres has been started. > 2016-03-19 10:34:49 Respawning postgres. OK. >> However, daemons can usually be told to write to syslog, which is more >> appropriate than writing things to stdout/stderr anyway.=20=20 > > It can always be unable to open syslog for some reason.=20 > > If shepherd can't/doesn't redirect stderr on its own, it would be nice to= have init write it somewhere and everyone else inherit it as default. Just= throwing it away is not nice.=20 Agreed. > But I agree, the chance of being able to write it to syslog is high. Btw:= How does guixsd know to start the syslog service before the postgres servi= ce? Syslogd is another Shepherd service, so all we need is to express this dependency. >>What=E2=80=99s the right command-line/configuration option to have postgr= esql use syslog? > > The option is > > log_destination =3D 'syslog' > > in postgresql.conf > > which is generated in gnu/services/databases.scm (%default-postgres-confi= g). Could you try this and report back? --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm index 6c3b829..690375e 100644 --- a/gnu/services/databases.scm +++ b/gnu/services/databases.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2015 David Thompson -;;; Copyright =C2=A9 2015 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2015, 2016 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2016 Leo Famulari ;;; ;;; This file is part of GNU Guix. @@ -56,6 +56,7 @@ host all all ::1/128 trust")) =20 (define %default-postgres-config (mixed-text-file "postgresql.conf" + "log_destination =3D 'syslog'\n" "hba_file =3D '" %default-postgres-hba "'\n" "ident_file =3D '" %default-postgres-ident "'\n")) =20 @@ -116,7 +117,7 @@ host all all ::1/128 trust")) (list (shepherd-service (provision '(postgres)) (documentation "Run the PostgreSQL daemon.") - (requirement '(user-processes loopback)) + (requirement '(user-processes loopback syslogd)) (start #~(make-forkexec-constructor #$start-script)) (stop #~(make-kill-destructor)))))))) =20 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > Does shepherd back off from respawing it eventually (if it respawns too f= ast) or will it log the same messages into syslog once every 0.1 s until my= disk is full? :-> Yes, it avoids respawning too fast. Thanks, Ludo=E2=80=99. --=-=-=-- From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 22 Mar 2016 19:30:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Cc: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.14586749615689 (code B ref 23064); Tue, 22 Mar 2016 19:30:03 +0000 Received: (at 23064) by debbugs.gnu.org; 22 Mar 2016 19:29:21 +0000 Received: from localhost ([127.0.0.1]:60438 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiRzU-0001Th-My for submit@debbugs.gnu.org; Tue, 22 Mar 2016 15:29:20 -0400 Received: from dd1012.kasserver.com ([85.13.128.8]:59720) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiRzT-0001Ta-PT for 23064@debbugs.gnu.org; Tue, 22 Mar 2016 15:29:20 -0400 Received: from localhost (77.119.131.26.wireless.dyn.drei.com [77.119.131.26]) by dd1012.kasserver.com (Postfix) with ESMTPSA id 666411CA028B; Tue, 22 Mar 2016 20:29:17 +0100 (CET) Date: Tue, 22 Mar 2016 20:29:16 +0100 From: Danny Milosavljevic Message-ID: <20160322202916.53b89904@scratchpost.org> In-Reply-To: <87shzkcjoi.fsf@gnu.org> References: <20160319161337.609ada55@scratchpost.org> <87lh5ef9uy.fsf@gnu.org> <20160319223921.107f27bc@scratchpost.org> <87shzkcjoi.fsf@gnu.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi Ludo, > Could you try this and report back? $ patch -p1 < ... $ sudo ./pre-inst-env guix system reconfigure /etc/config.scm $ sudo herd stop postgres $ sudo herd start postgres Service postgres has been started. $ tail -f /var/log/messages Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was created, adding watch Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was written to Mar 22 20:20:40 localhost postgres[2115]: [1-1] FATAL: database files are incompatible with server Mar 22 20:20:40 localhost postgres[2115]: [1-2] DETAIL: The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.5.1. ... Nice :-) Thanks! Although it would be nice to have shepherd have a field in "herd status" which tells you when it respawned the service in < 1 s intervals (or maybe if it respawned it < 1 s after the initial start) - or maybe just how long ago it was (last) started and when the actual herd start command was :-) From unknown Tue Aug 19 03:05:17 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Danny Milosavljevic Subject: bug#23064: closed (Re: bug#23064: herd has incomplete status reporting, not so helpful) Message-ID: References: <87poumxfjo.fsf@gnu.org> <20160319161337.609ada55@scratchpost.org> X-Gnu-PR-Message: they-closed 23064 X-Gnu-PR-Package: guix Reply-To: 23064@debbugs.gnu.org Date: Tue, 22 Mar 2016 23:25:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1458689102-8851-1" This is a multi-part message in MIME format... ------------=_1458689102-8851-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #23064: PostgreSQL logs to stdout/stderrr, which is lost which was filed against the guix package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 23064@debbugs.gnu.org. --=20 23064: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D23064 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1458689102-8851-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 23064-done) by debbugs.gnu.org; 22 Mar 2016 23:24:09 +0000 Received: from localhost ([127.0.0.1]:60583 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiVei-0002He-SO for submit@debbugs.gnu.org; Tue, 22 Mar 2016 19:24:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46403) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiVeg-0002H9-Ro for 23064-done@debbugs.gnu.org; Tue, 22 Mar 2016 19:24:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aiVeY-0005A7-Gg for 23064-done@debbugs.gnu.org; Tue, 22 Mar 2016 19:24:01 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37151) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aiVeY-00059y-DA; Tue, 22 Mar 2016 19:23:58 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:41738 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aiVeX-00052a-Pt; Tue, 22 Mar 2016 19:23:58 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Danny Milosavljevic Subject: Re: bug#23064: herd has incomplete status reporting, not so helpful References: <20160319161337.609ada55@scratchpost.org> <87lh5ef9uy.fsf@gnu.org> <20160319223921.107f27bc@scratchpost.org> <87shzkcjoi.fsf@gnu.org> <20160322202916.53b89904@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Germinal an 224 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Wed, 23 Mar 2016 00:23:55 +0100 In-Reply-To: <20160322202916.53b89904@scratchpost.org> (Danny Milosavljevic's message of "Tue, 22 Mar 2016 20:29:16 +0100") Message-ID: <87poumxfjo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 23064-done Cc: 23064-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Danny Milosavljevic skribis: >> Could you try this and report back? > > $ patch -p1 < ... > $ sudo ./pre-inst-env guix system reconfigure /etc/config.scm > $ sudo herd stop postgres > $ sudo herd start postgres > Service postgres has been started. > $ tail -f /var/log/messages=20 > Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was cr= eated, adding watch > Mar 22 20:15:51 localhost nscd: 213 monitored file `/etc/services` was wr= itten to > Mar 22 20:20:40 localhost postgres[2115]: [1-1] FATAL: database files ar= e incompatible with server > Mar 22 20:20:40 localhost postgres[2115]: [1-2] DETAIL: The data directo= ry was initialized by PostgreSQL version 9.3, which is not compatible with = this version 9.5.1. Great, thanks for testing! Pushed as 9b1cee97a5a75766cb52553111794c758c4f1651. > Although it would be nice to have shepherd have a field in "herd status" = which tells you when it respawned the service in < 1 s intervals (or maybe = if it respawned it < 1 s after the initial start) - or maybe just how long = ago it was (last) started and when the actual herd start command was :-) In trying to implement that, I realized there=E2=80=99s a bug in respawn de= lay handling. More on that later. Ludo=E2=80=99. ------------=_1458689102-8851-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 19 Mar 2016 15:13:53 +0000 Received: from localhost ([127.0.0.1]:53817 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIZc-0005z9-TR for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ahIZb-0005yw-Ne for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahIZV-0005XE-OH for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:46 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_50, FILL_THIS_FORM_FRAUD_PHISH,T_FILL_THIS_FORM_SHORT autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:42371) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZV-0005X9-MN for submit@debbugs.gnu.org; Sat, 19 Mar 2016 11:13:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37757) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZU-0004jx-Jb for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ahIZR-0005VO-AQ for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:44 -0400 Received: from dd1012.kasserver.com ([85.13.128.8]:37648) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ahIZR-0005VH-3c for bug-guix@gnu.org; Sat, 19 Mar 2016 11:13:41 -0400 Received: from localhost (91.141.2.247.wireless.dyn.drei.com [91.141.2.247]) by dd1012.kasserver.com (Postfix) with ESMTPSA id 1077D1CA0551 for ; Sat, 19 Mar 2016 16:13:39 +0100 (CET) Date: Sat, 19 Mar 2016 16:13:37 +0100 From: Danny Milosavljevic To: bug-guix@gnu.org Subject: herd has incomplete status reporting, not so helpful Message-ID: <20160319161337.609ada55@scratchpost.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.6 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) So there was a postgresql update where the data format was incompatible. That happens regularily and is ... acceptable, I guess (even more acceptable in guix because I can easily switch back to the old release). However, in order to find that out, I had to: $ psql ... something about a socket not being reachable $ sudo herd status postgres Status of postgres: It is started. Running value is 18438. It is enabled. Provides (postgres). Requires (user-processes loopback). Conflicts with (). Will be respawned. So it claimed that everything was just peachy. $ sudo herd stop postgres $ sudo herd start postgres ... which reported nothing out of the ordinary either. $ ps -ef |grep postgres [xxxx] Aha! finally an indication that something is amiss. So I tried to find the log entries, to no avail so far. (I found: /gnu/store/b073csgl56g2wnq7azj4wrswb6azsvc0-postgresql-9.5.1/bin/pg_ctl -D /var/lib/postgresql/data -l logfile start, not sure how that helps) $ man pg_ctl No manual entry for pg_ctl In the mean time, I decided I can just start up the postgres server process manually and it will tell me. $ postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data FATAL: data directory "/var/lib/postgresql/data" has wrong ownership HINT: The server must be started by the user that owns the data directory. $ sudo -i # su - postgres Password: errr... what password? # passwd postgres ... # su - postgres Password: su: Authentication failure # passwd -u postgres # su - postgres Password: This account is currently not available. Hmmm... $ sudo -u postgres postgres --config-file=/etc/postgresql.conf -D /var/lib/postgresql/data FATAL: database files are incompatible with server DETAIL: The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.5.1. $ Aha! It says so right there in the standard output (or error?). Where did that message go in the shepherd case? $ cd /var/log $ grep -r incompatible . $ ??? Anyway, it works again now but we shouldn't have to do this kind of digging. ------------=_1458689102-8851-1-- From unknown Tue Aug 19 03:05:17 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23064: herd has incomplete status reporting, not so helpful Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 23 Mar 2016 21:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23064 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Danny Milosavljevic Cc: 23064@debbugs.gnu.org Received: via spool by 23064-submit@debbugs.gnu.org id=B23064.14587694544936 (code B ref 23064); Wed, 23 Mar 2016 21:45:02 +0000 Received: (at 23064) by debbugs.gnu.org; 23 Mar 2016 21:44:14 +0000 Received: from localhost ([127.0.0.1]:34862 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiqZa-0001HY-24 for submit@debbugs.gnu.org; Wed, 23 Mar 2016 17:44:14 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47052) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiqZY-0001HN-TO for 23064@debbugs.gnu.org; Wed, 23 Mar 2016 17:44:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aiqZP-0006Tl-Ua for 23064@debbugs.gnu.org; Wed, 23 Mar 2016 17:44:07 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59399) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aiqZP-0006Th-RY; Wed, 23 Mar 2016 17:44:03 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:43920 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aiqZP-0004yr-8D; Wed, 23 Mar 2016 17:44:03 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <20160319161337.609ada55@scratchpost.org> <87lh5ef9uy.fsf@gnu.org> <20160319223921.107f27bc@scratchpost.org> <87shzkcjoi.fsf@gnu.org> <20160322202916.53b89904@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Germinal an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Wed, 23 Mar 2016 22:44:00 +0100 In-Reply-To: <20160322202916.53b89904@scratchpost.org> (Danny Milosavljevic's message of "Tue, 22 Mar 2016 20:29:16 +0100") Message-ID: <87vb4cuaxr.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi again, There was indeed a respawn bug fixed by Shepherd commit 8c8a010b425e56461289bae62a94ee401e5dad41. I also changed =E2=80=98herd=E2=80=99 so that =E2=80=99herd status foo=E2= =80=99 shows the last respawn time, as you suggested. I=E2=80=99ll cut a 0.3.1 release soon. Thanks! Ludo=E2=80=99.