From unknown Tue Jun 24 19:09:29 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#55441 <55441@debbugs.gnu.org> To: bug#55441 <55441@debbugs.gnu.org> Subject: Status: Use of 'primitive-fork' in (guix inferior) leads to hangs in 'cuirass evaluate' Reply-To: bug#55441 <55441@debbugs.gnu.org> Date: Wed, 25 Jun 2025 02:09:29 +0000 retitle 55441 Use of 'primitive-fork' in (guix inferior) leads to hangs in = 'cuirass evaluate' reassign 55441 guix submitter 55441 Maxim Cournoyer severity 55441 important thanks From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 23:49:53 2022 Received: (at submit) by debbugs.gnu.org; 16 May 2022 03:49:53 +0000 Received: from localhost ([127.0.0.1]:51285 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqRk1-0001vJ-2x for submit@debbugs.gnu.org; Sun, 15 May 2022 23:49:53 -0400 Received: from lists.gnu.org ([209.51.188.17]:51084) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqRjz-0001vA-D4 for submit@debbugs.gnu.org; Sun, 15 May 2022 23:49:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51124) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqRjy-0003gz-VV; Sun, 15 May 2022 23:49:51 -0400 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]:33342) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nqRjx-0006Nu-9x; Sun, 15 May 2022 23:49:50 -0400 Received: by mail-qv1-xf35.google.com with SMTP id j3so2024763qvn.0; Sun, 15 May 2022 20:49:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:cc:subject:references:to:cc:date:in-reply-to:message-id :user-agent:mime-version; bh=4zH24G1P6Yk80tSfIENeVF+uUHk44/HZVEZBTQ/Zauc=; b=Jm0AC9XDtG0wf7Lzolh6Hn+C8/piVr8qMDb5VFHBf9BbBoaUeo0I5McYG2JJXQak9A 6lM/w+stXqvQW8FbDVRZiRZxdGCOxWE/8yZDIFK1AWbP2RIoYA99IYuZ5/6sE+MgmNtC TPu4oyWhIwiadgVYecqlGkiUJdLCGu5Ak8JjDm+n5x8QMCxFGUHkF6VOQxRQr5GiAN4h 2cEV1yvZiHTOggU8wITLIYqFuTgCTSksaNNAjwF6QDIt6RRqJh6StqdjgVm++l0L34Y4 H3Gdf6VFiWs4aiwhCLyAF2BZYMcbCr6K99CgCzc4wNt2PBC8peNV1/yL+MMXIwI4nxwX Novg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:cc:subject:references:to:cc:date :in-reply-to:message-id:user-agent:mime-version; bh=4zH24G1P6Yk80tSfIENeVF+uUHk44/HZVEZBTQ/Zauc=; b=A/ALxKyDSV0Jj6gIFwvYQtq6KMufRn8PhN2apI+kAoEtI45i8+DdaS3x/3y5MwsLDb BLwgXG/SpSdHD0Oj5vhk/nMogu88qtEZ7VWGTTuOoQbSbttz9Rp8eqANQyC/VMXAxJve K5gip4ddZ4hBIlaMyWyRZBDQiyyzpbMnu7ULsb+fNISoDnGBvsLeHQBRPc/SD5Y7Aj6N 2bwK4PPIWWXyoZ+K5ff999MX41W0UfMBOmF67JU127L8t/IvqGS4BRkDgotp1f5TUSy5 2O4QZ+7SlZCHHdjvhL+nkFbOJnZn3T5gqIvkYMLMwtzsWxCFZXi/C0SYXALISnMYX1RM MD0Q== X-Gm-Message-State: AOAM531+AVrsGq+iO5m6xGeJpn7B6K57KY4BIR/+FtCTlbhSggrmTdVs YZ7ZP3g2Yo++/5rkuUe2h6+VQ213wKQf7A== X-Google-Smtp-Source: ABdhPJy2Q3CRMjcwusHCQezPyBb3zxY+JT3HYF2vBW/dggA26BKjnquLmSdcX4GUosrv+WAynbdB9w== X-Received: by 2002:ad4:4b65:0:b0:456:4582:597c with SMTP id m5-20020ad44b65000000b004564582597cmr13591936qvx.33.1652672987282; Sun, 15 May 2022 20:49:47 -0700 (PDT) Received: from hurd (dsl-10-136-209.b2b2c.ca. [72.10.136.209]) by smtp.gmail.com with ESMTPSA id n188-20020a37bdc5000000b0069fc13ce1desm5224154qkf.15.2022.05.15.20.49.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 May 2022 20:49:46 -0700 (PDT) From: Maxim Cournoyer Subject: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> to: bug-guix Date: Sun, 15 May 2022 23:49:45 -0400 In-Reply-To: <87mtfj174l.fsf@gnu.org> (Mathieu Othacehe's message of "Sun, 15 May 2022 11:23:54 +0200") Message-ID: <878rr2kug6.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::f35; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: submit Cc: Mathieu Othacehe , guix-sysadmin X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.3 (/) Hello Guix, Since a couple days, Craps exhibits a new problem, where the evaluations on master appear stalled, showing as "In progress..." forever. Eventually, it seems the maximum number of PostgreSQL connections allowed (300, if what Ricardo wrote on #guix is accurate :-)) is reached, and we get the following error: --8<---------------cut here---------------start------------->8--- Backtrace: In ice-9/boot-9.scm: 1752:10 11 (with-exception-handler _ _ #:unwind? _ # _) In unknown file: 10 (apply-smob/0 #) In ice-9/boot-9.scm: 724:2 9 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 8 (_ #(#(#))) In cuirass/ui.scm: 104:10 7 (run-cuirass-command _ . _) In cuirass/scripts/evaluate.scm: 101:7 6 (cuirass-evaluate _) In cuirass/utils.scm: 123:6 5 (make-worker-thread-channel _ #:parallelism _) In srfi/srfi-1.scm: 634:9 4 (for-each # ?) In cuirass/utils.scm: 125:21 3 (_ . _) In cuirass/scripts/evaluate.scm: 101:7 2 (_) In cuirass/database.scm: 375:13 1 (db-open #:database _) In squee.scm: 257:10 0 (connect-to-postgres-paramstring _) squee.scm:257:10: In procedure connect-to-postgres-paramstring: Throw to key `psql-connect-error' with args `(connection-bad "FATAL: remaining connection slots are reserved for non-replication superuser connections\n")'. --8<---------------cut here---------------end--------------->8--- (see: https://ci.guix.gnu.org/eval/323648/log/raw) Restarting postgresql workarounds the above error, but it doesn't fix the hang, and the problem connection problem recurs. I've logged at the Cuirass and messages logs, but I haven't spot anything abnormal. Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 03:32:50 2022 Received: (at submit) by debbugs.gnu.org; 16 May 2022 07:32:50 +0000 Received: from localhost ([127.0.0.1]:51561 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqVDl-0007xH-VP for submit@debbugs.gnu.org; Mon, 16 May 2022 03:32:50 -0400 Received: from lists.gnu.org ([209.51.188.17]:52836) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqVDk-0007xA-Kz for submit@debbugs.gnu.org; Mon, 16 May 2022 03:32:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqVDj-00072o-J9; Mon, 16 May 2022 03:32:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37200) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqVDi-0001Yg-IN; Mon, 16 May 2022 03:32:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=DdZcq8jK6Gxd+M8buWDz9yjBTsHeo1FPSrwvgTcUPMo=; b=JE0M7e9ePg9PZePzPySm Ga2VvwLyb4WTjSG82DNsMWfBnJsYIHdGIncrJEWviu9egdAniS0ygkSqftWiD2HR+XdcecR2KPSi9 29QON0ZkY2oJyakkoqk7hmbHS5+0+bl9wrwZIcNOiVNPG8Eh0w13ZsMmS8Ry3q/bxE5n0JwAJfSgU YDF+06Pzaq7R0L0jmU8VN5eMpaMsU2oaoEjx2uzPzX2+tPoRcH9N/b21cwMFGHO8efuOO5ryawnbp VrDAG23O1TsC0MnJIyykEcMiqH+wb18UWOllN+6TvY3ho/HXNcUBbTYAfkVf/cqiXz8vxQg0rUiz5 GBTgTHj0aCffjA==; Received: from 2a02-8429-81d2-3d01-94c9-8097-ea5c-2774.rev.sfr.net ([2a02:8429:81d2:3d01:94c9:8097:ea5c:2774]:36304 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqVDi-0001Ue-0z; Mon, 16 May 2022 03:32:46 -0400 From: Mathieu Othacehe To: Maxim Cournoyer Subject: Re: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> Date: Mon, 16 May 2022 09:32:43 +0200 In-Reply-To: <878rr2kug6.fsf_-_@gmail.com> (Maxim Cournoyer's message of "Sun, 15 May 2022 23:49:45 -0400") Message-ID: <87pmke6ig4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , bug-guix , guix-sysadmin X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Maxim, Thanks for the report, the root cause here is the evaluation hanging. The Postgres slots starvation is just a symptom caused by having many concurrent (stuck) evaluation processes using slots. I noticed this error: https://ci.guix.gnu.org/eval/320682/log/raw. The following lines can be caused by having a multi-threaded program: (cuirass scripts evaluate) because of its par-for-each procedure, call "primitive-fork" in open-bidirectional-pipe of (guix inferior). --8<---------------cut here---------------start------------->8--- warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. --8<---------------cut here---------------end--------------->8--- Not sure that it could cause the observed hang, Ludo what do you think? Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 04:40:37 2022 Received: (at control) by debbugs.gnu.org; 16 May 2022 08:40:37 +0000 Received: from localhost ([127.0.0.1]:51679 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqWHN-0001GK-Fu for submit@debbugs.gnu.org; Mon, 16 May 2022 04:40:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqWHM-0001G2-1r for control@debbugs.gnu.org; Mon, 16 May 2022 04:40:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:38108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqWHF-0003ro-QV for control@debbugs.gnu.org; Mon, 16 May 2022 04:40:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=j6vt3NsJM56dDvB1eLSmtzeCuAlKZ+qUa0ontP0aseY=; b=QwvMf7ktTZmI2r y+zHhhVWX4UNokgePYHxouSHqjxo+bc5x+nIr3F7ozlIzkDMnymchPvlrWMRFNTERoCUBw91Jv0GL 3pjd9bYusJ7BTJ1GITxkZSWl//d5VayGweXiFJ4j2slHWXG+AGyM8L6SBjQ40PSYcmii3sEYn4JOt RnK+4JfyD0jLd0wnjCUKZVCgVc+h5Pxwx4/dq+7xN8qIWpD7gFzDsINCMSTX3Dc5HuQyI3TVeDlBL UvIqEvYLk7Z1XX1Y7n4PrJcv7A3Oy9NPMjr0zdA1hU1qI1JMkcY9vvXfcNpgtmeFSS8NbbFr7nya5 rI6ekJLDKtow9dyUYKbw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:55938 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqWH2-00049J-0y for control@debbugs.gnu.org; Mon, 16 May 2022 04:40:20 -0400 Date: Mon, 16 May 2022 10:40:14 +0200 Message-Id: <874k1pc1ld.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #55441 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) severity 55441 important quit From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 05:13:31 2022 Received: (at 55441) by debbugs.gnu.org; 16 May 2022 09:13:31 +0000 Received: from localhost ([127.0.0.1]:51732 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqWnC-0004K8-F6 for submit@debbugs.gnu.org; Mon, 16 May 2022 05:13:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqWnA-0004Ju-Iq for 55441@debbugs.gnu.org; Mon, 16 May 2022 05:13:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:38312) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqWn5-0000VP-BP; Mon, 16 May 2022 05:13:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=AxgMu2MwwSE1MdNbmVqw+FC3/3YhTm9q8GnKeOkwoJk=; b=IVuQC03w4ILhyBeiqAZp xedEy3Lqwly486MtiKfuYZymJipUP3/3YCfbDBT68doXCMj7F5Wgj6OIghyTM8nl3ZMh4XFwN65ln k8vUYfNLzwjeCxLiFRyoUikKDCQjF1Nlvi7gQWhuPwD1ttX9v6tYy0ul6PcabHuYyHN6lkl/I90rn Hy04C5cYG1kvyjUEykgsijIfvKd+v/ApKzsfBGqCNvFfnWepN/E9qtjoWovVe1aZakJNGTvFy+n7d AGcKEy4flXUwVl4yKmD2KqNOCBux9O5hkXjOiYpOnH6WK+YWh8RvJmdZMKR1oX29MhZfCfvSIGY1F gB8jprJTqe9yBw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:59284 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqWn2-0007Oc-Fe; Mon, 16 May 2022 05:13:20 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> Date: Mon, 16 May 2022 11:13:17 +0200 In-Reply-To: <87pmke6ig4.fsf@gnu.org> (Mathieu Othacehe's message of "Mon, 16 May 2022 09:32:43 +0200") Message-ID: <87o7zxalhu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, guix-sysadmin@gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Mathieu Othacehe skribis: > I noticed this error: https://ci.guix.gnu.org/eval/320682/log/raw. > > The following lines can be caused by having a multi-threaded program: > (cuirass scripts evaluate) because of its par-for-each procedure, call > "primitive-fork" in open-bidirectional-pipe of (guix inferior). > > warning: call to primitive-fork while multiple threads are running; > further behavior unspecified. See "Processes" in the > manual, for more information. > > Not sure that it could cause the observed hang, Ludo what do you think? As you write, the warning comes from =E2=80=98open-bidirectional-pipe=E2=80= =99, added in commit bd86bbd300474204878e927f6cd3f0defa1662a5. I noticed it back then but as far as I could see, it was harmless (but definitely suboptimal): https://issues.guix.gnu.org/48007#11 Indeed, =E2=80=9Cmake cuirass-jobs=E2=80=9D works for me. I decided to go ahead and run =E2=80=98cuirass evaluate=E2=80=99 by hand to= gather more info: --8<---------------cut here---------------start------------->8--- ludo@berlin ~$ sudo su - cuirass -s /bin/sh -c "/gnu/store/qamrzl942khb7w0c= b4csyflnxfkwvvxq-cuirass-1.1.0-11.9f08035/bin/cuirass evaluate dbname=3Dcui= rass 323183" Computing Guix derivation for 'x86_64-linux'... | warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. 2022-05-16T09:56:43 Registering builds for evaluation 323183. 2022-05-16T10:08:47 Registering builds for evaluation 323183. --8<---------------cut here---------------end--------------->8--- I straced it in another terminal. It did its work for a long while, and eventually activity stopped: --8<---------------cut here---------------start------------->8--- read(228, "nu/store/7gi4qsakz775z5wcmddg8xan7q806sj1-findutils-4.8.0\")) (#= :nix-name . \"findutils-4.8.0\") (#:sys"..., 4096) =3D 4096 read(228, "nrd-gcc-cross-x86_64-w64-mingw32-10.3.0.drv\" \"/gnu/store/ykrks= w421rd22zh4w3pc751r90wc881n-patch-2.7."..., 4096) =3D 4096 read(228, "wm-binutils-cross-x86_64-w64-mingw32-2.37.drv\" \"/gnu/store/xfh= 4965w2amrsl7x0h3dslrhndzn5bbz-linux-li"..., 4096) =3D 4096 read(228, "1ir-gawk-5.1.0.drv\" \"/gnu/store/p1d2w4fg1ljbyfs0ayv854snahcima= mk-libunistring-0.9.10.drv\" \"/gnu/stor"..., 4096) =3D 4096 read(228, "3ysc-guile-2.2.7.drv\" \"/gnu/store/b0ma6l61ad4dfb3030sl1ngvdlic= c523-glibc-utf8-locales-2.33.drv\" \"/gn"..., 4096) =3D 4096 read(228, "j6a5mc7w13xiw8136y4brv7w-xz-5.2.5.drv\" \"/gnu/store/w1bxsf02c4y= 0hj0nd39awv7i55ych641-libffi-3.3.drv\" "..., 4096) =3D 4096 read(228, "tore/qbqgamcpwgs2sw8lcr6pxl5y38fpyz0d-sed-4.8.drv\" \"/gnu/store= /qmsbxbgb758h6f7018snxlwn69w9w9i4-diff"..., 4096) =3D 4096 read(228, "oconf-2.69.drv\" \"/gnu/store/6n5mn2vkl4x4fab37sx1rmqyqxqcwd4y-l= ibidn2-2.3.1.drv\" \"/gnu/store/6q5cp0m2"..., 4096) =3D 4096 mmap(0x7fddd26af000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS= , -1, 0) =3D 0x7fddd268f000 read(228, "-binutils-cross-x86_64-w64-mingw32-2.37.drv\" \"/gnu/store/wdmk9= llhdzx5y10mziss1i5z6q037a7z-guile-3.0."..., 4096) =3D 954 futex(0x7fdf015381f0, FUTEX_WAKE_PRIVATE, 1) =3D 1 futex(0x7fdf01546b00, FUTEX_WAIT_PRIVATE, 2, NULL) =3D -1 EAGAIN (Resource = temporarily unavailable) futex(0x7fdf015381f4, FUTEX_WAKE_PRIVATE, 1) =3D 1 futex(0x7fdf01546b00, FUTEX_WAKE_PRIVATE, 1) =3D 1 futex(0x7fdf01538f74, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NU= LL, FUTEX_BITSET_MATCH_ANY) =3D ? ERESTARTSYS (To be restarted if SA_RESTAR= T is set) --- SIGPWR {si_signo=3DSIGPWR, si_code=3DSI_TKILL, si_pid=3D64599, si_uid= =3D997} --- rt_sigsuspend(~[INT QUIT ABRT BUS SEGV TERM XCPU RTMIN RT_1], 8) =3D ? ERES= TARTNOHAND (To be restarted if no handler) --- SIGXCPU {si_signo=3DSIGXCPU, si_code=3DSI_TKILL, si_pid=3D64599, si_uid= =3D997} --- rt_sigreturn({mask=3D~[INT QUIT ABRT BUS KILL SEGV TERM STOP RTMIN RT_1]}) = =3D -1 EINTR (Interrupted system call) rt_sigreturn({mask=3D[]}) =3D 202 futex(0x7fdf01538f74, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NU= LL, FUTEX_BITSET_MATCH_ANY) =3D ? ERESTARTSYS (To be restarted if SA_RESTAR= T is set) --- SIGPWR {si_signo=3DSIGPWR, si_code=3DSI_TKILL, si_pid=3D64599, si_uid= =3D997} --- rt_sigsuspend(~[INT QUIT ABRT BUS SEGV TERM XCPU RTMIN RT_1], 8) =3D ? ERES= TARTNOHAND (To be restarted if no handler) --- SIGXCPU {si_signo=3DSIGXCPU, si_code=3DSI_TKILL, si_pid=3D64599, si_uid= =3D997} --- rt_sigreturn({mask=3D~[INT QUIT ABRT BUS KILL SEGV TERM STOP RTMIN RT_1]}) = =3D -1 EINTR (Interrupted system call) rt_sigreturn({mask=3D[]}) =3D 202 futex(0x7fdf01538f74, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NU= LL, FUTEX_BITSET_MATCH_ANY) =3D ? ERESTARTSYS (To be restarted if SA_RESTAR= T is set) [=E2=80=A6] --- SIGXCPU {si_signo=3DSIGXCPU, si_code=3DSI_TKILL, si_pid=3D64599, si_uid= =3D997} --- rt_sigreturn({mask=3D~[INT QUIT ABRT BUS KILL SEGV TERM STOP RTMIN RT_1]}) = =3D -1 EINTR (Interrupted system call) rt_sigreturn({mask=3D[]}) =3D 202 futex(0x7fdf01538f74, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NU= LL, FUTEX_BITSET_MATCH_ANY) =3D 0 futex(0x7fdf01546c00, FUTEX_WAKE_PRIVATE, 1) =3D 0 close(224) =3D 0 futex(0x7fdf01538f70, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NU= LL, FUTEX_BITSET_MATCH_ANY [stops here] --8<---------------cut here---------------end--------------->8--- The =E2=80=98read=E2=80=99 calls are about getting the jobset alists; FD 22= 8 is a socket (to a =E2=80=98guix repl=E2=80=99 subprocess I presume). SIGPWR/SIGXCPU su= ggests attempts to make a stop-the-world GC. GDB shows that the process has 120 threads (it=E2=80=99s a 96-core machine,= but still=E2=80=A6) and most threads are stuck in pthread_cond_wait: --8<---------------cut here---------------start------------->8--- (gdb) bt #0 0x00007fdf01d62afa in __futex_abstimed_wait_common64 (futex_word=3Dfute= x_word@entry=3D0x7fded20cd8b4,=20 expected=3Dexpected@entry=3D0, clockid=3Dclockid@entry=3D0, abstime=3Da= bstime@entry=3D0x0, private=3Dprivate@entry=3D0,=20 cancel=3Dcancel@entry=3Dtrue) at ../sysdeps/nptl/futex-internal.c:74 #1 0x00007fdf01d62b5b in __GI___futex_abstimed_wait_cancelable64 (futex_wo= rd=3Dfutex_word@entry=3D0x7fded20cd8b4,=20 expected=3Dexpected@entry=3D0, clockid=3Dclockid@entry=3D0, abstime=3Da= bstime@entry=3D0x0, private=3Dprivate@entry=3D0) at ../sysdeps/nptl/futex-internal.c:123 #2 0x00007fdf01d5cab0 in __pthread_cond_wait_common (abstime=3D0x0, clocki= d=3D0, mutex=3D0x7fdef9c15fc0,=20 cond=3Dcond@entry=3D0x7fded20cd888) at pthread_cond_wait.c:504 #3 __pthread_cond_wait (cond=3Dcond@entry=3D0x7fded20cd888, mutex=3Dmutex@= entry=3D0x7fdef9c15fc0) at pthread_cond_wait.c:619 #4 0x00007fdf01ea2a55 in scm_pthread_cond_wait (cond=3Dcond@entry=3D0x7fde= d20cd888, mutex=3Dmutex@entry=3D0x7fdef9c15fc0) at threads.c:1605 #5 0x00007fdf01ea7b6b in block_self ( queue=3D((#) # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #= # # # # # # # # = # # # # # # # # #= # # # # # # # #<= smob thread 7fdedfa52e90> # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #), mutex=3Dmutex@entry=3D0x7fdef9c15fc0, waittime=3Dwa= ittime@entry=3D0x0) at threads.c:312 #6 0x00007fdf01eaa1cd in timed_wait (waittime=3D0x0, current_thread=3D0x7f= ded20cd6c0, c=3D0x7fdef9c10d20,=20 m=3D0x7fdef9c15fc0, kind=3DSCM_MUTEX_STANDARD) at threads.c:1331 #7 scm_timed_wait_condition_variable (cond=3D#,=20 mutex=3D#, timeout=3D#) at thre= ads.c:1424 #8 0x00007fded169a7f3 in ?? () #9 0x0000000000b83688 in ?? () #10 0x00007fded169a750 in ?? () #11 0x00007fdeec009e98 in ?? () #12 0x00007fdf01e57ccc in scm_jit_enter_mcode (thread=3D0x7fded20cd6c0, mco= de=3D0xb8369c "\034\330\003") at jit.c:6038 #13 0x00007fdf01eacf3c in vm_regular_engine (thread=3D0x7fded20cd6c0) at vm= -engine.c:360 #14 0x00007fdf01eba5e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #15 0x00007fdf01e1ea0e in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:406 #16 0x00007fdf01ead336 in vm_regular_engine (thread=3D0x7fded20cd6c0) at vm= -engine.c:972 #17 0x00007fdf01eba5e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #18 0x00007fdf01ea9be6 in really_launch (d=3D0x7fdef730c860) at threads.c:7= 78 #19 0x00007fdf01e2085a in c_body (d=3D0x7fdec8c44d80) at continuations.c:430 #20 0x00007fded2874532 in ?? () #21 0x00007fdef9a227e0 in ?? () #22 0x00007fdef7d8c370 in ?? () #23 0x0000000000000048 in ?? () #24 0x00007fdf01e57ccc in scm_jit_enter_mcode (thread=3D0x7fded20cd6c0, mco= de=3D0xb81874 "\034<\003") at jit.c:6038 #25 0x00007fdf01eacf3c in vm_regular_engine (thread=3D0x7fded20cd6c0) at vm= -engine.c:360 #26 0x00007fdf01eba5e9 in scm_call_n (proc=3D, argv=3D, nargs=3D2) at vm.c:1608 #27 0x00007fdf01e2209a in scm_call_2 (proc=3D, arg1=3D, arg2=3D) at eval.c:503 #28 0x00007fdf01ed8752 in scm_c_with_exception_handler.constprop.0 (type=3D= #t,=20 handler_data=3Dhandler_data@entry=3D0x7fdec8c44d10, thunk_data=3Dthunk_= data@entry=3D0x7fdec8c44d10,=20 thunk=3D, handler=3D) at exceptions.c:170 #29 0x00007fdf01eaa88f in scm_c_catch (tag=3D, body=3D, body_data=3D,=20 handler=3D, handler_data=3D, pre_unwind_h= andler=3D,=20 pre_unwind_handler_data=3D0x7fdef9b97040) at throw.c:168 #30 0x00007fdf01e22e66 in scm_i_with_continuation_barrier (pre_unwind_handl= er=3D0x7fdf01e22b80 ,=20 pre_unwind_handler_data=3D0x7fdef9b97040, handler_data=3D0x7fdec8c44d80= , handler=3D0x7fdf01e298b0 ,=20 body_data=3D0x7fdec8c44d80, body=3D0x7fdf01e20850 ) at continua= tions.c:368 #31 scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:464 #32 0x00007fdf01ea9b39 in with_guile (base=3D0x7fdec8c44e08, data=3D0x7fdec= 8c44e30) at threads.c:645 #33 0x00007fdf01d800ba in GC_call_with_stack_base () from /gnu/store/2lczkxbdbzh4gk7wh91bzrqrk7h5g1dl-libgc-8.0.4/lib/libgc.s= o.1 #34 0x00007fdf01ea216d in scm_i_with_guile (dynamic_state=3D= , data=3D0x7fdef730c860,=20 func=3D0x7fdf01ea9b70 ) at threads.c:688 #35 launch_thread (d=3D0x7fdef730c860) at threads.c:787 #36 0x00007fdf01d56d7e in start_thread (arg=3D0x7fdec8c45640) at pthread_cr= eate.c:473 #37 0x00007fdf01954eff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clo= ne.S:95 --8<---------------cut here---------------end--------------->8--- There=E2=80=99s also one thread stuck in read(2): --8<---------------cut here---------------start------------->8--- (gdb) bt #0 __libc_read (nbytes=3D1, buf=3D0x7fdedf97bdd0, fd=3D7) at ../sysdeps/un= ix/sysv/linux/read.c:26 #1 __libc_read (fd=3D7, buf=3Dbuf@entry=3D0x7fdedf97bdd0, nbytes=3Dnbytes@= entry=3D1) at ../sysdeps/unix/sysv/linux/read.c:24 #2 0x00007fdf01e382e8 in fport_read (port=3D, dst=3D, start=3D, count=3D1) at fports.c:597 #3 0x00007fdf01e78d22 in scm_i_read_bytes (port=3Dport@entry=3D# 7fdef59b7180>,=20 dst=3D"#" =3D {...}, start=3Dstart@entry=3D0, count=3D1) at = ports.c:1566 #4 0x00007fdf01e7b1c7 in scm_fill_input (port=3Dport@entry=3D# 7fdef59b7180>,=20 minimum_size=3D1, minimum_size@entry=3D0, cur_out=3Dcur_out@entry=3D0x7= fde86fcc3e8,=20 avail_out=3Davail_out@entry=3D0x7fde86fcc3e0) at ports.c:2693 #5 0x00007fdf01e7b434 in peek_byte_or_eof (port=3Dport@entry=3D# 7fdef59b7180>,=20 buf_out=3Dbuf_out@entry=3D0x7fde86fcc448, cur_out=3Dcur_out@entry=3D0x7= fde86fcc440) at ports.c:1526 #6 0x00007fdf01e7c1b5 in maybe_consume_bom (port=3Dport@entry=3D# 7fdef59b7180>,=20 bom=3Dbom@entry=3D0x7fdf01ee56ad "\357\273\277scm_i_read= _bytes", bom_len=3Dbom_len@entry=3D3) at ports.c:2468 #7 0x00007fdf01e7b324 in port_clear_stream_start_for_bom_read (io_mode=3DB= OM_IO_TEXT,=20 port=3D# 7fdef59b7180>) at ports.c:= 2504 #8 scm_fill_input (port=3Dport@entry=3D# 7fdef59b7180>, minimum_size=3D1,=20 minimum_size@entry=3D0, cur_out=3Dcur_out@entry=3D0x7fde86fcc4e8, avail= _out=3Davail_out@entry=3D0x7fde86fcc4e0) at ports.c:2648 #9 0x00007fdf01e7b434 in peek_byte_or_eof (port=3Dport@entry=3D# 7fdef59b7180>,=20 buf_out=3Dbuf_out@entry=3D0x7fde86fcc548, cur_out=3Dcur_out@entry=3D0x7= fde86fcc540) at ports.c:1526 #10 0x00007fdf01e7b97c in peek_utf8_codepoint (port=3D# 7fdef59b7180>,=20 buf_out=3Dbuf_out@entry=3D0x7fde86fcc5a8, cur_out=3Dcur_out@entry=3D0x7= fde86fcc5a0, len_out=3Dlen_out@entry=3D0x7fde86fcc598) at ports.c:1790 #11 0x00007fdf01e7bee4 in peek_codepoint (len=3D0x7fde86fcc598, cur=3D0x7fd= e86fcc5a0, buf=3D0x7fde86fcc5a8,=20 port=3D) at ports.c:1984 #12 scm_peek_char (port=3D) at ports.c:2202 #13 0x00007fdef997b1ab in ?? () #14 0x0000000000b82ae0 in ?? () #15 0x00007fdef997b120 in ?? () #16 0x00007fdec4292c48 in ?? () #17 0x00007fdf01e57ccc in scm_jit_enter_mcode (thread=3D0x7fded1366000, mco= de=3D0xb82af0 "\034\234\003") at jit.c:6038 #18 0x00007fdf01eacf3c in vm_regular_engine (thread=3D0x7fded1366000) at vm= -engine.c:360 #19 0x00007fdf01eba5e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #20 0x00007fdf01e1ea0e in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:406 --8<---------------cut here---------------end--------------->8--- A plausible scenario is a synchronization issue: =E2=80=98cuirass evaluate= =E2=80=99 got all the data it asked for from =E2=80=98guix repl=E2=80=99, but it=E2=80=99= s still waiting on read(2) even though there=E2=80=99s nothing left. (There are two child =E2= =80=98guix repl=E2=80=99 processes and both are waiting on read(0, =E2=80=A6).) Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 06:32:18 2022 Received: (at 55441) by debbugs.gnu.org; 16 May 2022 10:32:18 +0000 Received: from localhost ([127.0.0.1]:51831 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqY1R-0006Wx-QH for submit@debbugs.gnu.org; Mon, 16 May 2022 06:32:17 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53528) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqY1Q-0006Wj-AD for 55441@debbugs.gnu.org; Mon, 16 May 2022 06:32:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39032) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqY1K-0004Xz-V5; Mon, 16 May 2022 06:32:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=fry40eOl0ZuHZEZKGbbbwGpp23c9Gn78XbczXdqSR1A=; b=oOegRBb0oQ/gMZIJI2lR RJYkCppAw0Vw78XtIcRe4yZun3lwC4udrkwgrIdLUvHKjJXIU6RWUYFarFQ/kX0xd7FXfziSYrmY9 762kQ/75YYWAIZyJhwBy7AKqn2BnaAu0wbZVgrwCgiBcnbFyjZUnnEYijt4Yt5CHxgSdLcR8uOto0 s7VPjy1EVD/J1dCCSF8bdpdVQnZaliXzsto6Qsvk5Y9vx6KtQvs+mDkQbqNM7MAzSvatR1asfvSbe xkrwJoexNEf0RNw9wWsfYFZxnMwTldsdh4F0oPJ+4kCGMQ1eeF3e12HfOJmWgwKZfgTs9RMpiX7A+ o72UVj/DO/TPxg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:62756 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqY1K-0002aP-Fb; Mon, 16 May 2022 06:32:10 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> Date: Mon, 16 May 2022 12:32:07 +0200 In-Reply-To: <87o7zxalhu.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 16 May 2022 11:13:17 +0200") Message-ID: <87k0alahug.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, guix-sysadmin@gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > A plausible scenario is a synchronization issue: =E2=80=98cuirass evaluat= e=E2=80=99 got > all the data it asked for from =E2=80=98guix repl=E2=80=99, but it=E2=80= =99s still waiting on > read(2) even though there=E2=80=99s nothing left. (There are two child = =E2=80=98guix > repl=E2=80=99 processes and both are waiting on read(0, =E2=80=A6).) This reminds me of: https://github.com/UMCUGenetics/hpcguix-web/commit/88b07caaf3084f48b181c3= f3f84af27dc9b834ec I=E2=80=99ve added a missing call to =E2=80=98close-inferior=E2=80=99. It= =E2=80=99s a good idea, though I=E2=80=99m not entirely convinced yet it=E2=80=99ll solve the problem: https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3Df087= aaf685dbc7cc18f0254895f4a4b0dfaba631 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 08:20:03 2022 Received: (at 55441) by debbugs.gnu.org; 16 May 2022 12:20:04 +0000 Received: from localhost ([127.0.0.1]:51989 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqZhj-0005ZR-B7 for submit@debbugs.gnu.org; Mon, 16 May 2022 08:20:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48130) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqZhg-0005Yg-1p for 55441@debbugs.gnu.org; Mon, 16 May 2022 08:20:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40536) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqZha-000644-Pm; Mon, 16 May 2022 08:19:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=1SEo6lGUgX4LIuSSQVTOBTrIj1kd5ZmUcELum758c5s=; b=QfX5S7K4hZji8n5YGo/Q BN57vLSNMKPdZdBZW+GK4EFXHtlApQu6hvpTyPaYpeRMXVrywC1xdZaoVJN1rgQwK6OOniDkT+UaT xu5kuIy+GWeZCeVPgdUFJEPfu2t1Vz107LrClgQ2X5EOkX/zS8xYXb8upzdPPhhFvv9VVLWk4GWMM DrREUZIOWgbnAFIZbgihRJMtXtV7JdPpWZoe9j8sWjqV3WC1KfMb5Y7ZgbhwxOumN7o3CKhciIzR+ Qc6SsR0j/sTyZRjtSaMH8uXlqaY4spNIZib0kkBdeLHRsgSonE9B7b7y0yr3ekiF59aoUPSG0M3dW kT5Tkem6uYhRrQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53358 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqZha-00069H-5v; Mon, 16 May 2022 08:19:54 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> Date: Mon, 16 May 2022 14:19:51 +0200 In-Reply-To: <87k0alahug.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 16 May 2022 12:32:07 +0200") Message-ID: <87fsl9acuw.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, guix-sysadmin@gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > I=E2=80=99ve added a missing call to =E2=80=98close-inferior=E2=80=99. I= t=E2=80=99s a good idea, though > I=E2=80=99m not entirely convinced yet it=E2=80=99ll solve the problem: > > https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3Df0= 87aaf685dbc7cc18f0254895f4a4b0dfaba631 I tested it like this and it ran to completion without hanging: --8<---------------cut here---------------start------------->8--- ludo@berlin ~$ guix build cuirass --with-branch=3Dcuirass=3Dmaster [=E2=80=A6] /gnu/store/sqsf2q3qf9d485mcw6lm14abwr54na01-cuirass-git.master ludo@berlin ~$ sudo su - cuirass -s /bin/sh -c "/gnu/store/sqsf2q3qf9d485m= cw6lm14abwr54na01-cuirass-git.master/bin/cuirass evaluate dbname=3Dcuirass = 323183" Password: Computing Guix derivation for 'x86_64-linux'... \ 2022-05-16T12:39:18 Registering builds for evaluation 323183. 2022-05-16T12:40:34 Registering builds for evaluation 323183. GC Warning: Repeated allocation of very large block (appr. size 14385152): May lead to memory leak and poor performance 2022-05-16T12:41:25 Registering builds for evaluation 323183. 2022-05-16T12:44:35 Registering builds for evaluation 323183. --8<---------------cut here---------------end--------------->8--- Should we try and deploy this commit on berlin? Maxim, how can we proceed? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 08:49:38 2022 Received: (at 55441) by debbugs.gnu.org; 16 May 2022 12:49:38 +0000 Received: from localhost ([127.0.0.1]:52065 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqaAL-0002Op-KK for submit@debbugs.gnu.org; Mon, 16 May 2022 08:49:37 -0400 Received: from mail-qt1-f180.google.com ([209.85.160.180]:44898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqaAJ-0002OY-MX for 55441@debbugs.gnu.org; Mon, 16 May 2022 08:49:36 -0400 Received: by mail-qt1-f180.google.com with SMTP id i20so11897867qti.11 for <55441@debbugs.gnu.org>; Mon, 16 May 2022 05:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=VOJpJazdtnu+Tqquzi4x/W4a4MRB4ugyc+6jZm6qky4=; b=Yo2LaC8ZjnR/1wrQAcdrqiCgO2xVSLyFbKT7J9Z/a3pCqBl9mw+rOCmQ5/Vq7nN4Bx rxf9SvUMZRY3brm9OHfefTEnqH0QcR11VJq7HvOO3jan/69WYGgYjWOv/0gdU0/GmJ+i li5MAbePd8iLEig5FhuPpS1/CenA7pJgNJ2acNg1HdU6IV6EnvyaHFeWD3IG+LsaPvrK oo8Lftx0E1zsxd1qpEqFdC+VFcr14ctr1nowL/V3pHW/HOCmbVuQagVWXWdnrldR+HX0 gDWN6fRnvlGyuoYZq3eLzfNriv756E4knwGJvJErD1DjvDIvpepd+HVxPxTY4ptMjEAe CNig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=VOJpJazdtnu+Tqquzi4x/W4a4MRB4ugyc+6jZm6qky4=; b=Fb5BEh2k3y412jiBCMCJhc2T6nQrRcxUO6kTR0Zq3X/2IGZ3Q+hF151kPbx+RpnEGc N7kknYNR+tRdCtlQDkHm3onNCxmZYxPD2KJliDZtxEXrGgFRQIHjUbPYgmWyAe21lIcG yBKDbvUWcthUm4COSeUXmd8gPnpAm+MOwJ4aoy1AYJ5vsZfgx5tadHHFnMYrmWxt53Lo HVespEJcM+soQlJODzPUgOQHLppCWExE7JQfaHP0mxXOwoEpk+ZRAifQyE569YGJlgyF 18XKB80pdNemITDFrIOSM0U2hjmQnrmqRaDmg4QipEKzOu26IRM6w9MZwj1h6L9acSFL u8SQ== X-Gm-Message-State: AOAM530oHmS6xZANoW94fhnmKTnjlpJj30iD9Bh/sRZyHCHoMnvP7kGc vrJ6XncNquXLGqDFvfnJIqoPrt5F7Gd8tA== X-Google-Smtp-Source: ABdhPJyCaa+OqHYJQcPhu35LhQ3+DyElf9suo7KYuDSviUMIMetI++TJFXIL+a9gXAnCu9EOJ4MltA== X-Received: by 2002:ac8:5b04:0:b0:2f3:ec8f:1b0a with SMTP id m4-20020ac85b04000000b002f3ec8f1b0amr14990340qtw.327.1652705369930; Mon, 16 May 2022 05:49:29 -0700 (PDT) Received: from hurd (dsl-10-136-209.b2b2c.ca. [72.10.136.209]) by smtp.gmail.com with ESMTPSA id v7-20020ac873c7000000b002f39b99f692sm5691471qtp.44.2022.05.16.05.49.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 May 2022 05:49:29 -0700 (PDT) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> Date: Mon, 16 May 2022 08:49:28 -0400 In-Reply-To: <87o7zxalhu.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 16 May 2022 11:13:17 +0200") Message-ID: <87h75pk5gn.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo and Mathieu! Ludovic Court=C3=A8s writes: > Hi! > > Mathieu Othacehe skribis: > >> I noticed this error: https://ci.guix.gnu.org/eval/320682/log/raw. >> >> The following lines can be caused by having a multi-threaded program: >> (cuirass scripts evaluate) because of its par-for-each procedure, call >> "primitive-fork" in open-bidirectional-pipe of (guix inferior). >> >> warning: call to primitive-fork while multiple threads are running; >> further behavior unspecified. See "Processes" in the >> manual, for more information. >> >> Not sure that it could cause the observed hang, Ludo what do you think? > > As you write, the warning comes from =E2=80=98open-bidirectional-pipe=E2= =80=99, added in > commit bd86bbd300474204878e927f6cd3f0defa1662a5. I noticed it back then > but as far as I could see, it was harmless (but definitely suboptimal): > > https://issues.guix.gnu.org/48007#11 Thanks for investigating it! I'm not yet very knowledgeable about POSIX threads and shortcomings, but this looks like a dangerous trap to have lying around :-). Is there an alternative implementation we could come up with that would avoid the potential problem (and warnings) entirely? Otherwise, if we are sure there are no practical problems in a multi-threaded scenario, I think a comment explaining the reasoning in the code (of why it's considered safe despite the warnings) and a test exercising the behavior would be in order to ensure it continues working as intended in the face of changes. Thoughts? Thanks again, Maxim From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 13:32:36 2022 Received: (at 55441) by debbugs.gnu.org; 16 May 2022 17:32:36 +0000 Received: from localhost ([127.0.0.1]:54974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqeaB-0000K8-QY for submit@debbugs.gnu.org; Mon, 16 May 2022 13:32:36 -0400 Received: from mail-qv1-f53.google.com ([209.85.219.53]:38721) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqeaA-0000Jv-Bs for 55441@debbugs.gnu.org; Mon, 16 May 2022 13:32:34 -0400 Received: by mail-qv1-f53.google.com with SMTP id n10so12639618qvi.5 for <55441@debbugs.gnu.org>; Mon, 16 May 2022 10:32:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=AyDulEWULlZfb/9I8wOv1T8PzgKap1nzqvShncYRotQ=; b=c18Ch8TbFdQItk46MvQIOvka0CxeJvl4obbX1IV2QjgeKNIeT443pv7Mx58O/ZXYhs VLf0ifIdDeKfI18iNyyv3swsFIByvUKVv2Hfiuk1gPkfvn0Bl49PBhob3AalPrU5LTsa h/XSwQ791hbO0H0As1cHst5MtsBxln7eNSnG9ADYQrUggoUrp3HbupQ4LsFJr5p2iGD0 H2RGUzYYhrv9TBApd3FwrwpmI3xvQ9ra1GrmzghrjIdQviuZe65PtdA3YoqU/c4lydoQ n6PX9lwP7LV1z/8m7SM7BTXE3HIjggzWXpnuBdMyf7TYJ+90X0Eh/YWYw2lIoCTUTRIG qInQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=AyDulEWULlZfb/9I8wOv1T8PzgKap1nzqvShncYRotQ=; b=KqfIe51h5jZXVH0SxLWgSLbq/MywIzcakUnKml/0h97fApNFBquSm23R9/UQKxWL3K ohknE9CfpaUzNgZaHxsS8wD56kwjeTi71fPZ30U5gmsl9T/0TlWsvi4DU9j7rRz87Pns x8nME5RJVfPrAJwcTuinlywErD/dMn+bHvfODRpE1izJvu0Yo3q1SNBUzW8lytdxYgGB rIwhxQv95YQLF/3n5u3rVDrVUTZ3qhJ8icCsFgFK8BNrIGA5y48tmBAcUgCtRg4h+N+g 5b/u60OyS7PXrLprOuon73t2p/oWDrdGm/7icWoqrTk5r4P87IPERlf5U0oY2f4yuL9w EmPg== X-Gm-Message-State: AOAM533L1nr3PwKDNmM8C2zOghy6scZvw4PiEJWpu8U9lOffEgFXOdwl UMq5wREmQVDZAuzyNqX2WV1Lsn6LAfi79A== X-Google-Smtp-Source: ABdhPJzW6cnl7kIXnF7thKJdwkXK/f2AJIB5f27yKh6egJsAhqZ0aAK7d49gfsVap7zvnFo7IGB6wQ== X-Received: by 2002:a05:6214:410c:b0:45a:7a94:ee04 with SMTP id kc12-20020a056214410c00b0045a7a94ee04mr16353863qvb.63.1652722348407; Mon, 16 May 2022 10:32:28 -0700 (PDT) Received: from hurd (dsl-10-136-209.b2b2c.ca. [72.10.136.209]) by smtp.gmail.com with ESMTPSA id l2-20020a37bb02000000b0069fc13ce21esm5972708qkf.79.2022.05.16.10.32.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 May 2022 10:32:28 -0700 (PDT) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> Date: Mon, 16 May 2022 13:32:26 -0400 In-Reply-To: <87fsl9acuw.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 16 May 2022 14:19:51 +0200") Message-ID: <878rr1jsd1.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Ludovic Court=C3=A8s writes: > Ludovic Court=C3=A8s skribis: > >> I=E2=80=99ve added a missing call to =E2=80=98close-inferior=E2=80=99. = It=E2=80=99s a good idea, though >> I=E2=80=99m not entirely convinced yet it=E2=80=99ll solve the problem: >> >> https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3Df= 087aaf685dbc7cc18f0254895f4a4b0dfaba631 > > I tested it like this and it ran to completion without hanging: > > ludo@berlin ~$ guix build cuirass --with-branch=3Dcuirass=3Dmaster > [=E2=80=A6] > /gnu/store/sqsf2q3qf9d485mcw6lm14abwr54na01-cuirass-git.master > ludo@berlin ~$ sudo su - cuirass -s /bin/sh -c "/gnu/store/sqsf2q3qf9d48= 5mcw6lm14abwr54na01-cuirass-git.master/bin/cuirass evaluate dbname=3Dcuiras= s 323183" > Password: > Computing Guix derivation for 'x86_64-linux'... \ > 2022-05-16T12:39:18 Registering builds for evaluation 323183. > 2022-05-16T12:40:34 Registering builds for evaluation 323183. > GC Warning: Repeated allocation of very large block (appr. size 14385152): > May lead to memory leak and poor performance > 2022-05-16T12:41:25 Registering builds for evaluation 323183. > 2022-05-16T12:44:35 Registering builds for evaluation 323183. > > Should we try and deploy this commit on berlin? > > Maxim, how can we proceed? Berlin was reconfigured with this commit of Cuirass, and is now running the derivations with it, but so far still "In progress..." after more than 100 minutes [0] [0] https://ci.guix.gnu.org/eval/325592 Looking in /var/log/cuirass.log, I can see: --8<---------------cut here---------------start------------->8--- 2022-05-16 15:57:41 2022-05-16T15:57:41 next evaluation in 300 seconds 2022-05-16 15:59:02 Uncaught exception in fiber ##f: 2022-05-16 15:59:02 In cuirass/base.scm: 2022-05-16 15:59:02 726:13 3 (_) 2022-05-16 15:59:02 In ice-9/boot-9.scm: 2022-05-16 15:59:02 1752:10 2 (with-exception-handler _ _ #:unwind? _ # = _) 2022-05-16 15:59:02 1685:16 1 (raise-exception _ #:continuable? _) 2022-05-16 15:59:02 1683:16 0 (raise-exception _ #:continuable? _) 2022-05-16 15:59:02 ice-9/boot-9.scm:1683:16: In procedure raise-exception: 2022-05-16 15:59:02 ERROR: 2022-05-16 15:59:02 1. &evaluation-error: 2022-05-16 15:59:02 name: "core" 2022-05-16 15:59:02 id: 325375 2022-05-16 16:02:41 2022-05-16T16:02:41 Fetching channels for spec 'core'. --8<---------------cut here---------------end--------------->8--- But it seems it should only affect another job specification ('core') ? Before, there was also: --8<---------------cut here---------------start------------->8--- 2022-05-16 15:41:34 Uncaught exception in fiber ##f: 2022-05-16 15:41:34 In cuirass/base.scm: 2022-05-16 15:41:34 726:13 3 (_) 2022-05-16 15:41:34 In ice-9/boot-9.scm: 2022-05-16 15:41:34 1752:10 2 (with-exception-handler _ _ #:unwind? _ # = _) 2022-05-16 15:41:34 1685:16 1 (raise-exception _ #:continuable? _) 2022-05-16 15:41:34 1683:16 0 (raise-exception _ #:continuable? _) 2022-05-16 15:41:34 ice-9/boot-9.scm:1683:16: In procedure raise-exception: 2022-05-16 15:41:34 ERROR: 2022-05-16 15:41:34 1. &evaluation-error: 2022-05-16 15:41:34 name: "purge-python2-packages" 2022-05-16 15:41:34 id: 325332 --8<---------------cut here---------------end--------------->8--- and --8<---------------cut here---------------start------------->8--- 2022-05-16 15:05:27 Uncaught exception in fiber ##f: 2022-05-16 15:05:27 In cuirass/base.scm: 2022-05-16 15:05:27 726:13 3 (_) 2022-05-16 15:05:27 In ice-9/boot-9.scm: 2022-05-16 15:05:27 1752:10 2 (with-exception-handler _ _ #:unwind? _ # = _) 2022-05-16 15:05:27 1685:16 1 (raise-exception _ #:continuable? _) 2022-05-16 15:05:27 1683:16 0 (raise-exception _ #:continuable? _) 2022-05-16 15:05:27 ice-9/boot-9.scm:1683:16: In procedure raise-exception: 2022-05-16 15:05:27 ERROR: 2022-05-16 15:05:27 1. &evaluation-error: 2022-05-16 15:05:27 name: "master" 2022-05-16 15:05:27 id: 324938 --8<---------------cut here---------------end--------------->8--- and --8<---------------cut here---------------start------------->8--- 2022-05-16 13:02:07 Uncaught exception in fiber ##f: 2022-05-16 13:02:07 In cuirass/base.scm: 2022-05-16 13:02:07 726:13 3 (_) 2022-05-16 13:02:07 In ice-9/boot-9.scm: 2022-05-16 13:02:07 1752:10 2 (with-exception-handler _ _ #:unwind? _ # = _) 2022-05-16 13:02:07 1685:16 1 (raise-exception _ #:continuable? _) 2022-05-16 13:02:07 1683:16 0 (raise-exception _ #:continuable? _) 2022-05-16 13:02:07 ice-9/boot-9.scm:1683:16: In procedure raise-exception: 2022-05-16 13:02:07 ERROR: 2022-05-16 13:02:07 1. &evaluation-error: 2022-05-16 13:02:07 name: "guix" 2022-05-16 13:02:07 id: 324937 --8<---------------cut here---------------end--------------->8--- I don't know if these are related or not; probably not, as their timestamps are older by more than 3 hours, while the last derivations were started less than 1h30 ago. Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Tue May 17 03:46:05 2022 Received: (at 55441) by debbugs.gnu.org; 17 May 2022 07:46:05 +0000 Received: from localhost ([127.0.0.1]:55692 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqru8-0003QV-Td for submit@debbugs.gnu.org; Tue, 17 May 2022 03:46:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqru6-0003Pt-PP for 55441@debbugs.gnu.org; Tue, 17 May 2022 03:46:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54090) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqru1-0002fA-DU; Tue, 17 May 2022 03:45:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=JcteS0APi5I+vA9mFPQ5hZ63zVVlxsSFn3uYQeNDeYA=; b=W02ufWeKcK1lelvRHj1p AAoKCEXXgi/FxO6zad7we05bINPJvdvRQZmuyaLY+7aMDmJn9vsofRefpOLXKc2hgVdAAaRGZomVK za5tmOsIrdBJhCIBcWoM+v3jhxrnAQgxTCsHZNTaMqfPeKUlJu5BnFI7fY4suaZ/selv2QlU1ylhp 7F2y7d65EcRM35AIGdiOxRxLpLLDrB9x5mfy/zrRg0HY4yAMyVOycD0vRu21C2iQT09MHZRL3Ch2Z WophmnmxJjrlgw0GT+r5He73jC4scYy+jsQ7pZx0uA5UAY4Ftk5mozP0XHUTJoGrl8StHUlYgYJsG iNN0PRKDmn+cPA==; Received: from [193.50.110.244] (port=52728 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqrtx-0001dI-5L; Tue, 17 May 2022 03:45:53 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> Date: Tue, 17 May 2022 09:45:50 +0200 In-Reply-To: <878rr1jsd1.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 16 May 2022 13:32:26 -0400") Message-ID: <87fsl87gb5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Maxim Cournoyer skribis: > Berlin was reconfigured with this commit of Cuirass, and is now running > the derivations with it, but so far still "In progress..." after more > than 100 minutes [0] Yeah, I=E2=80=99m not sure about the backtrace you report, however there=E2= =80=99s again a bunch of =E2=80=98cuirass evaluate=E2=80=99 processes hanging, this time = with the main thread stuck on =E2=80=98waitpid=E2=80=99 (presumably from the =E2=80=98clo= se-inferior=E2=80=99 call): --8<---------------cut here---------------start------------->8--- #0 0x00007f0886310f27 in __GI___wait4 (pid=3D86099, stat_loc=3Dstat_loc@en= try=3D0x7ffea0cb849c, options=3Doptions@entry=3D0, usage=3Dusage@entry=3D0x= 0) at ../sysdeps/unix/sysv/linux/wait4.c:30 #1 0x00007f0886310ea7 in __GI___waitpid (pid=3D, stat_loc= =3Dstat_loc@entry=3D0x7ffea0cb849c, options=3Doptions@entry=3D0) at waitpid.c:38 #2 0x00007f08868ae25e in scm_waitpid (pid=3D86099, options=3D) at posix.c:727 #3 0x00007f088689a336 in vm_regular_engine (thread=3D0x7f08861fad80) at vm= -engine.c:972 #4 0x00007f08868a75e9 in scm_call_n (proc=3D, argv=3D, nargs=3D1) at vm.c:1608 #5 0x00007f088680f457 in scm_primitive_eval (exp=3D, exp@entry=3D((@ (ice-9 control) %) (begin ((@@ (ice-9 command-line) loa= d/lang) "/gnu/store/z8haznhwck4bjm4gxqy25wvwv4041wvx-cuirass-1.1.0-12.f087a= af/bin/.cuirass-real") (main (command-line)) (quit)))) at eval.c:671 #6 0x00007f08868154b6 in scm_eval ( exp=3D((@ (ice-9 control) %) (begin ((@@ (ice-9 command-line) load/lang= ) "/gnu/store/z8haznhwck4bjm4gxqy25wvwv4041wvx-cuirass-1.1.0-12.f087aaf/bin= /.cuirass-real") (main (command-line)) (quit))), module_or_state=3D"#" =3D {...}) at eval.c:705 #7 0x00007f08868793b6 in scm_shell (argc=3D9, argv=3D0x7ffea0cb8c98) at sc= ript.c:357 --8<---------------cut here---------------end--------------->8--- Process 86099 (the one it=E2=80=99s waiting for) is indeed =E2=80=98guix re= pl=E2=80=99 and it=E2=80=99s waiting for input in read(0, =E2=80=A6). There=E2=80=99s a second thread stuck in =E2=80=98waitpid=E2=80=99: --8<---------------cut here---------------start------------->8--- (gdb) thread 27 [Switching to thread 27 (LWP 86002)] #0 0x00007f0886310f27 in __GI___wait4 (pid=3D86100, stat_loc=3Dstat_loc@en= try=3D0x7f085393c60c, options=3Doptions@entry=3D0, usage=3Dusage@entry=3D0x= 0) at ../sysdeps/unix/sysv/linux/wait4.c:30 30 ../sysdeps/unix/sysv/linux/wait4.c: No such file or directory. (gdb) bt #0 0x00007f0886310f27 in __GI___wait4 (pid=3D86100, stat_loc=3Dstat_loc@en= try=3D0x7f085393c60c, options=3Doptions@entry=3D0, usage=3Dusage@entry=3D0x= 0) at ../sysdeps/unix/sysv/linux/wait4.c:30 #1 0x00007f0886310ea7 in __GI___waitpid (pid=3D, stat_loc= =3Dstat_loc@entry=3D0x7f085393c60c, options=3Doptions@entry=3D0) at waitpid.c:38 #2 0x00007f08868ae25e in scm_waitpid (pid=3D86100, options=3D) at posix.c:727 #3 0x00007f088689a336 in vm_regular_engine (thread=3D0x7f087d54e240) at vm= -engine.c:972 #4 0x00007f08868a75e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #5 0x00007f088680ba0e in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:406 #6 0x00007f088689a336 in vm_regular_engine (thread=3D0x7f087d54e240) at vm= -engine.c:972 #7 0x00007f08868a75e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 --8<---------------cut here---------------end--------------->8--- and then a couple of threads in =E2=80=98read=E2=80=99: --8<---------------cut here---------------start------------->8--- #0 __libc_read (nbytes=3D1, buf=3D0x7f085a69ab90, fd=3D5) at ../sysdeps/un= ix/sysv/linux/read.c:26 #1 __libc_read (fd=3D5, buf=3Dbuf@entry=3D0x7f085a69ab90, nbytes=3Dnbytes@= entry=3D1) at ../sysdeps/unix/sysv/linux/read.c:24 #2 0x00007f08868252e8 in fport_read (port=3D, dst=3D, start=3D, count=3D1) at fports.c:597 #3 0x00007f0886865d22 in scm_i_read_bytes (port=3Dport@entry=3D# 7f0854af6d20>, dst=3D"#" =3D {...},=20 start=3Dstart@entry=3D0, count=3D1) at ports.c:1566 #4 0x00007f08868681c7 in scm_fill_input (port=3Dport@entry=3D# 7f0854af6d20>,=20 minimum_size=3Dminimum_size@entry=3D1, cur_out=3Dcur_out@entry=3D0x7f08= 5313b5e0, avail_out=3Davail_out@entry=3D0x7f085313b588) at ports.c:2693 #5 0x00007f0886868d5c in peek_iconv_codepoint (port=3D# 7f0854af6d20>, buf=3Dbuf@entry=3D0x7f085313b5e8,=20 cur=3Dcur@entry=3D0x7f085313b5e0, len=3Dlen@entry=3D0x7f085313b5d8) at = ports.c:1944 #6 0x00007f0886868e4a in peek_codepoint (len=3D0x7f085313b5d8, cur=3D0x7f0= 85313b5e0, buf=3D0x7f085313b5e8, port=3D) at ports.c:1988 #7 scm_peek_char (port=3D) at ports.c:2202 #8 0x00007f087e62582b in ?? () #9 0x0000000000857760 in ?? () #10 0x00007f087e6257a0 in ?? () #11 0x00007f087d850c48 in ?? () #12 0x00007f0886844ccc in scm_jit_enter_mcode (thread=3D0x7f087d54e000, mco= de=3D0x857770 "\034\234\003") at jit.c:6038 #13 0x00007f0886899f3c in vm_regular_engine (thread=3D0x7f087d54e000) at vm= -engine.c:360 --8<---------------cut here---------------end--------------->8--- Normally we call =E2=80=98waitpid=E2=80=99 once the pipe has been closed: --8<---------------cut here---------------start------------->8--- (define* (open-inferior directory #:key (command "bin/guix") (error-port (%make-void-port "w"))) "Open the inferior Guix in DIRECTORY, running 'DIRECTORY/COMMAND repl' or equivalent. Return #f if the inferior could not be launched." (let ((pipe pid (inferior-pipe directory command error-port))) (port->inferior pipe (lambda (port) (close-port port) (waitpid pid))))) ;<----- here --8<---------------cut here---------------end--------------->8--- =E2=80=A6 and when the pipe is closed, the child =E2=80=98guix repl=E2=80= =99 process gets EOF and exits. So I=E2=80=99m not sure why the =E2=80=98guix repl=E2=80=99 process would s= tick around. Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue May 17 08:52:16 2022 Received: (at 55441) by debbugs.gnu.org; 17 May 2022 12:52:16 +0000 Received: from localhost ([127.0.0.1]:56034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqwgR-0001K9-Or for submit@debbugs.gnu.org; Tue, 17 May 2022 08:52:15 -0400 Received: from eggs.gnu.org ([209.51.188.92]:36772) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqwgQ-0001Jv-01 for 55441@debbugs.gnu.org; Tue, 17 May 2022 08:52:14 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58394) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqwgK-0007Ci-LU; Tue, 17 May 2022 08:52:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=wtOnMrHFJHd9NOAKA9oYgQqtwp0cg6jSJderYy+r2pI=; b=L9L3Npgewi7v3oYrg+7x +OyV72iJUGOxlVXSeu6ueD/zjTtzyR5LcPDeIxjlu7VMcEoBrdQmJsX/lDKm9xxFgXRCDR7vxr77L M3I0CpSY61XuWcn9iTyAp19IS9KyacXAnp521Idk991DRxQtY4psH0dEO7cK7lUrXlLPV3Kzz4bRE pwozv8ysX5lSOnNWO7EWE9B1mUnrjcRE/rYLKPHQt84cqx3Y0e1JlOn+nKJlHnlNuN38lb5QPCYYQ 5EyZXbQT7F5MHd4JhSfCuaPSAHZAgdfdX9/6rpck13Rl3WMp1JoPx+JxtKmQ7mOg45faXVuhe+quC bmMslX4vyc5wUQ==; Received: from [193.50.110.244] (port=52732 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqwgK-000811-78; Tue, 17 May 2022 08:52:08 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87h75pk5gn.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 28 =?utf-8?Q?Flor=C3=A9al?= an 230 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 17 May 2022 14:52:06 +0200 In-Reply-To: <87h75pk5gn.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 16 May 2022 08:49:28 -0400") Message-ID: <87pmkc5nk9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Maxim Cournoyer skribis: > I'm not yet very knowledgeable about POSIX threads and shortcomings, but > this looks like a dangerous trap to have lying around :-). Is there an > alternative implementation we could come up with that would avoid the > potential problem (and warnings) entirely? It *is* a dangerous trap, mitigated by the fact that our code execs soon after fork, but that=E2=80=99s not great (=E2=80=9CA fork() on the Road=E2= =80=9D explains the many shortcomings of =E2=80=98fork=E2=80=99, including how it interacts with multi-threaded code.) As discussed on IRC yesterday, the longer-term approach is to augment Guile is a posix_spawn-like primitive implemented in C. If it=E2=80=99s in= C, we can make it do the right thing safely, as is done with =E2=80=98open-pip= e=E2=80=99. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed May 18 03:44:56 2022 Received: (at 55441) by debbugs.gnu.org; 18 May 2022 07:44:56 +0000 Received: from localhost ([127.0.0.1]:58822 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nrEMa-00030L-FO for submit@debbugs.gnu.org; Wed, 18 May 2022 03:44:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nrEMZ-000307-7a for 55441@debbugs.gnu.org; Wed, 18 May 2022 03:44:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52732) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrEMS-0000hD-FM; Wed, 18 May 2022 03:44:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=kMKHn58GXMVX1KMaJKDhUPJqpMbfeLi7XWhaat8g1Cg=; b=GEjWfh/dzXpzKIhbnAX4 Dwv0mNJXIZDCod7YGEc5l2mLvFuHQwl97SUbAt/SYt/c+Kk3VhPpGGbQ08Vd18qLqLviwBYnK3BVQ tJml9lMXAJc7xt3QDsy0cpB9N/eJ/Yn8LbKbOagxoLiGC0lCT/8BXcrZ6Nj6EiJGR8UjKd7iAEDDJ 2wTe6sFN01POfWTOpcMxNiEA504c8LeG+HrXNSfpyK5mc+P6NCYY3sQlLDED4GQ+q3Y0oSohhHm/y htSFxcVhlOJ8I5qbYyyUuDf9qjV9dZwd/BJkUJMDaJc0ckZ2KW4A6FimAQwKsFjHz6dFJJ2Ma//0R 11LnTlNu306dFg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53220 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrEMS-0003ee-28; Wed, 18 May 2022 03:44:48 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> Date: Wed, 18 May 2022 09:44:44 +0200 In-Reply-To: <87fsl87gb5.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Tue, 17 May 2022 09:45:50 +0200") Message-ID: <874k1n5loz.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.8 (-) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.8 (--) Hi, Ludovic Court=C3=A8s skribis: > Hi, > > Maxim Cournoyer skribis: > >> Berlin was reconfigured with this commit of Cuirass, and is now running >> the derivations with it, but so far still "In progress..." after more >> than 100 minutes [0] > > Yeah, I=E2=80=99m not sure about the backtrace you report, however there= =E2=80=99s again > a bunch of =E2=80=98cuirass evaluate=E2=80=99 processes hanging, this tim= e with the main > thread stuck on =E2=80=98waitpid=E2=80=99 (presumably from the =E2=80=98c= lose-inferior=E2=80=99 call): An update: I ran this by hand: --8<---------------cut here---------------start------------->8--- $ sudo su - cuirass -s /bin/sh -c "/gnu/store/z8haznhwck4bjm4gxqy25wvwv4041= wvx-cuirass-1.1.0-12.f087aaf/bin/cuirass evaluate dbname=3Dcuirass 326988" Computing Guix derivation for 'x86_64-linux'... -^- warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; furthwarning: call to primitive-fork while multiple threads are ru= nning; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. warning: call to primitive-fork while multiple threads are running; further behavior unspecified. See "Processes" in the manual, for more information. --8<---------------cut here---------------end--------------->8--- =E2=80=A6 and attached =E2=80=98strace=E2=80=99, which shows things are wor= king as intended: --8<---------------cut here---------------start------------->8--- read(228, "r6x878i3fyr12lf9v496cl9x5jh6-gzip-1.10.drv\" \"/gnu/store/6280r7= vqfp5qkv6cdib02203xw1sk7h3-binutils-cr"..., 4096) =3D 4096 read(228, "pper-0.drv\" \"/gnu/store/45wsr6x878i3fyr12lf9v496cl9x5jh6-gzip-= 1.10.drv\" \"/gnu/store/6280r7vqfp5qkv6c"..., 4096) =3D 4096 read(228, "8qy2pnpn2030xxfmm56h71-gettext-0.21-doc\") (\"out\" . \"/gnu/sto= re/gmrqdvy9v42hfbs6b7vaxfrwf8b5qhqy-gett"..., 4096) =3D 4096 read(228, "\" \"/gnu/store/k5889v7ms3f5x1rjr3php71k4743fn19-bash-minimal-5.= 1.8.drv\" \"/gnu/store/lbyw1h3xz33qnb08j"..., 4096) =3D 4096 read(228, "x2g7ysm16xqnrpbkqdc4hhni1r42dgg-patch-2.7.6.drv\" \"/gnu/store/3= fy0f7gy85ddy6rpa4mmhjygns8qzk03-findut"..., 4096) =3D 4096 read(228, "08s1nz9bpv6k6a56idv6l7r2zjqphszl-file-5.39.drv\" \"/gnu/store/1x= 2g7ysm16xqnrpbkqdc4hhni1r42dgg-patch-2"..., 4096) =3D 4096 read(228, "tore/azdhavgwpjc527n81iqwrf91v8sv3x23-help2man-1.48.5.drv\" \"/g= nu/store/b1pi5ssin56s4mqq6964hsh2bdkyd"..., 4096) =3D 4096 read(228, ") (#:nix-name . \"guix-1.3.0-25.c1719a0\") (#:system . \"x86_64-= linux\") (#:max-silent-time . 3600) (#:t"..., 4096) =3D 121 close(228) =3D 0 wait4(83882, # hangs forever --8<---------------cut here---------------end--------------->8--- The question to me becomes: why is =E2=80=98guix repl=E2=80=99 not getting = EOF from its read(0, =E2=80=A6) call? FWIW, here=E2=80=99s an estimate of the amount of data transferred from the inferior to the main process: --8<---------------cut here---------------start------------->8--- $ grep 'read(228, .* =3D 4096$' 8--- I=E2=80=99m trying to reproduce that synthetically. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri May 20 14:33:08 2022 Received: (at 55441) by debbugs.gnu.org; 20 May 2022 18:33:08 +0000 Received: from localhost ([127.0.0.1]:40105 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ns7Qy-00076G-5H for submit@debbugs.gnu.org; Fri, 20 May 2022 14:33:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51078) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ns7Qw-00075i-Ak for 55441@debbugs.gnu.org; Fri, 20 May 2022 14:33:07 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ns7Qq-0000pN-VV; Fri, 20 May 2022 14:33:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=b88byOjNIUyAGIDFj98iGqV1UFb0R98q8pwcotwKtNc=; b=HOO5iq6Q/C3JBmgFnpcc nAq0rI0xoCFpm+vnB7scx+5rUjYf1wL86Eqk6PG0vfKEaG5BOm7U9kh0Q3ou0vLXL1I6RWH0hdlc9 aZG6vxcJqZDAAKpFDumhVhgMxbP1TBf/ZH2UtBuRxOg9E3L/e9wo6BrKfIM0l/E6BvRphPi2+WVPs trCFdKqmp60vwTD13gR//zaBMLrTNr+B854OVlM69ceHZZmvID5wqFcxK7ywe10AxIHpghsR3jwuT bII3p0CAPS006kN2CFUsaKhbiRaEBBSbcHFL5vK2BzV4nOSfm9jZvdJaucJNi2fL8ua8ITYVeMiBD Hn34/eWfk+tHYQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:59147 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ns7Qq-0005mg-I5; Fri, 20 May 2022 14:33:00 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> Date: Fri, 20 May 2022 20:32:58 +0200 In-Reply-To: <874k1n5loz.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 18 May 2022 09:44:44 +0200") Message-ID: <87r14ovyud.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Ludovic Court=C3=A8s skribis: > wait4(83882, # hangs forever > > > The question to me becomes: why is =E2=80=98guix repl=E2=80=99 not gettin= g EOF from its > read(0, =E2=80=A6) call? The answer was that another child process (an inferior) was holding a copy of the socketpair file descriptor connected to that =E2=80=98guix repl= =E2=80=99 standard input. Fixed in Guix commit a4994d739306abcf3f36706012fb88b35a970e6b with a test that reproduces the issue. Commit d02b7abe24fac84ef1fb1880f51d56fc9fb6cfef updates the =E2=80=98guix= =E2=80=99 package so we should be able to reconfigure berlin now and hopefully (crossing fingers!) be done with it. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 17:02:36 2022 Received: (at 55441) by debbugs.gnu.org; 24 May 2022 21:02:36 +0000 Received: from localhost ([127.0.0.1]:53598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ntbfn-0001d8-Nc for submit@debbugs.gnu.org; Tue, 24 May 2022 17:02:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45980) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ntbfZ-0001cj-Gz for 55441@debbugs.gnu.org; Tue, 24 May 2022 17:02:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:46874) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ntbfU-0002aQ-8b; Tue, 24 May 2022 17:02:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=On9l6mGBjxbAgR7oNeJ2PRCQnsn+3LphuMNCEJryBAE=; b=DKQ4s9iwRlPqHzMgb+G9 vGzmVM2Vaj6lTRSJTXxj/VRvtDvr6rikC/0ozAiv2t3KJdqE7DJyuGFUx5GOiABh7aZY/Dllux4nI lqX0bF/w4uj/x+CAAiMJfdlr4ENEJEsDRsSzZFNSmX9q7blC0SMbK/Ryqbt1S48L3EQopWCEvX4ud gzB3H6BkDCgjqLqilerRoMZI0SqU6uTNqfoKzcZojBxJyiQ8MGj+hAFXFWBQKkN52ukJOh4/Z9lKU Mot8EwyeqySn5IKLaKyCbsYQjgRCYcuA68XTHSwaG0e4kajjyJOuxaaORAPx4vPUFmtpd9dz9nWgG dH1kZCkrf4B9QQ==; Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=37742 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ntbfT-0004Ng-KL; Tue, 24 May 2022 17:02:15 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> Date: Tue, 24 May 2022 23:02:13 +0200 In-Reply-To: <87r14ovyud.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Fri, 20 May 2022 20:32:58 +0200") Message-ID: <87k0aaiqzu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Ludovic Court=C3=A8s skribis: > Fixed in Guix commit a4994d739306abcf3f36706012fb88b35a970e6b with a > test that reproduces the issue. > > Commit d02b7abe24fac84ef1fb1880f51d56fc9fb6cfef updates the =E2=80=98guix= =E2=80=99 > package so we should be able to reconfigure berlin now and hopefully > (crossing fingers!) be done with it. An update: Cuirass is now up-to-date on berlin.guix, built from Guix commit adf5ae5a412ed13302186dd4ce8e2df783d4515d. Unfortunately, while evaluations now run to completion, child processes of =E2=80=98cuirass evaluate=E2=80=99 stick around at the end: --8<---------------cut here---------------start------------->8--- (gdb) bt #0 futex_wait (private=3D0, expected=3D2, futex_word=3D0x7f5b1d054f08) at = ../sysdeps/nptl/futex-internal.h:146 #1 __lll_lock_wait (futex=3Dfutex@entry=3D0x7f5b1d054f08, private=3D0) at = lowlevellock.c:52 #2 0x00007f5b1d873ef3 in __GI___pthread_mutex_lock (mutex=3Dmutex@entry=3D= 0x7f5b1d054f08) at ../nptl/pthread_mutex_lock.c:80 #3 0x00007f5b1d995303 in scm_c_weak_set_remove_x (pred=3D, = closure=3D0x7f5b13dd8d00, raw_hash=3D1824276156261873434, set=3D#) at weak-set.c:794 #4 scm_weak_set_remove_x (obj=3D# 7f5b= 13dd8d00>, set=3D#) at weak-set.c:817 #5 close_port (explicit=3D, port=3D# 7f5b13dd8d00>) at ports.c:891 #6 close_port (port=3D# 7f5b13dd8d00>,= explicit=3D) at ports.c:874 #7 0x00007f5af3a7df82 in ?? () #8 0x0000000000dbd860 in ?? () #9 0x00007f5af3a7df60 in ?? () #10 0x0000000000db82b8 in ?? () #11 0x00007f5b1d972ccc in scm_jit_enter_mcode (thread=3D0x7f5b157bf240, mco= de=3D0xdbd86c "\034\217\003") at jit.c:6038 #12 0x00007f5b1d9c7f3c in vm_regular_engine (thread=3D0x7f5b157bf240) at vm= -engine.c:360 #13 0x00007f5b1d9d55e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #14 0x00007f5b1d939a0e in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:406 #15 0x00007f5b1d9c8336 in vm_regular_engine (thread=3D0x7f5b157bf240) at vm= -engine.c:972 #16 0x00007f5b1d9d55e9 in scm_call_n (proc=3D, argv=3D, nargs=3D0) at vm.c:1608 #17 0x00007f5b1d9c4be6 in really_launch (d=3D0x7f5aebccac80) at threads.c:7= 78 #18 0x00007f5b1d93b85a in c_body (d=3D0x7f5aea691d80) at continuations.c:430 #19 0x00007f5aeeb118c2 in ?? () #20 0x00007f5b1553d7e0 in ?? () #21 0x00007f5b138a7370 in ?? () #22 0x0000000000000048 in ?? () #23 0x00007f5b1d972ccc in scm_jit_enter_mcode (thread=3D0x7f5b157bf240, mco= de=3D0xdbc874 "\034<\003") at jit.c:6038 #24 0x00007f5b1d9c7f3c in vm_regular_engine (thread=3D0x7f5b157bf240) at vm= -engine.c:360 #25 0x00007f5b1d9d55e9 in scm_call_n (proc=3D, argv=3D, nargs=3D2) at vm.c:1608 #26 0x00007f5b1d93d09a in scm_call_2 (proc=3D, arg1=3D, arg2=3D) at eval.c:503 #27 0x00007f5b1d9f3752 in scm_c_with_exception_handler.constprop.0 (type=3D= #t, handler_data=3Dhandler_data@entry=3D0x7f5aea691d10, thunk_data=3Dthunk_= data@entry=3D0x7f5aea691d10, thunk=3D, handler=3D) at exceptions.c:170 #28 0x00007f5b1d9c588f in scm_c_catch (tag=3D, body=3D, body_data=3D, handler=3D, handler_= data=3D, pre_unwind_handler=3D, pre_unwind_handler_data=3D0x7f5b1= 56b2040) at throw.c:168 #29 0x00007f5b1d93de66 in scm_i_with_continuation_barrier (pre_unwind_handl= er=3D0x7f5b1d93db80 , pre_unwind_handler_data=3D0x7f5b1= 56b2040, handler_data=3D0x7f5aea691d80, handler=3D0x7f5b1d9448b0 , body_data=3D0x7f5aea691d80, body= =3D0x7f5b1d93b850 ) at continuations.c:368 #30 scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:464 #31 0x00007f5b1d9c4b39 in with_guile (base=3D0x7f5aea691e08, data=3D0x7f5ae= a691e30) at threads.c:645 #32 0x00007f5b1d89b0ba in GC_call_with_stack_base () from /gnu/store/2lczkx= bdbzh4gk7wh91bzrqrk7h5g1dl-libgc-8.0.4/lib/libgc.so.1 #33 0x00007f5b1d9bd16d in scm_i_with_guile (dynamic_state=3D= , data=3D0x7f5aebccac80, func=3D0x7f5b1d9c4b70 ) at threads.= c:688 #34 launch_thread (d=3D0x7f5aebccac80) at threads.c:787 #35 0x00007f5b1d871d7e in start_thread (arg=3D0x7f5aea692640) at pthread_cr= eate.c:473 #36 0x00007f5b1d46feff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clo= ne.S:95 (gdb) info threads Id Target Id Frame * 1 process 53801 "guile" futex_wait (private=3D0, expected=3D2, futex_w= ord=3D0x7f5b1d054f08) at ../sysdeps/nptl/futex-internal.h:146 --8<---------------cut here---------------end--------------->8--- Notice there=E2=80=99s a single thread: it very much looks like the random results one gets when forking a multithreaded process (in this case, this one thread is a finalization thread, except it=E2=80=99s running in a process that doesn=E2=80=99t actually have the other Guile threads). The fork+threads problem is already manifesting, after all. I=E2=80=99ll try and come up with a solution to that, if nobody beats me at= it. What=E2=80=99s annoying is that it=E2=80=99s not easy to test: the problem = doesn=E2=80=99t manifest on my 4-core laptop, but it does on the 96-core berlin. To be continued=E2=80=A6 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 12:32:25 2022 Received: (at 55441) by debbugs.gnu.org; 25 May 2022 16:32:26 +0000 Received: from localhost ([127.0.0.1]:56334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nttvt-0004Gl-Nb for submit@debbugs.gnu.org; Wed, 25 May 2022 12:32:25 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49944) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nttvr-0004GY-Kg for 55441@debbugs.gnu.org; Wed, 25 May 2022 12:32:24 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45764) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nttvd-0001B7-DY; Wed, 25 May 2022 12:32:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=2HQrJQcJpW7F8Pr0nVLZvpeXv0STCsdASfZUgtvedxU=; b=dUKUlf5l70FZzP1qLSTq daXMFAISArZK0O/TTblOKqbtMk/fZUd26rNdlANS8r7MNchmkG7FPV9NiWXLYTQWkLkOU84W80Sqp 9qCZMeKCn+7DTxmlXjnwggQNmPBt33dG/AW+K9uoKd/VO4zKPhsHiaecx41dgB+sbQynpfQ6GCto5 GrQxJxPl2CcBFp5nCLNvHAXTp9CBZ7BN0UvYo5RJ4mk4M2deBr/RotIPYvyfbRryyw0A550Ot1dNX BOEQSzpn7rwJ6OEAXpoB+Ty8hd39B95FFN+PXq6y67baAphabA+RCHzVSgrbuA3Ilj9QQavt+LLPU m8kW0CMpbRJFfw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:63824 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nttvb-0000KQ-Tr; Wed, 25 May 2022 12:32:08 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> Date: Wed, 25 May 2022 18:32:05 +0200 In-Reply-To: <87k0aaiqzu.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Tue, 24 May 2022 23:02:13 +0200") Message-ID: <87bkvlh8u2.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > Notice there=E2=80=99s a single thread: it very much looks like the random > results one gets when forking a multithreaded process (in this case, > this one thread is a finalization thread, except it=E2=80=99s running in a > process that doesn=E2=80=99t actually have the other Guile threads). The > fork+threads problem is already manifesting, after all. An update: The =E2=80=98with-database=E2=80=99 used by (cuirass scripts eva= luate), creates threads unconditionally and leaves them around when its body has exited. So there=E2=80=99s currently no way in (cuirass scripts evaluate) = to arrange so that =E2=80=98open-inferior=E2=80=99 is called in a single-threa= ded context (I tried that in =E2=80=98wip-bug-55441=E2=80=99). Bummer. Looks like we=E2=80=99re left with unpleasant solutions, such as writing bindings for =E2=80=98posix_spawn=E2=80=99 and using that in (guix = inferior). Mathieu, any ideas? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 14:21:42 2022 Received: (at 55441) by debbugs.gnu.org; 25 May 2022 18:21:43 +0000 Received: from localhost ([127.0.0.1]:56433 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ntvde-0007NG-Fb for submit@debbugs.gnu.org; Wed, 25 May 2022 14:21:42 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ntvdZ-0007Mt-MH for 55441@debbugs.gnu.org; Wed, 25 May 2022 14:21:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47620) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ntvdU-0002G0-9q; Wed, 25 May 2022 14:21:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=0hciw+GW15xtp2xDTW5UmsuRf5jHRpk4pDSun1y5Gjc=; b=jfr6+a2xed1KCL5bXd90 OzLJo3I1FhZdv6Xv9YI43lAaBcUUrgXzk1DUXYWsKKohxAgk9QY8xroGATcmft0G+8at2qzl11VoY 0XT/4D8N3WmRaCspmqXPa6hGPq5R/+rSkUqasGUlIdmYgKdxll7R9x5uF/cKq7aLD3KukfMKMovI/ LOzGNiEun4Sbk9CVGlYZ2klturC+t3zhVn4rs/B6qo0QfA1U3huSKDAzfGUn6tUmXk2Rmsa6H4XrO JDNdzITJQgN0aM58qwAM5qCTkZOWdoa2wEzvVe4dNB4VNYeIG5JjQ+LPFEuvGlU7U66j16pxbm2aJ QmNm0yRFwdJa2Q==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:50459 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ntvdT-0003hQ-RQ; Wed, 25 May 2022 14:21:32 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> Date: Wed, 25 May 2022 20:21:29 +0200 In-Reply-To: <87bkvlh8u2.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 25 May 2022 18:32:05 +0200") Message-ID: <875ylth3rq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable For now, I=E2=80=99m going to go with the solution below, which is to use an older revision of Guix (one where =E2=80=98open-inferior=E2=80=99 was using =E2=80=98open-pipe*=E2=80=99) as the dependency of the =E2=80=98cuirass=E2= =80=99 package. I=E2=80=99m running =E2=80=9Ccuirass evaluate=E2=80=9D manually on berlin t= o make sure it actually works. If everything goes well, I=E2=80=99ll push it and reconfig= ure berlin later today or tomorrow. Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/ci.scm b/gnu/packages/ci.scm index ab421fe870..9cd2f6884e 100644 --- a/gnu/packages/ci.scm +++ b/gnu/packages/ci.scm @@ -164,7 +164,14 @@ (define-public cuirass ;; FIXME: this is propagated by "guile-git", but it needs to be among ;; the inputs to add it to GUILE_LOAD_PATH. guile-bytestructures - guix)) + + ;; FIXME: The 'cuirass evaluate' command is multithreaded, but it + ;; uses 'open-inferior', which calls 'primitive-fork', thus + ;; potentially creating child processes that deadlock. To work + ;; around that, use the last revision of Guix where + ;; 'open-inferior' was using 'open-pipe*' rather than + ;; 'primitive-fork'. See . + guix-for-cuirass)) (native-inputs (list autoconf automake pkg-config texinfo ephemeralpg)) (native-search-paths diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index 619458a0d3..16ad4fe9db 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -489,6 +489,30 @@ (define code (license license:gpl3+) (properties '((ftp-server . "alpha.gnu.org")))))) +(define-public guix-for-cuirass + ;; Known-good revision before commit + ;; bd86bbd300474204878e927f6cd3f0defa1662a5, which introduced + ;; 'primitive-fork' in 'open-inferior'. + (let ((version "1.3.0") + (commit "a27e47f9d1e22dc32bb250cfeef88cfacb930e23") + (revision 23)) + (package + (inherit guix) + (version (string-append version "-" + (number->string revision) + "." (string-take commit 7))) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://git.savannah.gnu.org/git/guix.git") + (commit commit))) + (sha256 + (base32 + "12jmvagbw05hmmlrb82i0qazhlv7mcfnl4dmknwx3a9hd760g9y1")) + (file-name (string-append "guix-" version "-checkout")))) + (properties `((hidden? . #t) + ,@(package-properties guix)))))) + (define-public guix-daemon ;; This package is for internal consumption: it allows us to quickly build ;; the 'guix-daemon' program and use that in (guix self), used by 'guix --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 19:36:55 2022 Received: (at 55441) by debbugs.gnu.org; 25 May 2022 23:36:55 +0000 Received: from localhost ([127.0.0.1]:56644 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nu0Yh-0003lD-4F for submit@debbugs.gnu.org; Wed, 25 May 2022 19:36:55 -0400 Received: from mira.cbaines.net ([212.71.252.8]:41300) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nu0Yb-0003kz-V2 for 55441@debbugs.gnu.org; Wed, 25 May 2022 19:36:53 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id DEF2C27BBEA; Thu, 26 May 2022 00:36:48 +0100 (BST) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 785aa7e3; Wed, 25 May 2022 23:36:47 +0000 (UTC) References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> User-agent: mu4e 1.6.10; emacs 27.2 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections Date: Thu, 26 May 2022 00:13:33 +0100 In-reply-to: <875ylth3rq.fsf@gnu.org> Message-ID: <87h75dtcaa.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > For now, I=E2=80=99m going to go with the solution below, which is to use= an > older revision of Guix (one where =E2=80=98open-inferior=E2=80=99 was usi= ng > =E2=80=98open-pipe*=E2=80=99) as the dependency of the =E2=80=98cuirass= =E2=80=99 package. > > I=E2=80=99m running =E2=80=9Ccuirass evaluate=E2=80=9D manually on berlin= to make sure it > actually works. If everything goes well, I=E2=80=99ll push it and reconf= igure > berlin later today or tomorrow. To put in an email something I put on IRC earlier. Maybe the store connection caching could be optional when calling inferior-eval-with-store, and that could also switch between using open-pipe* and primitive-fork for starting the inferior process. I'm guessing the use of primitive-fork for starting the inferior process is causing problems with Cuirass in some cases, and it's possible that it'll affect the data service in a similar way as well. I don't think the connection caching actually benefits Cuirass though, since it only calls inferior-eval-with-store once per inferior. Additionally, on the data service side, the caching functionality is actually undesirable as it leads to the inferior process running out of memory, so currently the cache is manually cleared in various places [1]. 1: http://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=3Dff11= 6d5e6437ffb916aa4bc5d1458a142297a900 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmKOvY1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XciYQ//WfemT9xfTUCL0aKz8Ew73mNfhH0gmNfd tCZZKBOZPoVNgEXPpt3W5JEsr8r+/mhF5ymFn+N++xetwCkRDlCCRepp96oXs7RX wqpTfCIX3VG9dpUbUgb3OGkuzPvjkzx6OQ/A7o8jPKgKv1nu4M4JD78le7ZjMeQE 7S8YbSZRtSsAQaYoQfWAAflytqyRq5NeH/QopqC72sxOqDOtdb/6ZaRC1f8sIx8D AdhEHjQGpMinUAMbu0T14Qvwle6n/AkxXRMSp0D9bs2pXhqw3X5tz8OM+d07K/dl BGqjQ82ZY+DAnEq3y/GsvffV3yVha8YuI3qhsDAY8xEASS5l9iYgsYSmOA1+GLhK NBdhL5q9SXX6MEyEu8C8FSyDOBN4SAXMmsPVfjeOJYwBQJbg8XWAuBpGwexz1cD0 g5Ke6BduC/Wl9c0VK6eJv3rsEXSAFr9C0qRCVmkdqKj6Ij1kLKOIUjsGsiCr0esw mSYsx2Bi4hN3Ssku0dNJvztMG3Izq7FywJp8gqQodUEaK1vNxL71fWP5WTNR6cjD ksEJcQ/4KXtPa4fP9pAzNIs6ay17ESj87T7JW6CHFBro8/FUkBbHXxRgo7QdWktG aqASbbbVkcvDmuNdUuVmJbFBOfMSkUbEdAjVA0dWDLevPeYbt3N2SfA11M0MnEak qR2SUYz1ptU= =xr7r -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 05:44:24 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 09:44:24 +0000 Received: from localhost ([127.0.0.1]:57082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuA2Z-0006SC-WC for submit@debbugs.gnu.org; Thu, 26 May 2022 05:44:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48256) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuA2U-0006Rw-S0 for 55441@debbugs.gnu.org; Thu, 26 May 2022 05:44:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58954) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuA2P-0005Cn-EZ; Thu, 26 May 2022 05:44:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=HJ+WDjHN/Kdc4QXZh5Gtmgzo+pvycE+E7bemXLp5FUg=; b=QmqBU6/ftEmCYZ08sn2Z iGEFt0b/OLhduZcxNmUxRGcoPasI2KGrmA5Zd60/1zZ5vvA0weJoxerjSfziYAk8oxDfErTvvfLnl ltIrgFgzgrpmeflrCDJqOCI91MZNNoRiOhe/pEsAFn+N+f2gnQIYIdANCoUNehxCYklmvtP+09BlR hODxJ6vhRWgWQIbNMAmAl1L/9IOAM2Y6Zqa7eD9JxGZatkUdc3fbJwym6OC/9XWTgvGw/G+GLNoYj 95odllKerGsT57YzKfqwCfv+jetf9jbB2W5RyPjiALlSBEwjolxxl/zrItQSP7Q6En2CddV2VjIf0 LkpQ4FbYvZjOaQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:54734 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuA2N-000334-SK; Thu, 26 May 2022 05:44:13 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> <87h75dtcaa.fsf@cbaines.net> Date: Thu, 26 May 2022 11:44:09 +0200 In-Reply-To: <87h75dtcaa.fsf@cbaines.net> (Christopher Baines's message of "Thu, 26 May 2022 00:13:33 +0100") Message-ID: <871qwghbme.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Christopher Baines skribis: > Maybe the store connection caching could be optional when calling > inferior-eval-with-store, and that could also switch between using > open-pipe* and primitive-fork for starting the inferior process. I don=E2=80=99t think that=E2=80=99s desirable as an interface. To me, the= proper fix will be to use an interface like =E2=80=98posix_spawn=E2=80=99 so we can ha= ve both correctness and speed, rather than let users choose. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 08:29:19 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 12:29:19 +0000 Received: from localhost ([127.0.0.1]:57259 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuCc6-0006kj-Qp for submit@debbugs.gnu.org; Thu, 26 May 2022 08:29:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51546) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuCc5-0006kX-7y for 55441@debbugs.gnu.org; Thu, 26 May 2022 08:29:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60914) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuCbz-0003cW-VA; Thu, 26 May 2022 08:29:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=PC4kbDCi90WOjjC+EhuDKAFmhZJw6CkBG0z6fK4uaZg=; b=h074/uzwyAPnDyVoYKIZ mVXHjDv3ofOukvOPx20JoySpQajVrnmQgFyfwof0l0DTTLUA5qlSLLI477yECUbkChG5a9+TiUyaG YWMOs20Hoy2U2FIsCrm4O4noudRJGYCOV/ULp6AIRPSLkTE5pjY/vEUVfg2uhCoPpo8oyI8CFN6wD TFuo1bqRgsYlj1RMFhnd9PPa1td8bfJT/YlybP/owcKM5VjCEQNiIyZIKUOD0RA4fuQDq80o9C9dd m3Qk7hxMzmY6ldpvSU/3mOrQRjvRm+iLATpplBgl0Ol/dAx/A2NNRRhtlSgbpVJdgOu8qUdZh3SC5 f/LOGwJQzZLAXA==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:52982 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuCbz-00062c-Dm; Thu, 26 May 2022 08:29:07 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> Date: Thu, 26 May 2022 14:29:05 +0200 In-Reply-To: <875ylth3rq.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 25 May 2022 20:21:29 +0200") Message-ID: <87sfowfpf2.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, guix-sysadmin@gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > For now, I=E2=80=99m going to go with the solution below, which is to use= an > older revision of Guix (one where =E2=80=98open-inferior=E2=80=99 was usi= ng > =E2=80=98open-pipe*=E2=80=99) as the dependency of the =E2=80=98cuirass= =E2=80=99 package. > > I=E2=80=99m running =E2=80=9Ccuirass evaluate=E2=80=9D manually on berlin= to make sure it > actually works. If everything goes well, I=E2=80=99ll push it and reconf= igure > berlin later today or tomorrow. Pushed commit as dcb7ce1eb6911f9d503e7cd2bfe380058cee956b. I reconfigured berlin a couple of hours ago: --8<---------------cut here---------------start------------->8--- ludo@berlin ~/src/maintenance/hydra$ guix system describe Generation 576 May 26 2022 12:50:31 (current) file name: /var/guix/profiles/system-576-link canonical file name: /gnu/store/dd4p4cw67p5wbwwv7szgz0zxsvkq78zb-system label: GNU with Linux-Libre 5.17.9 bootloader: grub root device: label: "my-root" kernel: /gnu/store/si7fz3pxwq5aiqpdwkw0qs4jmsqh89n5-linux-libre-5.17.9/bz= Image channels: guix: repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 2262ee2c63fd92a17836ff7450623db76ed39d92 configuration file: /gnu/store/sd59z8h741jilaa3cdf16441f84xghil-configura= tion.scm ludo@berlin ~/src/maintenance/hydra$ git log |head -5 commit 07f587cb266f284dee6ac2e744161973d3512b04 Author: Ludovic Court=C3=A8s Date: Tue May 24 14:57:36 2022 +0200 hydra: Add missing service type 'description' field. --8<---------------cut here---------------end--------------->8--- There=E2=80=99s been a couple of evaluations (346596 and 346651 in the =E2= =80=98master=E2=80=99 jobset) and so far things seem to work smoothly. Let=E2=80=99s keep monitoring. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:49:08 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:49:08 +0000 Received: from localhost ([127.0.0.1]:60135 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKPs-0005iN-5b for submit@debbugs.gnu.org; Thu, 26 May 2022 16:49:08 -0400 Received: from jpoiret.xyz ([206.189.101.64]:48662) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKPq-0005iF-6i for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:49:07 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 767F7184BF2; Thu, 26 May 2022 20:49:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1653598144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RiyxE1pds2gI5ksnkZdQHXjO4ZTa7MbeR07Cdv4zIi0=; b=pD5VZcaQbZvJQz1at2Ck6OpjVa9Kl/BCfu+4lquDIQdaO5TvtCh2tQj9ChvyEnWgCJxi68 QGYSNDARhBNn3izS94g4PHDH/Q0ivJxZVaXzYehM9s2ML/RFTni9odSjkjDP1On04hOd6G FE11mON4fHJmbxgvE2qGlob0dy4P4KYn9udIdxNNuO3vTlPBVodCcX+y8vRrQoBKex+AQK khN0cUE69HIEGdTWsrdqQnnqlcKWNdyjG7JGgK9r6k+BY8MqAWD9Qoz38bi4V7wgc5D/nT Ll9yaV+JTe7Ru0NyYOlk4Y269bbAwmXjuQWZKhmdQoupRbBFsf+o5v6TRlWrzA== From: Josselin Poiret To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Christopher Baines Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections In-Reply-To: <871qwghbme.fsf@gnu.org> References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> <87h75dtcaa.fsf@cbaines.net> <871qwghbme.fsf@gnu.org> Date: Thu, 26 May 2022 22:49:03 +0200 Message-ID: <87sfow10lc.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Type: text/plain Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hello everyone, With all the recent platform changes, I was looking at `make cuirass-jobs` for the first time and it left some deadlocked inferiors as well, so I figured I could try my own solution, since I've alread [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hello everyone, With all the recent platform changes, I was looking at `make cuirass-jobs` for the first time and it left some deadlocked inferiors as well, so I figured I could try my own solution, since I've alread [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager Hello everyone, With all the recent platform changes, I was looking at `make cuirass-jobs` for the first time and it left some deadlocked inferiors as well, so I figured I could try my own solution, since I've already dabbled in Guile POSIX internals! Here's a patch that rewrites open-bidirectional-pipe with a more thread-safe alternative, `piped-process`, but this also requires some Guile patches, that I will post as well. With these, `make cuirass-jobs` doesn't hang at all anymore for me. Let me know what you think of these Best, -- Josselin Poiret From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:50:26 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:50:26 +0000 Received: from localhost ([127.0.0.1]:60140 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKR8-0005kQ-Fd for submit@debbugs.gnu.org; Thu, 26 May 2022 16:50:26 -0400 Received: from jpoiret.xyz ([206.189.101.64]:48866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKR6-0005kI-MT for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:50:25 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 9F3BD184F6C; Thu, 26 May 2022 20:50:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1653598224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZNNvuGSlLHpGEsUl5ZBc26eelUowqCS/1hVsxj6DLfc=; b=rNyBzEVJXg0l4DIzgSijgG8MkV8k4Me/V++REj9EKG6mR+uejy8JAS8EI3683WyCDQEmgl 9rMXYGSJJ+C+5xKg4p69rSeJQn2SvuTDGv1m1pAqHtXWccWkXV14QLF37p67oVHG1+qsyp WNU30YAs9h+eNP3OWufWMI+sUjDC/ekwJJp79KzIdzc2dO+btIbfpKmlIWLRnOJRgWxuLv SySlIG4lfYEW3+D+k+cEYLGeIcg8gpDjUuxHQdvk7h0VHTAby09RYYbk+2lP9lLASNkjP9 TYvnwvKIHoahU2ZUTNb6BGOJEaqaPJDAjzmAlmimPYbY09r2HaFZSmrYsTJqFA== From: Josselin Poiret To: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Christopher Baines Subject: [PATCH] guix: inferior: Make open-bidirectional-pipe use piped-process. Date: Thu, 26 May 2022 22:50:16 +0200 Message-Id: <20220526205016.23899-1-dev@jpoiret.xyz> In-Reply-To: <87sfow10lc.fsf@jpoiret.xyz> References: <87sfow10lc.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/inferior.scm: Load up scm_init_popen extension to be able to use piped-process. (open-bidirectional-pipe): Remove custom fork+exec code and use piped-process instead. --- guix/inferior.scm | 39 [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Josselin Poiret X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/inferior.scm: Load up scm_init_popen extension to be able to use piped-process. (open-bidirectional-pipe): Remove custom fork+exec code and use piped-process instead. --- guix/inferior.scm | 39 [...] Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager * guix/inferior.scm: Load up scm_init_popen extension to be able to use piped-process. (open-bidirectional-pipe): Remove custom fork+exec code and use piped-process instead. --- guix/inferior.scm | 39 ++++++++++++++++++--------------------- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/guix/inferior.scm b/guix/inferior.scm index 54200b75e4..a4a4c1d40e 100644 --- a/guix/inferior.scm +++ b/guix/inferior.scm @@ -134,6 +134,10 @@ (define (write-inferior inferior port) (set-record-type-printer! write-inferior) +(eval-when (expand load eval) + (load-extension (string-append "libguile-" (effective-version)) + "scm_init_popen")) + (define (open-bidirectional-pipe command . args) "Open a bidirectional pipe to COMMAND invoked with ARGS and return it, as a regular file port (socket). @@ -147,27 +151,20 @@ (define (open-bidirectional-pipe command . args) ;; the REPL process wouldn't get EOF on standard input. (match (socketpair AF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0) ((parent . child) - (match (primitive-fork) - (0 - (dynamic-wind - (lambda () - #t) - (lambda () - (close-port parent) - (close-fdes 0) - (close-fdes 1) - (dup2 (fileno child) 0) - (dup2 (fileno child) 1) - ;; Mimic 'open-pipe*'. - (unless (file-port? (current-error-port)) - (close-fdes 2) - (dup2 (open-fdes "/dev/null" O_WRONLY) 2)) - (apply execlp command command args)) - (lambda () - (primitive-_exit 127)))) - (pid - (close-port child) - (values parent pid)))))) + (let* ((proc + (lambda () + (piped-process command args + ;; Use port->fdes on child to increase the + ;; revealed count, so that the fd does not get + ;; closed again when child gets gc'd, as + ;; piped-process already closes it. + (cons (fileno parent) (port->fdes child)) + (cons (fileno child) (fileno parent))))) + (pid + (if (file-port? (current-error-port)) + (proc) + (with-error-to-file "/dev/null" proc)))) + (values parent pid))))) (define* (inferior-pipe directory command error-port) "Return two values: an input/output pipe on the Guix instance in DIRECTORY -- 2.36.0 From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:50:55 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:50:55 +0000 Received: from localhost ([127.0.0.1]:60146 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKRa-0005lD-P0 for submit@debbugs.gnu.org; Thu, 26 May 2022 16:50:55 -0400 Received: from jpoiret.xyz ([206.189.101.64]:49004) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKRY-0005kz-IL for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:50:53 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 85933184F6C; Thu, 26 May 2022 20:50:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1653598251; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OW5G34BGS52mVKO3cQqsmLZEBpDsOcbo7KLGJEeipfk=; b=eP3xvCwxG4dQSsGhevRjiiXzrSYaI0sVZkT1dfkzB4bDfQjd0ScoSJFLQDZOa5TlvnqaMp LXmUjv/OtQhWRrBSkrcuMhNmpW2TUB696zCbIZqJTIjZLF4+X9mq6NToaQNongXaMZ2LjG 7Vr0PpGDvnAFg2kSSdySnsooK0n2wyYwqNe0/VFEFBg537pUT50/FDuewmadlezHz+ptry grkALWdFKVJ+5ooi0mV7Z4VImv8DKqliluJVMwelEepi/wJFeuP7wvYk1RDfOlgErwWbj5 bsdr0lGS7gS7mBCRwoW7HA2qNzB2C0yhLjT2N7aXt9CXiRGK63n6TRqTEh5S3w== From: Josselin Poiret To: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Christopher Baines Subject: [PATCH 1/2] Fix child spawning closing standard fds prematurely Date: Thu, 26 May 2022 22:50:46 +0200 Message-Id: <20220526205047.23960-1-dev@jpoiret.xyz> In-Reply-To: <87sfow10lc.fsf@jpoiret.xyz> References: <87sfow10lc.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * libguile/posix.c (renumber_file_descriptor): Refactor it as dup_handle_error. (dup_handle_error, dup2_handle_error): New functions that wrap around dup and dup2 by retrying on EINTR or EBUSY, as wel [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Josselin Poiret X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * libguile/posix.c (renumber_file_descriptor): Refactor it as dup_handle_error. (dup_handle_error, dup2_handle_error): New functions that wrap around dup and dup2 by retrying on EINTR or EBUSY, as wel [...] Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager * libguile/posix.c (renumber_file_descriptor): Refactor it as dup_handle_error. (dup_handle_error, dup2_handle_error): New functions that wrap around dup and dup2 by retrying on EINTR or EBUSY, as well as erroring out on other errors. (start_child): Close standard file descriptors only after all of them have been dup2'd. --- libguile/posix.c | 82 +++++++++++++++++++++++++++++++----------------- 1 file changed, 53 insertions(+), 29 deletions(-) diff --git a/libguile/posix.c b/libguile/posix.c index 3ab12b99e..dc3080b3c 100644 --- a/libguile/posix.c +++ b/libguile/posix.c @@ -1280,14 +1280,14 @@ SCM_DEFINE (scm_fork, "primitive-fork", 0, 0, 0, #endif /* HAVE_FORK */ #ifdef HAVE_FORK -/* 'renumber_file_descriptor' is a helper function for 'start_child' - below, and is specialized for that particular environment where it - doesn't make sense to report errors via exceptions. It uses dup(2) - to duplicate the file descriptor FD, closes the original FD, and - returns the new descriptor. If dup(2) fails, print an error message - to ERR and abort. */ +/* 'dup_handle_error' is a helper function for 'start_child' below, and + is specialized for that particular environment where it doesn't make + sense to report errors via exceptions. It uses dup(2) to duplicate + the file descriptor FD, does *not* close the original FD, and returns + the new descriptor. If dup(2) fails, print an error message to ERR + and abort. */ static int -renumber_file_descriptor (int fd, int err) +dup_handle_error (int fd, int err) { int new_fd; @@ -1304,7 +1304,33 @@ renumber_file_descriptor (int fd, int err) _exit (127); /* Use exit status 127, as with other exec errors. */ } - close (fd); + return new_fd; +} + +/* 'dup2_handle_error' is a helper function for 'start_child' below, and + is specialized for that particular environment where it doesn't make + sense to report errors via exceptions. It uses dup2(2) to duplicate + the file descriptor FD, does *not* close the original FD, and returns + the new descriptor. If dup2(2) fails, print an error message to ERR + and abort. */ +static int +dup2_handle_error (int fd, int to, int err) +{ + int new_fd; + + do + new_fd = dup2 (fd, to); + while (new_fd == -1 && (errno == EINTR || errno == EBUSY)); + + if (new_fd == -1) + { + /* At this point we are in the child process before exec. We + cannot safely raise an exception in this environment. */ + const char *msg = strerror (errno); + fprintf (fdopen (err, "a"), "start_child: dup failed: %s\n", msg); + _exit (127); /* Use exit status 127, as with other exec errors. */ + } + return new_fd; } #endif /* HAVE_FORK */ @@ -1357,27 +1383,25 @@ start_child (const char *exec_file, char **exec_argv, if (err == -1) err = open ("/dev/null", O_WRONLY); - if (in > 0) - { - if (out == 0) - out = renumber_file_descriptor (out, err); - if (err == 0) - err = renumber_file_descriptor (err, err); - do dup2 (in, 0); while (errno == EINTR); - close (in); - } - if (out > 1) - { - if (err == 1) - err = renumber_file_descriptor (err, err); - do dup2 (out, 1); while (errno == EINTR); - close (out); - } - if (err > 2) - { - do dup2 (err, 2); while (errno == EINTR); - close (err); - } + /* Dup each non-yet-dup2'd fd that's in the way to the next available fd, + so that we can safely dup2 to 0/1/2 without potentially overwriting + in/out/err. Note that dup2 doesn't do anything if its arguments are + equal. */ + if (out == 0) + out = dup_handle_error (out, err); + if (err == 0) + err = dup_handle_error (err, err); + dup2_handle_error (in, 0, err); + + if (err == 1) + err = dup_handle_error (err, err); + dup2_handle_error (out, 1, err); + + dup2_handle_error (err, 2, err); + + if (in > 2) close (in); + if (out > 2) close (out); + if (err > 2) close (err); execvp (exec_file, exec_argv); -- 2.36.0 From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:50:55 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:50:55 +0000 Received: from localhost ([127.0.0.1]:60148 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKRb-0005lG-73 for submit@debbugs.gnu.org; Thu, 26 May 2022 16:50:55 -0400 Received: from jpoiret.xyz ([206.189.101.64]:49112) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKRZ-0005l6-PF for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:50:54 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 92C3C1851E0; Thu, 26 May 2022 20:50:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1653598253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jyI1w3v4SM61nS6P+eq8N3bLtYPmZXK0BiZU/x8DsbI=; b=puupIP5hAJWOCvg9tlwBEGnWvJDaYub9r7//d8kxqTSopNzcE2YQIhgYzebof+eHOiO0yh UnIOQUMQjyy2UViL0E2NlrKA/mugmLwYZ85fBrCeiOZiGqbXsiXVTsE9ku/W3tTp5gF1Ko 3wqUjNxFUAXEd4xpIXE0iZVLyfxybquDfPljp4Xjzp0ZxViuJJmeyecgyW+x80m+k/zlSN W/wC2XIzafcHOTxrTnIz5nH6CrkCGVvVk6NcKYkeaiCvy46ydwvDiT21kzQyqqkDei1aIe RmxUVOVDCkNNcf+EIDrjPUO3WakRVLpn5it3aCQS/v0UCrW2WqPxGzro+J3iXA== From: Josselin Poiret To: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Christopher Baines Subject: [PATCH 2/2] Improve thread safety of piped-process. Date: Thu, 26 May 2022 22:50:47 +0200 Message-Id: <20220526205047.23960-2-dev@jpoiret.xyz> In-Reply-To: <87sfow10lc.fsf@jpoiret.xyz> References: <87sfow10lc.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * libguile/posix.c (scm_piped_process): Avoid double close. --- libguile/posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libguile/posix.c b/libguile/posix.c index dc3080b3c..e134408e3 100644 --- a/libguile/posix.c +++ b/libguile/posix.c @@ -1486,7 +1486,7 @@ scm_piped_process (SCM prog, SCM args, SCM from, [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Josselin Poiret X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * libguile/posix.c (scm_piped_process): Avoid double close. --- libguile/posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libguile/posix.c b/libguile/posix.c index dc3080b3c..e134408e3 100644 --- a/libguile/posix.c +++ b/libguile/posix.c @@ -1486,7 +1486,7 @@ scm_piped_process (SCM prog, SCM args, SCM from, [...] Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager * libguile/posix.c (scm_piped_process): Avoid double close. --- libguile/posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libguile/posix.c b/libguile/posix.c index dc3080b3c..e134408e3 100644 --- a/libguile/posix.c +++ b/libguile/posix.c @@ -1486,7 +1486,7 @@ scm_piped_process (SCM prog, SCM args, SCM from, SCM to) SCM_SYSERROR; } - if (reading) + if (reading && !(writing && c2p[1] == p2c[0])) close (c2p[1]); if (writing) close (p2c[0]); -- 2.36.0 From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:55:29 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:55:29 +0000 Received: from localhost ([127.0.0.1]:60173 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKW1-0005sz-Gd for submit@debbugs.gnu.org; Thu, 26 May 2022 16:55:29 -0400 Received: from michel.telenet-ops.be ([195.130.137.88]:38238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKVy-0005sp-Ul for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:55:28 -0400 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by michel.telenet-ops.be with bizsmtp id bYvR270014UW6Th06YvRRS; Thu, 26 May 2022 22:55:25 +0200 Message-ID: <4e763e7f1dafa427a1cf82df3241b90282085931.camel@telenet.be> Subject: Re: bug#55441: [PATCH 1/2] Fix child spawning closing standard fds prematurely From: Maxime Devos To: Josselin Poiret , Ludovic =?ISO-8859-1?Q?Court=E8s?= , Christopher Baines Date: Thu, 26 May 2022 22:55:17 +0200 In-Reply-To: <20220526205047.23960-1-dev@jpoiret.xyz> References: <87sfow10lc.fsf@jpoiret.xyz> <20220526205047.23960-1-dev@jpoiret.xyz> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-n/c6yq61ysNVHcOesppq" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1653598525; bh=PG3nwvgBkstb95cZuMbaOp02wbuZSEh0Bj+UPl03qSk=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=JvvLUWGejbmmtOHf81a1Vpw1zb0yKvKyXI9ClwR6RnyTmTIOV1T6bVcZ6EHefa/lR SDQN4xmgcy6/2C8NIxF6T3CaFXuJCZTmvHs/Gmd6rM9c/jjFJOGBtWCMN/7lmU3qdc 7EXJEvwsVLodaquEpF0A0id4fw8JKN/yT5rdBbD/zWpPsljU8H9S9+wBp76Bn7ddn9 zUyoTrcfgcDz5BE4e8oKNJw1PoXD/KY1CbEQABRVh4WiNS4eq4jpsG2cuWWVpF2Mfd SfRVzungQyOutOVCDaP9l9Ecyn9GhVqIxuUwTp2PY41zodJrjzgxL+xW5nN2jInMDH XdCvYWAruygKw== X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-n/c6yq61ysNVHcOesppq Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Josselin Poiret via Bug reports for GNU Guix schreef op do 26-05-2022 om 22:50 [+0200]: > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fprintf (fdopen (err, "a"), "start_child:= dup failed: %s\n", msg); fdopen can fail due to out-of-memory problems, so you'll have to check the return value, or use dprintf instead which accepts a file descriptor. Greetings, Maxime. --=-n/c6yq61ysNVHcOesppq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYo/pNRccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7talAP9oVkIikXdtWSzxvOeu2raaTF02 ReTzA5XqSaNnJhyCQwEAtzuFYJrpDBQj5uTwIE14lv97DjuYJHQl9SYhjRmK5Qk= =wFJT -----END PGP SIGNATURE----- --=-n/c6yq61ysNVHcOesppq-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 16:58:46 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 20:58:46 +0000 Received: from localhost ([127.0.0.1]:60180 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKZB-0005xg-VB for submit@debbugs.gnu.org; Thu, 26 May 2022 16:58:46 -0400 Received: from laurent.telenet-ops.be ([195.130.137.89]:52786) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKZA-0005xX-2C for 55441@debbugs.gnu.org; Thu, 26 May 2022 16:58:44 -0400 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by laurent.telenet-ops.be with bizsmtp id bYyi270074UW6Th01YyipR; Thu, 26 May 2022 22:58:42 +0200 Message-ID: <16f8aa929bcba3536da4701f9b1152f28ca4b893.camel@telenet.be> Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections From: Maxime Devos To: Josselin Poiret , Ludovic =?ISO-8859-1?Q?Court=E8s?= , Christopher Baines Date: Thu, 26 May 2022 22:58:42 +0200 In-Reply-To: <87sfow10lc.fsf@jpoiret.xyz> References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> <87h75dtcaa.fsf@cbaines.net> <871qwghbme.fsf@gnu.org> <87sfow10lc.fsf@jpoiret.xyz> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-tDvHSVanuBYZZSO6fzkn" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1653598722; bh=B/uQ9HA4c9b6DW+VT9cNUfc/PT4DSBLzltKA40gvS10=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=lkxMfj0Hi/1R5mrCANmSEsTuMcRLtRjyJEJZ90z1QqwWa3pWOaU2Bk3z54qSlIUx+ lbLG7sz4xLlc1hH96WrswVFnAKrYAeoPErs88qKXAHQQuVrnsft13CfE0jiCk2sRj3 4D420PQ5tv7ASus9cfcHI4GmjWx8XbwChvf4fgRBxKsGirwQI3D9xc6axERA3QEAi/ 4kQTUjU8d/HSgQF6hEjizEynShmwuak46Wjk8JVmmaETvWwGDVuOgKTUCZzp+iJSfr 8kdZ2TlpRTJFUdTqnnN0l7OtceOlejbl6HD2t9TGP4JI0u40ROcn7ZGla4CKGNcUHY zslylv+cEhrQw== X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-tDvHSVanuBYZZSO6fzkn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Josselin Poiret via Bug reports for GNU Guix schreef op do 26-05-2022 om 22:49 [+0200]: > Hello everyone, >=20 > With all the recent platform changes, I was looking at `make > cuirass-jobs` for the first time and it left some deadlocked inferiors > as well, so I figured I could try my own solution, since I've already > dabbled in Guile POSIX internals! Here's a patch that rewrites > open-bidirectional-pipe with a more thread-safe alternative, > `piped-process`, but this also requires some Guile patches, that I will > post as well. With these, `make cuirass-jobs` doesn't hang at all > anymore for me. >=20 > Let me know what you think of these >=20 > Best, The process opening code looks fragile, so I think some tests are required to avoid causing regressions later. Greetings, Maxime. --=-tDvHSVanuBYZZSO6fzkn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYo/qAhccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7qOgAP9m6yapGzXa40adLWf3FBueIL8/ PFrPZk7zLX9PlvzUswD/Wf7op9MzB9rWvwEBRppdN55wbFD2SzQk4DgPI9Ru0wE= =WKTD -----END PGP SIGNATURE----- --=-tDvHSVanuBYZZSO6fzkn-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 17:02:08 2022 Received: (at 55441) by debbugs.gnu.org; 26 May 2022 21:02:08 +0000 Received: from localhost ([127.0.0.1]:60185 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKcS-00064I-Df for submit@debbugs.gnu.org; Thu, 26 May 2022 17:02:08 -0400 Received: from albert.telenet-ops.be ([195.130.137.90]:45582) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nuKcQ-000649-0a for 55441@debbugs.gnu.org; Thu, 26 May 2022 17:02:07 -0400 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by albert.telenet-ops.be with bizsmtp id bZ24270084UW6Th06Z244R; Thu, 26 May 2022 23:02:04 +0200 Message-ID: <273851b9eabb7f2fa0ae02ec6aaf074292e35580.camel@telenet.be> Subject: Re: bug#55441: [PATCH] guix: inferior: Make open-bidirectional-pipe use piped-process. From: Maxime Devos To: Josselin Poiret , Ludovic =?ISO-8859-1?Q?Court=E8s?= , Christopher Baines Date: Thu, 26 May 2022 23:02:04 +0200 In-Reply-To: <20220526205016.23899-1-dev@jpoiret.xyz> References: <87sfow10lc.fsf@jpoiret.xyz> <20220526205016.23899-1-dev@jpoiret.xyz> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-zyfw3vLhTHyJPAP+xOod" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1653598924; bh=IX+SAS7HySYGpwkaJOZx06VJZZFay0Uu5mEa9+4+pWQ=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=aQhHxKOny5tSirJ16CGF96SKm4DF4nbZOJ2fsdKjyitoxrcl8bceloErGGJuXtdvM FsSID923I2mD0GQn/otmniS8kTpreLhJ9LcHPATWsWXYMkboep9A7+tI8LTm5Pa/8/ MgNjNpBbyZ3ebq6Z+Eo22kmfiSuAlI31ckwOO7du26wHzNLkWd/F3JhlEnHww0xg9C ZVse73dZvYfSXOUvJ/Am3reng2sbMnl+2jK2fpfB1SuioDogIQcCnRLYaWvIsRx2UR Iy5vKh4BEV2TebI4ha/ZljQ1dZjM6Xm/hjQlFc1WjLPws1xKgi5EVGCnkIdDVEuFpU uerPaZ8Q4V4vA== X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-zyfw3vLhTHyJPAP+xOod Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Josselin Poiret via Bug reports for GNU Guix schreef op do 26-05-2022 om 22:50 [+0200]: > * guix/inferior.scm: Load up scm_init_popen extension to be able to > use piped-process. Can we export the procedure instead from (ice-9 popen) + use @@ for now, instead of doing this low-level thing? Greetings, Maxime. --=-zyfw3vLhTHyJPAP+xOod Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYo/qzBccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rTyAP4qrXSn/6ihdtnwDVrh6Bp0dHLR M+JUowRjeiry8rvnVwD8C34OeGVOxxEXceful4LCdf3XYJyNANUC7iNclxHu7gM= =W4a0 -----END PGP SIGNATURE----- --=-zyfw3vLhTHyJPAP+xOod-- From debbugs-submit-bounces@debbugs.gnu.org Sat May 28 10:02:45 2022 Received: (at 55441) by debbugs.gnu.org; 28 May 2022 14:02:45 +0000 Received: from localhost ([127.0.0.1]:38638 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nux1g-0003Nw-N6 for submit@debbugs.gnu.org; Sat, 28 May 2022 10:02:45 -0400 Received: from jpoiret.xyz ([206.189.101.64]:37732) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nux1c-0003Nl-4B for 55441@debbugs.gnu.org; Sat, 28 May 2022 10:02:43 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 80CE71851D2; Sat, 28 May 2022 14:02:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1653746555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=O4PnMkEPSIB3dqyg3Ljq2OLeCxdRbVjjB1u+cqFaM/M=; b=JIrQe93dEJXiw1pBzDRRQfI5Y9gs8SNktOWEbV0yYLrcFZ7nFFvA0rLZIh7DDD6x140iXv oIL97+eMumKH/5aQlQKs9zIPxqCqWOdfWlMfEgpcbt6VOU9yoR05Rj86O/9qw9i4eDHaL3 whSqgqTl8wGJ+PVPSKu5lOguNCc7lIbxjfZFI0ATNMKVN6FAXn5EX2ESpvUY+s7Qg24MC4 lF1uWJ4OKeQDSKFVjJmA5+RdVFnW2angVGhP8VX/KS+tNz2J6KLoQBfjIuwT7VsUcZi2Dm xYy9as6yNtBzwPkLj7QkZYkR1vQB060NM549g2ycYnPpegZ/Iw2WxVwh42D8iw== From: Josselin Poiret To: Maxime Devos , Ludovic =?utf-8?Q?Court=C3=A8s?= , Christopher Baines Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections In-Reply-To: <16f8aa929bcba3536da4701f9b1152f28ca4b893.camel@telenet.be> References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> <87h75dtcaa.fsf@cbaines.net> <871qwghbme.fsf@gnu.org> <87sfow10lc.fsf@jpoiret.xyz> <16f8aa929bcba3536da4701f9b1152f28ca4b893.camel@telenet.be> Date: Sat, 28 May 2022 16:02:33 +0200 Message-ID: <87ilpphi12.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Type: text/plain Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hello Maxime, Thanks for your review. I took your comments into account and actually wrote some more patches to clean up start_child of potential thread unsafe practices. Since these are not guix-specific, I sent t [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hello Maxime, Thanks for your review. I took your comments into account and actually wrote some more patches to clean up start_child of potential thread unsafe practices. Since these are not guix-specific, I sent t [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager Hello Maxime, Thanks for your review. I took your comments into account and actually wrote some more patches to clean up start_child of potential thread unsafe practices. Since these are not guix-specific, I sent the whole patchset to guile-devel instead [1]. Maxime Devos writes: > The process opening code looks fragile, so I think some tests are > required to avoid causing regressions later. How would you suggest we do that? Since these are mostly racing issues between threads, I don't know how we could reproduce these reliably in tests. [1] v4 PATCH at https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52835 (20220528124634.17353-1-dev@jpoiret.xyz) Best, -- Josselin Poiret From debbugs-submit-bounces@debbugs.gnu.org Sat May 28 13:16:04 2022 Received: (at 55441) by debbugs.gnu.org; 28 May 2022 17:16:04 +0000 Received: from localhost ([127.0.0.1]:38822 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nv02m-000233-Hh for submit@debbugs.gnu.org; Sat, 28 May 2022 13:16:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48352) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nv02i-00022O-9e for 55441@debbugs.gnu.org; Sat, 28 May 2022 13:16:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59774) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nv02c-0001tr-Ue; Sat, 28 May 2022 13:15:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=uFjOD3MUq/scYgp+dmb/nUAzbix+2kxtbaEgpgC7nj0=; b=P2hyTOwVYKG0GLRtNzLd 52kd9Kcvc9qlnAezOtB2FsvFJsh/C+tArpO8+Y/LnkHbtdJeBDXUcPRJzhbRWf/1xJL/QCJqO27x2 huij8DrsOwSmq1m7m1Cl6DwaieNPgOCnn11MNmYsyMeyRYgLmqrM6dsYfeKhIXpDpjEWnPpRDM9ae NkMc7kuwFdGWATbzOXnR/0V6uM1CislpxaMaRhKoBAajDbEZ03rFt10hJuqusPZxsHdg7U4tL/Ond ujWpqDQKyRp8jO1Ik0xyVcHrvKyDna7ktAJmBn39UrtxHwbH6uqRybc6fiBSNivunq5Acfz7wCJQB Fi1dxBa1j/1gcQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:52050 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nv02c-0000ta-Im; Sat, 28 May 2022 13:15:54 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: [PATCH] guix: inferior: Make open-bidirectional-pipe use piped-process. References: <87sfow10lc.fsf@jpoiret.xyz> <20220526205016.23899-1-dev@jpoiret.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 9 Prairial an 230 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 28 May 2022 19:15:52 +0200 In-Reply-To: <20220526205016.23899-1-dev@jpoiret.xyz> (Josselin Poiret's message of "Thu, 26 May 2022 22:50:16 +0200") Message-ID: <87h759bmt3.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Christopher Baines X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi, Josselin Poiret skribis: > * guix/inferior.scm: Load up scm_init_popen extension to be able to > use piped-process. > (open-bidirectional-pipe): Remove custom fork+exec code and use > piped-process instead. [...] > +(eval-when (expand load eval) > + (load-extension (string-append "libguile-" (effective-version)) > + "scm_init_popen")) To me this is not an option because it=E2=80=99s using an internal function= and it=E2=80=99s too fragile (that C function is not meant to be called more th= an once). That said, I agree that in principle we should be doing something along these lines: calling out to a C function that does fork+dup+exec in a safe way. Thanks for looking into this! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 26 09:40:10 2023 Received: (at 55441) by debbugs.gnu.org; 26 Jan 2023 14:40:10 +0000 Received: from localhost ([127.0.0.1]:32848 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Q9-0004df-PR for submit@debbugs.gnu.org; Thu, 26 Jan 2023 09:40:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50654) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Q8-0004dT-Bf for 55441@debbugs.gnu.org; Thu, 26 Jan 2023 09:40:08 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Q1-0001Uu-L8; Thu, 26 Jan 2023 09:40:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=bbW++DzBHB1EgyrLQFHlL4byUi+a21VRChm1KUmeDpw=; b=rr4BIBvUo/ie7Df09LG8 Hdyhmst6FipNwvt/6ng4HhNOXewbR2DzFmyREBm3EeIyzUMT2/BodLdbbwkGmYRjVS4AZ3GRSP4Jo M+dYQhdxc2BGea3RyqJFyxzlBYHMxarpTddCCPrOyKQpinNlNwhCTvHGBkifEcZGhk6PlQO9s5Nkg JQiApYGgnAUyh2WxZPJRaroDj+bItN/K6rVloPy5ayK6A0R0SnAaskFg2D30qymXFJ/3ZUFYmmbek nBg9UtvyUwjk1fEiVnShYLPd0ryDa+5TqmDPDqvai/V/p+H30HhgaFJgFMEcK4uKZo/hGxb7WXaSU IblYUF0yRHxCww==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Q0-0000yv-4u; Thu, 26 Jan 2023 09:40:00 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> <878rr2kug6.fsf_-_@gmail.com> <87pmke6ig4.fsf@gnu.org> <87o7zxalhu.fsf@gnu.org> <87k0alahug.fsf@gnu.org> <87fsl9acuw.fsf@gnu.org> <878rr1jsd1.fsf@gmail.com> <87fsl87gb5.fsf@gnu.org> <874k1n5loz.fsf@gnu.org> <87r14ovyud.fsf@gnu.org> <87k0aaiqzu.fsf@gnu.org> <87bkvlh8u2.fsf@gnu.org> <875ylth3rq.fsf@gnu.org> <87h75dtcaa.fsf@cbaines.net> <871qwghbme.fsf@gnu.org> <87sfow10lc.fsf@jpoiret.xyz> <16f8aa929bcba3536da4701f9b1152f28ca4b893.camel@telenet.be> <87ilpphi12.fsf@jpoiret.xyz> Date: Thu, 26 Jan 2023 15:39:57 +0100 In-Reply-To: <87ilpphi12.fsf@jpoiret.xyz> (Josselin Poiret's message of "Sat, 28 May 2022 16:02:33 +0200") Message-ID: <87pmb1nyv6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55441 Cc: 55441@debbugs.gnu.org, Mathieu Othacehe , Christopher Baines , Maxime Devos X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi there! I have some good news! Time has passed but we didn=E2=80=99t remain idle := -) and the scary bug at is finding a happy end! First, Josselin introduced bindings to =E2=80=98posix_spawn=E2=80=99=C2=B9 = in Guile, to use as the foundation of a new =E2=80=98spawn=E2=80=99 procedure, but also for = =E2=80=98system*=E2=80=99, =E2=80=98open-pipe*=E2=80=99, & co.: https://issues.guix.gnu.org/52835 That=E2=80=99s big deal because =E2=80=98posix_spawn=E2=80=99 is usable in = a multi-threaded context (POSIX doesn=E2=80=99t explicitly say so, but it stems from the fact that what happens in the child process before =E2=80=98exec=E2=80=99 is =E2= =80=9Csafe=E2=80=9D, and in particular Guile code doesn=E2=80=99t get a chance to run). Guile 3.0.9 released yesterday includes those changes. Then we have this commit, where I changed =E2=80=98open-bidirectional-pipe= =E2=80=99 in (guix inferior) to use =E2=80=98spawn=E2=80=99 where available: fed3953d70 inferior: Use 'spawn' on Guile 3.0.9+. That solves the problem that led to this bug. Last, we have this update: 230de2e94b gnu: guix: Update to d5fece6. It gives us a =E2=80=98guix=E2=80=99 package that contains the =E2=80=98spa= wn=E2=80=99 fix and removes =E2=80=98guix-for-cuirass=E2=80=99, which is now longer necessary. Thanks everyone for the team work, very nice when working on touch issues like this one! Ludo=E2=80=99. =C2=B9 https://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_spa= wn.html From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 26 09:41:52 2023 Received: (at control) by debbugs.gnu.org; 26 Jan 2023 14:41:52 +0000 Received: from localhost ([127.0.0.1]:32853 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Ro-0004gG-As for submit@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46942) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Rm-0004g3-2x for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:50 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Rg-00021J-SY for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=CRbZhrvSRqtDJFHJq6SH/JOK4KZnveidjabYwdjieeU=; b=Afcy7h79htIOvr 8MIuHi51edDfAs4GI1Mt80cVYrPKwvYWsu50a5i9S89tp/qcvKErjqWiwR7xqAdFiMP3q3o6lWnGv RLtU5qMBXoqeQJz3o23guVXfH7rcV0WHWwGRgptoReeIUtkE4nk1FfAFWbEQ320tT471LevCvFtCV 47mfw/zreTHauoUtCFstThlhz08e/Ppc2QrJ4FRK9tgwfXsgU7Eqdkr/yxluMZ/AB/mDzNYY3Oh6V hc4k16pGnDvkpPX1Hd9OQBfTMLelIX1rpi8e+DUHLNCu3TicXXijON86O0//sJiEmwO7tM9Aa2XeU yYLf7FnIl3gSm/DDHDqA==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Rg-0001Ay-Bg for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:44 -0500 Date: Thu, 26 Jan 2023 15:41:43 +0100 Message-Id: <87o7qlnys8.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #55441 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) retitle 55441 Use of 'primitive-fork' in (guix inferior) leads to hangs in 'cuirass evaluate' quit From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 26 09:41:56 2023 Received: (at control) by debbugs.gnu.org; 26 Jan 2023 14:41:56 +0000 Received: from localhost ([127.0.0.1]:32857 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Rs-0004gb-HS for submit@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pL3Rq-0004g8-IX for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:54 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Rl-00021o-CF for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=jnA8vz6ilT1uE/QWjWfr0iQURcFdub4Yn0yONtdZb2s=; b=iKCcxczUT4bhyu bj4AkHQtW895Q2XplcAUKsh9U4IIsD217iEUhSXtpOuqgO5w2Qy1rufCU6S9FASgWpZ2EeEwaFszu XWYMaF2v/8CwjEwg0gZGaHJKhTqYebZA0/MujN+cjmrKHezgV0EDkVdLLiWThBIN0MbUFLa49Ci19 fDkvdgQlMRlY/r33wSoB6Keni8NU06CJVlOhw/+mS/HygHN4sr/0Ab8nQYzRo0hLpTDyToZjyNURu U2PdFuWoDZIQyfmEN3/wjnfk8edbNbc7RQ2OC4bkOamX49bEblgARdjQCqKET0BUgS46k4gVTuTDM LOi6oRra4XH7qmmlDWpw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pL3Rl-0001Bd-01 for control@debbugs.gnu.org; Thu, 26 Jan 2023 09:41:49 -0500 Date: Thu, 26 Jan 2023 15:41:47 +0100 Message-Id: <87mt65nys4.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #55441 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 55441 quit From unknown Tue Jun 24 19:09:29 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 24 Feb 2023 12:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator