From unknown Tue Jun 17 20:13:22 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#71299 <71299@debbugs.gnu.org> To: bug#71299 <71299@debbugs.gnu.org> Subject: Status: Better handle "updating substitutes" messages in CI pipelines and dumb terminals Reply-To: bug#71299 <71299@debbugs.gnu.org> Date: Wed, 18 Jun 2025 03:13:22 +0000 retitle 71299 Better handle "updating substitutes" messages in CI pipelines= and dumb terminals reassign 71299 guix submitter 71299 Richard Sent severity 71299 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Fri May 31 18:10:56 2024 Received: (at submit) by debbugs.gnu.org; 31 May 2024 22:10:56 +0000 Received: from localhost ([127.0.0.1]:55457 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDASb-0002he-Qc for submit@debbugs.gnu.org; Fri, 31 May 2024 18:10:56 -0400 Received: from lists.gnu.org ([209.51.188.17]:33112) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDASL-0002hG-Gc for submit@debbugs.gnu.org; Fri, 31 May 2024 18:10:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sDAS9-0004n8-T5 for bug-guix@gnu.org; Fri, 31 May 2024 18:10:25 -0400 Received: from mail-108-mta168.mxroute.com ([136.175.108.168]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sDAS7-0000Kw-3Y for bug-guix@gnu.org; Fri, 31 May 2024 18:10:25 -0400 Received: from filter006.mxroute.com ([136.175.111.3] filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta168.mxroute.com (ZoneMTA) with ESMTPSA id 18fd0b42e36000e2b6.001 for (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384); Fri, 31 May 2024 22:10:20 +0000 X-Zone-Loop: 4d0879a6072874e0ea87624e48d0df5e3a876a7bcd15 X-Originating-IP: [136.175.111.3] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=freakingpenguin.com; s=x; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=G/rzBepNdZOQEywkLouCH33yngZdd4QvPW7J9/RWEJA=; b=lFd4tXSd1IaW/a6yyH5vnBu54n oe6cNh5gJZOI2vv267InQ+3qZSlyTArEMOPRFnZXhLm/tMiw5lxIfur8zNreDWSyH4YHD2I+Vp4YZ /8jA4Q/74FfMktbwKz+3J3qT3cjuWOybbd42NwNp8iwC9itV0emlEvqcFX6RKUVPmcZaV0PiBDwbj FlVvFxplZvGLuYG+8f/n9UvMVXY2NLaHPoWxqGmutEUvWIC9mE8XtS0X7rKECqUz8Fido9vIQGutY svcSMC52r1Gtzwsx3pccZMiI06F3ff5heM3xpHgw4VupN+hYkvbKjXceqWDWjW1s1aZMpTP3JBO1O rcKZ53dA==; From: Richard Sent To: bug-guix@gnu.org Subject: Better handle "updating substitutes" messages in CI pipelines and dumb terminals Date: Fri, 31 May 2024 18:10:16 -0400 Message-ID: <87ikyt8wfr.fsf@freakingpenguin.com> MIME-Version: 1.0 Content-Type: text/plain X-Authenticated-Id: richard@freakingpenguin.com Received-SPF: pass client-ip=136.175.108.168; envelope-from=richard@freakingpenguin.com; helo=mail-108-mta168.mxroute.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi Guix! Due to it's nature as an incremental counter (from 0% to 100%), Guix prints many messages to the terminal when it is updating substitutes. This hurts CI log files because large portions of the output is filled with cruft [1]. This problem is made worse because Guix updates substitutes multiple times during a build. verbosity=0 can be used to disable ALL output messages. However, this isn't ideal for automated build pipelines where people need to monitor the status and review what may have gone wrong. Because "updating substitutes" prints so many lines to the terminal that aren't typically relevant from a log reviewer's perspective that hide useful information, it should be possible to disable just those messages. Alternatively, I believe Guix has code to detect when it's attached to a "dumb" terminal. Perhaps these messages should be conditionally removed or replaced with a pair of static messages like the following: --8<---------------cut here---------------start------------->8--- substitute: Updating subsitutes from ..... substitute: Finished updating substitutes from .... --8<---------------cut here---------------end--------------->8--- This would make log files much easier to parse. Instead of having potentially thousands of lines printed to the console per "build stage", it would be limited to 2. [1]: https://builds.sr.ht/query/log/1238029/update-readme/log -- Take it easy, Richard Sent Making my computer weirder one commit at a time. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 01 14:40:25 2024 Received: (at 71299) by debbugs.gnu.org; 1 Jun 2024 18:40:25 +0000 Received: from localhost ([127.0.0.1]:57399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDTeS-0005KT-TP for submit@debbugs.gnu.org; Sat, 01 Jun 2024 14:40:25 -0400 Received: from mail-108-mta11.mxroute.com ([136.175.108.11]:37651) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDTeQ-0005KE-NG for 71299@debbugs.gnu.org; Sat, 01 Jun 2024 14:40:23 -0400 Received: from filter006.mxroute.com ([136.175.111.3] filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta11.mxroute.com (ZoneMTA) with ESMTPSA id 18fd51a0ced000e2b6.001 for <71299@debbugs.gnu.org> (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384); Sat, 01 Jun 2024 18:40:05 +0000 X-Zone-Loop: fa16059d81f9e63e5bc6e223a39a19e8b988a005b3b7 X-Originating-IP: [136.175.111.3] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=freakingpenguin.com; s=x; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=cPYm57D9oDlRWVTypfXkF82qqliKZ7rUMbMszWiWbzg=; b=SvqBA0SMMdpQZOmYfhMSRitshB +wnHiIB6wFuwOoAc5aUnqPU+yyXXYLvCUxCmsY+oSJoDS4NFnq28FU8+No6xVJ3OIoLjkiKhct2fy D3YB1yL7Neye2FMepriNb9S/gLoaZd+nmsv13xr6bXidUCYxAD6oz++XFbWiNs6QjAKfLuP6CC+cm 2pJCNOPIVBX+Us3iqH+LucI/+cw65GEp7inPYetIKPGvf3tR3tP+XeUdU6VFoOgSlk07sR94xGl+l 1Iigq1LZdhyMxyQsrVi9IN8ZE2tHTf0g4qij6TYyuo5o7Ye1UDaegP22JPW0i5K5PZRSdYoGcCTDa o/S6HfNQ==; From: Richard Sent To: 71299@debbugs.gnu.org Subject: Investigation Date: Sat, 01 Jun 2024 14:39:55 -0400 Message-ID: <87a5k48q2s.fsf@freakingpenguin.com> MIME-Version: 1.0 Content-Type: text/plain X-Authenticated-Id: richard@freakingpenguin.com X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 71299 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Guix! I started looking into this problem and unfortunately it looks like it might be more complicated than I thought. My initial thought was having two progress-reporters and selecting on based on a simple conditional if current-error-port was a tty or not. This would be in guix/scripts/substitute. Unfortunately, this might be a bit more challenging. Unlike the other scripts, guix/scripts/substitute is invoked by the daemon directly. Furthermore, it isn't passed a file descriptor to the invokers stderr. Instead, it goes through the build daemon. See nix/libstore/local-store.cc:LocalStore::getLineFromSubstituter. >From my testing calling istty? on that error port always returns #f, ergo the "simple" progress-reporter is always used and we lose the interactive progress bars. Testing this is also a pain as well because it involves changing the build daemon. In my experience the easiest way is: 1. Change the commit field for the guix package in package-management.scm to the commit you want to check 2. Update the url field for the guix package to "file:///path/to/guix/clone" 3. Forcefully disable authentication in guix/build-system/channel.scm 4. Generate an installer image via $ ./pre-inst-env guix system image gnu/system/install.scm --image-type=iso9660, then boot from that image and observe the output. So, that leaves two options that I can see: 1. The daemon stops capturing stderr from the substituter and merely passes it the daemon's stderr. 1. Challenging to get right and this may have unforseen consequences. Two processes writing freely to the same output at once is a bad idea. 2. The daemon has a isatty? check and sets a flag for the build agent which is passed along to scripts/substitute. -- Take it easy, Richard Sent Making my computer weirder one commit at a time. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 01 15:06:46 2024 Received: (at 71299) by debbugs.gnu.org; 1 Jun 2024 19:06:46 +0000 Received: from localhost ([127.0.0.1]:57417 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDU3x-00060r-ME for submit@debbugs.gnu.org; Sat, 01 Jun 2024 15:06:45 -0400 Received: from mail-108-mta173.mxroute.com ([136.175.108.173]:38015) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sDU3v-00060i-Aj for 71299@debbugs.gnu.org; Sat, 01 Jun 2024 15:06:44 -0400 Received: from filter006.mxroute.com ([136.175.111.3] filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta173.mxroute.com (ZoneMTA) with ESMTPSA id 18fd5322996000e2b6.001 for <71299@debbugs.gnu.org> (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384); Sat, 01 Jun 2024 19:06:25 +0000 X-Zone-Loop: 9b96ca194b6fd87bac658e37764504c93ea9ad6fa0aa X-Originating-IP: [136.175.111.3] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=freakingpenguin.com; s=x; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=NOUBRKkFKknnxvbNtBmEIVChgDfUoIedZVMWLnFKpJo=; b=pIdlWnw1nuK81oupN6VFH+AmqS 5Hn8netpe5JCv9kd2DVonjq3RKbFwvInE29pooaa0W2PCL/d2VkGkYg/fAiQ15yFxiipWVFPIPX8j cYnT7exUr0z4rKmYLy4IWnUnN0grc2CNN9WRbORSJEBVlLjUaolaAXLXZIHkYfzcC9KU0K6qOxqiy va4f7EjGZTtfsMNgiBcex4ITlQwvA5IcRDEFfsvqwZl3FBF4UBJDY0N15D5onFMGHGGBmp1NB+0lO NXTPHzeaYMsSEQQy/jvsx4sAqrByDaqryqu8yOgxFgxw4XUMIfsVT9ZR/jEm6lm5RxAYrSfDGk90x KSyaMa5w==; From: Richard Sent To: 71299@debbugs.gnu.org Subject: Fixed link for demonstration of the problem Date: Sat, 01 Jun 2024 15:06:21 -0400 Message-ID: <875xus8ouq.fsf@freakingpenguin.com> MIME-Version: 1.0 Content-Type: text/plain X-Authenticated-Id: richard@freakingpenguin.com X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 71299 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I was informed that the link I mentioned in the first issue is nonpublic. Here's the definitely public build output that demonstrates the problem, albeit in trimmed form: https://builds.sr.ht/~freakingpenguin/job/1238029 -- Take it easy, Richard Sent Making my computer weirder one commit at a time.