From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 0/4] Avoid 'fdatasync' calls for cache files Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-To: guix-patches@gnu.org X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by submit@debbugs.gnu.org id=B.174420852328405 (code B ref -1); Wed, 09 Apr 2025 14:23:03 +0000 Received: (at submit) by debbugs.gnu.org; 9 Apr 2025 14:22:03 +0000 Received: from localhost ([127.0.0.1]:40893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WJW-0007O3-KU for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:22:03 -0400 Received: from lists.gnu.org ([2001:470:142::17]:46164) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WJT-0007NU-NI for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:22:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WJO-0002w8-8b for guix-patches@gnu.org; Wed, 09 Apr 2025 10:21:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WJM-0002dN-Lg; Wed, 09 Apr 2025 10:21:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=oirkk7njtJ8auGsVs5bco3S8SASX5ZLFC1nVurNaJq4=; b=naB2GWCXE/1AnL NWeN91rYToYeZNRu3y669t+xz4KdzjTdPtzdtJ7XJz5fMvkasUwtok6ivVkeKjE1F9NmDW4dMwPFv 6WNi7KsvqGr99GflYigXGiBJmEI0iUI1RIzam2eyn/FITzOke6ZB9JKtMx8iN7TlVPI/8rJRjKrB5 l+GALr5Si9w+WP5JDEbFM8mOY3vnghrBULlJ9hqvYjrm8kdkfRIU+W9Evo2Ra/UK3WmBZOnTTJoRy udcw4+rsaCytf8gH7pMeAGSXlpuhL+lOMMD303yOM5sqoVV0b9JYKKAnO+0hH32pM/bxLegRPZF0x lhSpOp3Ty24JXr1RsA3g==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:21:27 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, As Chris found out, calling ‘fdatasync’ for each and every narinfo cache file created by ‘guix substitute’ (and likewise for other caches) is too expensive and pointless. This patch series fixes that. Thoughts? Ludo’. Ludovic Courtès (4): cache: Remove unused import. utils: Add #:sync? parameter to ‘with-atomic-file-output’. substitutes: Ignore corrupt cached narinfos. guix: Avoid ‘fdatasync’ call for caches and regular files. guix/cache.scm | 1 - guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 8 +++++--- guix/utils.scm | 14 ++++++++++---- 7 files changed, 26 insertions(+), 15 deletions(-) base-commit: c88f98bb3ca2616baea6b1e452cc900cc9c87503 -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 1/4] cache: Remove unused import. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:24:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174420864128700 (code B ref 77680); Wed, 09 Apr 2025 14:24:05 +0000 Received: (at 77680) by debbugs.gnu.org; 9 Apr 2025 14:24:01 +0000 Received: from localhost ([127.0.0.1]:40914 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WLN-0007Sg-Tg for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54602) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WLK-0007SH-Qi for 77680@debbugs.gnu.org; Wed, 09 Apr 2025 10:23:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WLE-0002n8-UF; Wed, 09 Apr 2025 10:23:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=Q/Ab67I5C3aE19soHcWl7rJqJU9p3eqirmYrVfEQqVM=; b=KkyGmWEckP39aNlmlxPj mB3N3MSb6wjRodgfJslnWofY+GTII1YAOAKmDJO7OuCxmQARxt8VyvUZiy23jU0VKH3DHHWxDMyjl l0W8P3GTWR0fRWiMxxp96h2T+d0Y3OUmwASnZ0lcX8LK1GJ28sTOwpLk0AIBdoBQJg/2WFgU4A1/j /j15uQ4LcHQWemZAM790hDqU51aA63y0CExR1dADiofKNl3OjJrD7CLHo25++i/Hyeh0OeB9fJ6Oa yHsFsmo19PF+dz0VpiKYTKsOjdVAdTrDKtA4ULV4z/by+yzxdQeOjLMJJPn0+eBb3RUOoslJgriw2 WYvfiEEm8C64Fw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:23:36 +0200 Message-ID: <3ff0009f0332d9b28191de3a986b0dbc26330221.1744208418.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/cache.scm: Remove unused import. Change-Id: I06956534aa5342a084ebf71b61518c515ce9288c --- guix/cache.scm | 1 - 1 file changed, 1 deletion(-) diff --git a/guix/cache.scm b/guix/cache.scm index 8b12312c77..5d8a0edbaa 100644 --- a/guix/cache.scm +++ b/guix/cache.scm @@ -18,7 +18,6 @@ ;;; along with GNU Guix. If not, see . (define-module (guix cache) - #:use-module ((guix utils) #:select (with-atomic-file-output)) #:autoload (guix build syscalls) (lock-file unlock-file) #:use-module (srfi srfi-19) #:use-module (srfi srfi-26) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 2/4] utils: Add #:sync? parameter to =?UTF-8?Q?=E2=80=98with-atomic-file-output=E2=80=99.?= Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:24:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174420864328716 (code B ref 77680); Wed, 09 Apr 2025 14:24:05 +0000 Received: (at 77680) by debbugs.gnu.org; 9 Apr 2025 14:24:03 +0000 Received: from localhost ([127.0.0.1]:40916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WLR-0007Ss-6m for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51656) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WLL-0007SK-Si for 77680@debbugs.gnu.org; Wed, 09 Apr 2025 10:23:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WLG-0002o5-Ht; Wed, 09 Apr 2025 10:23:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=aGeG7j1Bmh1K9z0mhsJ66S4LuOy7cbMuuYtHZrDZ4qI=; b=MJ1DL1H7+txCwONAk5aX aRkDmkTreGMPgyJMnT5eNMTPI5FgjGeOgjCnnzMTJqE8rohIVpbqHgK5OqFf2m6LIefiCkp3EJaSw uHBLGJDzTDi05/OOpib2ixPA+SBXtCK+EqivaRE0wM31nllAo5wozzrWLaVVh/sKhY+QIP/9jUGu8 uu+hE1dFMqal4TJz4r9jf91wpz/aTHD2rgvbnGrSHmkaTSeQmhT/S4XXVSN9NMF6//fmQJxsuJRhx HOtojnONCLwNzTgjGAudo7pYIgLy4E5zR+h9Mk38aNKGVhuZilsu8vqOY6k9HjUNTg+gCL3+RAFCB 7pzH4oOCbHV29g==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:23:37 +0200 Message-ID: <0b4f5c1d6e1be1ed563be7aca347817f725a6b10.1744208418.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/utils.scm (with-atomic-file-output): Add #:sync? and honor it. Add ‘force-output’ call. Change-Id: I2479778ae55360c0fab3389ac9249045a27b3568 --- guix/utils.scm | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/guix/utils.scm b/guix/utils.scm index c7c23d9d5b..7ae98096c2 100644 --- a/guix/utils.scm +++ b/guix/utils.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012-2022, 2024 Ludovic Courtès +;;; Copyright © 2012-2022, 2024-2025 Ludovic Courtès ;;; Copyright © 2013, 2014, 2015 Mark H Weaver ;;; Copyright © 2014 Eric Bavier ;;; Copyright © 2014 Ian Denhardt @@ -1057,16 +1057,22 @@ (define (call-with-temporary-directory proc) (lambda () (false-if-exception (delete-file-recursively tmp-dir)))))) -(define (with-atomic-file-output file proc) +(define* (with-atomic-file-output file proc #:key (sync? #t)) "Call PROC with an output port for the file that is going to replace FILE. Upon success, FILE is atomically replaced by what has been written to the -output port, and PROC's result is returned." +output port, and PROC's result is returned. + +When SYNC? is true, call 'fdatasync' on the temporary file before renaming it +to FILE; set it to #false for caches and temporary files to improve +performance." (let* ((template (string-append file ".XXXXXX")) (out (mkstemp! template))) (with-throw-handler #t (lambda () (let ((result (proc out))) - (fdatasync out) + (when sync? + (force-output out) + (fdatasync out)) (close-port out) (rename-file template file) result)) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 3/3] guix: Avoid =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= call for caches and regular files. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:25:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174420864728769 (code B ref 77680); Wed, 09 Apr 2025 14:25:01 +0000 Received: (at 77680) by debbugs.gnu.org; 9 Apr 2025 14:24:07 +0000 Received: from localhost ([127.0.0.1]:40921 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WLV-0007Tc-R4 for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51672) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WLN-0007SO-PI for 77680@debbugs.gnu.org; Wed, 09 Apr 2025 10:23:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WLI-0002ol-Fk; Wed, 09 Apr 2025 10:23:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=CFWlQzJKg8wPYd056P1FcdvdSzjkzGQkiRf6938Wn6s=; b=mOFSb/auaYJsYazw2gbe ske0hL8iqJqkCnMYVXKT5zD+oCzIS+gy99BnhK2yUBVp2Jb9DjF+s3Z5JkcwuOb9Ws2nuj1jMdc6T pvMjXDk/wgGQx3m6HHg8rlekNNR02mFn9oKfiCJHPVFCebd6aEF9jNzhj/YiIOFJC/1VhvxVaqh5k +cRIVft9LHqXAv2/3pXCFeEHy0Cx7IW3VwJCqjyH/b7N3oUN968ewUwtl3ChUd3Ix5h6ZI5xn0JX/ rfZK9pjP65NR7GWpJ9q4nGD/Z+b4BGJ4WcCBXHaU0w8ANTSzicDW1VCE8qF2JrUrtfHgbR7pPbo3h GjvYXoYpK4pGfg==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:23:38 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Fixes . Calling ‘fdatasync’ for each and every narinfo file created by ‘guix substitute’ proved to be too expensive on spinning HDDs and/or under load (from 0.1s to 1.3s for the ‘fdatasync’ call alone). * guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f. * guix/http-client.scm (http-fetch/cached): Likewise. * guix/scripts/discover.scm (write-publish-file): Likewise. * guix/scripts/style.scm (format-whole-file): Likewise. * guix/substitutes.scm (cache-narinfo!): Likewise. Reported-by: Christopher Baines Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf --- guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 3 ++- 5 files changed, 13 insertions(+), 8 deletions(-) diff --git a/guix/git-authenticate.scm b/guix/git-authenticate.scm index 37c69d0880..3cd1175c32 100644 --- a/guix/git-authenticate.scm +++ b/guix/git-authenticate.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2019, 2020, 2021, 2022 Ludovic Courtès +;;; Copyright © 2019-2022, 2025 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id) (chmod port #o600) (display ";; List of previously-authenticated commits.\n\n" port) - (pretty-print lst port)))))) + (pretty-print lst port))) + #:sync? #f))) ;;; diff --git a/guix/http-client.scm b/guix/http-client.scm index 9138a627ac..4e0cc59e91 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012-2018, 2020-2022 Ludovic Courtès +;;; Copyright © 2012-2018, 2020-2022, 2025 Ludovic Courtès ;;; Copyright © 2015 Mark H Weaver ;;; Copyright © 2012, 2015 Free Software Foundation, Inc. ;;; Copyright © 2017 Tobias Geerinckx-Rice @@ -346,7 +346,8 @@ (define* (http-fetch/cached uri #:key (ttl (%http-cache-ttl)) text? (when cache-port (close-port cache-port)) (with-atomic-file-output file - (cut write-cache port <>)) + (cut write-cache port <>) + #:sync? #f) (close-port port) (open-input-file file)))) diff --git a/guix/scripts/discover.scm b/guix/scripts/discover.scm index 32bf6085a5..cbe01638f5 100644 --- a/guix/scripts/discover.scm +++ b/guix/scripts/discover.scm @@ -87,7 +87,8 @@ (define* (write-publish-file #:key (file (%publish-file))) (format port "http://~a:~a~%" (avahi-service-address service) (avahi-service-port service))) - %publish-services))) + %publish-services)) + #:sync? #f) (chmod file #o644)) (define* (read-substitute-urls #:key (file (%publish-file))) diff --git a/guix/scripts/style.scm b/guix/scripts/style.scm index 4801529f7e..c45bdd4458 100644 --- a/guix/scripts/style.scm +++ b/guix/scripts/style.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021-2024 Ludovic Courtès +;;; Copyright © 2021-2025 Ludovic Courtès ;;; Copyright © 2024 Herman Rimm ;;; ;;; This file is part of GNU Guix. @@ -554,7 +554,8 @@ (define* (format-whole-file file order? #:rest rest) (apply pretty-print-with-comments/splice port lst #:format-comment canonicalize-comment #:format-vertical-space canonicalize-vertical-space - rest)))))) + rest)) + #:sync? #f)))) ;;; diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 86b9f5472a..a7a2dda0dc 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -127,7 +127,8 @@ (define (cache-narinfo! cache-url path narinfo ttl) (mkdir-p (dirname file)) (with-atomic-file-output file (lambda (out) - (write (cache-entry cache-url narinfo) out)))) + (write (cache-entry cache-url narinfo) out)) + #:sync? #f)) narinfo) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 3/4] substitutes: Ignore corrupt cached narinfos. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174420866028809 (code B ref 77680); Wed, 09 Apr 2025 14:25:02 +0000 Received: (at 77680) by debbugs.gnu.org; 9 Apr 2025 14:24:20 +0000 Received: from localhost ([127.0.0.1]:40933 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WLk-0007Ua-DL for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46216) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WLh-0007UF-Ns for 77680@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WLc-0002rN-9i; Wed, 09 Apr 2025 10:24:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=rhI9CwQ2rXZA7G2pqrpoNcM35bWjAz2JiDILrj36Odg=; b=MZSZsNnd5TStxyaisKXK WSbzXFdrRBFUtFlGT9Xz4yRM1BG/SdxPjRwk0++uJgUPti8xEH5Vueevqpu0cT67ewDFwpyyVnOi/ 9A0V3wGAK/eMMJqF4b5EiYtksRtPDi1uew64kL8qe81psziiyyu2Dd9czlFk2GEfuSF6jn3ziGPdE 0Or8UgvMJKMSMSCCfiXpNJhes3XQauIbyOskFYfhMdkwTDe70kFSGIJ8gwmgfW+rLScOIGhLONDlq rNwkVmAfGLKX1DxPvj2UzkFPXvFa090z7/hhUZ82dB8vmIryOb4aQmW1nOUFHcUt8d7pV+dDpP2Sr 8iW/ScCoERWGJA==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:23:39 +0200 Message-ID: <2774fbdd96e93995194d5f56bc3d328dc5905326.1744208418.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/substitutes.scm (cached-narinfo): Wrap ‘read’ call in ‘false-if-exception’. Add catch-all ‘match’ clause. Change-Id: I2d776f64b6521f778b4ab3f956b35cdef2ddb383 --- guix/substitutes.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 86b9f5472a..45c165e03e 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -300,7 +300,7 @@ (define (cached-narinfo cache-url path) (lambda () (call-with-input-file cache-file (lambda (p) - (match (read p) + (match (false-if-exception (read p)) (('narinfo ('version 2) ('cache-uri cache-uri) ('date date) ('ttl ttl) ('value #f)) @@ -315,7 +315,8 @@ (define (cached-narinfo cache-url path) (if (obsolete? date now ttl) (values #f #f) (values #t (string->narinfo value cache-uri)))) - (('narinfo ('version v) _ ...) + (_ + ;; Corrupt or incompatible cached entry. (values #f #f)))))) (lambda _ (values #f #f)))) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 4/4] guix: Avoid =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= call for caches and regular files. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Wed, 09 Apr 2025 14:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174420866828838 (code B ref 77680); Wed, 09 Apr 2025 14:25:02 +0000 Received: (at 77680) by debbugs.gnu.org; 9 Apr 2025 14:24:28 +0000 Received: from localhost ([127.0.0.1]:40937 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WLq-0007V2-Ss for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46218) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WLn-0007UX-FO for 77680@debbugs.gnu.org; Wed, 09 Apr 2025 10:24:24 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WLh-0002sO-WC; Wed, 09 Apr 2025 10:24:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ouRxti9NXZfHmqYVd1tC9kAwXFEbokPZzUgA5jlW6Og=; b=bmRnynJ4h+ZN0NZlOZGz RtpT4e0F7zqSRD9TQzsCRX6AqU5zyh1E+KbAMWy1GfV2lrC5kgQL4aoS8sZzTmoF0QFQF6msWRWm0 o0ufXBk2/Yz+sb/jG10omBIwzxhWTOweJNZD9HJwPCUxex2EYoAoIFSCmZaOngRXpxEHz2mzfD72u PvyUc2BzG5Axk7+ddwBJllNYyO/GMtoIZHWGoc1isMbTFoJ8jgBCrhUAC0NgT+bqglEBl8Eg83kuN eAg3eCKHX3gPOt/T3t+PwIU6MGUVqyTqow3tfaMD9tYUcEEVnJXGhw7zEzV/UdlC1IpH4Tu5DNapp X8sGzxWASMBHZQ==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 9 Apr 2025 16:23:40 +0200 Message-ID: <812224bace6510526d09f6cf661bc147ef157c92.1744208418.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Fixes . Calling ‘fdatasync’ for each and every narinfo file created by ‘guix substitute’ proved to be too expensive on spinning HDDs and/or under load (from 0.1s to 1.3s for the ‘fdatasync’ call alone). * guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f. * guix/http-client.scm (http-fetch/cached): Likewise. * guix/scripts/discover.scm (write-publish-file): Likewise. * guix/scripts/style.scm (format-whole-file): Likewise. * guix/substitutes.scm (cache-narinfo!): Likewise. Reported-by: Christopher Baines Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf --- guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 3 ++- 5 files changed, 13 insertions(+), 8 deletions(-) diff --git a/guix/git-authenticate.scm b/guix/git-authenticate.scm index 37c69d0880..3cd1175c32 100644 --- a/guix/git-authenticate.scm +++ b/guix/git-authenticate.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2019, 2020, 2021, 2022 Ludovic Courtès +;;; Copyright © 2019-2022, 2025 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id) (chmod port #o600) (display ";; List of previously-authenticated commits.\n\n" port) - (pretty-print lst port)))))) + (pretty-print lst port))) + #:sync? #f))) ;;; diff --git a/guix/http-client.scm b/guix/http-client.scm index 9138a627ac..4e0cc59e91 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012-2018, 2020-2022 Ludovic Courtès +;;; Copyright © 2012-2018, 2020-2022, 2025 Ludovic Courtès ;;; Copyright © 2015 Mark H Weaver ;;; Copyright © 2012, 2015 Free Software Foundation, Inc. ;;; Copyright © 2017 Tobias Geerinckx-Rice @@ -346,7 +346,8 @@ (define* (http-fetch/cached uri #:key (ttl (%http-cache-ttl)) text? (when cache-port (close-port cache-port)) (with-atomic-file-output file - (cut write-cache port <>)) + (cut write-cache port <>) + #:sync? #f) (close-port port) (open-input-file file)))) diff --git a/guix/scripts/discover.scm b/guix/scripts/discover.scm index 32bf6085a5..cbe01638f5 100644 --- a/guix/scripts/discover.scm +++ b/guix/scripts/discover.scm @@ -87,7 +87,8 @@ (define* (write-publish-file #:key (file (%publish-file))) (format port "http://~a:~a~%" (avahi-service-address service) (avahi-service-port service))) - %publish-services))) + %publish-services)) + #:sync? #f) (chmod file #o644)) (define* (read-substitute-urls #:key (file (%publish-file))) diff --git a/guix/scripts/style.scm b/guix/scripts/style.scm index 4801529f7e..c45bdd4458 100644 --- a/guix/scripts/style.scm +++ b/guix/scripts/style.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021-2024 Ludovic Courtès +;;; Copyright © 2021-2025 Ludovic Courtès ;;; Copyright © 2024 Herman Rimm ;;; ;;; This file is part of GNU Guix. @@ -554,7 +554,8 @@ (define* (format-whole-file file order? #:rest rest) (apply pretty-print-with-comments/splice port lst #:format-comment canonicalize-comment #:format-vertical-space canonicalize-vertical-space - rest)))))) + rest)) + #:sync? #f)))) ;;; diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 45c165e03e..44d666f851 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -127,7 +127,8 @@ (define (cache-narinfo! cache-url path narinfo ttl) (mkdir-p (dirname file)) (with-atomic-file-output file (lambda (out) - (write (cache-entry cache-url narinfo) out)))) + (write (cache-entry cache-url narinfo) out)) + #:sync? #f)) narinfo) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 3/4] substitutes: Ignore corrupt cached narinfos. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 10 Apr 2025 10:32:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 77680@debbugs.gnu.org Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.17442810925654 (code B ref 77680); Thu, 10 Apr 2025 10:32:01 +0000 Received: (at 77680) by debbugs.gnu.org; 10 Apr 2025 10:31:32 +0000 Received: from localhost ([127.0.0.1]:44050 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2pBz-0001T8-Q8 for submit@debbugs.gnu.org; Thu, 10 Apr 2025 06:31:32 -0400 Received: from mira.cbaines.net ([212.71.252.8]:44170) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2pBx-0001Sv-3I for 77680@debbugs.gnu.org; Thu, 10 Apr 2025 06:31:29 -0400 Received: from localhost (unknown [IPv6:2a02:6b67:e390:8b00::1ce5]) by mira.cbaines.net (Postfix) with ESMTPSA id A305727BC49; Thu, 10 Apr 2025 11:31:27 +0100 (BST) Received: from fang (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id e23a94d3; Thu, 10 Apr 2025 10:31:27 +0000 (UTC) From: Christopher Baines In-Reply-To: <2774fbdd96e93995194d5f56bc3d328dc5905326.1744208418.git.ludo@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 9 Apr 2025 16:23:39 +0200") References: <2774fbdd96e93995194d5f56bc3d328dc5905326.1744208418.git.ludo@gnu.org> User-Agent: mu4e 1.12.9; emacs 29.4 Date: Thu, 10 Apr 2025 11:31:27 +0100 Message-ID: <87wmbsxq80.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > * guix/substitutes.scm (cached-narinfo): Wrap =E2=80=98read=E2=80=99 call= in > =E2=80=98false-if-exception=E2=80=99. Add catch-all =E2=80=98match=E2=80= =99 clause. > > Change-Id: I2d776f64b6521f778b4ab3f956b35cdef2ddb383 > --- > guix/substitutes.scm | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/guix/substitutes.scm b/guix/substitutes.scm > index 86b9f5472a..45c165e03e 100644 > --- a/guix/substitutes.scm > +++ b/guix/substitutes.scm > @@ -300,7 +300,7 @@ (define (cached-narinfo cache-url path) > (lambda () > (call-with-input-file cache-file > (lambda (p) > - (match (read p) > + (match (false-if-exception (read p)) > (('narinfo ('version 2) > ('cache-uri cache-uri) > ('date date) ('ttl ttl) ('value #f)) > @@ -315,7 +315,8 @@ (define (cached-narinfo cache-url path) > (if (obsolete? date now ttl) > (values #f #f) > (values #t (string->narinfo value cache-uri)))) > - (('narinfo ('version v) _ ...) > + (_ > + ;; Corrupt or incompatible cached entry. > (values #f #f)))))) > (lambda _ > (values #f #f)))) I believe the particular failure case we're introducing here is that the cache file is empty, so maybe we could match that specific case here and handle it silently. I'm a bit wary of handling any and all cache issues silently since that could make it harder to spot that the caching is entirely broken. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmf3nf9fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XdT+A/9Fxe2RI3U+pYXBgP9mPuuKzngaBTWZgpZ yuRgMYoQrIqu5pBbEE9ifKNWN+eZAGXEvBTelvLUZ4h8nIzkpc66Ri++i7VcUJNH RUEkwPJ5qY1VVzn3GfUnR0tMoQltKaTvxfLPK6TPrBi9sKHRo/z5s6nYOS8I3TSW w+Z+jH8jzzt+QhxjuXqoTM/96wJJipWcNKGj3tMgAlg8YSzoC2dMXqpyA+Gc8QRL UDxc8ykZvDMbs/A3+01kMFxQbKuZ06jFYk6b4ikynAomQeKCYTexgrHD2IOoH9SA laru2WiQEOdkBBuWxKdMD0DMjoDeV05hPiYTwJysmNvx8+K4G/XdM+0ecr5RCuCA 5BbAxO0wgGxAdWv3MN4l/uPZDokQoGhedHClby7OwYk5pDbgtKzM/JdxV9ATQpBE LO2aMSW/O2gR2ExpjXJy1v8EDc1g5lGR9/0pCh167TUp8gGYe3D+X55RNjkJwaSe NQlfIATgcOXlusPvOD8YTbz5pJNymY3JEo34GV0vhJRJ1jjglE4rITBgqagMD2uh mcGfpK69ufq7wrzvIso03bqsMB8qbx5wH+DygIi3y4Y38T9k7xkceidFfYT3vQ7I dQGOVDl5Cn5cbL1DBtzGYZZTxCvXcQr4eJXgMngu2tKRfVE2+aZyFB1vmaodCxW3 R2+5TFF5HGs= =/VWK -----END PGP SIGNATURE----- --=-=-=-- From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 4/4] guix: Avoid =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= call for caches and regular files. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 10 Apr 2025 10:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 77680@debbugs.gnu.org Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.17442812986078 (code B ref 77680); Thu, 10 Apr 2025 10:35:01 +0000 Received: (at 77680) by debbugs.gnu.org; 10 Apr 2025 10:34:58 +0000 Received: from localhost ([127.0.0.1]:44068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2pFK-0001Zy-2z for submit@debbugs.gnu.org; Thu, 10 Apr 2025 06:34:58 -0400 Received: from mira.cbaines.net ([212.71.252.8]:44172) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2pFI-0001Zo-BV for 77680@debbugs.gnu.org; Thu, 10 Apr 2025 06:34:56 -0400 Received: from localhost (unknown [IPv6:2a02:6b67:e390:8b00::1ce5]) by mira.cbaines.net (Postfix) with ESMTPSA id EFC8E27BC49; Thu, 10 Apr 2025 11:34:55 +0100 (BST) Received: from fang (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 39152921; Thu, 10 Apr 2025 10:34:55 +0000 (UTC) From: Christopher Baines In-Reply-To: <812224bace6510526d09f6cf661bc147ef157c92.1744208418.git.ludo@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 9 Apr 2025 16:23:40 +0200") References: <812224bace6510526d09f6cf661bc147ef157c92.1744208418.git.ludo@gnu.org> User-Agent: mu4e 1.12.9; emacs 29.4 Date: Thu, 10 Apr 2025 11:34:55 +0100 Message-ID: <87r020xq28.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Fixes . > > Calling =E2=80=98fdatasync=E2=80=99 for each and every narinfo file creat= ed by =E2=80=98guix > substitute=E2=80=99 proved to be too expensive on spinning HDDs and/or un= der > load (from 0.1s to 1.3s for the =E2=80=98fdatasync=E2=80=99 call alone). > > * guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f. > * guix/http-client.scm (http-fetch/cached): Likewise. > * guix/scripts/discover.scm (write-publish-file): Likewise. > * guix/scripts/style.scm (format-whole-file): Likewise. > * guix/substitutes.scm (cache-narinfo!): Likewise. > > Reported-by: Christopher Baines > Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf > --- > guix/git-authenticate.scm | 5 +++-- > guix/http-client.scm | 5 +++-- > guix/scripts/discover.scm | 3 ++- > guix/scripts/style.scm | 5 +++-- > guix/substitutes.scm | 3 ++- > 5 files changed, 13 insertions(+), 8 deletions(-) > > diff --git a/guix/git-authenticate.scm b/guix/git-authenticate.scm > index 37c69d0880..3cd1175c32 100644 > --- a/guix/git-authenticate.scm > +++ b/guix/git-authenticate.scm > @@ -1,5 +1,5 @@ > ;;; GNU Guix --- Functional package management for GNU > -;;; Copyright =C2=A9 2019, 2020, 2021, 2022 Ludovic Court=C3=A8s > +;;; Copyright =C2=A9 2019-2022, 2025 Ludovic Court=C3=A8s > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id) > (chmod port #o600) > (display ";; List of previously-authenticated commits.\n\n" > port) > - (pretty-print lst port)))))) > + (pretty-print lst port))) > + #:sync? #f))) This introduces (or maybe increases) the risk that this file will be empty, does this happen often enough to justify the change? --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmf3ns9fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xcgdw//UzMWoOSXtwE93Dm8TA2m5ddTgihmrQp0 huFVYi77LY2i76H2L/4ETsNMfGKpBKr0IZ73qXR+Gpu00YuxCneXE3QK93LlTqTw t+iri0m2uRQiji8FCeMVIzs81a6Tz5/lQ0poiiS2sp6kClOlsIajJCFGYkYQv80f OLqwsypoRp6YZ4wWwKssSOYNu35mDZsYM31lHGqrPWF3mIop1X8dyuYY0qRknW3h f6nd5DMKDavEig49Yhp0CbuToiL6yxwK3q87ZBzOlutAkIvdKWVO5qfXg27+akLZ 0D4fQ2f2LOVDgYs75IJhZr8EfbNWxlLScIECzku2bZ2tl1VACm4n/semlwkeeT2B DRrp/zsC2+KVnO4IFggwpJVFrxyt4sD8O2ewGnztQHzufSNCsxlSyrUAYOoLvd0s yXea4M84k1t8967b/afkZ7tHjbdYtuIM1QAjwKnlqm/Dq3a4dUZXGfugVYoOS8mf RPNHuI0OhFQKyArC5yKjJ+DFuhdhO3Ni9Zo6JZEnhim8DCmnKOuAe0dZt+vB+Z9w a7nfdS5edFzO2EXr+ZI21tCV973Ks482D9vPBamr0m94KVIlug+TlQnZvOwXa5Yh IdlKssG+ba1L1Q4IztR50oO0wroxzyc597l04KP8Pew3gbIIvxt40sMp8Fa5unA7 yN6//30truY= =1fj1 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH 4/4] guix: Avoid =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= call for caches and regular files. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 10 Apr 2025 12:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 77680@debbugs.gnu.org Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.17442885478151 (code B ref 77680); Thu, 10 Apr 2025 12:36:02 +0000 Received: (at 77680) by debbugs.gnu.org; 10 Apr 2025 12:35:47 +0000 Received: from localhost ([127.0.0.1]:44273 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2r8F-00027O-3k for submit@debbugs.gnu.org; Thu, 10 Apr 2025 08:35:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50246) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2r8C-000273-UF for 77680@debbugs.gnu.org; Thu, 10 Apr 2025 08:35:45 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2r87-0001tg-Cg; Thu, 10 Apr 2025 08:35:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=shpPmwGnwxqyOH2Ci986UYCuVw+30FGNq48pQdmhLho=; b=AdDL+isNIvGbRtD/GOw8 i8I0xugGhkdpmGqD8IkHWqBqe3BYWeS0/dPmzcQoACDmkFUicKnrprnbfJGGdfO4lrS0vq6KZCeE9 4PpBhvRRwZ3Yz7TCNJAFzpgEuxNMgNAaTcyUdCw7oAys1Dxsh5svAhWZk3uGb0Zt1SiJxg8wfwFPJ dPPs1iAGBPM6F5/fO26GIIQSeAGJpS4J02EJ6eZ+fe6Gg1iCS5jtR3WV3a96gROcnOp7R4QGmWw0k 8qOj1BbdVimnafNYPdrZ1vPKp1POtZp0B7Yr+h/c8E5U+vxKsWwcZz/YR5Qs5kKo1hKQ1rYFCNKOm jRWx2Zfc5IXjDw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <87r020xq28.fsf@cbaines.net> (Christopher Baines's message of "Thu, 10 Apr 2025 11:34:55 +0100") References: <812224bace6510526d09f6cf661bc147ef157c92.1744208418.git.ludo@gnu.org> <87r020xq28.fsf@cbaines.net> User-Agent: mu4e 1.12.9; emacs 29.4 X-URL: https://people.bordeaux.inria.fr/lcourtes/ X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu X-Revolutionary-Date: Primidi 21 Germinal an 233 de la =?UTF-8?Q?R=C3=A9volution,?= jour du Gainier Date: Thu, 10 Apr 2025 13:08:02 +0200 Message-ID: <87mscotgtp.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: >> @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id) >> (chmod port #o600) >> (display ";; List of previously-authenticated commits.\n\n" >> port) >> - (pretty-print lst port)))))) >> + (pretty-print lst port))) >> + #:sync? #f))) > > This introduces (or maybe increases) the risk that this file will be > empty, does this happen often enough to justify the change? My thinking is that it=E2=80=99s just a cache, so it shouldn=E2=80=99t need= special treatment as if it were a precious file (which is what the =E2=80=98fdatasy= nc=E2=80=99 call was initially added for); it=E2=80=99s OK if we lose this file. But it=E2=80=99s rarely modified so from that perspective keeping the =E2=80=98fdatasync=E2=80=99 call cannot hurt. Ludo=E2=80=99. From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH v2 0/4] Avoid 'fdatasync' calls for cache files References: In-Reply-To: Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 11 Apr 2025 13:56:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174437974329856 (code B ref 77680); Fri, 11 Apr 2025 13:56:02 +0000 Received: (at 77680) by debbugs.gnu.org; 11 Apr 2025 13:55:43 +0000 Received: from localhost ([127.0.0.1]:49884 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3Er6-0007lD-Jz for submit@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57566) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3Er3-0007jG-Kx for 77680@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3Eqx-0003nm-LE; Fri, 11 Apr 2025 09:55:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=yTGXpjEQYAhhi8MXZa0kDUhpWZkv1rZbB11oQIUVvtk=; b=jJqZd9fCjEJYkv y/E8cEttcuRv1E+h4WisuqieUK9k9jLXb5X5vCMjvzSKDw+XNtPYzuagmcY+tty88D61RRq5P1RXt t9rVEx1j5lTOlo/21HbzsfF3bZdwi399QXM8f2yfrST7xArkuub+bwkr0+CppeaIM8O7S1w1damV8 HVERoo1p/zj0479Ljkv0NPVFc6hwkqbWqn72H4z0JY4geO4SlNheJzdn8DZvHTu9oUBiC8yRuxYT4 1wAcpHtCzQZnKTqDi1NmVqxTCP2MU0oC+W6LaltZ1lSoAI08zCXD8Z1eksxiQ475v7YUf+9XAeSSO V5CuyvA5UCCTHVqlrTgg==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Apr 2025 15:54:56 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Changes since v1: • Patch #3 only tolerates empty narinfo files instead of doing (false-if-exception (read p)), as suggested by Chris. Ludo'. Ludovic Courtès (4): cache: Remove unused import. utils: Add #:sync? parameter to ‘with-atomic-file-output’. substitutes: Ignore corrupt cached narinfos. guix: Avoid ‘fdatasync’ call for caches and regular files. guix/cache.scm | 1 - guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 5 ++++- guix/utils.scm | 14 ++++++++++---- 7 files changed, 25 insertions(+), 13 deletions(-) base-commit: 772b70455d0d5972fdad80d8529647dce20f409a -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH v2 1/4] cache: Remove unused import. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 11 Apr 2025 13:56:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174437974429876 (code B ref 77680); Fri, 11 Apr 2025 13:56:03 +0000 Received: (at 77680) by debbugs.gnu.org; 11 Apr 2025 13:55:44 +0000 Received: from localhost ([127.0.0.1]:49893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3Er8-0007lT-Nl for submit@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57572) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3Er4-0007jN-H1 for 77680@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3Eqz-0003o0-35; Fri, 11 Apr 2025 09:55:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=Q/Ab67I5C3aE19soHcWl7rJqJU9p3eqirmYrVfEQqVM=; b=IlijlwMP9Ne3KRtaRe9v qEBrsEzgfPHkTwCmSTcdBdh3gtYOSZnpJrXiLz8QeTz6ekMO2K2rjHgrtEIzeePR9FLO4bZrHdosG u6rYLxtsqcbA7oqkpim3zQwMEF3ZsklBFrukMu9t/X6YGDQwjgBqzfc9HB92PMruELpN8vhYeZA+7 zkwv0kDwaFMmhUaaqXO986KeZ6zSObZ0eC3E2f4X5p2MFa3pUQCbP2LYJsKN7fT116L6oeZ57DW8k DxLnrCKDJ7e24gpJhZBI9NC334JAzRhkfdDJLC4WbKoWjoTkreK/5VokPztlu8YuEDCl0Os3iTWPV vwVX/Ul56xVxOw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Apr 2025 15:54:57 +0200 Message-ID: <103687fb70792c8e2971614c4a2d1546abad9901.1744379552.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/cache.scm: Remove unused import. Change-Id: I06956534aa5342a084ebf71b61518c515ce9288c --- guix/cache.scm | 1 - 1 file changed, 1 deletion(-) diff --git a/guix/cache.scm b/guix/cache.scm index 8b12312c77..5d8a0edbaa 100644 --- a/guix/cache.scm +++ b/guix/cache.scm @@ -18,7 +18,6 @@ ;;; along with GNU Guix. If not, see . (define-module (guix cache) - #:use-module ((guix utils) #:select (with-atomic-file-output)) #:autoload (guix build syscalls) (lock-file unlock-file) #:use-module (srfi srfi-19) #:use-module (srfi srfi-26) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH v2 2/4] utils: Add #:sync? parameter to =?UTF-8?Q?=E2=80=98with-atomic-file-output=E2=80=99.?= Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 11 Apr 2025 13:56:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174437975329960 (code B ref 77680); Fri, 11 Apr 2025 13:56:04 +0000 Received: (at 77680) by debbugs.gnu.org; 11 Apr 2025 13:55:53 +0000 Received: from localhost ([127.0.0.1]:49929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3ErI-0007mz-5D for submit@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57574) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3Er5-0007jW-Fg for 77680@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3Er0-0003o9-6J; Fri, 11 Apr 2025 09:55:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=aGeG7j1Bmh1K9z0mhsJ66S4LuOy7cbMuuYtHZrDZ4qI=; b=p0PC9Z5oOt+d52HC19OT HJBFWnnHLPuBmDPhi11R3YKAbClbyCxjOx5Vq7vBuOWh1DnQEX2eXXOL0ZK+LdVBJQCxHFilvIU/y pHlX8JCF6/5aCv1l90U2mvP5vmi9qN4a64K1e3Aqz46bH3E0il28S2O5t0A276hx8gAerahYEKBa+ N3r3oDQhu0YiJLFNMOYz6pxs/p5uSDZF/w3iDeuxeLp5hmPIg3fCbanknRT9SdGeEtuDttODXEqH3 q6cMFRrE6ZxayLPIDDfelIB0FglFEKddFP4rJhctgRymNafoFFDxLa46k9kAUpFduwNOwvL7vqJO9 WdI4XqBInAmqcw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Apr 2025 15:54:58 +0200 Message-ID: <6ad42a95eed510eecf1f9a264c24714ebe9ff2d4.1744379552.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/utils.scm (with-atomic-file-output): Add #:sync? and honor it. Add ‘force-output’ call. Change-Id: I2479778ae55360c0fab3389ac9249045a27b3568 --- guix/utils.scm | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/guix/utils.scm b/guix/utils.scm index c7c23d9d5b..7ae98096c2 100644 --- a/guix/utils.scm +++ b/guix/utils.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012-2022, 2024 Ludovic Courtès +;;; Copyright © 2012-2022, 2024-2025 Ludovic Courtès ;;; Copyright © 2013, 2014, 2015 Mark H Weaver ;;; Copyright © 2014 Eric Bavier ;;; Copyright © 2014 Ian Denhardt @@ -1057,16 +1057,22 @@ (define (call-with-temporary-directory proc) (lambda () (false-if-exception (delete-file-recursively tmp-dir)))))) -(define (with-atomic-file-output file proc) +(define* (with-atomic-file-output file proc #:key (sync? #t)) "Call PROC with an output port for the file that is going to replace FILE. Upon success, FILE is atomically replaced by what has been written to the -output port, and PROC's result is returned." +output port, and PROC's result is returned. + +When SYNC? is true, call 'fdatasync' on the temporary file before renaming it +to FILE; set it to #false for caches and temporary files to improve +performance." (let* ((template (string-append file ".XXXXXX")) (out (mkstemp! template))) (with-throw-handler #t (lambda () (let ((result (proc out))) - (fdatasync out) + (when sync? + (force-output out) + (fdatasync out)) (close-port out) (rename-file template file) result)) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH v2 3/4] substitutes: Ignore corrupt cached narinfos. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 11 Apr 2025 13:56:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174437975429970 (code B ref 77680); Fri, 11 Apr 2025 13:56:05 +0000 Received: (at 77680) by debbugs.gnu.org; 11 Apr 2025 13:55:54 +0000 Received: from localhost ([127.0.0.1]:49936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3ErJ-0007n9-4W for submit@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57582) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3Er6-0007kF-Md for 77680@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3Er1-0003oH-BX; Fri, 11 Apr 2025 09:55:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=Ne1HOXPWHatNbg07D1V8su4FxxAGuHIu/62yGrBWMw4=; b=mFD7xSC9MHv2E+gNeflC qJyIm3GXqqHDAmdqfGUCurFfHDPBStqcpg3GPgpkE8lLBct/6TqlLMQFVirC84yRQgRH27GuLs6y8 7wI/GNuK/o41hgTJHa1FZoSlwjK/Ha/gFaiaaY++tsFqP0uLxvivwrRTFf//H1nSp9mZ0Y9x1b+H2 p81FYFqGR06eZnHj6r8s5zH48s1n2/gUAWnxLypZ/e1MXcL5XsD+QWSmZl1+UIEjCPZYKpJWZqxU9 5e5CvQnn3t06v7EwRDOWzDJ5Tkzs4oOUniUeD0aFz+Vx3AGd4i8tNg7um0ZZjS7p8XyG6HfutXiXJ DxY7Hk9Qvi0Iag==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Apr 2025 15:54:59 +0200 Message-ID: <0e7a18395128304038c4ac654033321f3e32950e.1744379552.git.ludo@gnu.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/substitutes.scm (cached-narinfo): Add ‘eof-object?’ clause. Change-Id: I2d776f64b6521f778b4ab3f956b35cdef2ddb383 --- guix/substitutes.scm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 86b9f5472a..9edce5b2f8 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -316,6 +316,8 @@ (define (cached-narinfo cache-url path) (values #f #f) (values #t (string->narinfo value cache-uri)))) (('narinfo ('version v) _ ...) + (values #f #f)) + ((? eof-object?) ;corrupt file (values #f #f)))))) (lambda _ (values #f #f)))) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#77680] [PATCH v2 4/4] guix: Avoid =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= call for caches and regular files. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 11 Apr 2025 13:56:07 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77680 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77680@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 77680-submit@debbugs.gnu.org id=B77680.174437975529989 (code B ref 77680); Fri, 11 Apr 2025 13:56:07 +0000 Received: (at 77680) by debbugs.gnu.org; 11 Apr 2025 13:55:55 +0000 Received: from localhost ([127.0.0.1]:49941 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3ErJ-0007nL-Qs for submit@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57590) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3Er8-0007kT-97 for 77680@debbugs.gnu.org; Fri, 11 Apr 2025 09:55:44 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3Er2-0003oU-88; Fri, 11 Apr 2025 09:55:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ZNbXfyv6zSSXmyetNqAg/zXk2AZKSUrEP/ACqQlCS04=; b=aj5B2M3fhQQFyrH1+mNM gH5mxjvZqbsisKVEkj8EyCyFRj0CfWMFX0IzfUCVDLnfuwyad0fbB+YQ/wW+v3RtFU4RoCSYe4PdJ cA3ibq+mLqGHlVB+K7ByuPgWjjeFMoJlPHqkvgRW9a8YAej7y+/TdRwNRx+BUZb3VZhfyXN34Qdde x22D04A1cT5FeLi8sLtw46dgX8dVNP4CTcCLA/ttQB3Mq16uFKji29/sWRJcbcHsfIUVcPjA78t3W 4OAE0IgEip2zCnxnsWaLFzOhP9hLUW8FYs8+5vJatRC90Qvlauryg3cNY4cwxhqP9iG6p2ZWCat/g HftUQVAMG+QSzg==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Apr 2025 15:55:00 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Fixes . Calling ‘fdatasync’ for each and every narinfo file created by ‘guix substitute’ proved to be too expensive on spinning HDDs and/or under load (from 0.1s to 1.3s for the ‘fdatasync’ call alone). * guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f. * guix/http-client.scm (http-fetch/cached): Likewise. * guix/scripts/discover.scm (write-publish-file): Likewise. * guix/scripts/style.scm (format-whole-file): Likewise. * guix/substitutes.scm (cache-narinfo!): Likewise. Reported-by: Christopher Baines Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf --- guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 3 ++- 5 files changed, 13 insertions(+), 8 deletions(-) diff --git a/guix/git-authenticate.scm b/guix/git-authenticate.scm index 37c69d0880..3cd1175c32 100644 --- a/guix/git-authenticate.scm +++ b/guix/git-authenticate.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2019, 2020, 2021, 2022 Ludovic Courtès +;;; Copyright © 2019-2022, 2025 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id) (chmod port #o600) (display ";; List of previously-authenticated commits.\n\n" port) - (pretty-print lst port)))))) + (pretty-print lst port))) + #:sync? #f))) ;;; diff --git a/guix/http-client.scm b/guix/http-client.scm index 9138a627ac..4e0cc59e91 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012-2018, 2020-2022 Ludovic Courtès +;;; Copyright © 2012-2018, 2020-2022, 2025 Ludovic Courtès ;;; Copyright © 2015 Mark H Weaver ;;; Copyright © 2012, 2015 Free Software Foundation, Inc. ;;; Copyright © 2017 Tobias Geerinckx-Rice @@ -346,7 +346,8 @@ (define* (http-fetch/cached uri #:key (ttl (%http-cache-ttl)) text? (when cache-port (close-port cache-port)) (with-atomic-file-output file - (cut write-cache port <>)) + (cut write-cache port <>) + #:sync? #f) (close-port port) (open-input-file file)))) diff --git a/guix/scripts/discover.scm b/guix/scripts/discover.scm index 32bf6085a5..cbe01638f5 100644 --- a/guix/scripts/discover.scm +++ b/guix/scripts/discover.scm @@ -87,7 +87,8 @@ (define* (write-publish-file #:key (file (%publish-file))) (format port "http://~a:~a~%" (avahi-service-address service) (avahi-service-port service))) - %publish-services))) + %publish-services)) + #:sync? #f) (chmod file #o644)) (define* (read-substitute-urls #:key (file (%publish-file))) diff --git a/guix/scripts/style.scm b/guix/scripts/style.scm index 4801529f7e..c45bdd4458 100644 --- a/guix/scripts/style.scm +++ b/guix/scripts/style.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021-2024 Ludovic Courtès +;;; Copyright © 2021-2025 Ludovic Courtès ;;; Copyright © 2024 Herman Rimm ;;; ;;; This file is part of GNU Guix. @@ -554,7 +554,8 @@ (define* (format-whole-file file order? #:rest rest) (apply pretty-print-with-comments/splice port lst #:format-comment canonicalize-comment #:format-vertical-space canonicalize-vertical-space - rest)))))) + rest)) + #:sync? #f)))) ;;; diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 9edce5b2f8..24b7873ce2 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -127,7 +127,8 @@ (define (cache-narinfo! cache-url path narinfo ttl) (mkdir-p (dirname file)) (with-atomic-file-output file (lambda (out) - (write (cache-entry cache-url narinfo) out)))) + (write (cache-entry cache-url narinfo) out)) + #:sync? #f)) narinfo) -- 2.49.0 From unknown Sat Aug 16 11:40:59 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Subject: bug#77680: closed (Re: [bug#77680] [PATCH v2 0/4] Avoid 'fdatasync' calls for cache files) Message-ID: References: <87semalad9.fsf@gnu.org> X-Gnu-PR-Message: they-closed 77680 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 77680@debbugs.gnu.org Date: Mon, 14 Apr 2025 21:09:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1744664942-7689-1" This is a multi-part message in MIME format... ------------=_1744664942-7689-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #77680: [PATCH 0/4] Avoid 'fdatasync' calls for cache files which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 77680@debbugs.gnu.org. --=20 77680: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D77680 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1744664942-7689-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 77680-done) by debbugs.gnu.org; 14 Apr 2025 21:08:32 +0000 Received: from localhost ([127.0.0.1]:49287 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u4R2d-0001z5-JY for submit@debbugs.gnu.org; Mon, 14 Apr 2025 17:08:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55678) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u4R2a-0001yp-Hi for 77680-done@debbugs.gnu.org; Mon, 14 Apr 2025 17:08:28 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4R2U-0005ge-6F; Mon, 14 Apr 2025 17:08:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=XNzLWjQihcZ2eJQ8j6o8ia365fvNNn6B+WJBO7e2khA=; b=MtqiiA3ToL/F7ySyD44b wBawUvl+sP9AByOj7oYWaHCqItJIh/UqLSw2ciIQ1gRoRcBOp/cCfjSdtbV0pshN/OZbtobkSnnBp nZFjK0H7uaPu/OImUFkmx5kR91x9w8yVsbvuoLrgJOApgi85kN5eOYXZpMVTmnlOgCGRjk3pI/YMm DvdvfGucS3/3OI6InJ8f6PiBxbw7+37uF/appvyktqG+2SKM+eNxiIirj54SqdGO5iMactVgArfpI g47WpdMUy0GzQGp8D1Ty7xFAefl3SpClBgpGtvuvPnvtGsVaKVSRwgRi9OQBCtX74T+T1CVruhLxf qUZc9ec13wf+aQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 77680-done@debbugs.gnu.org Subject: Re: [bug#77680] [PATCH v2 0/4] Avoid 'fdatasync' calls for cache files In-Reply-To: ("Ludovic =?utf-8?Q?Court?= =?utf-8?Q?=C3=A8s=22's?= message of "Fri, 11 Apr 2025 15:54:56 +0200") References: Date: Mon, 14 Apr 2025 23:08:18 +0200 Message-ID: <87semalad9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 77680-done Cc: Tobias Geerinckx-Rice , Christopher Baines , Josselin Poiret , Simon Tournier , Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s writes: > Changes since v1: > > =E2=80=A2 Patch #3 only tolerates empty narinfo files instead of > doing (false-if-exception (read p)), as suggested by Chris. > > Ludo'. > > Ludovic Court=C3=A8s (4): > cache: Remove unused import. > utils: Add #:sync? parameter to =E2=80=98with-atomic-file-output=E2=80= =99. > substitutes: Ignore corrupt cached narinfos. > guix: Avoid =E2=80=98fdatasync=E2=80=99 call for caches and regular fil= es. Pushed as dfac0a5a9526805334bd7f72926b77b5d46f59db, following our discussion on #guix today. ------------=_1744664942-7689-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 9 Apr 2025 14:22:03 +0000 Received: from localhost ([127.0.0.1]:40893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2WJW-0007O3-KU for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:22:03 -0400 Received: from lists.gnu.org ([2001:470:142::17]:46164) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2WJT-0007NU-NI for submit@debbugs.gnu.org; Wed, 09 Apr 2025 10:22:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WJO-0002w8-8b for guix-patches@gnu.org; Wed, 09 Apr 2025 10:21:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u2WJM-0002dN-Lg; Wed, 09 Apr 2025 10:21:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=oirkk7njtJ8auGsVs5bco3S8SASX5ZLFC1nVurNaJq4=; b=naB2GWCXE/1AnL NWeN91rYToYeZNRu3y669t+xz4KdzjTdPtzdtJ7XJz5fMvkasUwtok6ivVkeKjE1F9NmDW4dMwPFv 6WNi7KsvqGr99GflYigXGiBJmEI0iUI1RIzam2eyn/FITzOke6ZB9JKtMx8iN7TlVPI/8rJRjKrB5 l+GALr5Si9w+WP5JDEbFM8mOY3vnghrBULlJ9hqvYjrm8kdkfRIU+W9Evo2Ra/UK3WmBZOnTTJoRy udcw4+rsaCytf8gH7pMeAGSXlpuhL+lOMMD303yOM5sqoVV0b9JYKKAnO+0hH32pM/bxLegRPZF0x lhSpOp3Ty24JXr1RsA3g==; From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/4] Avoid 'fdatasync' calls for cache files Date: Wed, 9 Apr 2025 16:21:27 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Debbugs-Cc: Christopher Baines , Josselin Poiret , Ludovic Courtès , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, As Chris found out, calling ‘fdatasync’ for each and every narinfo cache file created by ‘guix substitute’ (and likewise for other caches) is too expensive and pointless. This patch series fixes that. Thoughts? Ludo’. Ludovic Courtès (4): cache: Remove unused import. utils: Add #:sync? parameter to ‘with-atomic-file-output’. substitutes: Ignore corrupt cached narinfos. guix: Avoid ‘fdatasync’ call for caches and regular files. guix/cache.scm | 1 - guix/git-authenticate.scm | 5 +++-- guix/http-client.scm | 5 +++-- guix/scripts/discover.scm | 3 ++- guix/scripts/style.scm | 5 +++-- guix/substitutes.scm | 8 +++++--- guix/utils.scm | 14 ++++++++++---- 7 files changed, 26 insertions(+), 15 deletions(-) base-commit: c88f98bb3ca2616baea6b1e452cc900cc9c87503 -- 2.49.0 ------------=_1744664942-7689-1--