From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 26 12:33:14 2019 Received: (at submit) by debbugs.gnu.org; 26 Dec 2019 17:33:14 +0000 Received: from localhost ([127.0.0.1]:56180 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikX0b-0006gd-S9 for submit@debbugs.gnu.org; Thu, 26 Dec 2019 12:33:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:49205) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikX0Z-0006gV-ST for submit@debbugs.gnu.org; Thu, 26 Dec 2019 12:33:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53649) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ikX0Y-0000pN-QG for guix-patches@gnu.org; Thu, 26 Dec 2019 12:33:11 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ikX0X-00018Y-Ob for guix-patches@gnu.org; Thu, 26 Dec 2019 12:33:10 -0500 Received: from mira.cbaines.net ([212.71.252.8]:55184) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ikX0X-00018N-J2 for guix-patches@gnu.org; Thu, 26 Dec 2019 12:33:09 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id 5DFC517995 for ; Thu, 26 Dec 2019 17:33:08 +0000 (GMT) Received: from phact (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 47db1fab for ; Thu, 26 Dec 2019 17:33:08 +0000 (UTC) User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: guix-patches@gnu.org Subject: [PATCH 0/2] Speed up the derivation linter. Date: Thu, 26 Dec 2019 17:33:07 +0000 Message-ID: <8736d7rnzg.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 212.71.252.8 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --=-=-= Content-Type: text/plain These patches speed up the derivation linter by reducing the number of times a connection to the guix-daemon is established. In testing on my computer, the time to lint all guix packages with the derivation linter drops from around 6 minutes to around 3 minutes. Christopher Baines (2): guix: lint: Add an optional parameter for a store connection. scripts: lint: Set the %link-checker-store-connection parameter. guix/lint.scm | 42 ++++++++++++++++++++++++++---------------- guix/scripts/lint.scm | 22 +++++++++++++--------- 2 files changed, 39 insertions(+), 25 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl4E7tNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xfy4RAAueFqD//gxRswuA8ghEF+ejdpH0L83kWS2cHDqhz86CgWA31+15NmweJB 1DQbBvs8c/vABpCIoLTnlGb+4K61XrFeP/YWNujBVgV+sWbw8Eg681ET1uAwwIAL GJEkkuM9jGt6EkO6TB8EG5eiYwCpGFCPNSCkOljDkXmJM9ah49WuX2THVTFeFkqz hcV86ElvFiVHL8TPMH/DEckGkFwLWMZNBkCX5/h8fD/yv+hqqMOMMDnIJKM3E4Pg HR/phgmS7puRhSQWm/BoVjH3T5fc1gS+WUV/kTZd6dAEMzQvy2WGdIXKRd5YsREg 49draeO1J2FSDfAukt0rHGEt3JeJ9xhxdUP52bG48qudz/UGWjDeGpkMKcBs7kM0 V2A6YFzcmDhww2aE0zBJT8+cY+B+xrX9RjlF4+jMN+BWJHLUE61lP/zTwjjGLXzg Rf9c9knTo1Lay4xe2Eyl6gmQxpM77U2U1oyyYboN+hGJzM7mIpXt/12UyXJJf0Cj DMH17EIWcPIGQiijPzmmG3LUq2rSG51iL+VMwxHesLsw8jR9gCQDOz2+fIFANxru cegH3iQbB8SDtsSCyy/zQC4Vhc5VBY5IL/B4uLBDGSmoT4Rxc8kWPXAT+9PoVcHq svuNl6STjW6FHSt+0M9zg64oWbQWRYUwLC/iAGMdQa/wGqZ33Xg= =XnOR -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 26 13:01:09 2019 Received: (at 38754) by debbugs.gnu.org; 26 Dec 2019 18:01:09 +0000 Received: from localhost ([127.0.0.1]:56213 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRd-0007fF-40 for submit@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:09 -0500 Received: from mira.cbaines.net ([212.71.252.8]:45644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRZ-0007f0-Uv for 38754@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:07 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id DE2F617928 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id c15c81ca for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 1/2] guix: lint: Add an optional parameter for a store connection. Date: Thu, 26 Dec 2019 18:01:03 +0000 Message-Id: <20191226180104.10888-1-mail@cbaines.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <8736d7rnzg.fsf@cbaines.net> References: <8736d7rnzg.fsf@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Previously, the derivation lint checker establishes a connection to the store for each supported system of each package. This change uses the same store connection for all supported systems, with the option of setting a parameter for a store connection which will be used instead of establishing a new connection. Previously, running the derivation linter for all packages would take around 6 and a half minutes, with this change, without setting the %lint-checker-store-connection parameter, the time is reduced to around 4 minutes. * guix/lint.scm (%lint-checker-store-connection): New parameter. (check-derivation): Arrange the code so that it's possible to either run with the store from the new parameter, or open a new connection via the with-store syntax. --- guix/lint.scm | 42 ++++++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/guix/lint.scm b/guix/lint.scm index cd2ea571ed..19498db857 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -100,7 +100,9 @@ lint-checker? lint-checker-name lint-checker-description - lint-checker-check)) + lint-checker-check + + %lint-checker-store-connection)) ;;; @@ -142,6 +144,9 @@ ((_ package (G_ message) rest ...) (%make-warning package message rest ...)))) +(define %lint-checker-store-connection + (make-parameter #f)) + ;;; ;;; Checkers @@ -887,7 +892,7 @@ descriptions maintained upstream." (define (check-derivation package) "Emit a warning if we fail to compile PACKAGE to a derivation." - (define (try system) + (define (try store system) (catch #t (lambda () (guard (c ((store-protocol-error? c) @@ -900,25 +905,30 @@ descriptions maintained upstream." (G_ "failed to create ~a derivation: ~a") (list system (condition-message c))))) - (with-store store - ;; Disable grafts since it can entail rebuilds. - (parameterize ((%graft? #f)) - (package-derivation store package system #:graft? #f) - - ;; If there's a replacement, make sure we can compute its - ;; derivation. - (match (package-replacement package) - (#f #t) - (replacement - (package-derivation store replacement system - #:graft? #f))))))) + ;; Disable grafts since it can entail rebuilds. + (parameterize ((%graft? #f)) + (package-derivation store package system #:graft? #f) + + ;; If there's a replacement, make sure we can compute its + ;; derivation. + (match (package-replacement package) + (#f #t) + (replacement + (package-derivation store replacement system + #:graft? #f)))))) (lambda args (make-warning package (G_ "failed to create ~a derivation: ~s") (list system args))))) - (filter lint-warning? - (map try (package-supported-systems package)))) + (define (check-with-store store) + (filter lint-warning? + (map (cut try store <>) (package-supported-systems package)))) + + (or (and=> (%lint-checker-store-connection) + check-with-store) + (with-store store + (check-with-store store)))) (define (check-license package) "Warn about type errors of the 'license' field of PACKAGE." -- 2.24.1 From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 26 13:01:11 2019 Received: (at 38754) by debbugs.gnu.org; 26 Dec 2019 18:01:11 +0000 Received: from localhost ([127.0.0.1]:56215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRf-0007fU-Cx for submit@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:11 -0500 Received: from mira.cbaines.net ([212.71.252.8]:45646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRZ-0007f1-Tx for 38754@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:09 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id 021F717992 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 5f0a2294 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. Date: Thu, 26 Dec 2019 18:01:04 +0000 Message-Id: <20191226180104.10888-2-mail@cbaines.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191226180104.10888-1-mail@cbaines.net> References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) If set, this parameter provides a store connection used by the derivation linter. Without this being set, the derivation linter establishes a new connection for each package. With this change, I saw the time taken to lint all packages with the derivation linter drop from over 4 minutes to around 3 minutes. * guix/scripts/lint.scm (guix-lint): Set the %lint-checker-store-connection) parameter. --- guix/scripts/lint.scm | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm index 8d08c484f5..47c104217d 100644 --- a/guix/scripts/lint.scm +++ b/guix/scripts/lint.scm @@ -28,6 +28,7 @@ (define-module (guix scripts lint) #:use-module (guix packages) + #:use-module (guix store) #:use-module (guix lint) #:use-module (guix ui) #:use-module (guix scripts) @@ -167,12 +168,15 @@ run the checkers on all packages.\n")) (_ #f)) (reverse opts))) (checkers (or (assoc-ref opts 'checkers) %all-checkers))) - (cond - ((assoc-ref opts 'list?) - (list-checkers-and-exit checkers)) - ((null? args) - (fold-packages (lambda (p r) (run-checkers p checkers)) '())) - (else - (for-each (lambda (spec) - (run-checkers (specification->package spec) checkers)) - args))))) + (with-store store + (parameterize + ((%lint-checker-store-connection store)) + (cond + ((assoc-ref opts 'list?) + (list-checkers-and-exit checkers)) + ((null? args) + (fold-packages (lambda (p r) (run-checkers p checkers)) '())) + (else + (for-each (lambda (spec) + (run-checkers (specification->package spec) checkers)) + args))))))) -- 2.24.1 From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 17:01:33 2019 Received: (at 38754) by debbugs.gnu.org; 30 Dec 2019 22:01:33 +0000 Received: from localhost ([127.0.0.1]:33569 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im36T-0003ph-2x for submit@debbugs.gnu.org; Mon, 30 Dec 2019 17:01:33 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im36Q-0003pU-Kx for 38754@debbugs.gnu.org; Mon, 30 Dec 2019 17:01:31 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41976) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1im36L-0005cU-9g; Mon, 30 Dec 2019 17:01:25 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44554 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1im36I-0005aE-VK; Mon, 30 Dec 2019 17:01:24 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#38754] [PATCH 0/2] Speed up the derivation linter. References: <8736d7rnzg.fsf@cbaines.net> Date: Mon, 30 Dec 2019 23:01:21 +0100 In-Reply-To: <8736d7rnzg.fsf@cbaines.net> (Christopher Baines's message of "Thu, 26 Dec 2019 17:33:07 +0000") Message-ID: <8736d11nim.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Christopher Baines skribis: > These patches speed up the derivation linter by reducing the number of > times a connection to the guix-daemon is established. In testing on my > computer, the time to lint all guix packages with the derivation linter > drops from around 6 minutes to around 3 minutes. Neat. > Christopher Baines (2): > guix: lint: Add an optional parameter for a store connection. > scripts: lint: Set the %link-checker-store-connection parameter. LGTM, thanks! :-) Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 17:12:54 2019 Received: (at 38754) by debbugs.gnu.org; 30 Dec 2019 22:12:55 +0000 Received: from localhost ([127.0.0.1]:33589 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im3HS-000463-Lf for submit@debbugs.gnu.org; Mon, 30 Dec 2019 17:12:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:57927) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im3HR-00045q-Sr for 38754@debbugs.gnu.org; Mon, 30 Dec 2019 17:12:54 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42149) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1im3HL-0007Vv-Ko; Mon, 30 Dec 2019 17:12:47 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44600 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1im3HK-0006c3-W7; Mon, 30 Dec 2019 17:12:47 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> Date: Mon, 30 Dec 2019 23:12:45 +0100 In-Reply-To: <20191226180104.10888-2-mail@cbaines.net> (Christopher Baines's message of "Thu, 26 Dec 2019 18:01:04 +0000") Message-ID: <87lfqtzcma.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: > + (with-store store > + (parameterize > + ((%lint-checker-store-connection store)) Actually it means that now =E2=80=98guix lint=E2=80=99 systematically conne= cts to the daemon. I wonder if we could arrange to open the connection lazily, and to somehow carry state across linter invocations. Perhaps =E2=80=98check-derivation=E2=80=99 should be monadic, with a field in indicating that. Sounds complicated though. Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 18:34:29 2019 Received: (at 38754) by debbugs.gnu.org; 30 Dec 2019 23:34:29 +0000 Received: from localhost ([127.0.0.1]:33617 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im4YO-00063U-HX for submit@debbugs.gnu.org; Mon, 30 Dec 2019 18:34:28 -0500 Received: from mira.cbaines.net ([212.71.252.8]:49100) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1im4YL-00063C-QJ for 38754@debbugs.gnu.org; Mon, 30 Dec 2019 18:34:26 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id C775D179BA; Mon, 30 Dec 2019 23:34:24 +0000 (GMT) Received: from phact (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 4aa9c908; Mon, 30 Dec 2019 23:34:24 +0000 (UTC) References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> <87lfqtzcma.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. In-reply-to: <87lfqtzcma.fsf@gnu.org> Date: Mon, 30 Dec 2019 23:34:21 +0000 Message-ID: <878smtqtfm.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> + (with-store store >> + (parameterize >> + ((%lint-checker-store-connection store)) > > Actually it means that now =E2=80=98guix lint=E2=80=99 systematically con= nects to the > daemon. I guess that's the effect, were you meaning this would make a better message in the commit? > I wonder if we could arrange to open the connection lazily, and to > somehow carry state across linter invocations. Perhaps > =E2=80=98check-derivation=E2=80=99 should be monadic, with a field in > indicating that. Sounds complicated though. > > Thoughts? I did wonder if the code could somehow transparently be made more efficient. Quite often database clients manage a pool of connections, and when you perform a database operation, a connection from the pool is checked out, and then returned once you're finished. But as you say, this could be complicated. I think parameters can be set with connections, and I'm not quiet sure what the interface should be. I also did think about somehow passing the store connection in to the lint checker more explicitly, but I'm not sure how to generalise that. At least to me, a parameter to store the connection in seemed like a simple if inelegant approach. I'm quite happy to look in to other approaches though if you have thoughts on what might be good. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl4KiX1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XfrTQ/6Ay69KW0ECA9FZxCvpfiPLk5KRXnoW6L9TvGyB+hG+9k3z78UrjZ3Ht7T tyebnY6Kslywxnp/uuOOIiYQbw4BhUhjTQkaqYYA7SqULL3COHat1xI4IZRSxuJu F3XUBR+upGYaO4TnftFaF2RIj4HmLJrvdHcNCmNJ74Di9nu/zsqfkwclAhm0b0lx LTWN0jQA8cMHYMxiR1Qwdb1EJh3ZgmvrR3To9uYbFLrfz5UuvyYbLfevOISOmCZC lo+fXSfP4eOlw2Fi/NseW5MnhQ1fEmX/iXE9qow3CRPx825St/WE+85teSipUsJ8 GVnYQrIdSAaLPgp1p6HD0InymkWgTfoOF7cJLOoPOMu9VooG/5eRpeUNvHp8D+jV 78pySy+O6ThEqg97FwUXohGEXW83JpvpdklIHuopmfkv3hkZ/0v/EVDMo4plfZQH sWwc1BjU28OigSXLGpaUQkrU0Pb7/vvhJM55PFJsGFzK5owrSMyCJWe93YdtPG0q rEedkkUzrMH3yZovBvhX5CTZUOPtIRANEk5feUPVMkVWHD937JeVFDuBVr8bFsrH P7sDDEJz9ku2VOkpr8D6RVY6a96oENuZHb8Jv0MSrmMfxKmHvdc9YLgOw+MURONI QHy7b2asPmLuPhiAaxK+9yqQTuIOpr5NsptrO7ZHaKyXBndD8XQ= =kQoX -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 31 13:15:49 2019 Received: (at 38754) by debbugs.gnu.org; 31 Dec 2019 18:15:49 +0000 Received: from localhost ([127.0.0.1]:35088 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1imM3Z-0005xb-Di for submit@debbugs.gnu.org; Tue, 31 Dec 2019 13:15:49 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1imM3X-0005xN-CA for 38754@debbugs.gnu.org; Tue, 31 Dec 2019 13:15:47 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53567) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1imM3R-0005sp-W2; Tue, 31 Dec 2019 13:15:42 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46138 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1imM3J-0006lh-PI; Tue, 31 Dec 2019 13:15:35 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> <87lfqtzcma.fsf@gnu.org> <878smtqtfm.fsf@cbaines.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 11 =?utf-8?Q?Niv=C3=B4se?= an 228 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 31 Dec 2019 19:15:32 +0100 In-Reply-To: <878smtqtfm.fsf@cbaines.net> (Christopher Baines's message of "Mon, 30 Dec 2019 23:34:21 +0000") Message-ID: <87lfqsxsxn.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Chris! Christopher Baines skribis: > Ludovic Court=C3=A8s writes: > >> Christopher Baines skribis: >> >>> + (with-store store >>> + (parameterize >>> + ((%lint-checker-store-connection store)) >> >> Actually it means that now =E2=80=98guix lint=E2=80=99 systematically co= nnects to the >> daemon. > > I guess that's the effect, were you meaning this would make a better > message in the commit? I mean that it=E2=80=99s a visible change. Before, you could run all the linters but this one without having a daemon running; now you need a daemon up and running. >> I wonder if we could arrange to open the connection lazily, and to >> somehow carry state across linter invocations. Perhaps >> =E2=80=98check-derivation=E2=80=99 should be monadic, with a field in >> indicating that. Sounds complicated though. >> >> Thoughts? > > I did wonder if the code could somehow transparently be made more > efficient. Quite often database clients manage a pool of connections, > and when you perform a database operation, a connection from the pool is > checked out, and then returned once you're finished. But as you say, > this could be complicated. I think parameters can be set with > connections, and I'm not quiet sure what the interface should be. > > I also did think about somehow passing the store connection in to the > lint checker more explicitly, but I'm not sure how to generalise that. There could be a field indicating either that (1) the procedure takes an optional store parameter, or that (2) the procedure is monadic in =E2=80=98%store-monad=E2=80=99. #2 seems more complicated to implement that #1 though. For #1, =E2=80=98guix lint=E2=80=99 could check whether: (any checker-require-store? checkers) is true, and if it is, it could open a connection and pass it on as needed. WDYT? If that seems good to you, I guess you can go ahead with it (let=E2=80=99s = just not lose our hair on it!). Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 15 17:06:36 2020 Received: (at 38754) by debbugs.gnu.org; 15 Mar 2020 21:06:36 +0000 Received: from localhost ([127.0.0.1]:35700 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSy-00009Z-6I for submit@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:36 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36346) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSw-000099-9d for 38754@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:34 -0400 Received: from localhost (unknown [46.237.160.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 8FF7F27BBEE for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:33 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id bbdd5819 for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:31 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 4/4] scripts: lint: Handle store connections for lint checkers. Date: Sun, 15 Mar 2020 21:06:31 +0000 Message-Id: <20200315210631.5334-4-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200315210631.5334-1-mail@cbaines.net> References: <87lfqsxsxn.fsf@gnu.org> <20200315210631.5334-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Rather than individual checkers opening up a connection to the store for each package to check, if any checker requires a store connection, open a connection and pass it to all checkers that would use it. This makes running the derivation checker much faster for multiple packages. * guix/scripts/lint.scm (run-checkers): Add a #:store argument, and pass the store to checkers if they require a store connection. (guix-lint): Establish a store connection if any checker requires one, and pass it through to run-checkers. --- guix/scripts/lint.scm | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm index 8d08c484f5..97ffd57301 100644 --- a/guix/scripts/lint.scm +++ b/guix/scripts/lint.scm @@ -30,6 +30,7 @@ #:use-module (guix packages) #:use-module (guix lint) #:use-module (guix ui) + #:use-module (guix store) #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (gnu packages) @@ -53,7 +54,7 @@ (lint-warning-message lint-warning)))) warnings)) -(define (run-checkers package checkers) +(define* (run-checkers package checkers #:key store) "Run the given CHECKERS on PACKAGE." (let ((tty? (isatty? (current-error-port)))) (for-each (lambda (checker) @@ -63,7 +64,9 @@ (lint-checker-name checker)) (force-output (current-error-port))) (emit-warnings - ((lint-checker-check checker) package))) + (if (lint-checker-requires-store? checker) + ((lint-checker-check checker) package #:store store) + ((lint-checker-check checker) package)))) checkers) (when tty? (format (current-error-port) "\x1b[K") @@ -167,12 +170,27 @@ run the checkers on all packages.\n")) (_ #f)) (reverse opts))) (checkers (or (assoc-ref opts 'checkers) %all-checkers))) - (cond - ((assoc-ref opts 'list?) + + (when (assoc-ref opts 'list?) (list-checkers-and-exit checkers)) - ((null? args) - (fold-packages (lambda (p r) (run-checkers p checkers)) '())) - (else - (for-each (lambda (spec) - (run-checkers (specification->package spec) checkers)) - args))))) + + (let ((any-lint-checker-requires-store? + (any lint-checker-requires-store? checkers))) + + (define (call-maybe-with-store proc) + (if any-lint-checker-requires-store? + (with-store store + (proc store)) + (proc #f))) + + (call-maybe-with-store + (lambda (store) + (cond + ((null? args) + (fold-packages (lambda (p r) (run-checkers p checkers + #:store store)) '())) + (else + (for-each (lambda (spec) + (run-checkers (specification->package spec) checkers + #:store store)) + args)))))))) -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 15 17:06:36 2020 Received: (at 38754) by debbugs.gnu.org; 15 Mar 2020 21:06:37 +0000 Received: from localhost ([127.0.0.1]:35702 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSy-00009b-Ez for submit@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:36 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSw-00009A-AQ for 38754@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:35 -0400 Received: from localhost (unknown [46.237.160.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 801CC27BBEB for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:33 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id e4b16d6a for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:31 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 3/4] lint: Add a #:store argument to check-derivation Date: Sun, 15 Mar 2020 21:06:30 +0000 Message-Id: <20200315210631.5334-3-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200315210631.5334-1-mail@cbaines.net> References: <87lfqsxsxn.fsf@gnu.org> <20200315210631.5334-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This can then be used to avoid opening up a store connection each time a package needs checking. * guix/lint.scm (check-derivation): Add a #:store argument, and pull the handling of the store connection out of the try function. --- guix/lint.scm | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/guix/lint.scm b/guix/lint.scm index b20510b45d..cfe3be2302 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -917,9 +917,9 @@ descriptions maintained upstream." (define exception-with-kind-and-args? (const #f)))) -(define (check-derivation package) +(define* (check-derivation package #:key store) "Emit a warning if we fail to compile PACKAGE to a derivation." - (define (try system) + (define (try store system) (catch #t ;TODO: Remove 'catch' when Guile 2.x is no longer supported. (lambda () (guard (c ((store-protocol-error? c) @@ -938,25 +938,29 @@ descriptions maintained upstream." (G_ "failed to create ~a derivation: ~a") (list system (condition-message c))))) - (with-store store - ;; Disable grafts since it can entail rebuilds. - (parameterize ((%graft? #f)) - (package-derivation store package system #:graft? #f) - - ;; If there's a replacement, make sure we can compute its - ;; derivation. - (match (package-replacement package) - (#f #t) - (replacement - (package-derivation store replacement system - #:graft? #f))))))) + (parameterize ((%graft? #f)) + (package-derivation store package system #:graft? #f) + + ;; If there's a replacement, make sure we can compute its + ;; derivation. + (match (package-replacement package) + (#f #t) + (replacement + (package-derivation store replacement system + #:graft? #f)))))) (lambda args (make-warning package (G_ "failed to create ~a derivation: ~s") (list system args))))) - (filter lint-warning? - (map try (package-supported-systems package)))) + (define (check-with-store store) + (filter lint-warning? + (map (cut try store <>) (package-supported-systems package)))) + + ;; For backwards compatability, don't rely on store being set + (or (and=> store check-with-store) + (with-store store + (check-with-store store)))) (define (check-license package) "Warn about type errors of the 'license' field of PACKAGE." -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 15 17:06:37 2020 Received: (at 38754) by debbugs.gnu.org; 15 Mar 2020 21:06:37 +0000 Received: from localhost ([127.0.0.1]:35704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSy-00009j-Sv for submit@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:37 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36344) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSw-000098-B6 for 38754@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:35 -0400 Received: from localhost (unknown [46.237.160.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 7032327BBEA for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:33 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 58ee8cbb for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:31 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 2/4] lint: Mark the derivation checker as requiring a store connection. Date: Sun, 15 Mar 2020 21:06:29 +0000 Message-Id: <20200315210631.5334-2-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200315210631.5334-1-mail@cbaines.net> References: <87lfqsxsxn.fsf@gnu.org> <20200315210631.5334-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * guix/lint.scm (%local-checkers): Mark the derivation checker as requiring a store connection. --- guix/lint.scm | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/guix/lint.scm b/guix/lint.scm index 2a084382c6..b20510b45d 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -1330,9 +1330,10 @@ or a list thereof") (description "Check for autogenerated tarballs") (check check-source-unstable-tarball)) (lint-checker - (name 'derivation) - (description "Report failure to compile a package to a derivation") - (check check-derivation)) + (name 'derivation) + (description "Report failure to compile a package to a derivation") + (check check-derivation) + (requires-store? #t)) (lint-checker (name 'patch-file-names) (description "Validate file names and availability of patches") -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 15 17:06:37 2020 Received: (at 38754) by debbugs.gnu.org; 15 Mar 2020 21:06:37 +0000 Received: from localhost ([127.0.0.1]:35706 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSz-00009q-6S for submit@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:37 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36342) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDaSw-000097-9b for 38754@debbugs.gnu.org; Sun, 15 Mar 2020 17:06:35 -0400 Received: from localhost (unknown [46.237.160.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 5DA5427BBE4 for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:33 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id dbed325b for <38754@debbugs.gnu.org>; Sun, 15 Mar 2020 21:06:31 +0000 (UTC) From: Christopher Baines To: 38754@debbugs.gnu.org Subject: [PATCH 1/4] lint: Add a requires-store? field to the checker record. Date: Sun, 15 Mar 2020 21:06:28 +0000 Message-Id: <20200315210631.5334-1-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <87lfqsxsxn.fsf@gnu.org> References: <87lfqsxsxn.fsf@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 38754 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This can then be used to mark checkers that require a store connection, which will enable passing a connection in, avoiding the overhead of establishing a connection inside the check function when it's run for lots of different packages. * guix/lint.scm (): Add requires-store? to the record type. --- guix/lint.scm | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/guix/lint.scm b/guix/lint.scm index 24fbf05202..2a084382c6 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -100,7 +100,8 @@ lint-checker? lint-checker-name lint-checker-description - lint-checker-check)) + lint-checker-check + lint-checker-requires-store?)) ;;; @@ -155,7 +156,9 @@ ;; 'certainty' level. (name lint-checker-name) (description lint-checker-description) - (check lint-checker-check)) + (check lint-checker-check) + (requires-store? lint-checker-requires-store? + (default #f))) (define (properly-starts-sentence? s) (string-match "^[(\"'`[:upper:][:digit:]]" s)) -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 15 17:35:36 2020 Received: (at 38754) by debbugs.gnu.org; 15 Mar 2020 21:35:36 +0000 Received: from localhost ([127.0.0.1]:35730 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDav1-0000wD-NY for submit@debbugs.gnu.org; Sun, 15 Mar 2020 17:35:35 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36446) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jDav0-0000w5-5Q for 38754@debbugs.gnu.org; Sun, 15 Mar 2020 17:35:34 -0400 Received: from localhost (unknown [46.237.160.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 40BD327BBEA; Sun, 15 Mar 2020 21:35:33 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id c5909441; Sun, 15 Mar 2020 21:35:31 +0000 (UTC) References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> <87lfqtzcma.fsf@gnu.org> <878smtqtfm.fsf@cbaines.net> <87lfqsxsxn.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. In-reply-to: <87lfqsxsxn.fsf@gnu.org> Date: Sun, 15 Mar 2020 21:35:28 +0000 Message-ID: <87mu8hcndr.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi Chris! > > Christopher Baines skribis: > >> Ludovic Court=C3=A8s writes: >> >>> Christopher Baines skribis: >>> >>>> + (with-store store >>>> + (parameterize >>>> + ((%lint-checker-store-connection store)) >>> >>> Actually it means that now =E2=80=98guix lint=E2=80=99 systematically c= onnects to the >>> daemon. >> >> I guess that's the effect, were you meaning this would make a better >> message in the commit? > > I mean that it=E2=80=99s a visible change. Before, you could run all the > linters but this one without having a daemon running; now you need a > daemon up and running. Ah, yeah, that's a good point. >>> I wonder if we could arrange to open the connection lazily, and to >>> somehow carry state across linter invocations. Perhaps >>> =E2=80=98check-derivation=E2=80=99 should be monadic, with a field in <= checker> >>> indicating that. Sounds complicated though. >>> >>> Thoughts? >> >> I did wonder if the code could somehow transparently be made more >> efficient. Quite often database clients manage a pool of connections, >> and when you perform a database operation, a connection from the pool is >> checked out, and then returned once you're finished. But as you say, >> this could be complicated. I think parameters can be set with >> connections, and I'm not quiet sure what the interface should be. >> >> I also did think about somehow passing the store connection in to the >> lint checker more explicitly, but I'm not sure how to generalise that. > > There could be a field indicating either that (1) the > procedure takes an optional store parameter, or that (2) the procedure > is monadic in =E2=80=98%store-monad=E2=80=99. > > #2 seems more complicated to implement that #1 though. > > For #1, =E2=80=98guix lint=E2=80=99 could check whether: > > (any checker-require-store? checkers) > > is true, and if it is, it could open a connection and pass it on as > needed. > > WDYT? > > If that seems good to you, I guess you can go ahead with it (let=E2=80=99= s just > not lose our hair on it!). I've finally got around to looking at this again. I've sent some new patches which add a field to the record, and adjust the running of lint checkers as well as the derivation checker to use a single store connection. Let me know what you think? Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5un6BfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XfnhQ//TgII6dZOglvxrSPVzYYMcpwCByot52rMbcVnO6K2KAL8adBxS5BnGa9V r9G+PbhImKMUsZPnjz6WLX750P/KgY7kYpIlwTdhKOt2WwfuJKVu/O9ZZOy2rEtI rNkgWOpAxz7lymjjoHY3dGvQ3I0f1tPhZYrD8VrxhsJIFR2nzJslRwdsOAosBKmE eQwQTo/oZPh2+D+RcGcK38DlzhsziNoJ7s8oUAzrbi7TYHK+fJn8Vym8F+cZJrvI xNocdgQwmyzCdkwNYnwEmOS/7pmWCP/e7y2qSoGwNbs7Fdfb7fTjCXCMSDOq7Jpg 0kDv1uGdK7IGPz0lV1jmi2X+PTq03t0V9UjxftpEvOO4Tsccz1Ut3dMkwrHZf3JA JhvNbnbp+bvJ4QhT64JEWZyS3Gjhc0hl/PWS6kMGhwVKGv04DHYV3QwRCbQp7O/e HTGGVsay2XQGS3m4y3OHhJFI7AcR/laufpR45yAoOQpu3AaAo4dHOHqxbuVMUWPr TrUu/i34IDkqoh2YUZjkEeZRBKJqMIoMWERK6IofrdgzMID3IwJgudupp1+aCtH+ SCLsYuPHcPyoDhYXmwMb07NmAiF5fDH9iwHFk85LzLFtIBQXm5aeaVUUpUv+YY3c 94HcuWVtCPfjExCuCe5VILnMp9I+de2nTrfhxfHzOPxYpsB9+EM= =YGen -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 24 06:20:41 2020 Received: (at 38754) by debbugs.gnu.org; 24 Mar 2020 10:20:41 +0000 Received: from localhost ([127.0.0.1]:53918 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGgfp-0002ou-3y for submit@debbugs.gnu.org; Tue, 24 Mar 2020 06:20:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:36651) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGgfo-0002oj-3O for 38754@debbugs.gnu.org; Tue, 24 Mar 2020 06:20:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44833) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jGgfi-0006tb-PV; Tue, 24 Mar 2020 06:20:34 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44464 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jGgfg-0006ZW-Rx; Tue, 24 Mar 2020 06:20:34 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> <87lfqtzcma.fsf@gnu.org> <878smtqtfm.fsf@cbaines.net> <87lfqsxsxn.fsf@gnu.org> <87mu8hcndr.fsf@cbaines.net> Date: Tue, 24 Mar 2020 11:20:31 +0100 In-Reply-To: <87mu8hcndr.fsf@cbaines.net> (Christopher Baines's message of "Sun, 15 Mar 2020 21:35:28 +0000") Message-ID: <87imiunjf4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 38754 Cc: 38754@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi, Christopher Baines skribis: > I've finally got around to looking at this again. I've sent some new > patches which add a field to the record, and adjust the > running of lint checkers as well as the derivation checker to use a > single store connection. > > Let me know what you think? LGTM, thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 24 15:50:13 2020 Received: (at 38754-done) by debbugs.gnu.org; 24 Mar 2020 19:50:13 +0000 Received: from localhost ([127.0.0.1]:55624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGpYz-0003OM-G2 for submit@debbugs.gnu.org; Tue, 24 Mar 2020 15:50:13 -0400 Received: from mira.cbaines.net ([212.71.252.8]:49588) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGpYx-0003OD-MP for 38754-done@debbugs.gnu.org; Tue, 24 Mar 2020 15:50:12 -0400 Received: from localhost (unknown [46.237.162.18]) by mira.cbaines.net (Postfix) with ESMTPSA id AC1BA27BBE4; Tue, 24 Mar 2020 19:50:10 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 326b24f8; Tue, 24 Mar 2020 19:50:08 +0000 (UTC) References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> <20191226180104.10888-2-mail@cbaines.net> <87lfqtzcma.fsf@gnu.org> <878smtqtfm.fsf@cbaines.net> <87lfqsxsxn.fsf@gnu.org> <87mu8hcndr.fsf@cbaines.net> <87imiunjf4.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. In-reply-to: <87imiunjf4.fsf@gnu.org> Date: Tue, 24 Mar 2020 19:50:07 +0000 Message-ID: <87eeth5y8g.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 38754-done Cc: 38754-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi, > > Christopher Baines skribis: > >> I've finally got around to looking at this again. I've sent some new >> patches which add a field to the record, and adjust the >> running of lint checkers as well as the derivation checker to use a >> single store connection. >> >> Let me know what you think? > > LGTM, thank you! Great, I've pushed this to master now as 57e12aad6dfc2d12567164144dd15161e66f32d5. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl56ZG9fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdxuQ/9E6/4L+ZXd+KADKgTGYK6WKrlFa1/Xlgdz1mWSkTfiqcKjwzp9Fuor9Mu MnHJ0bT6TTWGlshlp2pXrEhLtbLvdZWWj3yFLROyrFCNhwm5+83Xxr4toqbl11dD XFsLaRL78IRLyJIHpd+re1AlTF/oZh2bCHFIaVwaaXiLqfcxdE804YcBIJ49Qcdn z/Y9Pi99JMzI25ukdAnQlG/0j2Yp2PJFoH3BAruMde6u9uKlhClgCrn9nanjqyKS zcJ2xOptXzZjRIDNFgln/lMAPZN0sNcC5C0GZsb7Wzvm9sI7vnxDzIUw6zre9nNv kEAHEd6Eex1krkV5yCEJFIT6DupfqBLhMfD0FBZVJ9XOzhAb08IBlPS6y0yN4zFF aHAuhOBLykenQCxB6eTAqwrYQvHgIW6yamuBYG0GtPr88cUZXv5gQkysKWCLImll lwupttu1YyUZNRAEIkmJAI2cylJS/5qBNtTZKzqa8MiIApX6ZHtDZp/xsMd5s7kR AOgatLJtY2+xT/93BWJzoCf+Vgyye+YNSqUhqsamBgMgljnny58jXqQXRSykzdPE 3IxGfHDcG6xGIvTx4/NA5varuw3I0wdTM8bcH7gngtDNOcPkmag8vE+5e+NT3TfS TUabja7yCAj7MlwLrTLO8dED/MaYUQOSLR6TO+8TpXMx1WgYc8o= =Re4d -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Jun 13 10:25:18 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 22 Apr 2020 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator