From unknown Fri Sep 05 09:11:28 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#44193 <44193@debbugs.gnu.org> To: bug#44193 <44193@debbugs.gnu.org> Subject: Status: [PATCH 0/1] 'guix publish --cache' can publish items not yet cached Reply-To: bug#44193 <44193@debbugs.gnu.org> Date: Fri, 05 Sep 2025 16:11:28 +0000 retitle 44193 [PATCH 0/1] 'guix publish --cache' can publish items not yet = cached reassign 44193 guix-patches submitter 44193 Ludovic Court=C3=A8s severity 44193 normal tag 44193 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 24 10:49:44 2020 Received: (at submit) by debbugs.gnu.org; 24 Oct 2020 14:49:44 +0000 Received: from localhost ([127.0.0.1]:34253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrY-0001E7-6W for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:44 -0400 Received: from lists.gnu.org ([209.51.188.17]:43150) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrV-0001Dx-Ba for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39644) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWKrV-0006Gn-12 for guix-patches@gnu.org; Sat, 24 Oct 2020 10:49:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47624) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWKrS-0006RB-73; Sat, 24 Oct 2020 10:49:39 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51756 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWKrQ-0004dT-Hx; Sat, 24 Oct 2020 10:49:37 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/1] 'guix publish --cache' can publish items not yet cached Date: Sat, 24 Oct 2020 16:49:29 +0200 Message-Id: <20201024144929.4529-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! The ‘--cache’ mode of ‘guix publish’ is nice for many reasons. Until now though, it would only return 200 to narinfo and nar requests corresponding to items already in cache—already “baked”. Thus, the first narinfo request for an item would always return 404; one would have to wait until the item is baked to get 200 and download the substitute. If you’re fetching substitutes for popular packages on a popular instance, the extra delay is most likely invisible. However, if you’re using an instance with few users, or if you’re interesting in substitutes for an package that’s not popular, the behavior described above is a showstopper. (Many people here have criticized it in the past. :-)) This patch changes the behavior of ‘--cache’: if a store item is not yet in cache, and if it’s “small enough”, then narinfo/nar requests for it immediately return 200, as in the no-cache mode. You’re trading possibly increased server resource usage for reduced publication delay. To put an upper bound on the extra resource usage, the ‘--cache-bypass-threshold’ allows users to control the maximum size of a store item that can receive this treatment. An interesting use case that can benefit from this change is .drv substitutes: .drv themselves can be substitutes, so you can run: guix build /gnu/store/….drv and the daemon will fetch the .drv and its closure and start building it (since commit 9c9982dc0c8c38ce3821b154b7e92509c1564317). It’s not something we use much so far, but maybe we could put it to good use in the future (I know Chris has been playing with it in the context of the Data Service). Thoughts? Ludo’. Ludovic Courtès (1): publish: Add '--cache-bypass-threshold'. doc/guix.texi | 24 +++++++++++- guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++-------- tests/publish.scm | 43 ++++++++++++++++++-- 3 files changed, 130 insertions(+), 22 deletions(-) -- 2.28.0 From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 24 10:54:33 2020 Received: (at 44193) by debbugs.gnu.org; 24 Oct 2020 14:54:33 +0000 Received: from localhost ([127.0.0.1]:34258 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKwC-0001La-TF for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:54:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKw9-0001LM-82 for 44193@debbugs.gnu.org; Sat, 24 Oct 2020 10:54:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47714) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWKw3-00074N-BM; Sat, 24 Oct 2020 10:54:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51772 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWKw2-00051z-S2; Sat, 24 Oct 2020 10:54:23 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 44193@debbugs.gnu.org Subject: [PATCH 1/1] publish: Add '--cache-bypass-threshold'. Date: Sat, 24 Oct 2020 16:54:16 +0200 Message-Id: <20201024145416.4691-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44193 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/scripts/publish.scm (show-help, %options): Add '--cache-bypass-threshold'. (low-compression): New procedure. (cache-bypass-threshold): New parameter. (bypass-cache?): New procedure. (render-narinfo/cached): Call 'render-narinfo' when 'bypass-cache?' returns true. (render-nar/cached): Call 'render-nar' when 'bypass-cache?' returns true. (guix-publish): Parameterize 'cache-bypass-threshold'. * tests/publish.scm ("with cache", "with cache, lzip + gzip") ("with cache, uncompressed"): Pass '--cache-bypass-threshold=0'. ("with cache, vanishing item"): Expect 200 for RESPONSE. ("with cache, cache bypass"): New test. --- doc/guix.texi | 24 +++++++++++- guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++-------- tests/publish.scm | 43 ++++++++++++++++++-- 3 files changed, 130 insertions(+), 22 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b5061877e2..633c974562 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -11977,13 +11977,20 @@ in advance, so @command{guix publish} does not add a prevents clients from knowing the amount of data being downloaded. Conversely, when @option{--cache} is used, the first request for a store -item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a +item (@i{via} a @code{.narinfo} URL) triggers a background process to @dfn{bake} the archive---computing its @code{.narinfo} and compressing the archive, if needed. Once the archive is cached in @var{directory}, subsequent requests succeed and are served directly from the cache, which guarantees that clients get the best possible bandwidth. +That first @code{.narinfo} request nonetheless returns 200, provided the +requested store item is ``small enough'', below the cache bypass +threshold---see @option{--cache-bypass-threshold} below. That way, +clients do not have to wait until the archive is baked. For larger +store items, the first @code{.narinfo} request returns 404, meaning that +clients have to wait until the archive is baked. + The ``baking'' process is performed by worker threads. By default, one thread per CPU core is created, but this can be customized. See @option{--workers} below. @@ -12009,6 +12016,21 @@ Additionally, when @option{--cache} is used, cached entries that have not been accessed for @var{ttl} and that no longer have a corresponding item in the store, may be deleted. +@item --cache-bypass-threshold=@var{size} +When used in conjunction with @option{--cache}, store items smaller than +@var{size} are immediately available, even when they are not yet in +cache. @var{size} is a size in bytes, or it can be prefixed by @code{M} +for megabytes and so on. The default is @code{10M}. + +``Cache bypass'' allows you to reduce the publication delay for clients +at the expense of possibly additional I/O and CPU use on the server +side: depending on the client access patterns, those store items can end +up being baked several times until a copy is available in cache. + +Increasing the threshold may be useful for sites that have few users, or +to guarantee that users get substitutes even for store items that are +not popular. + @item --nar-path=@var{path} Use @var{path} as the prefix for the URLs of ``nar'' files (@pxref{Invoking guix archive, normalized archives}). diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 4eaf961ab2..c0150c74da 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -81,6 +81,9 @@ Publish ~a over HTTP.\n") %store-directory) compress archives with METHOD at LEVEL")) (display (G_ " -c, --cache=DIRECTORY cache published items to DIRECTORY")) + (display (G_ " + --cache-bypass-threshold=SIZE + serve store items below SIZE even when not cached")) (display (G_ " --workers=N use N workers to bake items")) (display (G_ " @@ -134,6 +137,12 @@ if ITEM is already compressed." (list %no-compression) requested)) +(define (low-compression c) + "Return of the same type as C, but optimized for low CPU +usage." + (compression (compression-type c) + (min (compression-level c) 2))) + (define %options (list (option '(#\h "help") #f #f (lambda _ @@ -184,6 +193,10 @@ if ITEM is already compressed." (option '(#\c "cache") #t #f (lambda (opt name arg result) (alist-cons 'cache arg result))) + (option '("cache-bypass-threshold") #t #f + (lambda (opt name arg result) + (alist-cons 'cache-bypass-threshold (size->number arg) + result))) (option '("workers") #t #f (lambda (opt name arg result) (alist-cons 'workers (string->number* arg) @@ -434,7 +447,7 @@ items. Failing that, we could eventually have to recompute them and return (expiration-time file)))))) (define (hash-part->path* store hash cache) - "Like 'hash-part->path' but cached results under CACHE. This ensures we can + "Like 'hash-part->path' but cache results under CACHE. This ensures we can still map HASH to the corresponding store file name, even if said store item vanished from the store in the meantime." (let ((cached (hash-part-mapping-cache-file cache hash))) @@ -454,6 +467,18 @@ vanished from the store in the meantime." result)) (apply throw args)))))) +(define cache-bypass-threshold + ;; Maximum size of a store item that may be served by the '/cached' handlers + ;; below even when not in cache. + (make-parameter (* 10 (expt 2 20)))) + +(define (bypass-cache? store item) + "Return true if we allow ITEM to be downloaded before it is cached. ITEM is +interpreted as the basename of a store item." + (guard (c ((store-error? c) #f)) + (< (path-info-nar-size (query-path-info store item)) + (cache-bypass-threshold)))) + (define* (render-narinfo/cached store request hash #:key ttl (compressions (list %no-compression)) (nar-path "nar") @@ -513,9 +538,20 @@ requested using POOL." (nar-expiration-time ttl) #:delete-entry delete-entry #:cleanup-period ttl)))) - (not-found request - #:phrase "We're baking it" - #:ttl 300)) ;should be available within 5m + + ;; If ITEM passes 'bypass-cache?', render a temporary narinfo right + ;; away, with a short TTL. The narinfo is temporary because it + ;; lacks 'FileSize', for instance, which the cached narinfo will + ;; have. Chances are that the nar will be baked by the time the + ;; client asks for it. + (if (bypass-cache? store item) + (render-narinfo store request hash + #:ttl 300 ;temporary + #:nar-path nar-path + #:compressions compressions) + (not-found request + #:phrase "We're baking it" + #:ttl 300))) ;should be available within 5m (else (not-found request #:phrase ""))))) @@ -627,19 +663,31 @@ return it; otherwise, return 404. When TTL is true, use it as the 'Cache-Control' expiration time." (let ((cached (nar-cache-file cache store-item #:compression compression))) - (if (file-exists? cached) - (values `((content-type . (application/octet-stream - (charset . "ISO-8859-1"))) - ,@(if ttl - `((cache-control (max-age . ,ttl))) - '()) + (cond ((file-exists? cached) + (values `((content-type . (application/octet-stream + (charset . "ISO-8859-1"))) + ,@(if ttl + `((cache-control (max-age . ,ttl))) + '()) - ;; XXX: We're not returning the actual contents, deferring - ;; instead to 'http-write'. This is a hack to work around - ;; . - (x-raw-file . ,cached)) - #f) - (not-found request)))) + ;; XXX: We're not returning the actual contents, deferring + ;; instead to 'http-write'. This is a hack to work around + ;; . + (x-raw-file . ,cached)) + #f)) + ((let* ((hash (and=> (string-index store-item #\-) + (cut string-take store-item <>))) + (item (and hash + (guard (c ((store-error? c) #f)) + (hash-part->path store hash))))) + (and item (bypass-cache? store item))) + ;; Render STORE-ITEM live. We reach this because STORE-ITEM is + ;; being baked but clients are already asking for it. Thus, we're + ;; duplicating work, but doing so allows us to reduce delays. + (render-nar store request store-item + #:compression (low-compression compression))) + (else + (not-found request))))) (define (render-content-addressed-file store request name algo hash) @@ -1061,7 +1109,10 @@ methods, return the applicable compression." consider using the '--user' option!~%"))) (parameterize ((%public-key public-key) - (%private-key private-key)) + (%private-key private-key) + (cache-bypass-threshold + (or (assoc-ref opts 'cache-bypass-threshold) + (cache-bypass-threshold)))) (info (G_ "publishing ~a on ~a, port ~d~%") %store-directory (inet-ntop (sockaddr:fam address) (sockaddr:addr address)) diff --git a/tests/publish.scm b/tests/publish.scm index 1c3b2785fb..f081d016d3 100644 --- a/tests/publish.scm +++ b/tests/publish.scm @@ -412,7 +412,8 @@ References: ~%" (call-with-new-thread (lambda () (guix-publish "--port=6797" "-C2" - (string-append "--cache=" cache))))))) + (string-append "--cache=" cache) + "--cache-bypass-threshold=0")))))) (wait-until-ready 6797) (let* ((base "http://localhost:6797/") (part (store-path-hash-part %item)) @@ -461,7 +462,8 @@ References: ~%" (call-with-new-thread (lambda () (guix-publish "--port=6794" "-Cgzip:2" "-Clzip:2" - (string-append "--cache=" cache))))))) + (string-append "--cache=" cache) + "--cache-bypass-threshold=0")))))) (wait-until-ready 6794) (let* ((base "http://localhost:6794/") (part (store-path-hash-part %item)) @@ -516,7 +518,8 @@ References: ~%" (call-with-new-thread (lambda () (guix-publish "--port=6796" "-C2" "--ttl=42h" - (string-append "--cache=" cache))))))) + (string-append "--cache=" cache) + "--cache-bypass-threshold=0")))))) (wait-until-ready 6796) (let* ((base "http://localhost:6796/") (part (store-path-hash-part item)) @@ -580,12 +583,44 @@ References: ~%" (basename item) ".narinfo")) (response (http-get url))) - (and (= 404 (response-code response)) + (and (= 200 (response-code response)) ;we're below the threshold (wait-for-file cached) (begin (delete-paths %store (list item)) (response-code (pk 'response (http-get url)))))))))) +(test-equal "with cache, cache bypass" + 200 + (call-with-temporary-directory + (lambda (cache) + (let ((thread (with-separate-output-ports + (call-with-new-thread + (lambda () + (guix-publish "--port=6788" "-C" "gzip" + (string-append "--cache=" cache))))))) + (wait-until-ready 6788) + + (let* ((base "http://localhost:6788/") + (item (add-text-to-store %store "random" (random-text))) + (part (store-path-hash-part item)) + (narinfo (string-append base part ".narinfo")) + (nar (string-append base "nar/gzip/" (basename item))) + (cached (string-append cache "/gzip/" (basename item) + ".narinfo"))) + ;; We're below the default cache bypass threshold, so NAR and NARINFO + ;; should immediately return 200. The NARINFO request should trigger + ;; caching, and the next request to NAR should return 200 as well. + (and (let ((response (pk 'r1 (http-get nar)))) + (and (= 200 (response-code response)) + (not (response-content-length response)))) ;not known + (= 200 (response-code (http-get narinfo))) + (begin + (wait-for-file cached) + (let ((response (pk 'r2 (http-get nar)))) + (and (> (response-content-length response) + (stat:size (stat item))) + (response-code response)))))))))) + (test-equal "/log/NAME" `(200 #t application/x-bzip2) (let ((drv (run-with-store %store -- 2.28.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 25 09:11:48 2020 Received: (at 44193) by debbugs.gnu.org; 25 Oct 2020 13:11:48 +0000 Received: from localhost ([127.0.0.1]:35947 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWfoJ-00018M-N3 for submit@debbugs.gnu.org; Sun, 25 Oct 2020 09:11:48 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:34455) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWfoH-000181-Gf for 44193@debbugs.gnu.org; Sun, 25 Oct 2020 09:11:46 -0400 Received: by mail-wr1-f65.google.com with SMTP id i1so9490722wro.1 for <44193@debbugs.gnu.org>; Sun, 25 Oct 2020 06:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=kR3e8215gWcQXDbjaw9OvKf+RxxgDA49hiJPCQT4vMY=; b=GbDRLro1ahL30K1rJHMkfigmbGHOjyGfyqv0fIHK5TBwEp0Oc2poPpQ6M4oAhtQ1bL pxCAEq27pb6eoCFUnFQFNExYNJDe5cHQuJWiRxNiUNIuqT+pCDE2UBia6EyY+2u74sqN WNTHGEL6YIL/QV7jPICGhfALrdSIzRQcoVHXCm7BqPpl8SiF9EVoLmEZB+C48XvGmhQ8 yl2lZcnnu/nctx9pZ+jFPmXg6ahXBASOMkcDRNTivtL+rC3ev9qe0gC1KlPozfIe1Di8 r44DvZ8rCgOC9C1sUePrt1daOUi4ADu5oXsigd2g4VtHfBoqjiehKwxgok1O/XTdzDW8 Q5Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=kR3e8215gWcQXDbjaw9OvKf+RxxgDA49hiJPCQT4vMY=; b=ff5rQa/jKlwcsJZtcrJCRxEPNOadqwXYcYD7vmPHhdMyqireP2GxyOCiKnCWcwVaCu oilSU26pbMueQLB9zybsMWZk8gNm6cxfnrmkH3su25o5MC8b/2iMDGhY5mxyAq+aTTgh 5uHzeEwB6n7VHFNoo2NbW+Sd3avOhTvnh0j0O53k9wNSFWl9tQiCLQxr24oZqqPPk9pF C9IsvyJUffFWp0kNf9A/C+4xwLQFr1O/XZs+0pnw8mpO3cfqeyANUZKrdupDf2zge4tf ZLMP8BtU3QAOWG+gxKvQjTFZiNBHDZzmNcokkYXCOkcOmv80UQc3vilXxOc7rHwo17Aq K1xA== X-Gm-Message-State: AOAM533koh9wMP5RngGAHIIfvEVQOojWHGUg2JM9tjvHmz5BYW/vDtOz eZrQBYfYf/dYxAVVVTJpIFVqMGithTGJzw== X-Google-Smtp-Source: ABdhPJzFIf2KaohsB4qRqvSd+JSOq3g2px9P8PrJk8Vm972XrcyPz1WxGuPlrj02pr29VBMl/5+MAQ== X-Received: by 2002:adf:d849:: with SMTP id k9mr12248752wrl.332.1603631499408; Sun, 25 Oct 2020 06:11:39 -0700 (PDT) Received: from unfall (218.139.134.37.dynamic.jazztel.es. [37.134.139.218]) by smtp.gmail.com with ESMTPSA id w83sm16697255wmg.48.2020.10.25.06.11.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Oct 2020 06:11:38 -0700 (PDT) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> Date: Sun, 25 Oct 2020 14:11:37 +0100 In-Reply-To: <20201024144929.4529-1-ludo@gnu.org> ("Ludovic =?utf-8?Q?Cour?= =?utf-8?Q?t=C3=A8s=22's?= message of "Sat, 24 Oct 2020 16:49:29 +0200") Message-ID: <87v9ey5u2e.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Hi! Just one general comment about this issue: Ludovic Court=C3=A8s writes: > Thus, the first narinfo request for an item would always return 404; > one would have to wait until the item is baked to get 200 and download > the substitute. I'd argue that returning unconditionally the 404 is a problem. If the nar is getting baked, I guess that a 202[1] would be the appropriate answer, and I'd leave the 404 for invalid store paths[2]. This way the client could implement more policies: the classic timeout, but also, for example, it might check other servers before checking once again if nobody else has it, or directly wait until a 404 is reached. WDYT? Happy hacking! Miguel [1] Section 10.2.3 from https://www.ietf.org/rfc/rfc2616.txt [2] I understand that it isn't at all a bad usage of the 404, as it explicitly says that the condition might be temporary, but on the other hand I don't know how could that extra information be used by a rogue client in any way worse than it could do right now, as the server process is doing the same computation more or less in both cases. From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 25 12:49:58 2020 Received: (at 44193) by debbugs.gnu.org; 25 Oct 2020 16:49:58 +0000 Received: from localhost ([127.0.0.1]:37492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWjDS-0006vL-GU for submit@debbugs.gnu.org; Sun, 25 Oct 2020 12:49:58 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33152) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWjDQ-0006v8-BI for 44193@debbugs.gnu.org; Sun, 25 Oct 2020 12:49:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37398) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWjDK-0004RH-U1; Sun, 25 Oct 2020 12:49:51 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57292 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWjDA-0007GT-38; Sun, 25 Oct 2020 12:49:44 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> <87v9ey5u2e.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 25 Oct 2020 17:49:38 +0100 In-Reply-To: <87v9ey5u2e.fsf@gmail.com> ("Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas"'s message of "Sun, 25 Oct 2020 14:11:37 +0100") Message-ID: <87pn56dzdp.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Miguel =C3=81ngel Arruga Vivas skribis: > Ludovic Court=C3=A8s writes: >> Thus, the first narinfo request for an item would always return 404; >> one would have to wait until the item is baked to get 200 and download >> the substitute. > > I'd argue that returning unconditionally the 404 is a problem. If the > nar is getting baked, I guess that a 202[1] would be the appropriate > answer, and I'd leave the 404 for invalid store paths[2]. This way the > client could implement more policies: the classic timeout, but also, for > example, it might check other servers before checking once again if > nobody else has it, or directly wait until a 404 is reached. WDYT? Indeed, 202 seems more appropriate (and it=E2=80=99s precisely half of 404,= that tells something!). Unfortunately (guix scripts substitute) currently explicitly checks for 404 and 200 and considers anything else to be a transient error with a default TTL (in =E2=80=98handle-narinfo-response=E2=80=99). So we would ne= ed to adapt that first and then wait until some time has passed before =E2=80=98guix publish=E2=80=99 can return 202. :-/ I guess we can change (guix scripts substitute) with that in mind already. WDYT? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 25 13:30:53 2020 Received: (at 44193) by debbugs.gnu.org; 25 Oct 2020 17:30:53 +0000 Received: from localhost ([127.0.0.1]:37584 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWjr2-0000Ly-JK for submit@debbugs.gnu.org; Sun, 25 Oct 2020 13:30:52 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:46436) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWjr0-0000FB-D8 for 44193@debbugs.gnu.org; Sun, 25 Oct 2020 13:30:50 -0400 Received: by mail-wr1-f65.google.com with SMTP id n6so9882744wrm.13 for <44193@debbugs.gnu.org>; Sun, 25 Oct 2020 10:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=1MIbPplSb9D7PLmS+2AgsfxbLPW/085+CI+2/DbrNSI=; b=XBlBKkpkg0KC2IOT0upa4GSAmfCxnkl74F4HuyZAxOcFMUINgez63Bf3XLC25Fmzz2 wueiM7lfZeDfBUtIA9MIThsZzigluYGDnX/IQCRWRYJ7RzgEIyrMmWJeooOStCBXTEZy sbF/EkHGuN0zxHFSnUpfZ+RstRUdfhAgYv6zlk7VDctCY6bbBYxUKF6Eo3XQ/dNy9DXS iNI/0EyTccx5UOxSjNSpBZv6hRHd2caXhLpBRlRsJiVcd5QihO+Wul9O6xRJFGKAT/vR J5N5siJMC3bo1Dqew8x8FzG6YGqzE694ND+ZAIzAuB8kR8vNRMEMAZqZH4ciU/sg7CNg xKUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=1MIbPplSb9D7PLmS+2AgsfxbLPW/085+CI+2/DbrNSI=; b=RXl7SjCqXiFIN1ymOtSr8b5zpS0IWPYHaDyPGuqJ1YS9VriesnFIO5qhu4JsE6BKXz nMkGRvsVkiItj5Nkgt4rGMnNy/G2eMPiQZPp2MhlgxuqqqCh0Yr5y70SMwO9xmSZKyj+ 1wf4WVQa/XvXE3+tKxptvNW6a0bnUuNCnDNOTwgJ2Ui7HZnlMs5NT8UjafOyMJ6S161S LLAsHVt/xAlw+wz3FgZhC/9B8ZxGi4iMFCTa06YDNul3YBJOHghTXa/Go0EZxp7PmbHb 0AK6/7jYzsV3sVMZenQx9TUL4w1brfi8BWLlxpSvCNtd27XzX8LSh0UfoFTPskHzHssC hD/w== X-Gm-Message-State: AOAM530kpPKXfJrQUGwwF56NTN3SVRXAq6v3EBYwEMhXXv7W3FYrYQas 4czG58kxDFMNueICa5KH0h1F0GRxpaQEPQ== X-Google-Smtp-Source: ABdhPJyvNAzZA6aF84AqFM078EkYszIs2PtWbYdTshg8F+EdXBJc3HubH0SI7KmM2zCP28870CgZcQ== X-Received: by 2002:adf:f20f:: with SMTP id p15mr13795325wro.339.1603647043992; Sun, 25 Oct 2020 10:30:43 -0700 (PDT) Received: from unfall (218.139.134.37.dynamic.jazztel.es. [37.134.139.218]) by smtp.gmail.com with ESMTPSA id l1sm19751635wrb.1.2020.10.25.10.30.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Oct 2020 10:30:43 -0700 (PDT) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> <87v9ey5u2e.fsf@gmail.com> <87pn56dzdp.fsf@gnu.org> Date: Sun, 25 Oct 2020 18:30:41 +0100 In-Reply-To: <87pn56dzdp.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Sun, 25 Oct 2020 17:49:38 +0100") Message-ID: <874kmiqkla.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Hi, Ludo! Ludovic Court=C3=A8s writes: > Hi! > > Miguel =C3=81ngel Arruga Vivas skribis: > >> Ludovic Court=C3=A8s writes: >>> Thus, the first narinfo request for an item would always return 404; >>> one would have to wait until the item is baked to get 200 and download >>> the substitute. >> >> I'd argue that returning unconditionally the 404 is a problem. If the >> nar is getting baked, I guess that a 202[1] would be the appropriate >> answer, and I'd leave the 404 for invalid store paths[2]. This way the >> client could implement more policies: the classic timeout, but also, for >> example, it might check other servers before checking once again if >> nobody else has it, or directly wait until a 404 is reached. WDYT? > > Indeed, 202 seems more appropriate (and it=E2=80=99s precisely half of 40= 4, that > tells something!). :-) > Unfortunately (guix scripts substitute) currently explicitly checks for > 404 and 200 and considers anything else to be a transient error with a > default TTL (in =E2=80=98handle-narinfo-response=E2=80=99). So we would = need to adapt > that first and then wait until some time has passed before =E2=80=98guix > publish=E2=80=99 can return 202. :-/ I see, it uses 'max-age from the http response only when it's a 404. Nonetheless, correct me if I'm wrong, the difference is 5 vs 10 minutes, so I don't think we should wait too much to upgrade both sides. :-) > I guess we can change (guix scripts substitute) with that in mind > already. WDYT? I fully agree with that. Adding 202 together with 404 would be enough as an start, wouldn't it? -------------------------------8<----------------------------- (cache-narinfo! url (hash-part->path hash-part) #f (if (or (=3D 404 code) (=3D 202 code)) ttl %narinfo-transient-error-ttl)) ------------------------------->8----------------------------- Happy hacking! Miguel From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 26 06:50:27 2020 Received: (at 44193) by debbugs.gnu.org; 26 Oct 2020 10:50:27 +0000 Received: from localhost ([127.0.0.1]:38608 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kX055-0006gv-44 for submit@debbugs.gnu.org; Mon, 26 Oct 2020 06:50:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49030) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kX053-0006gg-8K for 44193@debbugs.gnu.org; Mon, 26 Oct 2020 06:50:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52038) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kX04x-0005xO-T9; Mon, 26 Oct 2020 06:50:19 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=60826 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kX04q-0006PQ-Rg; Mon, 26 Oct 2020 06:50:14 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> <87v9ey5u2e.fsf@gmail.com> <87pn56dzdp.fsf@gnu.org> <874kmiqkla.fsf@gmail.com> Date: Mon, 26 Oct 2020 11:50:10 +0100 In-Reply-To: <874kmiqkla.fsf@gmail.com> ("Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas"'s message of "Sun, 25 Oct 2020 18:30:41 +0100") Message-ID: <87r1plb6sd.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Miguel =C3=81ngel Arruga Vivas skribis: > I fully agree with that. Adding 202 together with 404 would be enough > as an start, wouldn't it? > -------------------------------8<----------------------------- > (cache-narinfo! url (hash-part->path hash-part) #f > (if (or (=3D 404 code) (=3D 202 code)) > ttl > %narinfo-transient-error-ttl)) > ------------------------------->8----------------------------- Yes, exactly! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 27 15:19:26 2020 Received: (at 44193) by debbugs.gnu.org; 27 Oct 2020 19:19:26 +0000 Received: from localhost ([127.0.0.1]:45912 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXUVC-000364-4y for submit@debbugs.gnu.org; Tue, 27 Oct 2020 15:19:26 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:40316) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXUVA-00035q-R8 for 44193@debbugs.gnu.org; Tue, 27 Oct 2020 15:19:25 -0400 Received: by mail-wm1-f68.google.com with SMTP id k18so2588386wmj.5 for <44193@debbugs.gnu.org>; Tue, 27 Oct 2020 12:19:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=d7PyFEerk8tzZbe281JFioEJ13Of2LKAs8e765+ccCY=; b=KA5poTJvhjmwByLkU/uVW2nxk8RdwsJ7pXjHa6fmeciqiTy9IJ0zw5R12mi4Z4YZYC iM8flIoApbD0qZ3Buey30EcXI+kHjLAtBa3SR2lDJcFuOLqhIOY7X0EijXMHrxmAI0Sp r1/bFMcoZOYq4Q2MDEDXCE874ZDPMGP2gmAIyy+iGiC1soBFXGLfqAnvUSVh/eOUxz9y N0Dzq/U24YHmRaWjJvxSVtoUzAOa+1/JmoIlr2BLmVzuFwoCq9EvM873aSU2Un6lVYrl 7TlKKW7Z/yT0YU6yAaRP4U8LMLpCYwFUyBHZq8MWEOmVRFkUsAbRNI0j+yfmjzp+c5cA BHBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=d7PyFEerk8tzZbe281JFioEJ13Of2LKAs8e765+ccCY=; b=mllDfKglTs34ViG5CpHSpOOZUXCYfvy51voEJfskahExBcBwKj37xs1i9xhN2IEXnF pUC8mPf8nm5BkOeUUaJ340N8ojI3Kp3NPCF2DSavzy2fmxacRZOGRdO63LYOfnpPbxyd GqvnJazSRa0D7qtvm60xmiYs+SoyrKvE7ts5JhahjmPq+/nXLb2wcW8r0/3PKrFC/QKQ TCI+Z6cN7xNwCOdZb1VPIU4pIswBYBPNyVVYNuo8K76InrnCG9WDb71wmHBClecFM3Qb FFb9xTM1JyCWlmMZIgNilrfNN2KMdk4Y4I8VNLwok8FTKhoqLunjlZC+vBR7t4+torsR R6xw== X-Gm-Message-State: AOAM5323i8P9KtkAALIXNR5/dOH0Wmyc5OqD5D48V+ZrtFP7R8sBEiBE +XxwH4fVSYreKnpwQjpkdCHnMNA1YqvCzg== X-Google-Smtp-Source: ABdhPJwi1hrX4IoxfkFWY8uAHkOAwyIsP9L7kDAOswhn9f1Qx2Hc35e/kgEgYT5yYCSEuc+Q7hKEqg== X-Received: by 2002:a1c:3503:: with SMTP id c3mr4183261wma.43.1603826358766; Tue, 27 Oct 2020 12:19:18 -0700 (PDT) Received: from unfall (218.139.134.37.dynamic.jazztel.es. [37.134.139.218]) by smtp.gmail.com with ESMTPSA id u3sm3302598wrq.19.2020.10.27.12.19.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Oct 2020 12:19:17 -0700 (PDT) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> <87v9ey5u2e.fsf@gmail.com> <87pn56dzdp.fsf@gnu.org> <874kmiqkla.fsf@gmail.com> <87r1plb6sd.fsf@gnu.org> Date: Tue, 27 Oct 2020 20:19:17 +0100 In-Reply-To: <87r1plb6sd.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 26 Oct 2020 11:50:10 +0100") Message-ID: <87eeljo4sq.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Hi, Ludovic Court=C3=A8s writes: > Hi, > > Miguel =C3=81ngel Arruga Vivas skribis: > >> I fully agree with that. Adding 202 together with 404 would be enough >> as an start, wouldn't it? >> -------------------------------8<----------------------------- >> (cache-narinfo! url (hash-part->path hash-part) #f >> (if (or (=3D 404 code) (=3D 202 code)) >> ttl >> %narinfo-transient-error-ttl)) >> ------------------------------->8----------------------------- > > Yes, exactly! Should I, or you, push this before the release? It's probably worth having it already for 1.2. The optimization could be cool too: IIUC could be only the other if branch the one returning the 202 when it's widely accepted, perhaps I should have explicitly pointed out that earlier instead of driving too much the conversation to the return code, sorry for that. :-( Happy hacking! Miguel From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 28 05:39:37 2020 Received: (at 44193) by debbugs.gnu.org; 28 Oct 2020 09:39:37 +0000 Received: from localhost ([127.0.0.1]:47206 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXhvd-0001ZS-KR for submit@debbugs.gnu.org; Wed, 28 Oct 2020 05:39:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53988) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXhvb-0001ZF-MH for 44193@debbugs.gnu.org; Wed, 28 Oct 2020 05:39:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40548) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kXhvW-0006RC-DX; Wed, 28 Oct 2020 05:39:30 -0400 Received: from vpn-0-27.aquilenet.fr ([2a0c:e300:4:27::]:38306 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kXhvV-0000rS-Q6; Wed, 28 Oct 2020 05:39:30 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas Subject: Re: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached References: <20201024144929.4529-1-ludo@gnu.org> <87v9ey5u2e.fsf@gmail.com> <87pn56dzdp.fsf@gnu.org> <874kmiqkla.fsf@gmail.com> <87r1plb6sd.fsf@gnu.org> <87eeljo4sq.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 28 Oct 2020 10:39:26 +0100 In-Reply-To: <87eeljo4sq.fsf@gmail.com> ("Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas"'s message of "Tue, 27 Oct 2020 20:19:17 +0100") Message-ID: <87lffq665t.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44193 Cc: 44193@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Miguel =C3=81ngel Arruga Vivas skribis: > Ludovic Court=C3=A8s writes: >> Hi, >> >> Miguel =C3=81ngel Arruga Vivas skribis: >> >>> I fully agree with that. Adding 202 together with 404 would be enough >>> as an start, wouldn't it? >>> -------------------------------8<----------------------------- >>> (cache-narinfo! url (hash-part->path hash-part) #f >>> (if (or (=3D 404 code) (=3D 202 code)) >>> ttl >>> %narinfo-transient-error-ttl)) >>> ------------------------------->8----------------------------- >> >> Yes, exactly! > > Should I, or you, push this before the release? It's probably worth > having it already for 1.2. Agreed, you can go ahead and push this change. > The optimization could be cool too: IIUC could be only the other if > branch the one returning the 202 when it's widely accepted, perhaps I > should have explicitly pointed out that earlier instead of driving too > much the conversation to the return code, sorry for that. :-( No problem! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 28 11:26:58 2020 Received: (at 44193-done) by debbugs.gnu.org; 28 Oct 2020 15:26:58 +0000 Received: from localhost ([127.0.0.1]:50169 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXnLm-0002gv-Br for submit@debbugs.gnu.org; Wed, 28 Oct 2020 11:26:58 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47022) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXnLj-0002ge-Vv for 44193-done@debbugs.gnu.org; Wed, 28 Oct 2020 11:26:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:46292) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kXnLe-0007hA-Oq for 44193-done@debbugs.gnu.org; Wed, 28 Oct 2020 11:26:50 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44672 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kXnLe-0004Yl-3Q for 44193-done@debbugs.gnu.org; Wed, 28 Oct 2020 11:26:50 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 44193-done@debbugs.gnu.org Subject: Re: [bug#44193] [PATCH 1/1] publish: Add '--cache-bypass-threshold'. References: <20201024144929.4529-1-ludo@gnu.org> <20201024145416.4691-1-ludo@gnu.org> Date: Wed, 28 Oct 2020 16:26:48 +0100 In-Reply-To: <20201024145416.4691-1-ludo@gnu.org> ("Ludovic =?utf-8?Q?Cour?= =?utf-8?Q?t=C3=A8s=22's?= message of "Sat, 24 Oct 2020 16:54:16 +0200") Message-ID: <87v9euz807.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44193-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > * guix/scripts/publish.scm (show-help, %options): Add > '--cache-bypass-threshold'. > (low-compression): New procedure. > (cache-bypass-threshold): New parameter. > (bypass-cache?): New procedure. > (render-narinfo/cached): Call 'render-narinfo' when 'bypass-cache?' > returns true. > (render-nar/cached): Call 'render-nar' when 'bypass-cache?' returns > true. > (guix-publish): Parameterize 'cache-bypass-threshold'. > * tests/publish.scm ("with cache", "with cache, lzip + gzip") > ("with cache, uncompressed"): Pass '--cache-bypass-threshold=3D0'. > ("with cache, vanishing item"): Expect 200 for RESPONSE. > ("with cache, cache bypass"): New test. > --- > doc/guix.texi | 24 +++++++++++- > guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++-------- > tests/publish.scm | 43 ++++++++++++++++++-- > 3 files changed, 130 insertions(+), 22 deletions(-) Pushed as ecaa102a58ad3ab0b42e04a3d10d7c761c05ec98. Ludo=E2=80=99. From unknown Fri Sep 05 09:11:28 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 26 Nov 2020 12:24:10 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator