GNU bug report logs - #50040
[PATCH 0/2] publish: Always render nar/narinfo during backing.

Previous Next

Full log

View this message in rfc822 format

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: 50040 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [bug#50040] [PATCH 0/2] publish: Always render nar/narinfo during backing.
Date: Fri, 13 Aug 2021 12:28:00 +0200

Hello,

This patchset does two things:

* It ensures that the narinfo creation is always done in a dedicated thread to
  prevent blocking the main publish thread and cause the timeout connection
  errors that some users experimented.

* It removes the cache bypass support and always renders temporary narinfo and
  nar files during baking. This way, the first user asking for a substitute
  won't get a 404 error. It also removes the need for Cuirass to pre-bake nar
  files. Finally it will make way easier to use a publish server with caching
  in a CI context.

The downside is that it makes the publish server code harder to understand and
that it consumes more server resources by duplicating the nar/narinfo file
creation.

In the long run, spawning new threads constantly won't scale. I think it would
be nice to rewrite the publish server using Fiber or an HTTP server framework.

Thanks,

Mathieu

Mathieu Othacehe (2):
  publish: Defer narinfo string creation to the http-write.
  publish: Remove cache bypass support.

 doc/guix.texi            |  15 -----
 gnu/services/base.scm    |  11 +---
 guix/scripts/publish.scm | 136 +++++++++++++++++++++++----------------
 tests/publish.scm        |  88 +++----------------------
 4 files changed, 93 insertions(+), 157 deletions(-)

-- 
2.32.0

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.