GNU bug report logs - #35880
[PATCH 0/7] Lzip support for 'guix publish' and 'guix substitute'

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#35880: closed ([PATCH 0/7] Lzip support for 'guix publish'
 and 'guix substitute')
Date: Mon, 27 May 2019 20:54:03 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 27 May 2019 22:53:11 +0200
with message-id <87lfyrr554.fsf <at> gnu.org>
and subject line Re: [bug#35880] [PATCH 1/7] lzlib: Add 'make-lzip-input-port/compressed'.
has caused the debbugs.gnu.org bug report #35880,
regarding [PATCH 0/7] Lzip support for 'guix publish' and 'guix substitute'
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
35880: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=35880
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: [PATCH 0/7] Lzip support for 'guix publish' and 'guix substitute'
Date: Fri, 24 May 2019 15:31:59 +0200

Hi!

As a followup to Pierre’s work on (guix lzlib), these patches implement
‘lzip’ support for ‘guix substitute’ and ‘guix publish’.  With these,
you can now run:

  ./pre-inst-env guix publish -Clzip …

on one side, and on another machine:

  ./pre-inst-env guix-daemon --build-users-group=guixbuild

and from there the client machine should be able to fetch
lzip-compressed substitutes.

These patches do not address the transitioning issue that we discussed
earlier, where we have clients lacking lzip support talking to an
lzip-capable server.  As discussed earlier, clients will have to send
a special HTTP header, ‘X-Guix-Accept-Encoding’.

For the server-side, I’m still hesitating between implementing it in ‘guix
publish’ or simply running two instances of ‘guix publish’ side-by-side
(one gzip and one lzip) and letting nginx dispatch between the two.  :-)

Note that we’ll have to adjust our nginx mirror configs to take that
header into account!

Comments?

Ludo’.

Ludovic Courtès (7):
  lzlib: Add 'make-lzip-input-port/compressed'.
  utils: Test 'compressed-port' and 'decompressed-port' for both gzip
    and xz.
  utils: Support compression and decompression with lzip.
  publish: Add support for lzip.
  self: Add dependency on lzlib.
  gnu: guix: Add dependency on lzlib.
  lzlib: 'lzread!' never returns more than it was asked for.

 .dir-locals.el                      |   2 +
 doc/guix.texi                       |  25 +++++--
 gnu/packages/package-management.scm |   1 +
 guix/lzlib.scm                      | 101 ++++++++++++++++++++++------
 guix/scripts/publish.scm            |  84 +++++++++++++++++------
 guix/self.scm                       |  13 +++-
 guix/tests.scm                      |   1 +
 guix/utils.scm                      |  27 ++++++--
 tests/lzlib.scm                     |  10 +++
 tests/publish.scm                   |  36 ++++++++++
 tests/utils.scm                     |  62 +++++++++++------
 11 files changed, 284 insertions(+), 78 deletions(-)

-- 
2.21.0

[Message part 3 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 35880-done <at> debbugs.gnu.org
Subject: Re: [bug#35880] [PATCH 1/7] lzlib: Add
 'make-lzip-input-port/compressed'.
Date: Mon, 27 May 2019 22:53:11 +0200

Hi!

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> That said, if the encoder buffer is not empty, I think lz-compress-read
> should always return something >0.

Yes, probably.  The docstring for ‘lz-compress-read’ says:

    "Read up to COUNT bytes from the encoder stream, storing the results in LZFILE-BV.
  Return the number of uncompressed bytes written, a strictly positive integer."
                                                     ^~~~~~~~~~~~~~~~~

However, the lzlib manual doesn’t say that for ‘LZ_compress_read’ (info
"(lzlib) Compression functions").

But that’s OK: the ‘read!’ method in ‘make-lzip-input-port/compressed’
can just call ‘lzwrite!’ again with more data when that happens, so I’ve
done that.

And I pushed the whole thing! :-)

I think it’d be good to let people play with it in their personal
setups.

Next up: multi-compression support in ‘guix publish’ (possibly?) so we
can smoothly transition on our build farms.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.