Subject: bug#35880: closed (Re: [bug#35880] [PATCH 1/7] lzlib: Add 'make-lzip-input-port/compressed'.)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 27 May 2019 20:54:04 +0000

Your bug report

#35880: [PATCH 0/7] Lzip support for 'guix publish' and 'guix substitute'

which was filed against the guix-patches package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
35880: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=35880
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: [bug#35880] [PATCH 1/7] lzlib: Add 'make-lzip-input-port/compressed'.
From: Ludovic Courtès <[email-masked]>
To: Pierre Neidhardt <[email-masked]>
Cc: [email-masked]
Date: Mon, 27 May 2019 22:53:11 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi!

Pierre Neidhardt <[email-masked]> skribis:

> That said, if the encoder buffer is not empty, I think lz-compress-read
> should always return something >0.

Yes, probably.  The docstring for ‘lz-compress-read’ says:

    "Read up to COUNT bytes from the encoder stream, storing the results in LZFILE-BV.
  Return the number of uncompressed bytes written, a strictly positive integer."
                                                     ^~~~~~~~~~~~~~~~~

However, the lzlib manual doesn’t say that for ‘LZ_compress_read’ (info
"(lzlib) Compression functions").

But that’s OK: the ‘read!’ method in ‘make-lzip-input-port/compressed’
can just call ‘lzwrite!’ again with more data when that happens, so I’ve
done that.

And I pushed the whole thing! :-)

I think it’d be good to let people play with it in their personal
setups.

Next up: multi-compression support in ‘guix publish’ (possibly?) so we
can smoothly transition on our build farms.

Thanks!

Ludo’.

Subject: [PATCH 0/7] Lzip support for 'guix publish' and 'guix substitute'
From: Ludovic Courtès <[email-masked]>
To: [email-masked]
Cc: Ludovic Courtès <[email-masked]>, Pierre Neidhardt <[email-masked]>
Date: Fri, 24 May 2019 15:31:59 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi!

As a followup to Pierre’s work on (guix lzlib), these patches implement
‘lzip’ support for ‘guix substitute’ and ‘guix publish’.  With these,
you can now run:

  ./pre-inst-env guix publish -Clzip …

on one side, and on another machine:

  ./pre-inst-env guix-daemon --build-users-group=guixbuild

and from there the client machine should be able to fetch
lzip-compressed substitutes.

These patches do not address the transitioning issue that we discussed
earlier, where we have clients lacking lzip support talking to an
lzip-capable server.  As discussed earlier, clients will have to send
a special HTTP header, ‘X-Guix-Accept-Encoding’.

For the server-side, I’m still hesitating between implementing it in ‘guix
publish’ or simply running two instances of ‘guix publish’ side-by-side
(one gzip and one lzip) and letting nginx dispatch between the two.  :-)

Note that we’ll have to adjust our nginx mirror configs to take that
header into account!

Comments?

Ludo’.

Ludovic Courtès (7):
  lzlib: Add 'make-lzip-input-port/compressed'.
  utils: Test 'compressed-port' and 'decompressed-port' for both gzip
    and xz.
  utils: Support compression and decompression with lzip.
  publish: Add support for lzip.
  self: Add dependency on lzlib.
  gnu: guix: Add dependency on lzlib.
  lzlib: 'lzread!' never returns more than it was asked for.

 .dir-locals.el                      |   2 +
 doc/guix.texi                       |  25 +++++--
 gnu/packages/package-management.scm |   1 +
 guix/lzlib.scm                      | 101 ++++++++++++++++++++++------
 guix/scripts/publish.scm            |  84 +++++++++++++++++------
 guix/self.scm                       |  13 +++-
 guix/tests.scm                      |   1 +
 guix/utils.scm                      |  27 ++++++--
 tests/lzlib.scm                     |  10 +++
 tests/publish.scm                   |  36 ++++++++++
 tests/utils.scm                     |  62 +++++++++++------
 11 files changed, 284 insertions(+), 78 deletions(-)

-- 
2.21.0