Subject: Re: [bug#60358] [PATCH] gnu: Add gnulib.

From: Simon Josefsson <[email-masked]>
Date: Thu, 29 Dec 2022 15:44:12 +0100

Vivien Kraus via Guix-patches via <[email-masked]> writes:

> Hello!
>
> Le mardi 27 décembre 2022 à 19:44 +0100, Simon Josefsson a écrit :
>> Hi.  Many packages needs a specific checkout of gnulib to work
>> reliably,
>> via --gnulib-refdir= (rather than --gnulib-srcdir=), would you
>> consider
>> installing the entire gnulib git archive instead of just the latest
>> checkout?  
>
> I did not know about gnulib-refdir.

It is newer than --gnulib-srcdir, but came about because --gnulib-srcdir
is often a fragile solution: you have no idea which gnulib version the
person building the package supplied.  Since gnulib is rolling,
compatibility becomes difficult.  The --gnulib-srcdir approach works if
you make sure to use the same gnulib git checkout as the project you
wish to build uses.  But then it becomes difficult to package gnulib: no
two projects are likely to rely on the same gnulib git commit.  So which
gnulib git version to package?

That's the motivation for packaging the gnulib git repository instead.
This may sound strange, but compare how gettext/autopoint ships a CVS
repository and checks out the particular files that are needed.

Yeah, I can agree that this approach is not ideal, and there are many
concerns with it.  I'm not convinced gnulib's idea of "source-level
library" is something that is viable long-term.  But it is what exists
today.

> Providing the entire gnulib archive is tempting, but there might be
> downsides. The contents of the .git directory depends on how trees and
> objects have been packed by git. As far as I understand, it is
> possible that pushing a commit in gnulib results in commit objects
> from previous commits to be re-packed. Thus, if I clone gnulib from
> the initial commit to a specific commit, the result might depend on
> other, unrelated commits. So, the content of .git is not reproducible.
>
> Maybe I could get around that by deleting all the refs, doing an
> aggressive garbage collection and then re-packing, but I’m not sure I
> would get a reproducible result.

Interesting -- I think researching this more would be useful.  It should
be possible to come up with a safe approach to produce a reproducible
checkout of a git repository.

Doesn't 'git archive' produce a reproducible output from a git
repository for a particular branch and commit?

> Now, I don’t know much about gnulib-refdir. How does gnulib-refdir
> work?

You supply --gnulib-refdir pointing to a local gnulib git repository
clone when you run ./bootstrap.  This avoid checking out the gnulib git
submodule from Savannah, and instead ./bootstrap will use the local git
repository instead.  I believe it should automatically extract the
intended gnulib git commit from the gnulib/ git submodule, and extract
that version from the local copy (please test -- may be bugs).

> Do you have an example of a package that uses this feature?

Packages wouldn't use it, but if they use gnulib's ./bootstrap script
the support this approach.  The idea is that people building projects
that use gnulib doesn't always have to fetch the gnulib git submodule,
but have a local copy for security or performance reasons.

> Can I convince the package to use a checkout instead of the gnulib
> repository, if I swear I have the exact checkout it wants, maybe by
> tweaking bootstrap.conf or something?

You can force a package to use another gnulib version by using
--gnulib-srcdir and point that to some other version of gnulib.  But the
project may not build.  The point of --gnulib-refdir is to actually get
the same version of gnulib that the project uses, why would you not want
that?  There is the GNULIB_REVISION environment variable that you can
set to something else, but this is probably not very well tested and
sounds like a bad idea (but I may be missing something).

/Simon