GNU bug report logs - #41602
texlive is actually substitutable

Previous Next

Full log

Message #24 received at 41602 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Morgan Arnold <morgan.arnold <at> proton.me>, 41602 <at> debbugs.gnu.org,
 Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#41602: texlive is actually substitutable
Date: Wed, 19 Feb 2025 01:08:55 +0900

Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hello,
>
> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Hi,
>>
>> Leo Famulari <leo <at> famulari.name> skribis:
>>
>>> On Fri, May 29, 2020 at 05:15:40PM +0200, Ludovic Courtès wrote:
>>>> The info suggests it won’t be substituted, but it’s eventually
>>>> substituted.  I wonder why, because the .drv has:
>>>> 
>>>>   ("allowSubstitutes","0")
>>>> 
>>>> and the daemon has:
>>>> 
>>>>   bool substitutesAllowed(const Derivation & drv)
>>>>   {
>>>>       return get(drv.env, "allowSubstitutes", "1") == "1";
>>>>   }
>>>> 
>>>> and:
>>>> 
>>>>   if (settings.useSubstitutes && substitutesAllowed(drv))
>>>>       foreach (PathSet::iterator, i, invalidOutputs)
>>>>           addWaitee(worker.makeSubstitutionGoal(*i, buildMode == bmRepair));
>>>> 
>>>> Thoughts?

It's odd, when using --dry-run, it seems the expected situation would
happen:

--8<---------------cut here---------------start------------->8---
$ guix build --no-grafts -n texlive
guix build --no-grafts -n texlive
La dérivation suivante serait compilée :
  /gnu/store/v10c5wzji81pkwq2fhj123gw3d8il0ic-texlivetexmf-20240312.drv
4 270,0 Mo seraient téléchargés :
  /gnu/store/pb6z5d5fx6s13cjzmvlw7dykpafp9x97-texlive-20240312-texmf.tar.xz
  /gnu/store/0gkx9q1kyys8cis93cw9qhlzyx584dr6-texlive-20240312
--8<---------------cut here---------------end--------------->8---

Only the private texlivetexmf is marked as non-substitutable, so the
above looks correct (though we see the intention of preserving bandwidth
wouldn't be achieved, given the source is as large as the final
package).

When removing the --dry-run, what happens though is that the output of
the texlivetexmf package, which is marked as non-substitutable, is downloaded:

--8<---------------cut here---------------start------------->8---
$ guix build --no-grafts texlive
La dérivation suivante sera compilée :
  /gnu/store/v10c5wzji81pkwq2fhj123gw3d8il0ic-texlivetexmf-20240312.drv
4 270,0 Mo seront téléchargés :
  /gnu/store/pb6z5d5fx6s13cjzmvlw7dykpafp9x97-texlive-20240312-texmf.tar.xz
  /gnu/store/0gkx9q1kyys8cis93cw9qhlzyx584dr6-texlive-20240312
substitution de /gnu/store/b5sn5ha961hab37r7vl5p2n6sf46x582-texlivetexmf-20240312...
téléchargement depuis https://bordeaux.guix.gnu.org/nar/lzip/b5sn5ha961hab37r7vl5p2n6sf46x582-texlivetexmf-20240312...
 texlivetexmf-20240312  3.95GiB                                     2.8MiB/s 00:06 ▕                  ▏   0.4%  C-c C-c^C
--8<---------------cut here---------------end--------------->8---

I've had some success as stopping in the daemon right before the downlod
starts, with this sequence:

Start the locally built guix-daemon in gdb, with a command like:

--8<---------------cut here---------------start------------->8---
sudo -E ./pre-inst-env gdb --args ./guix-daemon --debug --build-users-group=guixbuild --max-silent-time 3600 --timeout 86400 --log-compression none --discover=no --substitute-urls='https://bordeaux.guix.gnu.org https://ci.guix.gnu.org'
--8<---------------cut here---------------end--------------->8---

Then, to be able to reach the point in the execution where the downloads
are about to start, we need to break in the first fork (but not the 2nd,
which is the guile substituter), like follows at the GDB prompt:

--8<---------------cut here---------------start------------->8---
set follow-fork-mode child

# To break inside the first child process
break LocalStore::querySubstitutablePathInfos

run

## from a terminal, 'guix build texlive' to hit breakpoint

delete 1

# avoid entering the guile substituter fork
set follow-fork-mode parent  

break LocalStore::buildPaths

continue

-> should break in
Thread 2.1 "guix-daemon" hit Breakpoint 2.1, nix::LocalStore::buildPaths
(this=0x51f4d0, drvPaths=std::set with 1 element = {...},
buildMode=nix::bmNormal) at nix/libstore/build.cc:3627

Thread 2.1 "guix-daemon" hit Breakpoint 2.1, nix::LocalStore::buildPaths (this=0x51f4d0, drvPaths=std::set with 1 element = {...}, buildMode=nix::bmNormal) at nix/libstore/build.cc:3627
3627	{
(gdb) bt
#0  nix::LocalStore::buildPaths (this=0x51f4d0, drvPaths=std::set with 1 element = {...}, buildMode=nix::bmNormal) at nix/libstore/build.cc:3627
#1  0x00000000004253b6 in performOp (from=..., to=..., op=<optimized out>, clientVersion=356, trusted=false) at nix/nix-daemon/nix-daemon.cc:481
#2  processConnection (trusted=<optimized out>, userId=<optimized out>) at nix/nix-daemon/nix-daemon.cc:841
#3  0x000000000042805f in operator() (__closure=0x4e8f50) at nix/nix-daemon/nix-daemon.cc:1003
#4  0x00000000004280ac in std::__invoke_impl<void, acceptConnection(int)::<lambda()>&> (__f=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/invoke.h:61
#5  std::__invoke_r<void, acceptConnection(int)::<lambda()>&> (__fn=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/invoke.h:154
#6  std::_Function_handler<void(), acceptConnection(int)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/std_function.h:290
#7  0x0000000000495dbb in std::function<void()>::operator() (this=0x7fffffffbb00) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/std_function.h:590
#8  nix::startProcess (fun=..., dieWithParent=dieWithParent <at> entry=false, errorPrefix="unexpected build daemon error: ", runExitHandlers=runExitHandlers <at> entry=true) at nix/libutil/util.cc:1025
#9  0x00000000004237c2 in acceptConnection (fdSocket=<optimized out>) at nix/nix-daemon/nix-daemon.cc:977
#10 daemonLoop (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1055
#11 0x00000000004241b9 in run (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1064
#12 0x0000000000420635 in main (argc=<optimized out>, argv=<optimized out>) at nix/nix-daemon/guix-daemon.cc:569
--8<---------------cut here---------------end--------------->8---

I haven't investigated where in this execution path the
substitutesAllowed procedure is called, if at all, but this is what I
will try to understand next.

-- 
Thanks,
Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.