GNU bug report logs - #38100
'--with-input', '--with-git-url' etc. cause unnecessary rebuilds

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: pkill9 <pkill9 <at> runbox.com>
Subject: bug#42156: closed (Re: bug#38100: ‘--with-input’
 causes unintended rebuilds)
Date: Sun, 27 Sep 2020 21:47:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#38100: '--with-input', '--with-git-url' etc. cause unnecessary rebuilds

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 42156 <at> debbugs.gnu.org.

-- 
38100: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=38100
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: 38100-done <at> debbugs.gnu.org
Subject: Re: bug#38100: ‘--with-input’ causes
 unintended rebuilds
Date: Sun, 27 Sep 2020 23:46:22 +0200

Hey there!

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:
>
>> Indeed, evaluating:
>>
>>   (bag-transitive-inputs
>>    (package->bag ((package-input-rewriting '()) glib)))
>>
>> shows that we have two “python” packages there that are not ‘eq?’.
>
> The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
> ‘python-build-system’ and thus has ‘python’ as an implicit input.
>
> ‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
> that implicit ‘python’ untouched.
>
> Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
> equality, we end up with two “python” packages that are not ‘eq?’ but
> are functionally equivalent: the one produced by
> ‘package-input-rewriting’, and the implicit dependency of
> ‘python-libxml2’.  QED.
>
> (This is essentially the same as <https://bugs.gnu.org/30155>.)

Good news, this is fixed by 2bf6f962b91123b0474c0f7123cd17efe7f09a66,
which introduces package rewriting including implicit inputs!

Before getting there, this issue did get on my nerves for a while.  Here
are several ways to address this issue that I thought of:

  1. Have ‘package-input-rewriting/spec’ traverse implicit inputs, at
     least optionally.  We wouldn’t end up with an
     equivalent-but-not-eq? ‘python’ in the example above.  It does
     change the semantics though, and it may be nice to keep a “shallow”
     replacement option.  That’s what
     2bf6f962b91123b0474c0f7123cd17efe7f09a66 does.

  2. Do (delete-duplicates input-drvs) in ‘bag->derivation’.  That seems
     wise, but it’s unfortunately impossible on ‘master’ because of
     <https://issues.guix.gnu.org/43508>.

  3. ‘package-input-rewriting/spec’ preserves eq?-ness for packages not
     transformed; in the example above, the transformation result would
     be eq? to ‘glib’ because ‘--with-input=libreoffice=inkscape’ had no
     effect.  Tricky to implement efficiently, perhaps not worth it.

I think #2 might still be worth investigating, but it may have
undesirable implications too.  #3 is hardly doable.

All in all, I’m glad that #1 addresses the issue, because it’s also
something we wanted anyway.

Ludo’.

[Message part 3 (message/rfc822, inline)]

From: pkill9 <pkill9 <at> runbox.com>
To: bug-guix <at> gnu.org
Subject: Guix unnecessarily rebuilds packages when replacing a source with
 --with-git-url
Date: Wed, 1 Jul 2020 18:04:12 +0100

Running `guix build --with-git-url=youtube-dl=https://github.com/ytdl-org/youtube-dl mpv -n`
shows it will rebuild packages unecessarily:

```
The following derivations would be built:
   /gnu/store/3qhfgkacylwsvzxgz5fqdgxlx812c7nf-mpv-0.32.0.drv
   /gnu/store/2240y6fjiq2riiq1rj4apci5s7sl7rk6-youtube-dl-2020.06.16.1.drv
   /gnu/store/94dqd7xiy0sdmhzxikqrycllm921fcm5-libcaca-0.99.beta19.drv
   /gnu/store/r0p4y6b78x2lfdh51p8287al3i461pfr-ftgl-2.4.0.drv
   /gnu/store/an06c6fg9w0ja44sbmnssryklziyjsia-mpg123-1.26.1.drv
   /gnu/store/ddkwx46ksizbz59fc5xwfxb7bq5j246h-ffmpeg-4.3.drv
   /gnu/store/h28qpl97rs3fc2rc2n2vxcpy3pyxnbm8-frei0r-plugins-1.7.0.drv
   /gnu/store/nyk555wa6kjqz311ini6gji21vl9j6lz-sdl2-2.0.12.drv
   /gnu/store/5mh046xgl3qzdkmccvmi3g08cl2zh4g0-fcitx-4.2.9.7.drv
   /gnu/store/jxqzag3jb5jbj5z60md011gi4r6g97js-enchant-1.6.0.drv
   /gnu/store/mi4ris95502ixx7yybp8wxay4g53q0zn-ibus-1.5.22.drv
   /gnu/store/3vhw2hspgvskfq26zsk2iihjlavp16id-dconf-0.34.0.drv
   /gnu/store/aybh2nafr8v5k8w1ld28s1hzvg3mfjkg-libnotify-0.7.7.drv
   /gnu/store/jzxixdnm7cbfajfx901mypi1acidjj0i-orbit2-2.14.19.drv
   /gnu/store/cwm9rdd3c1l6j2x3gdkc6s8dxaba9hfb-libidl-0.8.14.drv
   /gnu/store/rams8llip6qlhw1w83sl7xmamy52fr68-gconf-3.2.6.drv
   /gnu/store/3s2isd4z1qzyansngg67ib0a5hswvjs0-dbus-glib-0.110.drv
   /gnu/store/pv6a3vfay83kzcszl0rcnv0d5v1v1cd7-openal-1.20.1.drv
   /gnu/store/ynpm7jd13yyciz2lpyr6q9rh2kx0g530-libass-0.14.0.drv
   /gnu/store/yyc2cqz28nh75h1zgvjsza2w1nkxn8ai-libva-2.7.1.drv
   /gnu/store/gh57iyp68zjs0xfwfcfp6a3jsshpgp2j-rsound-1.1.drv
   /gnu/store/hj1s2riv7zvyvmw1hbnzzkv2gfa0ncqn-ao-1.2.2-5-g20dc8ed.drv
   /gnu/store/k0g5pp30s1m32yii5fk9rbiy2h3jyjhy-vulkan-loader-1.2.140.drv
```

It can be seen that it's unnecessary by running `guix graph --path
youtube-dl ffmpeg` and it shows youtube-dl isn't in the package graph
for ffmpeg, yet it's still rebuilding it.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.