GNU bug report logs - #66305
Error with recursive git checkout

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 66305 in the body.
You can then email your comments to 66305 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: bug-guix <at> gnu.org
Subject: Error with recursive git checkout
Date: Mon, 02 Oct 2023 11:17:16 +0000

[Message part 1 (text/plain, inline)]

Hi.

With Guix at 47d0346553fdad9795c9390a60944ccaad7e5255, I'm unable to
build a package (see attached patch) requiring a recursive git-fetch to
get the sources:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build bladerf
The following derivations will be built:
  /gnu/store/982zz7z94va89fxn79hpjil5wp0v49pn-bladerf-2023.02.drv
  /gnu/store/5rlqf4srlnnymsv93ydxkgxwgfszkszw-bladerf-2023.02-checkout.drv
building /gnu/store/5rlqf4srlnnymsv93ydxkgxwgfszkszw-bladerf-2023.02-checkout.drv...
Initialized empty Git repository in /gnu/store/fhlm9zxs4r4cgapbngckpzrs8rnzf1l2-bladerf-2023.02-checkout/.git/
From https://github.com/Nuand/bladeRF
 * tag               2023.02    -> FETCH_HEAD
Note: switching to 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 41ef634 Revert "libbladeRF: update compatibility for FPGA v0.15.0 from libbladeRF 2.4.0 to 2.5.0"
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: basename: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: sed: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: basename: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: sed: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 292: uname: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: sed: command not found
/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: cmd_: command not found
git-fetch: '/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/bin/git submodule update --init --recursive' failed with exit code 127
--8<---------------cut here---------------end--------------->8---

[0001-gnu-Add-bladerf.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: 66305 <at> debbugs.gnu.org
Subject: Re: Error with recursive git checkout
Date: Mon, 02 Oct 2023 17:37:28 +0000

[Message part 1 (text/plain, inline)]

Workaround: by rebooting the machine to an older generation (and
therefore an older guix-daemon, with Guix at
4f35ff1275e05be31f5d41464ccf147e9dbfd016), the recursive git-fetch
works.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#66305: Error with recursive git checkout
Date: Tue, 03 Oct 2023 21:26:53 +0200

Hi Guillaume,

On Mon, 02 Oct 2023 at 11:17, Guillaume Le Vaillant <glv <at> posteo.net> wrote:

> With Guix at 47d0346553fdad9795c9390a60944ccaad7e5255, I'm unable to
> build a package (see attached patch) requiring a recursive git-fetch to
> get the sources:

[...]

> HEAD is now at 41ef634 Revert "libbladeRF: update compatibility for FPGA v0.15.0 from libbladeRF 2.4.0 to 2.5.0"
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: basename: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: sed: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: basename: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: sed: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 292: uname: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: sed: command not found
> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: cmd_: command not found
> git-fetch: '/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/bin/git submodule update --init --recursive' failed with exit code 127

It looks very similar as bug#65924:

        bug#65924: git searches coreutils and util-linux commands in PATH
        Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
        Wed, 13 Sep 2023 14:00:09 -0400
        id:87fs3iuf6e.fsf <at> gmail.com
        https://issues.guix.gnu.org//65924
        https://issues.guix.gnu.org/msgid/87fs3iuf6e.fsf <at> gmail.com
        https://yhetil.org/guix/87fs3iuf6e.fsf <at> gmail.com

I think it is a corollary from:

        bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
        Ludovic Courtès <ludo <at> gnu.org>
        Tue, 26 Sep 2023 17:44:22 +0200
        id:87fs312b3d.fsf_-_ <at> gnu.org
        https://issues.guix.gnu.org//65866
        https://issues.guix.gnu.org/msgid/87fs312b3d.fsf_-_ <at> gnu.org
        https://yhetil.org/guix/87fs312b3d.fsf_-_ <at> gnu.org

and if I am correct, such potential issue had been pointed by:

        Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
        Simon Tournier <zimon.toutoune <at> gmail.com>
        Thu, 14 Sep 2023 19:28:06 +0200
        id:CAJ3okZ0hzimVNtTcSsJKR-x=WKpPVtHYxshznGzecqxNHFWC5Q <at> mail.gmail.com
        https://lists.gnu.org/archive/html/guix-devel/2023-09
        https://yhetil.org/guix/CAJ3okZ0hzimVNtTcSsJKR-x=WKpPVtHYxshznGzecqxNHFWC5Q <at> mail.gmail.com

Cheers,
simon

Message #20 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Wed, 04 Oct 2023 18:25:56 +0200

[Message part 1 (text/plain, inline)]

Hello,

Simon Tournier <zimon.toutoune <at> gmail.com> skribis:

> On Mon, 02 Oct 2023 at 11:17, Guillaume Le Vaillant <glv <at> posteo.net> wrote:
>
>> With Guix at 47d0346553fdad9795c9390a60944ccaad7e5255, I'm unable to
>> build a package (see attached patch) requiring a recursive git-fetch to
>> get the sources:
>
> [...]
>
>> HEAD is now at 41ef634 Revert "libbladeRF: update compatibility for FPGA v0.15.0 from libbladeRF 2.4.0 to 2.5.0"
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: basename: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 7: sed: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: basename: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 77: sed: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-sh-setup: line 292: uname: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: sed: command not found
>> /gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/libexec/git-core/git-submodule: line 613: cmd_: command not found
>> git-fetch: '/gnu/store/y3vdq2pdkljrw63xxnc2vb6lz07ycar6-git-minimal-2.41.0/bin/git submodule update --init --recursive' failed with exit code 127
>
> It looks very similar as bug#65924:
>
>         bug#65924: git searches coreutils and util-linux commands in PATH
>         Maxim Cournoyer <maxim.cournoyer <at> gmail.com>

D’oh!  Thanks Simon and Guillaume for the heads-up (I had totally
overlooked that you raised this during the review, Simon; my bad).

Here’s a patch that fixes it for me:

[Message part 2 (text/x-patch, inline)]

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index 045dd84ad6..c869f19502 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -108,6 +108,10 @@ (define* (perform-git-download drv output
            (drv-output (assoc-ref (derivation-outputs drv) "out"))
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
+      ;; Commands such as 'git submodule' expect Coreutils and sed (among
+      ;; others) to be in $PATH.
+      (setenv "PATH" "/run/current-system/profile/bin:/bin:/usr/bin")
+
       (git-fetch-with-fallback url commit output
                                #:recursive? recursive?
                                #:git-command %git))))

[Message part 3 (text/plain, inline)]

To test it, you need to run the daemon from your checkout, with
something like:

  sudo herd stop guix-daemon
  sudo -E ./pre-inst-env guix-daemon --build-users-group=guixbuild &

(Once we’ve applied the fix, we’ll have to update the ‘guix’ package.)

I think we should eventually our ‘git’ package as discussed in
<https://issues.guix.gnu.org/65924>, but that won’t help on foreign
distros anyway, hence the fix above.

WDYT?

Ludo’.

Message #23 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Wed, 04 Oct 2023 20:16:06 +0200

Hi Ludo,

On Wed, 04 Oct 2023 at 18:25, Ludovic Courtès <ludo <at> gnu.org> wrote:

> diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
> index 045dd84ad6..c869f19502 100644
> --- a/guix/scripts/perform-download.scm
> +++ b/guix/scripts/perform-download.scm
> @@ -108,6 +108,10 @@ (define* (perform-git-download drv output
>             (drv-output (assoc-ref (derivation-outputs drv) "out"))
>             (algo       (derivation-output-hash-algo drv-output))
>             (hash       (derivation-output-hash drv-output)))
> +      ;; Commands such as 'git submodule' expect Coreutils and sed (among
> +      ;; others) to be in $PATH.
> +      (setenv "PATH" "/run/current-system/profile/bin:/bin:/usr/bin")
> +
>        (git-fetch-with-fallback url commit output
>                                 #:recursive? recursive?
>                                 #:git-command %git))))

LGTM.

Well, I would add a comment explicitly mentioning that’s a temporary fix
pointing this issue #66305.

And I have tested with:

--8<---------------cut here---------------start------------->8---
$ guix build  ocaml-ansiterminal -S --no-substitutes
$ guix build volk --no-substitutes --check -S
--8<---------------cut here---------------end--------------->8---


Cheers,
simon

Message #26 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Wed, 4 Oct 2023 20:18:10 +0200

On Wed, 4 Oct 2023 at 20:16, Simon Tournier <zimon.toutoune <at> gmail.com> wrote:

> And I have tested with:

On foreign distro I mean :-)

Message #29 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Thu, 05 Oct 2023 11:28:58 +0200

Hello,

Simon Tournier <zimon.toutoune <at> gmail.com> skribis:

> On Wed, 04 Oct 2023 at 18:25, Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
>> index 045dd84ad6..c869f19502 100644
>> --- a/guix/scripts/perform-download.scm
>> +++ b/guix/scripts/perform-download.scm
>> @@ -108,6 +108,10 @@ (define* (perform-git-download drv output
>>             (drv-output (assoc-ref (derivation-outputs drv) "out"))
>>             (algo       (derivation-output-hash-algo drv-output))
>>             (hash       (derivation-output-hash drv-output)))
>> +      ;; Commands such as 'git submodule' expect Coreutils and sed (among
>> +      ;; others) to be in $PATH.
>> +      (setenv "PATH" "/run/current-system/profile/bin:/bin:/usr/bin")
>> +
>>        (git-fetch-with-fallback url commit output
>>                                 #:recursive? recursive?
>>                                 #:git-command %git))))
>
> LGTM.
>
> Well, I would add a comment explicitly mentioning that’s a temporary fix
> pointing this issue #66305.

It’s not temporary in that it will still be needed for example by the
Debian package of Guix.  But yeah, I’ll clarify that in the comment.

> And I have tested with:
>
> $ guix build  ocaml-ansiterminal -S --no-substitutes
> $ guix build volk --no-substitutes --check -S

Awesome, thanks for checking!

Ludo’.

Message #32 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305 <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Thu, 05 Oct 2023 23:31:45 +0200

Hi!

Pushed as 762fdbdef52b4c17df578478cadc8655d56171a4.

Now to update ‘guix’…

Ludo’.

Message #37 received at 66305-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 66305-done <at> debbugs.gnu.org
Subject: Re: bug#66305: Error with recursive git checkout
Date: Thu, 12 Oct 2023 16:42:58 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Pushed as 762fdbdef52b4c17df578478cadc8655d56171a4.
>
> Now to update ‘guix’…

Done last week in 16fd9d6e3d626fc624c38cb3096331905a4161e4.

Closing!

Message #40 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Alexis Simon <alexis.simon <at> runbox.com>
To: 66305 <at> debbugs.gnu.org
Subject: Error with recursive git checkout
Date: Tue, 24 Oct 2023 21:53:35 -0700

Hi,

I think I'm hitting this bug trying to git-fetch with submodules.
I am on a foreign distro.

--8<---------------cut here---------------start------------->8---
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-submodule: 
line 7: /bin/basename: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-submodule: 
line 7: /bin/sed: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-sh-setup: 
line 77: /bin/basename: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-sh-setup: 
line 77: /bin/sed: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-sh-setup: 
line 292: /bin/uname: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-submodule: 
line 613: /bin/sed: Permission denied
/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/libexec/git-core/git-submodule: 
line 613 : cmd_: command not found
git-fetch: 
'/gnu/store/pmv37cxc4cg1s7x8yg8dkhikkwmwpncr-git-minimal-2.41.0/bin/git 
submodule update --init --recursive' failed with exit code 127
--8<---------------cut here---------------end--------------->8---

This is with a recent guix I think
--8<---------------cut here---------------start------------->8---
❯ guix describe
Generation 8    oct. 24 2023 21:31:58   (current)
  guix 0074731
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 00747316ee0e1a7962ffe226c727776ba7a8163b
--8<---------------cut here---------------end--------------->8---

Alexis

Message #43 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Alexis Simon <alexis.simon <at> runbox.com>
To: 66305 <at> debbugs.gnu.org
Subject: Error with recursive git checkout
Date: Wed, 25 Oct 2023 09:51:37 -0700

ah well it seems this is due to a selinux policy error

--8<---------------cut here---------------start------------->8---
SELinux is preventing git-submodule from execute access on the file 
/usr/bin/sed.

*****  Plugin catchall (100. confidence) suggests 
**************************

If you believe that git-submodule should be allowed execute access on 
the sed file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'git-submodule' --raw | audit2allow -M my-gitsubmodule
# semodule -X 300 -i my-gitsubmodule.pp


Additional Information:
Source Context                system_u:system_r:guix_daemon.guix_daemon_t:s0
Target Context                system_u:object_r:bin_t:s0
Target Objects                /usr/bin/sed [ file ]
Source                        git-submodule
Source Path                   git-submodule
Port                          <Unknown>
Host                          xps13
Source RPM Packages
Target RPM Packages           sed-4.8-12.fc38.x86_64
SELinux Policy RPM            selinux-policy-targeted-38.29-1.fc38.noarch
Local Policy RPM
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     xps13
Platform                      Linux xps13 6.5.7-200.fc38.x86_64 #1 SMP
                              PREEMPT_DYNAMIC Wed Oct 11 04:07:58 UTC 2023
                              x86_64
Alert Count                   460
First Seen                    2023-10-24 20:20:26 PDT
Last Seen                     2023-10-25 09:44:31 PDT
Local ID                      fa57086c-6738-4eec-8252-3abb66a9e249

Raw Audit Messages
type=AVC msg=audit(1698252271.150:513): avc:  denied  { execute } for 
pid=10644 comm="git-submodule" name="sed" dev="dm-0" ino=261979 
scontext=system_u:system_r:guix_daemon.guix_daemon_t:s0 
tcontext=system_u:object_r:bin_t:s0 tclass=file permissive=0


Hash: git-submodule,guix_daemon.guix_daemon_t,bin_t,file,execute
--8<---------------cut here---------------end--------------->8---

But trying to fix it does not seem to have any effect. I've added this 
to the guix-daemon.cil and re-applied
--8<---------------cut here---------------start------------->8---
(allow guix_daemon_t
       bin_t
       (file (execute)))
--8<---------------cut here---------------end--------------->8---

Alexis

Message #46 received at 66305 <at> debbugs.gnu.org (full text, mbox):

From: Alexis Simon <alexis.simon <at> runbox.com>
To: 66305 <at> debbugs.gnu.org
Subject: Error with recursive git checkout
Date: Wed, 25 Oct 2023 10:47:04 -0700

This is what was needed in the selinux policy to fix the errors

--8<---------------cut here---------------start------------->8---
(allow guix_daemon_t
       bin_t
       (file (execute execute_no_trans map)))
--8<---------------cut here---------------end--------------->8---

Alexis

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.