Subject: bug#56398: (guix git) fails to check out repos with nested submodules

From: Ludovic Courtès <[email-masked]>
Date: Fri, 08 Jul 2022 10:26:40 +0200

Hi!

André Batista <[email-masked]> skribis:

> Hi!
>
> ter 05 jul 2022 às 17:02:31 (1657051351), [email-masked] enviou:
>> It seems that ‘update-cached-checkout’ fails to handle nested Git
>> submodules, when a submodule itself has submodules:
>
> I think this may be actually a bug upstream. When refreshing, guix is
> able to recursively upgrade submodules under 'thirdparty/fbgemm' just
> fine, but it fails on thirdparty/gloo because the path declared on
> its .gitmodules does not exist on the repository:

Oh indeed.  The bug probably doesn’t have to do with nested submodules;
it manifests when checking out that gloo repo:

--8<---------------cut here---------------start------------->8---
scheme@(guix git)> (update-cached-checkout "https://github.com/facebookincubator/gloo" #:recursive? #t)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Git error: failed to resolve path '/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest/.git': No such file or directory

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
--8<---------------cut here---------------end--------------->8---

If we do this:

--8<---------------cut here---------------start------------->8---
scheme@(guix git)> (define r (repository-open "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/"))
scheme@(guix git)> (define mod (submodule-lookup r "third-party/googletest"))
scheme@(guix git)> mod
$13 = #<git-submodule 17d1220>
scheme@(guix git)> (submodule-update $13)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Git error: failed to resolve path '/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest/.git': No such file or directory

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
--8<---------------cut here---------------end--------------->8---

… the ‘submodule-update’ bit looks like this:

--8<---------------cut here---------------start------------->8---
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", {st_mode=S_IFREG|0644, st_size=116, ...}, 0) = 0
access("/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", F_OK) = 0
access("/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", R_OK) = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", {st_mode=S_IFREG|0644, st_size=116, ...}, 0) = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", {st_mode=S_IFREG|0644, st_size=116, ...}, 0) = 0
openat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", O_RDONLY|O_CLOEXEC) = 15
read(15, "[submodule \"third-party/googletest\"]\n\tpath = third-party/googletest\n\turl = https://github.com/google/googletest.git\n", 116) = 116
close(15)                               = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.gitmodules", {st_mode=S_IFREG|0644, st_size=116, ...}, 0) = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest", 0x7ffdd31497f0, 0) = -1 ENOENT (No such file or directory)
access("/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest/.git", F_OK) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/HEAD", {st_mode=S_IFREG|0644, st_size=21, ...}, 0) = 0
openat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/HEAD", O_RDONLY|O_CLOEXEC) = 15
read(15, "ref: refs/heads/main\n", 21)  = 21
close(15)                               = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/refs/heads/main", {st_mode=S_IFREG|0644, st_size=41, ...}, 0) = 0
openat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/refs/heads/main", O_RDONLY|O_CLOEXEC) = 15
read(15, "950c0e23819779a9e0c70b861db4c52b31d1d1b2\n", 41) = 41
close(15)                               = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/HEAD", {st_mode=S_IFREG|0644, st_size=21, ...}, 0) = 0
openat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/HEAD", O_RDONLY|O_CLOEXEC) = 15
read(15, "ref: refs/heads/main\n", 21)  = 21
close(15)                               = 0
newfstatat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/refs/heads/main", {st_mode=S_IFREG|0644, st_size=41, ...}, 0) = 0
openat(AT_FDCWD, "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/.git/refs/heads/main", O_RDONLY|O_CLOEXEC) = 15
read(15, "950c0e23819779a9e0c70b861db4c52b31d1d1b2\n", 41) = 41
close(15)                               = 0
--8<---------------cut here---------------end--------------->8---

Thus, looking at ‘git_submodule_update’ in libgit2, it looks as if this
condition was true:

  (submodule_status & GIT_SUBMODULE_STATUS_WD_UNINITIALIZED)

Hmm, thoughts?

Ludo’.