GNU bug report logs - #46424
ASDF build system sometimes fails to load .asd files properly

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46424 in the body.
You can then email your comments to 46424 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: bug-guix <at> gnu.org
Cc: Guillaume Le Vaillant <glv <at> posteo.net>
Subject: ASDF build system sometimes fails to load .asd files properly
Date: Wed, 10 Feb 2021 17:36:49 +0100

[Message part 1 (text/plain, inline)]

Yesterday I pushed 3aba721da73fbdc3382cc098c41596d5cfbb29eb
"gnu: sbcl-quri: Update to 20200209", which fixes the tests for sbcl-quri.

I noted:

--8<---------------cut here---------------start------------->8---
       ;; Test system must be loaded before, otherwise tests fail with:
       ;; Component QURI-ASD::QURI-TEST not found, required by #<SYSTEM
       ;; "quri">.
       '(#:asd-systems '("quri-test" "quri")))
--8<---------------cut here---------------end--------------->8---

Our build system should be able to do better, we should not even have to
specify the asd-systems here.  After all, ASDF is smart enough.

Note that:

- If we don't specify the systems, the tests are still run and
  succeed(!), it's only the phase that fails with the aforementioned
  error.

- If we specify the systems in the reverse order, it still fails!

We've got this issue for many SBCL packages.

Any clue?  Guillaume?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: bug-guix <at> gnu.org
Subject: Re: ASDF build system sometimes fails to load .asd files properly
Date: Fri, 12 Feb 2021 11:25:48 +0100

[Message part 1 (text/plain, inline)]

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Yesterday I pushed 3aba721da73fbdc3382cc098c41596d5cfbb29eb
> "gnu: sbcl-quri: Update to 20200209", which fixes the tests for sbcl-quri.
>
> I noted:
>
> --8<---------------cut here---------------start------------->8---
>        ;; Test system must be loaded before, otherwise tests fail with:
>        ;; Component QURI-ASD::QURI-TEST not found, required by #<SYSTEM
>        ;; "quri">.
>        '(#:asd-systems '("quri-test" "quri")))
> --8<---------------cut here---------------end--------------->8---
>
> Our build system should be able to do better, we should not even have to
> specify the asd-systems here.  After all, ASDF is smart enough.
>
> Note that:
>
> - If we don't specify the systems, the tests are still run and
>   succeed(!), it's only the phase that fails with the aforementioned
>   error.
>
> - If we specify the systems in the reverse order, it still fails!
>
> We've got this issue for many SBCL packages.
>
> Any clue?  Guillaume?

Currently we are using the 'asdf:load-asd' function to load system
definitions (see the 'build' function in
'guix/build/asdf-build-system.scm' and the 'compile-systems' function in
'guix/build/lisp-utils.scm').

This function can only load one '.asd' file at a time, which is the
cause of the issue. If we find how ASDF does the "lazy-load" of all the
'.asd' files in a directory tree, and if we use that instead of
'load-asd', I think it would solve the issue and we could get rid of the
'asd-files' parameter in Guix package definitions.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 46424 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 46424 <at> debbugs.gnu.org
Subject: Use load-systems or load-systems*
Date: Wed, 5 May 2021 20:55:21 +0000

Hi I've just checked source of ASDF and it looks like there is a
native option to load multiple systems at once

> --8<---------------cut here---------------start------------->8--->
  (defun load-systems* (systems &rest keys)
    "Loading multiple systems at once."
    (dolist (s systems) (apply 'load-system s keys)))

  (defun load-systems (&rest systems)
    "Loading multiple systems at once."
    (load-systems* systems))
--8<---------------cut here---------------end--------------->8---

https://gitlab.common-lisp.net/asdf/asdf/-/blob/master/operate.lisp#L155

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

Message #14 received at 46424 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: 46424 <at> debbugs.gnu.org
Cc: Pierre Neidhardt <mail <at> ambrevar.xyz>,
 Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: Re: bug#46424: Use load-systems or load-systems*
Date: Sat, 08 May 2021 09:37:58 +0000

[Message part 1 (text/plain, inline)]

In the case of quri, it looks like the problem comes from the
"(asdf:clear-system c)" call at the end of "quri-test.asd". When
removing it, the build succeeds even without the custom 'asd-systems'
argument in the Guix package definition.

So it looks like the lazy loading of system definitions is working in
asdf-build-system after all. I guess it's the default behaviour of
'load-asd'.

However I don't know why this 'clear-system' call messes up the ASDF
configuration in asdf-build-system, but it doesn't seem to cause
problems in Quicklisp...

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 46424 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>, 46424 <at> debbugs.gnu.org
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: Re: bug#46424: Use load-systems or load-systems*
Date: Sat, 08 May 2021 12:00:25 +0200

[Message part 1 (text/plain, inline)]

Interesting.
This reminds me of a puzzling issue I recently had while working on the
Nyxt .asd:

- (asdf:load-asd "/path/to/nyxt.asd")
- (asdf:make :nyxt/quicklisp)

The nyxt/quicklisp system would call ql:update-dist and then would fail,
saying it cannot find "nyxt/quicklisp" (which it is running!)
because ql:update-dist would lead ASDF to somehow forget about the .asd.

Adding (asdf:load-asd "/path/to/nyxt.asd") right after ql:update-dist
fixes it.

I've reworked the whole thing different on Nyxt's master branch, so this
is no longer an issue.

Maybe there is a hint lurking in there...

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 46424-done <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: 46424-done <at> debbugs.gnu.org
Subject: Re: ASDF build system sometimes fails to load .asd files properly
Date: Mon, 05 Sep 2022 09:10:49 +0000

[Message part 1 (text/plain, inline)]

This has been fixed with commit
6b5ef03a2582ab23228478018fd356e17db1daea.
Closing.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.