GNU bug report logs - #74532
with-extensions does not add native extensions to the load path

Previous Next

To reply to this bug, email your comments to 74532 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: bug-guix <at> gnu.org
Subject: with-extensions does not add native extensions to the load path
Date: Mon, 25 Nov 2024 18:49:53 +0100

[Message part 1 (text/plain, inline)]

The documentation for `with-extensions' says:

> In the same vein, sometimes you want to import not just pure-Scheme
> modules, but also “extensions” such as Guile bindings to C libraries
> or other “full-blown” packages.

However it does not actually add those C libraries into
(guile-extensions-path), which means that trying to actually use a
library imported this way will lead to an error.

I guess `load-path-expression' in (guix gexp) needs to also append to
the `guile-extensions-path' based on the search-paths of the imported
packages?

Reproduction is simple.  Make sure you *do not* have guile-yamlpp
installed in your profile.  Then build the `test' into a store item.

--8<---------------cut here---------------start------------->8---
(use-modules (gnu packages guile-xyz))
(define test
  (with-extensions (list guile-yamlpp)
    (program-file "test" #~(use-modules (yamlpp)))))
--8<---------------cut here---------------end--------------->8---

And try to run it:

--8<---------------cut here---------------start------------->8---
$ /gnu/store/nq0sahcnph5sxms4irq4inv2hq2hfbzr-test
Backtrace:
          19 (primitive-load "/gnu/store/nq0sahcnph5sxms4irq4inv2hq2hfbzr-test")
In ice-9/eval.scm:
   721:20 18 (primitive-eval _)
In ice-9/psyntax.scm:
  1229:36 17 (expand-top-sequence _ _ _ #f _ _ _)
  1221:19 16 (parse _ (("placeholder" placeholder)) ((top) #(ribcage () () ())) _ e (eval) #)
   259:10 15 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) (hygiene guile-user))
In ice-9/boot-9.scm:
  3935:20 14 (process-use-modules _)
   222:17 13 (map1 (((yamlpp))))
  3936:31 12 (_ ((yamlpp)))
  3327:17 11 (resolve-interface (yamlpp) #:select _ #:hide _ #:prefix _ #:renamer _ # _)
In ice-9/threads.scm:
    390:8 10 (_ _)
In ice-9/boot-9.scm:
  3253:13  9 (_)
In ice-9/threads.scm:
    390:8  8 (_ _)
In ice-9/boot-9.scm:
  3544:20  7 (_)
   2836:4  6 (save-module-excursion _)
  3564:26  5 (_)
In unknown file:
           4 (primitive-load-path "yamlpp" #<procedure 7f405e4fd560 at ice-9/boot-9.scm:35…>)
In yamlpp.scm:
     65:1  3 (_)
In unknown file:
           2 (load-extension "libguile-yamlpp" "init")
In system/foreign-library.scm:
   190:25  1 (load-foreign-library _ #:extensions _ #:search-ltdl-library-path? _ # _ # _ # …)
In unknown file:
           0 (dlopen "libguile-yamlpp.so" 1)

ERROR: In procedure dlopen:
In procedure dlopen: file "libguile-yamlpp.so", message "libguile-yamlpp.so: cannot open shared object file: No such file or directory"
--8<---------------cut here---------------end--------------->8---

Have a nice day,
Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Thu, 28 Nov 2024 08:54:02 +0100

Hi,

Tomas Volf <~@wolfsden.cz> skribis:

> The documentation for `with-extensions' says:
>
>> In the same vein, sometimes you want to import not just pure-Scheme
>> modules, but also “extensions” such as Guile bindings to C libraries
>> or other “full-blown” packages.

It’s not just bindings but also pure Scheme libraries like Guile-JSON.

> However it does not actually add those C libraries into
> (guile-extensions-path), which means that trying to actually use a
> library imported this way will lead to an error.

[...]

>    190:25  1 (load-foreign-library _ #:extensions _ #:search-ltdl-library-path? _ # _ # _ # …)
> In unknown file:
>            0 (dlopen "libguile-yamlpp.so" 1)
>
> ERROR: In procedure dlopen:
> In procedure dlopen: file "libguile-yamlpp.so", message "libguile-yamlpp.so: cannot open shared object file: No such file or directory"

Usually, packages like these (guile-gnutls, guile-git, guile-ssh, etc.)
have their .so absolute file name hard-coded, which sidesteps this
problem entirely.  I recommend doing that for guile-yamlpp as well.

That said, it would probably make sense to arrange for ‘with-extensions’
to set GUILE_EXTENSIONS_PATH.

Ludo’.

PS: Your MUA sets “Mail-Followup-To: bug-guix <at> gnu.org”, which is kinda
    annoying because that’s the wrong address when replying to a bug.
    :-)

Message #11 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Tue, 03 Dec 2024 00:40:42 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Tomas Volf <~@wolfsden.cz> skribis:
>
>> The documentation for `with-extensions' says:
>>
>>> In the same vein, sometimes you want to import not just pure-Scheme
>>> modules, but also “extensions” such as Guile bindings to C libraries
>>> or other “full-blown” packages.
>
> It’s not just bindings but also pure Scheme libraries like Guile-JSON.

True, and for those it works well.  However the text documentation
explicitly mentions "bindings to C libraries" as one of the use
cases. :)

>
>> However it does not actually add those C libraries into
>> (guile-extensions-path), which means that trying to actually use a
>> library imported this way will lead to an error.
>
> [...]
>
>>    190:25  1 (load-foreign-library _ #:extensions _ #:search-ltdl-library-path? _ # _ # _ # …)
>> In unknown file:
>>            0 (dlopen "libguile-yamlpp.so" 1)
>>
>> ERROR: In procedure dlopen:
>> In procedure dlopen: file "libguile-yamlpp.so", message "libguile-yamlpp.so: cannot open shared object file: No such file or directory"
>
> Usually, packages like these (guile-gnutls, guile-git, guile-ssh, etc.)
> have their .so absolute file name hard-coded, which sidesteps this
> problem entirely.  I recommend doing that for guile-yamlpp as well.

How it that usually done?  I cannot do that *before* build, because then
tests would not run (the library is not yet installed into the absolute
path), and I cannot do it after, because I would need to rebuild the .go
files after patching the source code.

Assuming I have a guile library that creates a new .so library during
the build, and the .so library must be loadable by the other modules in
the library during the build (and for the tests), how should I approach
that?

I am pretty sure I cannot just patch the source code, since the library
would not be installed into the absolute path yet during the build.

Honestly, setting GUILE_EXTENSIONS_PATH via pre-inst-env seemed like
fairly elegant solution.

>
> That said, it would probably make sense to arrange for ‘with-extensions’
> to set GUILE_EXTENSIONS_PATH.

That would be great.  My current work-around is

  (with-extensions (list guile-wolfsden)
    (program-file
     "audio-cycle-sinks"
     #~(begin
         ;; Bug 74532: Native extensions are not added to the load path.
         (eval-when (expand load eval)
           (let ((ext-path (@ (system foreign-library) guile-extensions-path)))
             ;; Just a temporary hack, we can live with duplicates in the path.
             (ext-path (cons #$(file-append guile-wolfsden
                                            "/lib/guile/3.0/extensions")
                             (ext-path)))))
         ...)))

which is anything but elegant.

>
> Ludo’.
>
> PS: Your MUA sets “Mail-Followup-To: bug-guix <at> gnu.org”, which is kinda
>     annoying because that’s the wrong address when replying to a bug.
>     :-)

Thanks for letting me know, I was not aware of that.  After reading
(message)Mailing Lists bit more carefully, adjusting the Posting Styles
and sending a bug fix to the Emacs' bug tracker, I believe it should not
happen anymore.  Sorry for the annoyance.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Z572 <zhengjunjie <at> iscas.ac.cn>
To: Tomas Volf <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Tue, 03 Dec 2024 11:10:15 +0800

[Message part 1 (text/plain, inline)]

Tomas Volf <~@wolfsden.cz> writes:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Hi,
>>
>> Tomas Volf <~@wolfsden.cz> skribis:
>>
>>> The documentation for `with-extensions' says:
>>>
>>>> In the same vein, sometimes you want to import not just pure-Scheme
>>>> modules, but also “extensions” such as Guile bindings to C libraries
>>>> or other “full-blown” packages.
>>
>> It’s not just bindings but also pure Scheme libraries like Guile-JSON.
>
> True, and for those it works well.  However the text documentation
> explicitly mentions "bindings to C libraries" as one of the use
> cases. :)
>
>>
>>> However it does not actually add those C libraries into
>>> (guile-extensions-path), which means that trying to actually use a
>>> library imported this way will lead to an error.
>>
>> [...]
>>
>>>    190:25  1 (load-foreign-library _ #:extensions _ #:search-ltdl-library-path? _ # _ # _ # …)
>>> In unknown file:
>>>            0 (dlopen "libguile-yamlpp.so" 1)
>>>
>>> ERROR: In procedure dlopen:
>>> In procedure dlopen: file "libguile-yamlpp.so", message "libguile-yamlpp.so: cannot open shared object file: No such file or directory"
>>
>> Usually, packages like these (guile-gnutls, guile-git, guile-ssh, etc.)
>> have their .so absolute file name hard-coded, which sidesteps this
>> problem entirely.  I recommend doing that for guile-yamlpp as well.
>
> How it that usually done?  I cannot do that *before* build, because then
> tests would not run (the library is not yet installed into the absolute
> path), and I cannot do it after, because I would need to rebuild the .go
> files after patching the source code.

you can use substitute* to adjust source.
e.g.

(or (false-if-exception (load-extension "/path/to/lib-some-object-file.so"))
    (load-extension "lib-some-object-file.so"))

>
> Assuming I have a guile library that creates a new .so library during
> the build, and the .so library must be loadable by the other modules in
> the library during the build (and for the tests), how should I approach
> that?
>
> I am pretty sure I cannot just patch the source code, since the library
> would not be installed into the absolute path yet during the build.
>
> Honestly, setting GUILE_EXTENSIONS_PATH via pre-inst-env seemed like
> fairly elegant solution.
>
>>
>> That said, it would probably make sense to arrange for ‘with-extensions’
>> to set GUILE_EXTENSIONS_PATH.
>
> That would be great.  My current work-around is
>
>   (with-extensions (list guile-wolfsden)
>     (program-file
>      "audio-cycle-sinks"
>      #~(begin
>          ;; Bug 74532: Native extensions are not added to the load path.
>          (eval-when (expand load eval)
>            (let ((ext-path (@ (system foreign-library) guile-extensions-path)))
>              ;; Just a temporary hack, we can live with duplicates in the path.
>              (ext-path (cons #$(file-append guile-wolfsden
>                                             "/lib/guile/3.0/extensions")
>                              (ext-path)))))
>          ...)))
>
> which is anything but elegant.
>
>>
>> Ludo’.
>>
>> PS: Your MUA sets “Mail-Followup-To: bug-guix <at> gnu.org”, which is kinda
>>     annoying because that’s the wrong address when replying to a bug.
>>     :-)
>
> Thanks for letting me know, I was not aware of that.  After reading
> (message)Mailing Lists bit more carefully, adjusting the Posting Styles
> and sending a bug fix to the Emacs' bug tracker, I believe it should not
> happen anymore.  Sorry for the annoyance.

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Z572 <zhengjunjie <at> iscas.ac.cn>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Fri, 13 Dec 2024 17:44:09 +0100

[Message part 1 (text/plain, inline)]

Hello,

Z572 <zhengjunjie <at> iscas.ac.cn> writes:

>> How it that usually done?  I cannot do that *before* build, because then
>> tests would not run (the library is not yet installed into the absolute
>> path), and I cannot do it after, because I would need to rebuild the .go
>> files after patching the source code.
>
> you can use substitute* to adjust source.
> e.g.
>
> (or (false-if-exception (load-extension "/path/to/lib-some-object-file.so"))
>     (load-extension "lib-some-object-file.so"))
>

Thank you for the suggestion.  I am not sure I like the
false-if-exception part, but temporary adjustment of the (@ (system
foreign-library) guile-extensions-path) should do the trick as well.

One thing I am curious about, I can see you have put the load from
absolute path as the first alternative.  But if it fails, it still falls
back to lookup based on the guile-extensions-path.  I wonder whether
putting it at the end of the list would not be better, that way you
could still replace the library in the spirit of LD_PRELOAD by setting
the appropriate environment variable.  And if reproducibility is the
priority, then even the `or' might not make sense.

Any thoughts on that?

Thanks,
Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Tue, 17 Dec 2024 14:55:06 +0100

Hi,

Tomas Volf <~@wolfsden.cz> skribis:

> Z572 <zhengjunjie <at> iscas.ac.cn> writes:
>
>>> How it that usually done?  I cannot do that *before* build, because then
>>> tests would not run (the library is not yet installed into the absolute
>>> path), and I cannot do it after, because I would need to rebuild the .go
>>> files after patching the source code.
>>
>> you can use substitute* to adjust source.
>> e.g.
>>
>> (or (false-if-exception (load-extension "/path/to/lib-some-object-file.so"))
>>     (load-extension "lib-some-object-file.so"))
>>
>
> Thank you for the suggestion.  I am not sure I like the
> false-if-exception part, but temporary adjustment of the (@ (system
> foreign-library) guile-extensions-path) should do the trick as well.

I think you can just do:

  (load-extension "/path/to/lib-some-object-file.so")

but you need to (1) do that in a post-install phase, and (2) re-run
‘make’ before ‘make install’ if necessary.

See ‘guile-gdbm-ffi’ as one example.

Ludo’.

Message #23 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: "Timo Wilken" <guix <at> twilken.net>
To: <74532 <at> debbugs.gnu.org>
Cc: Tomas Volf <~@wolfsden.cz>, Z572 <zhengjunjie <at> iscas.ac.cn>,
 Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: with-extensions does not add native extensions to the load path
Date: Sat, 15 Mar 2025 22:04:53 +0100

Hi,

Since I ran into this problem with guile-yamlpp today, I went ahead and
implemented Z572's suggestion.

It still feels wrong to do this on a per-package basis (shouldn't
`with-extension' take care of setting up the path?), but hey ho.

I needed the fallback (as suggested by Z572), else tests would fail (as
predicted by Tomas).

Like this, the whole chain works for me; in my case:

(computed-file "test"
  (with-extensions (list guile-yamlpp)
    #~(use-modules (yamlpp))))

Patch incoming.

Cheers,
Timo

Message #26 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Timo Wilken <guix <at> twilken.net>
To: 74532 <at> debbugs.gnu.org
Subject: [PATCH] gnu: guile-yamlpp: Allow usage with `with-extensions'
Date: Sat, 15 Mar 2025 22:10:09 +0100

* gnu/packages/guile-xyz.scm (guile-yamlpp) [arguments] <phases>:
  Add 'set-extension-file-name.

Change-Id: I49a7199b4e0fe4053822374bae59c6e1d0b7ead0
---
 gnu/packages/guile-xyz.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 30c8e847996..f43e06a3a7f 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -1997,6 +1997,19 @@ (define-public guile-yamlpp
        (sha256
         (base32 "0ik69y0vddg0myp0zdbkmklma0qkkrqzwlqwkij1zirklz6hl1ss"))))
     (build-system gnu-build-system)
+    (arguments
+     `(#:phases
+       ,#~(modify-phases %standard-phases
+            ;; In order to let Guile find the compiled library when used with
+            ;; Guix' `with-extension', hardcode the final path to the library.
+            ;; The fallback is needed for tests (i.e. before the 'install phase).
+            ;; See <https://issues.guix.gnu.org/74532>.
+            (add-after 'unpack 'set-extension-file-name
+              (lambda* (#:key outputs #:allow-other-keys)
+                (substitute* "scm/yamlpp.scm"
+                  (("\\(load-extension \"libguile-yamlpp\"(.*)\\)" all init)
+                   (format #f "(or (false-if-exception (load-extension \"~a/lib/guile/3.0/libguile-yamlpp\"~a)) ~a)"
+                           (assoc-ref outputs "out") init all))))))))
     (native-inputs (list autoconf automake libtool pkg-config))
     (inputs (list guile-3.0 yaml-cpp))
     (native-search-paths
-- 
2.48.1

Message #29 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Timo Wilken <guix <at> twilken.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: [PATCH] gnu: guile-yamlpp: Allow usage with
 `with-extensions'
Date: Sun, 16 Mar 2025 15:30:57 +0100

[Message part 1 (text/plain, inline)]

Hi!

Timo Wilken <guix <at> twilken.net> writes:

> * gnu/packages/guile-xyz.scm (guile-yamlpp) [arguments] <phases>:
>   Add 'set-extension-file-name.
>
> Change-Id: I49a7199b4e0fe4053822374bae59c6e1d0b7ead0
> ---
>  gnu/packages/guile-xyz.scm | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
> index 30c8e847996..f43e06a3a7f 100644
> --- a/gnu/packages/guile-xyz.scm
> +++ b/gnu/packages/guile-xyz.scm
> @@ -1997,6 +1997,19 @@ (define-public guile-yamlpp
>         (sha256
>          (base32 "0ik69y0vddg0myp0zdbkmklma0qkkrqzwlqwkij1zirklz6hl1ss"))))
>      (build-system gnu-build-system)
> +    (arguments
> +     `(#:phases
> +       ,#~(modify-phases %standard-phases
> +            ;; In order to let Guile find the compiled library when used with
> +            ;; Guix' `with-extension', hardcode the final path to the library.
> +            ;; The fallback is needed for tests (i.e. before the 'install phase).
> +            ;; See <https://issues.guix.gnu.org/74532>.
> +            (add-after 'unpack 'set-extension-file-name
> +              (lambda* (#:key outputs #:allow-other-keys)
> +                (substitute* "scm/yamlpp.scm"
> +                  (("\\(load-extension \"libguile-yamlpp\"(.*)\\)" all init)
> +                   (format #f "(or (false-if-exception (load-extension \"~a/lib/guile/3.0/libguile-yamlpp\"~a)) ~a)"
> +                           (assoc-ref outputs "out") init all))))))))

I am not sure it is optimal to hide *all* errors from load-extension.
Maybe check if file exists and then try to load-extension it without the
false-if-exception?

(Sure there is a TOCTTOU problem, but I think that can be ignored in
this case.)

>      (native-inputs (list autoconf automake libtool pkg-config))
>      (inputs (list guile-3.0 yaml-cpp))
>      (native-search-paths

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: "Timo Wilken" <guix <at> twilken.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org
Subject: Re: with-extensions does not add native extensions to the load path
Date: Sun, 16 Mar 2025 15:31:38 +0100

"Timo Wilken" <guix <at> twilken.net> writes:

> It still feels wrong to do this on a per-package basis (shouldn't
> `with-extension' take care of setting up the path?), but hey ho.

I definitely agree here...

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Message #35 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org,
 Timo Wilken <guix <at> twilken.net>
Subject: Re: bug#74532: [PATCH] gnu: guile-yamlpp: Allow usage with
 `with-extensions'
Date: Tue, 25 Mar 2025 11:10:26 +0100

Hi,

Tomas Volf <~@wolfsden.cz> skribis:

> Timo Wilken <guix <at> twilken.net> writes:

[...]

>> +                   (format #f "(or (false-if-exception (load-extension \"~a/lib/guile/3.0/libguile-yamlpp\"~a)) ~a)"
>> +                           (assoc-ref outputs "out") init all))))))))
>
> I am not sure it is optimal to hide *all* errors from load-extension.
> Maybe check if file exists and then try to load-extension it without the
> false-if-exception?

Or just unconditionally load from the absolute file name?  I don’t see
what the ‘or’ buys us.

Thanks,
Ludo’.

Message #38 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Timo Wilken <guix <at> twilken.net>
To: Tomas Volf <~@wolfsden.cz>,
 Ludovic Courtès <ludo <at> gnu.org>
Cc: Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: [PATCH] gnu: guile-yamlpp: Allow usage with
 `with-extensions'
Date: Tue, 25 Mar 2025 19:12:49 +0100

Hi Tomas, hi Ludo', hi Z572,

On Tue 25 Mar 2025 at 11:10 am CET, Ludovic Courtès wrote:
> Tomas Volf <~@wolfsden.cz> skribis:
>> Timo Wilken <guix <at> twilken.net> writes:
>> I am not sure it is optimal to hide *all* errors from load-extension.
>> Maybe check if file exists and then try to load-extension it without the
>> false-if-exception?

I think the v2 patch makes this moot -- see below.

> Or just unconditionally load from the absolute file name?  I don’t see
> what the ‘or’ buys us.

That only works if I move 'check after 'install -- else it fails with an error
in `load-foreign-library', saying the file doesn't exist.

The v2 patch does that, and works for me (as in, it compiles and I can use the
resulting package in `with-extensions').

Feedback is very welcome!

Cheers,
Timo

Message #41 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Timo Wilken <guix <at> twilken.net>
To: Tomas Volf <~@wolfsden.cz>,
 Ludovic Courtès <ludo <at> gnu.org>
Cc: Z572 <zhengjunjie <at> iscas.ac.cn>, 74532 <at> debbugs.gnu.org
Subject: [PATCH v2] gnu: guile-yamlpp: Allow usage with `with-extensions'
Date: Tue, 25 Mar 2025 19:12:53 +0100

* gnu/packages/guile-xyz.scm (guile-yamlpp) [arguments] <phases>:
  Add 'set-extension-file-name and move 'check after 'install.

Change-Id: I49a7199b4e0fe4053822374bae59c6e1d0b7ead0
---
 gnu/packages/guile-xyz.scm | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index f0277f430ec..cabc3101320 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -2020,6 +2020,23 @@ (define-public guile-yamlpp
        (sha256
         (base32 "0ik69y0vddg0myp0zdbkmklma0qkkrqzwlqwkij1zirklz6hl1ss"))))
     (build-system gnu-build-system)
+    (arguments
+     `(#:phases
+       ,#~(modify-phases %standard-phases
+            ;; In order to let Guile find the compiled library when used with
+            ;; Guix' `with-extension', hardcode the final path to the library.
+            ;; See <https://issues.guix.gnu.org/74532>.
+            (add-after 'unpack 'set-extension-file-name
+              (lambda* (#:key outputs #:allow-other-keys)
+                (substitute* "scm/yamlpp.scm"
+                  (("\\(load-extension \"libguile-yamlpp\"(.*)\\)" _ init)
+                   (format #f "(load-extension \"~a/lib/guile/3.0/libguile-yamlpp\"~a)"
+                           (assoc-ref outputs "out") init)))))
+            ;; Move 'check after installation, so that the foreign library is
+            ;; present where we expect it (as we hardcode its path above).
+            (delete 'check)
+            (add-after 'install 'check
+              (assoc-ref %standard-phases 'check)))))
     (native-inputs (list autoconf automake libtool pkg-config))
     (inputs (list guile-3.0 yaml-cpp))
     (native-search-paths
-- 
2.49.0

Message #46 received at 74532-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Timo Wilken <guix <at> twilken.net>
Cc: Tomas Volf <~@wolfsden.cz>, Z572 <zhengjunjie <at> iscas.ac.cn>,
 74532-done <at> debbugs.gnu.org
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Tue, 01 Apr 2025 09:49:15 +0200

Timo Wilken <guix <at> twilken.net> skribis:

> * gnu/packages/guile-xyz.scm (guile-yamlpp) [arguments] <phases>:
>   Add 'set-extension-file-name and move 'check after 'install.
>
> Change-Id: I49a7199b4e0fe4053822374bae59c6e1d0b7ead0

Applied, thanks!

Message #53 received at 74532-done <at> debbugs.gnu.org (full text, mbox):

From: "Timo Wilken" <guix <at> twilken.net>
To: <74532-done <at> debbugs.gnu.org>
Subject: Close #74532
Date: Tue, 08 Apr 2025 22:59:19 +0200

Somehow, this issue was magically reopened.

See the stanza beginning with "Did not alter fixed versions and reopened" at
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=74532>.

Message #56 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: control <at> debbugs.gnu.org
Cc: 74532 <at> debbugs.gnu.org
Subject: control message for bug #74532
Date: Wed, 09 Apr 2025 00:09:33 +0200

reopen 74532
quit

Look, I get that I must be annoying with the constant reopening of this
issue, but I promise, this is the last time.  When someone closes it
again, I will just give up.  But as I wrote in the control message last
time (where I have CC-ed Ludovic), I do not believe this issue is
resolved.  Great, guile-yamlpp works now, but that is not this issue is
about.

It is fine if you think that with-extensions should *not* touch
GUILE_EXTENSIONS_PATH.  I disagree, but fair enough, that is life.
However, it would be nice to know why you think handling just 2 out of 3
environment variable controlling the loading in Guile is desirable.

I get that Guix is a volunteer run project, and no one owns me an
explanation or something, but still...

Message #61 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Tomas Volf <~@wolfsden.cz>
Cc: 74532 <at> debbugs.gnu.org
Subject: Re: bug#74532: control message for bug #74532
Date: Fri, 11 Apr 2025 15:08:17 +0200

Hi,

On Wed, 09 Apr 2025 at 00:09, Tomas Volf <~@wolfsden.cz> wrote:

> It is fine if you think that with-extensions should *not* touch
> GUILE_EXTENSIONS_PATH.  I disagree, but fair enough, that is life.
> However, it would be nice to know why you think handling just 2 out of 3
> environment variable controlling the loading in Guile is desirable.

Well, bug#62008 [1] is a bit different but in the same spirit: somehow
inconsistencies – at least from my understanding – about Guile
packages. :-)

Cheers,
simon

1: [bug#62008] [PATCH 0/2] Update Disarchive to 0.5.0
Simon Tournier <zimon.toutoune <at> gmail.com>
Tue, 07 Mar 2023 11:43:35 +0100
id:87a60odep4.fsf <at> gmail.com
https://issues.guix.gnu.org/62008
https://issues.guix.gnu.org/msgid/87a60odep4.fsf <at> gmail.com
https://yhetil.org/guix/87a60odep4.fsf <at> gmail.com

Message #64 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: "Timo Wilken" <guix <at> twilken.net>
To: "Tomas Volf" <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 74532 <at> debbugs.gnu.org,
 Simon Tournier <zimon.toutoune <at> gmail.com>
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Mon, 14 Apr 2025 19:40:39 +0200

Hi Tomas,

> Look, I get that I must be annoying with the constant reopening of this
> issue, but I promise, this is the last time.  When someone closes it
> again, I will just give up.  But as I wrote in the control message last
> time (where I have CC-ed Ludovic), I do not believe this issue is
> resolved.  Great, guile-yamlpp works now, but that is not this issue is
> about.

I'm sorry, I didn't realise that the re-opening of the issue was an
intentional action by a human -- neither the debbugs web interface nor mumi
made this obvious to me. Your first control message doesn't seem to be
reproduced there.

> It is fine if you think that with-extensions should *not* touch
> GUILE_EXTENSIONS_PATH.  I disagree, but fair enough, that is life.
> However, it would be nice to know why you think handling just 2 out of 3
> environment variable controlling the loading in Guile is desirable.

I think I agree with you -- all of this should be handled properly through the
GUILE_* environment variables, without having to patch every single Guile
package individually. With this patch, I mostly wanted to get guile-yamlpp to
work since I needed it for something.

Guile packaging in Guix seems a little strange to me, in general -- e.g. the
treatment for disarchive in <https://issues.guix.gnu.org/62008> might need to
be applied to the Shepherd as well:

--8<---------------cut here--------------->8-
$ guix shell -CN --pure guile shepherd -- guile
scheme@(guile-user)> ,use (shepherd)
While executing meta-command:
no code for module (fibers)
--8<---------------cut here--------------->8-

Cheers,
Timo

Message #67 received at 74532 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: "Timo Wilken" <guix <at> twilken.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 74532 <at> debbugs.gnu.org,
 Simon Tournier <zimon.toutoune <at> gmail.com>
Subject: Re: bug#74532: with-extensions does not add native extensions to
 the load path
Date: Fri, 02 May 2025 15:38:13 +0200

"Timo Wilken" <guix <at> twilken.net> writes:

> Hi Tomas,
>
>> Look, I get that I must be annoying with the constant reopening of this
>> issue, but I promise, this is the last time.  When someone closes it
>> again, I will just give up.  But as I wrote in the control message last
>> time (where I have CC-ed Ludovic), I do not believe this issue is
>> resolved.  Great, guile-yamlpp works now, but that is not this issue is
>> about.
>
> I'm sorry, I didn't realise that the re-opening of the issue was an
> intentional action by a human -- neither the debbugs web interface nor mumi
> made this obvious to me. Your first control message doesn't seem to be
> reproduced there.

Oh, you are right!  Even when I know what message I am trying to find, I
fail to do so.  And since I have sent it using
debbugs-gnu-make-control-message, I do not even have it in my sent mail
group.  Hm.

>
>> It is fine if you think that with-extensions should *not* touch
>> GUILE_EXTENSIONS_PATH.  I disagree, but fair enough, that is life.
>> However, it would be nice to know why you think handling just 2 out of 3
>> environment variable controlling the loading in Guile is desirable.
>
> I think I agree with you -- all of this should be handled properly through the
> GUILE_* environment variables, without having to patch every single Guile
> package individually. With this patch, I mostly wanted to get guile-yamlpp to
> work since I needed it for something.
>
> Guile packaging in Guix seems a little strange to me, in general -- e.g. the
> treatment for disarchive in <https://issues.guix.gnu.org/62008> might need to
> be applied to the Shepherd as well:
>
> --8<---------------cut here--------------->8-
> $ guix shell -CN --pure guile shepherd -- guile
> scheme@(guile-user)> ,use (shepherd)
> While executing meta-command:
> no code for module (fibers)
> --8<---------------cut here--------------->8-

While we have *mostly* working approach for lot of cases, I still wonder
whether we should get something along the lines of rpath for non-elf
artifacts.  That that I have any idea how it would be supposed to work.
I *think* GCD 004 was supposed to address this somehow, but admit I was
not following the debate too closely.

I will put on my todo list to at least fix up the with-extensions.

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.