Subject: bug#73304: Python in relocatable guix pack leads to wrong sys.path

From: Rutherther <[email-masked]>
Date: Fri, 31 Jan 2025 11:59:45 +0000

"Rutherther" <[email-masked]> writes:

>> Hello,
>>
>> Konrad Hinsen <[email-masked]> skribis:
>>
>>> Konrad Hinsen <[email-masked]> writes:
>>>
>>>> I have changed my mind. In the sys.path outputs shown, there are no
>>>> paths from add-on packages. It's just the Python standard library.
>>>> Maybe our sitecustomize.py is not run at all, but if it is, it didn't do
>>>> anything to sys.path. There must be a bug somewhere else.
>>>
>>> Our sitecustomize.py is indeed not run at all, so this definitely is a
>>> different problem.
>>>
>>> Evidence: Run Rutherther's example, adding the -v option.  The long
>>> output is attached, both for "./profile/bin/python3 -v" and "$(realpath
>>> ./profile/bin/python3) -v". Search for "site-packages" to find the
>>> interesting parts. If you don't use realpath, large parts of the
>>> initialization are not done.
>>>
>>> There are lots of ../../ in the path shown in these log files. If Python
>>> resolves them lexically, as the normpath function does, that would
>>> probably explain most of these issues.
>
> I am not sure I understand this right. I am looking at the output, and
> it has `/tmp/b/profile/bin/../../hash-python-3.10.7R/{rest}`, omitting
> the gnu/store even here. Why is it significant how these paths are
> evaluated, if there is the gnu/store part missing already?
>
> Regards,
> Rutherther

Okay, I spent a bit of time on this, and I think I get it now.
This seems to boil down to something in Python wrongly handling
the argv0, where it looks at the path, being /tmp/a/profile/bin/python3,
substitutes the symlink /tmp/a/profile/bin/python3/../../hash-python-3.10.7/bin/python3, and then
probably calls something like normpath on it, and it becomes
/tmp/a/hash-python-3.10.7. Whereas the correct approach is to first get
the actual path /tmp/a/profile/bin/python3 is at, being
/tmp/a/gnu/store/hash-python-3.10.7R/bin/python3, and unwrap the symlink
from there. This happens even if python3 is called from PATH, I suppose
Python then looks through PATH and again finds the python3 file outside
of the store.

As a workaround,
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 074befde46..5ce8032ed8 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -654,6 +654,11 @@ main (int argc, char *argv[])
   assert (size > 0);
   self[size] = '\0';
 
+  for (int i = 0; i < argc; i++) {
+        printf("%s\n", argv[i]);
+  }
+  argv[0] = "@WRAPPED_PROGRAM@";
+
   /* SELF is something like "/home/ludo/.local/gnu/store/…-foo/bin/ls" and we
      want to extract "/home/ludo/.local/gnu/store".  */
   size_t index = strlen (self)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 7f5a5f2aa7..f73d92b678 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -1343,7 +1343,20 @@ (define* (wrapped-package package
                                      (+ (string-length (%store-directory))
                                         1)))))
               (mkdir-p (dirname result))
-              (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall"
+              (call-with-output-file (string-append result ".comp")
+                (lambda (port)
+                  (format port
+                          (apply string-append #$compiler "-std=gnu99" "-O0" "-g" "-Wall"
+                                         "run.c" "-o" result
+                                         (string-append "-DWRAPPER_PROGRAM=\""
+                                                        (canonicalize-path (dirname result)) "/"
+                                                        (basename result) "\"")
+                                         (append (if proot
+                                                     (list (string-append "-DPROOT_PROGRAM=\""
+                                                                          proot "\""))
+                                                     '())
+                                                 (elf-loader-compile-flags program))) )))
+              (apply invoke #$compiler "-std=gnu99" "-O0" "-g" "-Wall"
                      "run.c" "-o" result
                      (string-append "-DWRAPPER_PROGRAM=\""
                                     (canonicalize-path (dirname result)) "/"
@@ -1353,6 +1366,7 @@ (define* (wrapped-package package
                                                       proot "\""))
                                  '())
                              (elf-loader-compile-flags program)))
+              (copy-file "run.c" (string-append result ".run.c"))
               (delete-file "run.c")))
 
           (setvbuf (current-output-port) 'line)