Subject: [bug#73197] [PATCH] gnu: julia: Update to 1.9.3.

From: Simon Tournier <[email-masked]>
Date: Fri, 14 Feb 2025 16:22:15 +0100

Hi,

On Mon, 13 Jan 2025 at 19:31, Simon Tournier <[email-masked]> wrote:

> For the record, it’s about base/loading.jl:
>
> --8<---------------cut here---------------start------------->8---
>      try
>          toplevel_load[] = false
>          # perform the search operation to select the module file require intends to load
> -        path = locate_package(pkg)
> +        path = locate_package(pkg, env)
>          if path === nothing
>              throw(ArgumentError("""
>                  Package $pkg is required but does not seem to be installed:
> --8<---------------cut here---------------end--------------->8---

Welcome to a debugging session using the most advanced technique: print! :-)

Attached the base/loading.jl with many ’println’.

Well, that’s very interesting because it seems related to some Julia
internals.  Somehow, the order of LOAD_PATH matters!

In all the dance, the code enters once and it founds it!

--8<---------------cut here---------------start------------->8---
ENTRY _require: Call locate_package( ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9] , /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/ )

[...]

RESULT _require: path = /gnu/store/yg4bz2kb7dqb6haaj0v7hj4mrd8rn27l-julia-constructionbase-1.3.0/share/julia/loadpath/ConstructionBase/src/ConstructionBase.jl
--8<---------------cut here---------------end--------------->8---

Then, it enters later with the same call… And bang!

--8<---------------cut here---------------start------------->8---
ENTRY _require: Call locate_package( ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9] , /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/ )

[...]

RESULT _require: path = nothing
ERROR: LoadError: ArgumentError: Package ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.
--8<---------------cut here---------------end--------------->8---

What’s different?  The call loops over load_path(), something like:

--8<---------------cut here---------------start------------->8---
locate_package_env: START LOOP; over load_path() = ["/gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/environments/v1.9/Project.toml", "/gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9", "/gnu/store/yg4bz2kb7dqb6haaj0v7hj4mrd8rn27l-julia-constructionbase-1.3.0/share/julia/loadpath/", "/gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/"]

locate_package_env: loop = 1
locate_package_env: env = /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/environments/v1.9/Project.toml
locate_package_env: path = nothing

locate_package_env: loop = 2
locate_package_env: env = /gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9
locate_package_env: path = nothing

locate_package_env: loop = 3
locate_package_env: env = /gnu/store/yg4bz2kb7dqb6haaj0v7hj4mrd8rn27l-julia-constructionbase-1.3.0/share/julia/loadpath/
locate_package_env: path = /gnu/store/yg4bz2kb7dqb6haaj0v7hj4mrd8rn27l-julia-constructionbase-1.3.0/share/julia/loadpath/ConstructionBase/src/ConstructionBase.jl
--8<---------------cut here---------------end--------------->8---

And for the second case, tandam!

--8<---------------cut here---------------start------------->8---
locate_package_env: START LOOP; over load_path() = ["/tmp/.julia/environments/v1.9/Project.toml", "/gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9", "/gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/", "/gnu/store/yg4bz2kb7dqb6haaj0v7hj4mrd8rn27l-julia-constructionbase-1.3.0/share/julia/loadpath/"]

locate_package_env: loop = 1
locate_package_env: env = /tmp/.julia/environments/v1.9/Project.toml
locate_package_env: path = nothing

locate_package_env: loop = 2
locate_package_env: env = /gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9
locate_package_env: path = nothing

locate_package_env: loop = 3
locate_package_env: env = /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/
locate_package_env: path = nothing
locate_package_env: BREAK: /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/ == /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/
locate_package_env:      : loading_extension = false
locate_package_env:      : precompiling_extension = false
locate_package_env: END LOOP

locate_package_env: pkg = ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9]
locate_package_env: Sys.STDLIB = /gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9
locate_package_env: Call manifest_uuid_path( /gnu/store/cp0fbzd4j1znnp4fjpsknap0ksnp2xk1-julia-1.9.3/share/julia/stdlib/v1.9 , ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9] )
locate_package_env: mbypath = nothing String? false
locate_package_env: RETURN: path = nothing ; env = /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/
locate_package: RETURN: nothing
_require: Post locate_package( ConstructionBase [187b0558-2788-49d3-abe0-74a17ed4e7c9] , /gnu/store/garmz1dz382ib25p2ckl0kj9rjay4xd4-julia-unitful-1.12.2/share/julia/loadpath/ )
RESULT _require: path = nothing
--8<---------------cut here---------------end--------------->8---

The attentive reader notes that load_path() is a list of 4 elements –the
last contains what we want– but a test breaks the loop.

Well, I do not know the details about the reasons of all that.  Maybe
something worth to ask upstream. :-)

Well, I’ll (force) push shortly a trivial fix to the branch
wip-julia-upgrade. 

Cheers,
simon

PS:  For the record, some elements:

1. Checkout wip-julia-upgrade
2. cp $(./pre-inst-env guix build julia -S) /tmp/
3. Uncompress the source and copy (e.g., /home/simon/tmp/julia-1.9.3)
4. Tweak the base/loading.jl file (and other as base/initdefs.jl)
5. Use this source with:

    (source
     (local-file "/home/simon/tmp/julia-1.9.3"
                 #:recursive? #t))

6. guix shell -CPNWD guix --expose=/home/simon/tmp/=/home/simon/tmp/
7. Build: ./pre-inst-env guix build --no-grafts --without-tests=julia julia
8. Tweak guix/build-system/julia-build-system.scm
9. Loop ./pre-inst-env guix build --no-grafts --without-tests=julia julia-unitful

Hope that helps.

--
--- /tmp/julia-1.9.3/base/loading.jl	1970-01-01 01:00:01.000000000 +0100
+++ /home/simon/tmp/julia-1.9.3/base/loading.jl	2025-02-14 15:38:59.879047968 +0100
@@ -391,6 +391,7 @@
 identify_package(name::String)                = _nothing_or_first(identify_package_env(name))
 
 function locate_package_env(pkg::PkgId, stopenv::Union{String, Nothing}=nothing)
+    println("locate_package_env: ARGS: ", pkg, " , ", stopenv)
     cache = LOADING_CACHE[]
     if cache !== nothing
         pathenv = get(cache.located, (pkg, stopenv), nothing)
@@ -399,6 +400,7 @@
     path = nothing
     env′ = nothing
     if pkg.uuid === nothing
+        println("locate_package_env: Case: nothing... TODO")
         for env in load_path()
             env′ = env
             # look for the toplevel pkg `pkg.name` in this entry
@@ -418,9 +420,18 @@
             end
         end
     else
+        println("locate_package_env: pkg = ", pkg)
+        println("locate_package_env: START LOOP; over load_path() = ", load_path())
+        #println("locate_package_env: number of loop", length(load_path))
+        loop = 1
         for env in load_path()
+            println("locate_package_env: loop = ", loop)
+            println("locate_package_env: env = ", env)
             env′ = env
+            println("locate_package_env: Call manifest_uuid_path( ", env, " , ", pkg, " )")
             path = manifest_uuid_path(env, pkg)
+            println("locate_package_env: Post path =  manifest_uuid_path( ", env, " , ", pkg, " )")
+            println("locate_package_env: path = ", path)
             # missing is used as a sentinel to stop looking further down in envs
             if path === missing
                 path = nothing
@@ -431,14 +442,27 @@
                 @goto done
             end
             if !(loading_extension || precompiling_extension)
+                stopenv == env && println("locate_package_env: BREAK: ", stopenv, " == ", env)
+                stopenv == env && println("locate_package_env:      : loading_extension = ", loading_extension)
+                stopenv == env && println("locate_package_env:      : precompiling_extension = ", precompiling_extension)
                 stopenv == env && break
             end
+            loop = loop + 1
         end
+        println("locate_package_env: END LOOP; No more env")
         # Allow loading of stdlibs if the name/uuid are given
         # e.g. if they have been explicitly added to the project/manifest
+        println("locate_package_env: pkg = ", pkg)
+        println("locate_package_env: Sys.STDLIB = ", Sys.STDLIB)
+        println("locate_package_env: Call manifest_uuid_path( ", Sys.STDLIB, " , ", pkg, " )")
         mbypath = manifest_uuid_path(Sys.STDLIB, pkg)
+        println("locate_package_env: Post mbypath = manifest_uuid_path( ", Sys.STDLIB, " , ", pkg, " )")
+        println("locate_package_env: mbypath = ", mbypath, " String? ", mbypath isa String)
         if mbypath isa String
+            println("locate_package_env: Call entry_path( ", mbypath, " , ", pkg.name, " )")
             path = entry_path(mbypath, pkg.name)
+            println("locate_package_env: Post path = entry_path( ", mbypath, " , ", pkg.name, " )")
+            println("locate_package_env: path = ", path)
             @goto done
         end
     end
@@ -446,6 +470,7 @@
     if cache !== nothing
         cache.located[(pkg, stopenv)] = path, env′
     end
+    println("locate_package_env: RETURN: path = ", path, " ; env = ", env′)
     return path, env′
 end
 
@@ -464,7 +489,11 @@
 ```
 """
 function locate_package(pkg::PkgId, stopenv::Union{String, Nothing}=nothing)::Union{Nothing,String}
-    _nothing_or_first(locate_package_env(pkg, stopenv))
+    println("locate_package: ARGS: ", pkg, " , ", stopenv)
+    println("locate_package: Call locate_package_env(", pkg, " , ", stopenv, " )")
+    ret = _nothing_or_first(locate_package_env(pkg, stopenv))
+    println("locate_package: RETURN: ", ret)
+    return ret
 end
 
 """
@@ -581,9 +610,14 @@
 #  - `path`: the path of an explicit project file
 function env_project_file(env::String)::Union{Bool,String}
     @lock require_lock begin
+    println("env_project_file: ARGS: ", env)
     cache = LOADING_CACHE[]
     if cache !== nothing
+        println("env_project_file: cache = ", cache)
+        println("env_project_file: cache.env_project_file = ", cache.env_project_file)
         project_file = get(cache.env_project_file, env, nothing)
+        println("env_project_file: project_file = ", project_file)
+        project_file === nothing || println("env_project_file: RETURN: ", project_file)
         project_file === nothing || return project_file
     end
     if isdir(env)
@@ -596,6 +630,7 @@
     if cache !== nothing
         cache.env_project_file[env] = project_file
     end
+    println("env_project_file: RETURN: ", project_file)
     return project_file
     end
 end
@@ -651,20 +686,31 @@
 end
 
 function manifest_uuid_path(env::String, pkg::PkgId)::Union{Nothing,String,Missing}
+    println("manifest_uuid_path: ARGS: ", env, " , ", pkg)
+    println("manifest_uuid_path: Call env_project_file( ", env, " )")
     project_file = env_project_file(env)
+    println("manifest_uuid_path: Post project_file = env_project_file( ", env, " )")
+    println("manifest_uuid_path: project_file = ", project_file, " String? ", project_file isa String)
     if project_file isa String
         proj = project_file_name_uuid(project_file, pkg.name)
         if proj == pkg
             # if `pkg` matches the project, return the project itself
-            return project_file_path(project_file)
+            ret = project_file_path(project_file)
+            println("manifest_uuid_path: RETURN: (1) ", ret)
+            return
         end
         mby_ext = project_file_ext_path(project_file, pkg.name)
+        mby_ext === nothing || println("manifest_uuid_path: RETURN: (2) ", mby_ext)
         mby_ext === nothing || return mby_ext
         # look for manifest file and `where` stanza
-        return explicit_manifest_uuid_path(project_file, pkg)
+        ret = explicit_manifest_uuid_path(project_file, pkg)
+        println("manifest_uuid_path: RETURN: (3) ", ret)
+        return ret
     elseif project_file
         # if env names a directory, search it
-        return implicit_manifest_uuid_path(env, pkg)
+        ret = implicit_manifest_uuid_path(env, pkg)
+        println("manifest_uuid_path: RETURN: (4) ", ret)
+        return ret
     end
     return nothing
 end
@@ -1758,6 +1804,7 @@
 
 # Returns `nothing` or the new(ish) module
 function _require(pkg::PkgId, env=nothing)
+    println("_require: ARGS: ", pkg, " , ", env)
     assert_havelock(require_lock)
     # handle recursive calls to require
     loading = get(package_locks, pkg, false)
@@ -1772,7 +1819,10 @@
     try
         toplevel_load[] = false
         # perform the search operation to select the module file require intends to load
+        println("ENTRY _require: Call locate_package( ", pkg, " , ", env, " )")
         path = locate_package(pkg, env)
+        println("_require: Post locate_package( ", pkg, " , ", env, " )")
+        println("RESULT _require: path = ", path)
         if path === nothing
             throw(ArgumentError("""
                 Package $pkg is required but does not seem to be installed: