GNU bug report logs -
#74217
Bootstrapping Zig with no Binary Blobs
Previous Next
Full log
Message #45 received at 74217 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, Nov 14, 2024 at 12:46:59AM +0800, Hilton Chain wrote:
> Hello everyone,
>
> With a new forced push (and new rebuilds, sorry), wip-zig-bootstrap is mostly
> ready (for me) now. Please take a look at the first few commits, as I'm
> changing Zig's behavior there, here are some additional notes:
>
> To Efraim: Can adding pkg-config to native-inputs avoid the ncdu snippet?
Unfortunately no. The code reads:
exe.root_module.linkSystemLibrary("ncursesw", .{});
exe.root_module.linkSystemLibrary("libzstd", .{});
and so it searches (something like the following, I lost the log):
/usr/lib/liblibzstd.so
/usr/lib/liblibzstd.a
...
/gnu/store/...-glibc.../lib/liblibzstd.so
/gnu/store/...-glibc.../lib/liblibzstd.a
...
/gnu/store/...-zstd.../lib/liblibzstd.so
/gnu/store/...-zstd.../lib/liblibzstd.a
so it looks like it automatically adds the 'lib' at the front. When I've
built out to ncdu again I'll check it again.
> I'm building this branch on my personal Cuirass instance[1][2], for x86_64-linux
> and aarch64-linux (qemu-binfmt), in previous revisions I have indentified
> reproduciblity issue, and aarch64 builds timed-out. I haven't investigated them
> yet.
I've also been having berlin build zig on the wip-zig-bootstrap branch
for x86_64. Unfortunately there's quite a bit to go on aarch64 for it to
get there, but I can confirm from my machine that it's working.
> Then for what's the RUNPATH issue I have mentioned in commits:
> + For current zig <at> 0.10 on Guix master: glibc is missing from RUNPATH, which
> fails the validate-runpath check.
> + For zig <at> 0.11, some other inputs are missing, making the binary failing to run on Guix[3].
> + dan also mentioned privately to me that they needed to add paths from
> LIBRARY_PATH to RUNPATH for their own projects, so programs built by Zig is also
> affected.
I didn't test it. ncdu <at> 2.3 builds with zig-0.11, so that's an option for
testing it out.
https://dev.yorhel.nl/ncdu/changes2
> I think this is due to Zig's implemention of its own linking logic, which
> bypasses our ld-wrapper.
I wonder if switching from lld to make-lld-wrapper would make a
difference here.
> I'm not going to implement ld-wrapper within Zig. :) So my proposed workaround
> in wip-zig-bootstrap is to patch the handling logic added for Guix:
>
> (In lib/std/zig/system/NativePaths.zig)
> --8<---------------cut here---------------start------------->8---
> // Distros like guix don't use FHS, so they rely on environment
> // variables to search for headers and libraries.
> // We use os.getenv here since this part won't be executed on
> // windows, to get rid of unnecessary error handling.
> - if (std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| {
> + if (std.posix.getenv("CROSS_C_INCLUDE_PATH") orelse std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| {
> var it = mem.tokenizeScalar(u8, c_include_path, ':');
> while (it.next()) |dir| {
> try self.addIncludeDir(dir);
> }
> }
>
> - if (std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| {
> + if (std.posix.getenv("CROSS_CPLUS_INCLUDE_PATH") orelse std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| {
> var it = mem.tokenizeScalar(u8, cplus_include_path, ':');
> while (it.next()) |dir| {
> try self.addIncludeDir(dir);
> }
> }
>
> - if (std.posix.getenv("LIBRARY_PATH")) |library_path| {
> + if (std.posix.getenv("CROSS_LIBRARY_PATH") orelse std.posix.getenv("LIBRARY_PATH")) |library_path| {
> var it = mem.tokenizeScalar(u8, library_path, ':');
> while (it.next()) |dir| {
> try self.addLibDir(dir);
> + try self.addRPath(dir);
> }
> }
> }
> --8<---------------cut here---------------end--------------->8---
>
> Adding directories from CROSS_LIBRARY_PATH or LIBRARY_PATH to RUNPATH, "CROSS_"
> part is for our cross toolchain, I haven't tested it yet.
I like this, and it seems like it should make it work for cross
compiling zig programs. That's part of why I added the updated ncdu
commit, to use it for testing.
> I think this behavior change is reasonable since the search path
> (CROSS_)?LIBRARY_PATH is only automatically set by our compilers.
>
> I added this change to 0.9 as well to make all Zigs behave consistently. I also
> used shrink-runpath phase from meson-build-system in Zig and zig-build-system.
>
> I want to move shrink-runpath to (guix build utils) and export it too, so that
> it can be used easier. But I'm not sure if this change will trigger rebuilds of
> other packages, so I didn't do it.
I found that there was still the full LIBRARY_PATH embedded in the ncdu
binary as a string. So with that I'm not sure about using RPath instead
of LibDir for the LIBRARY_PATH.
> Thanks to Guile, for builds not managed by guix-daemon, something like the
> following script can be used, we can ship a program-file if we agree on this
> workaround.
> --8<---------------cut here---------------start------------->8---
> (use-modules (guix build meson-build-system))
>
> (define shrink-runpath
> (assoc-ref %standard-phases 'shrink-runpath))
>
> (define (main directories)
> (for-each (lambda (dir)
> (false-if-exception
> (shrink-runpath
> #:elf-directories '(".")
> #:outputs `(("out" . ,dir)))))
> directories))
>
> (main (cdr (command-line)))
> --8<---------------cut here---------------end--------------->8---
> Usage: guile <file-with-above-content> DIRECTORY...
>
>
> Thanks
> ---
> [1]: https://ci.boiledscript.com/jobset/guix-zig
> [2]: https://substitute.boiledscript.com, if you want to challenge it.
> [3]: https://github.com/ziglang/zig/issues/18434
I spent a bunch of time trying to get zig-0.10 or 0.10.0-610 to build on
riscv64 and ppc64le but haven't been able to crack it yet.
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
This bug report was last modified 193 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.