From unknown Sat Jun 21 03:27:23 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#52940 <52940@debbugs.gnu.org> To: bug#52940 <52940@debbugs.gnu.org> Subject: Status: [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. Reply-To: bug#52940 <52940@debbugs.gnu.org> Date: Sat, 21 Jun 2025 10:27:23 +0000 retitle 52940 [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. reassign 52940 guix-patches submitter 52940 Chris Marusich severity 52940 normal tag 52940 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 01 18:14:14 2022 Received: (at submit) by debbugs.gnu.org; 1 Jan 2022 23:14:15 +0000 Received: from localhost ([127.0.0.1]:59547 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n3nZm-00021J-6X for submit@debbugs.gnu.org; Sat, 01 Jan 2022 18:14:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:53142) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n3nZW-00020U-6d for submit@debbugs.gnu.org; Sat, 01 Jan 2022 18:14:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38524) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3nZW-0007gm-3J for guix-patches@gnu.org; Sat, 01 Jan 2022 18:13:58 -0500 Received: from [2607:f8b0:4864:20::434] (port=38872 helo=mail-pf1-x434.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n3nZR-0004yK-3o for guix-patches@gnu.org; Sat, 01 Jan 2022 18:13:56 -0500 Received: by mail-pf1-x434.google.com with SMTP id b22so26361869pfb.5 for ; Sat, 01 Jan 2022 15:13:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:user-agent:mime-version; bh=4V71Glf07hguPB/APnJtAywXNDnE+IBSH+JVsoeJ9FA=; b=mtCIXi9sK1cBP8hiIjOnOngzRkCO/na/zAvAP88m3ilXgEoIvoH+dN9Dzbv7rlzPKa ANWww2aTtQP7FpxdFPQemisyUfpDLZrKx0DcJx/Wl2cl+kDJzpi1cK87Z5yo45AAoT9I uK2zJ+adkf4Judt6xWaKEGHmooWQeJ2Poa07g3EDdJBe/Fj0YiHriMMTS9NU1WYjkZoU GU2zQ8KekNCFjc4nuYg28ovYZvgPLExzD6t/4TYSBTkB4LPvQS1cEAknbNQ51Z5KxD4P GpUXQeigLIW+BqezSaGLRgzFI4ZoL4ZTrlgdp05FARsXFMDBw91kRZGefI4322a2Ce47 MJzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=4V71Glf07hguPB/APnJtAywXNDnE+IBSH+JVsoeJ9FA=; b=VD1/7eZKGxSr39RNv5NrqvmFqn8WbM6dZdsCpSaB94cYlSUkJlEnhR79j9Sx5byCK6 FoG4rk2VeqRtNNqL7yyGNuVhlhpzqePUqs2nb9KxEf2foxoK7EEBoTbUvORcF1qbkAqJ WQTtnmuI9QuDuMGpL8rjsCk5Mllt6aSFkt2cHtnld/0sH8LtAysEzDQW7Ww0poVcUOCm Mg+yCVOcWnAoSFaSensFEV49XqR2mz8jFhpqDlSMAEMuTMQYL/uwFzXHwggB4AyNub+I 5s8UnAcfleR97SaXvVgLgqS4jPCrtsgORTQBphBm7oKpNWUygGnpJi8GDqm0/kDnevUV pDeQ== X-Gm-Message-State: AOAM530kUZhmvcEbofAt8XyfdT2Z8sCVamZpik6Z2usyCbZr6zKGDTYh tPc0SAv1KEsxzbNo0uEP8xwlXNBOS2c= X-Google-Smtp-Source: ABdhPJzV9vpTvWu1Em1P93/BlX3cDX1E5TgGi9y1rg8HUKphHgYwOUb0ScH25E6ad5YslwaCSjTHKg== X-Received: by 2002:a05:6a00:847:b0:4ba:f004:e380 with SMTP id q7-20020a056a00084700b004baf004e380mr40510756pfk.42.1641078828914; Sat, 01 Jan 2022 15:13:48 -0800 (PST) Received: from garuda-lan (c-24-18-44-142.hsd1.wa.comcast.net. [24.18.44.142]) by smtp.gmail.com with ESMTPSA id s14sm33912403pfk.171.2022.01.01.15.13.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Jan 2022 15:13:48 -0800 (PST) From: Chris Marusich To: guix-patches@gnu.org Subject: [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. Date: Sat, 01 Jan 2022 15:13:43 -0800 Message-ID: <87zgofaw2g.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::434 (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=cmmarusich@gmail.com; helo=mail-pf1-x434.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 2.3 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Guix, I've noticed that test file-needed/recursive in tests/gremlin.scm fails on master branch on powerpc64le-linux. It does not fail on x86_64-linux. I've attached a patch that attempts to fix the issue. T [...] Content analysis details: (2.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (cmmarusich[at]gmail.com) 0.0 T_SPF_TEMPERROR SPF: test of record failed (temperror) -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 1.3 SPOOFED_FREEMAIL No description available. 0.9 SPOOF_GMAIL_MID From Gmail but it doesn't seem to be... X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Hi Guix, I've noticed that test file-needed/recursive in tests/gremlin.scm fails on master branch on powerpc64le-linux. It does not fail on x86_64-linux. I've attached a patch that attempts to fix the issue. The primary issue is that it does not deduplicate entries in the same way as ld.so when there are multiple entries referring to the same shared object. The patch changes file-needed/recursive to behave more like ld.so. Here is the failing test output, including the test source: --8<---------------cut here---------------start------------->8--- ;;; (truth ("linux-vdso64.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile-3.0.7/lib/libguile-3.0.so.1" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkmsqs5pj09-libgc-8.0.4/lib/libgc.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libpthread.so.0" "/gnu/store/521riv2sgv0b0s4j0kzz6i52rf9rarh8-libffi-3.3/lib/../lib/libffi.so.7" "/gnu/store/xj20v8lk2wal0z1rla0yx3bjkasbx6mq-libunistring-0.9.10/lib/libunistring.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libm.so.6" "/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/libgcc_s.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/ld64.so.2")) ;;; (needed ("/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6" "/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/libgcc_s.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libm.so.6" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt.so.1" "/gnu/store/xj20v8lk2wal0z1rla0yx3bjkasbx6mq-libunistring-0.9.10/lib/libunistring.so.2" "/gnu/store/521riv2sgv0b0s4j0kzz6i52rf9rarh8-libffi-3.3/lib/../lib/libffi.so.7" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libpthread.so.0" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkmsqs5pj09-libgc-8.0.4/lib/libgc.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile-3.0.7/lib/libguile-3.0.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/ld64.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib/libc.so.6")) test-name: file-needed/recursive location: /home/marusich/guix-master/tests/gremlin.scm:70 source: + (test-assert + "file-needed/recursive" + (let* ((needed + (file-needed/recursive %guile-executable)) + (pipe (dynamic-wind + (lambda () + (setenv "LD_TRACE_LOADED_OBJECTS" "yup")) + (lambda () + (open-pipe* OPEN_READ %guile-executable)) + (lambda () (unsetenv "LD_TRACE_LOADED_OBJECTS"))))) + (define ldd-rx + (make-regexp + "^[[:blank:]]+([[:graph:]]+ => )?([[:graph:]]+) .*$")) + (define (read-ldd-output port) + (let loop ((result '())) + (match (read-line port) + ((? eof-object?) (reverse result)) + ((= (cut regexp-exec ldd-rx <>) m) + (if m + (loop (cons (match:substring m 2) result)) + (loop result)))))) + (define ground-truth + (remove + (cut string-prefix? "linux-vdso.so" <>) + (read-ldd-output pipe))) + (and (zero? (close-pipe pipe)) + (lset= string=? + (pk 'truth ground-truth) + (pk 'needed needed))))) actual-value: #f result: FAIL --8<---------------cut here---------------end--------------->8--- For reference, here is the actual dynamic section of of %guile-executable on this system, as reported by readelf: --8<---------------cut here---------------start------------->8--- $ readelf -d /gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile-3.0.7/bin/guile Dynamic section at offset 0xfc60 contains 37 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libguile-3.0.so.1] 0x0000000000000001 (NEEDED) Shared library: [libgc.so.1] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libffi.so.7] 0x0000000000000001 (NEEDED) Shared library: [libunistring.so.2] 0x0000000000000001 (NEEDED) Shared library: [libcrypt.so.1] 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile-3.0.7/lib:/gnu/store/7x2cjqbmpgwrgmnb234gsxkmsqs5pj09-libgc-8.0.4/lib:/gnu/store/521riv2sgv0b0s4j0kzz6i52rf9rarh8-libffi-3.3/lib/../lib:/gnu/store/xj20v8lk2wal0z1rla0yx3bjkasbx6mq-libunistring-0.9.10/lib:/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib:/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib:/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib:/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/gcc/powerpc64le-unknown-linux-gnu/7.5.0/../../../../lib] 0x000000000000000c (INIT) 0x10000980 0x000000000000000d (FINI) 0x10000ef4 0x0000000000000019 (INIT_ARRAY) 0x1001fc50 0x000000000000001b (INIT_ARRAYSZ) 8 (bytes) 0x000000000000001a (FINI_ARRAY) 0x1001fc58 0x000000000000001c (FINI_ARRAYSZ) 8 (bytes) 0x0000000000000004 (HASH) 0x10000268 0x000000006ffffef5 (GNU_HASH) 0x100002c0 0x0000000000000005 (STRTAB) 0x10000470 0x0000000000000006 (SYMTAB) 0x100002f0 0x000000000000000a (STRSZ) 891 (bytes) 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000015 (DEBUG) 0x0 0x0000000000000003 (PLTGOT) 0x10020000 0x0000000000000002 (PLTRELSZ) 216 (bytes) 0x0000000000000014 (PLTREL) RELA 0x0000000000000017 (JMPREL) 0x10000880 0x0000000070000000 (PPC64_GLINK) 0x10000eb0 0x0000000070000003 (PPC64_OPT) 0x0 0x0000000000000007 (RELA) 0x10000850 0x0000000000000008 (RELASZ) 48 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffffe (VERNEED) 0x10000810 0x000000006fffffff (VERNEEDNUM) 2 0x000000006ffffff0 (VERSYM) 0x100007ec 0x0000000000000000 (NULL) 0x0 --8<---------------cut here---------------end--------------->8--- Note that the RUNPATH above contains an entry for "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib" followed later by "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib". It seems that ld.so's tracing mechanism is smart enough to avoid printing the second entry. So, the test fails because the "needed" list is not set-equivalent to the "truth" list. There are two reasons why they are not set-equivalent: A) "truth" contains "linux-vdso64.so.1", but "needed" does not. B) "needed" contains "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib/libc.so.6", but "truth" does not. However, both contain "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6", which refers to the same file. Regarding (A), it seems to be an error in the test logic. The test code already filters out strings beginning with "linux-vdso.so" from the "truth" list.: (define ground-truth (remove (cut string-prefix? "linux-vdso.so" <>) (read-ldd-output pipe))) The intent seems to be to filter out the vdso shared object from the "truth" list. However, it fails to do so in this case, since the name of the vdso shared object is actually "linux-vdso64.so.1". My patch fixes this by filtering out strings that begin with "linux-vdso64.so", too. Regarding (B), it seems to occur because ld.so deduplicates entries. I checked the glibc source code, but I had a hard time figuring out exactly how exactly the deduplication works. In any case, based on ld.so's actual behavior, it seems that ld.so does in fact deduplicate entries, and file-needed/recursive does not. This explains the difference. What is a good solution for (B)? I can think of the following potential solutions: 1) Try to avoid introducing multiple entries referring to the same thing in the first place. Somehow, somewhere, something is adding the second entry to the dynamic section of Guile's ELF file. It happens on powerpc64le-linux but not on x86_64-linux. What code or tool is doing this? I don't know, but I guess I would start by looking at the gnu-build-system code. I'm not sure if it's a really problem, though, so I'm not eager to jump down this rabbit hole just yet. 2) Change the test so that it passes even if file-needed/recursive returns multiple entries referring to the same file. In other words, accept that the current behavior is OK, even if it means that the results returned by file-needed/recursive are not always exactly the same as the results returned by ld.so. 3) Try to change file-needed/recursive so that it does not return multiple entries referring to the same file. In other words, make it behave more like ld.so. I can't think of a reason why the current behavior of file-needed/recursive is bad, but it was simple enough to make it deduplicate entries similarly to ld.so. So, my patch implements solution (3). Hopefully it's good enough! -- Chris PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-gremlin-Mimic-ld.so-NEEDED-deduplication-behavior.patch Content-Transfer-Encoding: quoted-printable From=2067365d79afc7182aefbacf360941f338aea712b6 Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Sat, 1 Jan 2022 14:17:38 -0800 Subject: [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. Together, these two changes fix the file-needed/recursive test, which was failing on powerpc64le-linux. It was not failing on x86_64-linux. The test failure on powerpc64le-linux was caused by two issues. First, file-needed/recursive did not deduplicate entries in the same way as ld.so. The %guile-executable ELF file contains in its RUNPATH both "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib" and "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib". Altho= ugh ld.so deduplicates the second entry, file-needed/recursive did not. Second, the vdso shared library name is "linux-vdso64.so.1", but the test incorrect= ly assumed that the vdso shared library would always begin with "linux-vdso.so= ". * guix/build/gremlin.scm (file-needed/recursive)[contains-canonical-file?]: New procedure. Use it to deduplicate entries that refer to the same file. * tests/gremlin.scm (file-needed/recursive)[ground-truth]: In addition to strings that begin with "linux-vdso.so", remove strings that begin with "linux-vdso64.so". =2D-- guix/build/gremlin.scm | 12 +++++++++++- tests/gremlin.scm | 5 ++++- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/guix/build/gremlin.scm b/guix/build/gremlin.scm index 2a74d51dd9..e90e59679b 100644 =2D-- a/guix/build/gremlin.scm +++ b/guix/build/gremlin.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2015, 2018, 2020 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2022 Chris Marusich ;;; ;;; This file is part of GNU Guix. ;;; @@ -268,6 +269,10 @@ recursively, and the list of .so file names that could= not be found. File names are resolved by searching the RUNPATH of the file that NEEDs them. =20 This is similar to the info returned by the 'ldd' command." + (define (contains-canonical-file? file files) + (any (lambda (entry) + (string=3D? (canonicalize-path entry) (canonicalize-path file))) + files)) (let loop ((files (list file)) (result '()) (not-found '())) @@ -292,10 +297,15 @@ This is similar to the info returned by the 'ldd' com= mand." (not (libc-library? ne= eded)) needed)) needed resolved)) + ;; Deduplicate entries that refer to the same file. + ;; The actual ld.so tracing behavior is similar and + ;; will de-duplicate entries even if they have + ;; different names but refer to the same file. (needed (remove (lambda (value) (or (not value) ;; XXX: quadratic =2D (member value result))) + (contains-canonical-file? + value result))) resolved))) (loop (append rest needed) (append needed result) diff --git a/tests/gremlin.scm b/tests/gremlin.scm index 9af899c89a..86757e62b4 100644 =2D-- a/tests/gremlin.scm +++ b/tests/gremlin.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2015, 2018, 2020 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2022 Chris Marusich ;;; ;;; This file is part of GNU Guix. ;;; @@ -92,7 +93,9 @@ (loop result)))))) =20 (define ground-truth =2D (remove (cut string-prefix? "linux-vdso.so" <>) + (remove (lambda (entry) + (or (string-prefix? "linux-vdso.so" entry) + (string-prefix? "linux-vdso64.so" entry))) (read-ldd-output pipe))) =20 (and (zero? (close-pipe pipe)) =2D-=20 2.26.3 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJJBAEBCAAzFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAmHQ4CgVHGNtbWFydXNp Y2hAZ21haWwuY29tAAoJEN1AmhXYIkad8gEQAMmprHCJ5TziVbRREZHTKfs3CUpz Vzl2cU6fNoXyMuU/pJr3DY1aRQGemyfS2DeHgjdbxA1z/qdqmj++iRJxfXiwmiEI YOdmEeHCbh/4Uv3bqAt+U0Ox7pomNqHlK+D1ZZdve+NqxHNz3pcOgD6fSv9OvZ9B OYTx+CbrLepAzyZIq7uAdvFyb4/Zs1rhOq5k4v8sOzSvkkl7zDjVLaWyk1bVn2X0 GY4v/URFkTho/RSVYVarXqmWo0tRgodYC2sCjz+RbmpdYDYtRZIxgU9HvXwAjqkO 9mGnkIuombYh2Ko+wlHyabkcpCKRudlsv+6NuQW9vTBV7mCYovrmpeQAoSJFEUpt yboxoHeo194jj/1JVLiYil38cpqt/yIwsoYmLFog/9eagoFINqxE+wjBoojl6/za YAzlwhP1aXtH5YrnZYhhUPTn/ERADp62+O2/WxjLRM0Al/T19IpV4RxkhHgc5h2q JHKXuq9lWsvQsacY5L/ScC7Uv7IwA39PcwSJCmx70LgGNQZiT8k6FR2ajHpGRoiZ RtCjZijllGnChzep+ThWubxGTGfgLx9jzeOFXh2/OgvsMtP77EMCUjZXWbc7lc9l Uqbtei3hOStCKWeOzChrOKpBelFWxGjGNOs2vpH2rzEQuSqi6C40CwBfOg5397DH XlVpLYhhiEYCtJzH =Xlle -----END PGP SIGNATURE----- --==-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 05 14:07:26 2022 Received: (at 52940) by debbugs.gnu.org; 5 Jan 2022 19:07:26 +0000 Received: from localhost ([127.0.0.1]:41823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n5Bd7-0002rH-QH for submit@debbugs.gnu.org; Wed, 05 Jan 2022 14:07:26 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:42106) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n5Bd5-0002r0-08 for 52940@debbugs.gnu.org; Wed, 05 Jan 2022 14:07:24 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 6C1461B7; Wed, 5 Jan 2022 20:07:16 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24007vY1uIAk; Wed, 5 Jan 2022 20:07:15 +0100 (CET) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id DDEC8112; Wed, 5 Jan 2022 20:07:14 +0100 (CET) From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Chris Marusich Subject: Re: bug#52940: [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. References: <87zgofaw2g.fsf@gmail.com> Date: Wed, 05 Jan 2022 20:07:14 +0100 In-Reply-To: <87zgofaw2g.fsf@gmail.com> (Chris Marusich's message of "Sat, 01 Jan 2022 15:13:43 -0800") Message-ID: <87a6gaknml.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / Authentication-Results: hera.aquilenet.fr; none X-Rspamd-Server: hera X-Rspamd-Queue-Id: 6C1461B7 X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 52940 Cc: 52940@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Chris, Chris Marusich skribis: > Here is the failing test output, including the test source: > > ;;; (truth ("linux-vdso64.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr= 0zs-guile-3.0.7/lib/libguile-3.0.so.1" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkm= sqs5pj09-libgc-8.0.4/lib/libgc.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybj= a1dvqb-glibc-2.31/lib/libpthread.so.0" "/gnu/store/521riv2sgv0b0s4j0kzz6i52= rf9rarh8-libffi-3.3/lib/../lib/libffi.so.7" "/gnu/store/xj20v8lk2wal0z1rla0= yx3bjkasbx6mq-libunistring-0.9.10/lib/libunistring.so.2" "/gnu/store/sipyfs= 2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt.so.1" "/gnu/store/sipyfs= 2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl.so.2" "/gnu/store/sipyfs254= 0b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libm.so.6" "/gnu/store/ys7b4gr5nbq8s= fnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/libgcc_s.so.1" "/gnu/store/sipyfs2540= b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6" "/gnu/store/sipyfs2540b48b= 2sb9j8ypmybja1dvqb-glibc-2.31/lib/ld64.so.2")) > > ;;; (needed ("/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/= libc.so.6" "/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/l= ibgcc_s.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/l= ibm.so.6" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl= .so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt= .so.1" "/gnu/store/xj20v8lk2wal0z1rla0yx3bjkasbx6mq-libunistring-0.9.10/lib= /libunistring.so.2" "/gnu/store/521riv2sgv0b0s4j0kzz6i52rf9rarh8-libffi-3.3= /lib/../lib/libffi.so.7" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc= -2.31/lib/libpthread.so.0" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkmsqs5pj09-lib= gc-8.0.4/lib/libgc.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile= -3.0.7/lib/libguile-3.0.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-= glibc-2.31/lib/ld64.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glib= c-2.31/lib/../lib/libc.so.6")) [...] > Note that the RUNPATH above contains an entry for > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib" followed > later by > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib". It = seems > that ld.so's tracing mechanism is smart enough to avoid printing the > second entry. > > So, the test fails because the "needed" list is not set-equivalent to > the "truth" list. There are two reasons why they are not > set-equivalent: > > A) "truth" contains "linux-vdso64.so.1", but "needed" does not. > > B) "needed" contains > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib/libc.s= o.6", > but "truth" does not. However, both contain > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6", > which refers to the same file. [...] > What is a good solution for (B)? I can think of the following potential > solutions: > > 1) Try to avoid introducing multiple entries referring to the same thing > in the first place. Somehow, somewhere, something is adding the second > entry to the dynamic section of Guile's ELF file. It happens on > powerpc64le-linux but not on x86_64-linux. What code or tool is doing > this? I don't know, but I guess I would start by looking at the > gnu-build-system code. I'm not sure if it's a really problem, though, > so I'm not eager to jump down this rabbit hole just yet. > > 2) Change the test so that it passes even if file-needed/recursive > returns multiple entries referring to the same file. In other words, > accept that the current behavior is OK, even if it means that the > results returned by file-needed/recursive are not always exactly the > same as the results returned by ld.so. > > 3) Try to change file-needed/recursive so that it does not return > multiple entries referring to the same file. In other words, make it > behave more like ld.so. > > I can't think of a reason why the current behavior of > file-needed/recursive is bad, but it was simple enough to make it > deduplicate entries similarly to ld.so. So, my patch implements > solution (3). Hopefully it's good enough! Good catch! We could go fancy and have =E2=80=98loop=E2=80=99 in =E2=80=98file-needed/recursive=E2=80=99 thread a map of device/inode number= pairs to file names; when calling =E2=80=98search-path=E2=80=99, we=E2=80=99d check = whether the file we found already is in the set, possibly under a different name, and we=E2=80= =99d use that name instead of introducing a new one. That=E2=80=99d be more efficient that calling =E2=80=98canonicalize-path=E2=80=99, especially O(n= =C2=B3) times roughly. But=E2=80=A6 given that this is a corner case, that modifying (guix build gremlin) entails a full rebuild, and that there just should be that =E2=80=9C/lib/../lib=E2=80=9D entry in the first place, I=E2=80=99d lean to= wards leaving gremlin.scm unchanged. WDYT? However=E2=80=A6 > diff --git a/tests/gremlin.scm b/tests/gremlin.scm > index 9af899c89a..86757e62b4 100644 > --- a/tests/gremlin.scm > +++ b/tests/gremlin.scm > @@ -1,5 +1,6 @@ > ;;; GNU Guix --- Functional package management for GNU > ;;; Copyright =C2=A9 2015, 2018, 2020 Ludovic Court=C3=A8s > +;;; Copyright =C2=A9 2022 Chris Marusich > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -92,7 +93,9 @@ > (loop result)))))) >=20=20 > (define ground-truth > - (remove (cut string-prefix? "linux-vdso.so" <>) > + (remove (lambda (entry) > + (or (string-prefix? "linux-vdso.so" entry) > + (string-prefix? "linux-vdso64.so" entry))) > (read-ldd-output pipe))) >=20=20 > (and (zero? (close-pipe pipe)) =E2=80=A6 I think this part should definitely be committed (=E2=80=98master= =E2=80=99 is fine). Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 08 21:04:26 2022 Received: (at 52940-close) by debbugs.gnu.org; 9 Jan 2022 02:04:26 +0000 Received: from localhost ([127.0.0.1]:49000 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n6NZK-0001g0-Dj for submit@debbugs.gnu.org; Sat, 08 Jan 2022 21:04:26 -0500 Received: from mail-pl1-f175.google.com ([209.85.214.175]:41844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n6NZH-0001fj-DS for 52940-close@debbugs.gnu.org; Sat, 08 Jan 2022 21:04:24 -0500 Received: by mail-pl1-f175.google.com with SMTP id z3so9100599plg.8 for <52940-close@debbugs.gnu.org>; Sat, 08 Jan 2022 18:04:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=rSJh+cMMzSiUjTLnaaesS863SqXYhjsUBh00YzQ+uYM=; b=Hb7jALTaLd/VZzI2T9tQiHlhBcFy/aUxH5zV7HFj0FMKP6M9YfvVUxcZh10eVDhUgK whTsQFeXvNGVrla4HaGJkf6/lz3cCU52AS2vJoIPTUHzs2U38bh4LzTQ/KcrTrk0NslC d+yK+E9JI1t/n53u+vpBP9Ge3HrspSUTONrTrJNXBwtv0JWorR47CcbKzPVygWvgBXNZ jqkBRd9VImxo0gdVUTiwaS5Z/lriD0VCe2BK4pitn8dS1N6PJjSOgojCZOCXtPkHFn+Q GZa1lSI/+z+q8dNZBTITTfkLBaIryFp6afWbi/UROf0FRvYSMo10Q/4rgKVUfuKX1Vln 5ygw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=rSJh+cMMzSiUjTLnaaesS863SqXYhjsUBh00YzQ+uYM=; b=RlUuYWQyRfw6sFrdmqGCwhM0FRn4IS8ztRRJU2j5bSxd3VroMY3LixjgtdZJcsR3iO 6SsrZT8uyxjQaVLDzahmADkhK3DmxUebe/dkBo1b6G/isYbuS2QDzRfyqRW/dy5BM8cg cctYzMTCFFaUPWsoekSXplRUIPhPHAucSiGXbRaAwrJHczmS1lYqByiAUSMd3K8X9ghy roJJpi0dhqi8tLtaxaURzRAxCWUnu+985QbOdH6ugQmamlRlqxt9Jn+2p3tAGem7eY4Q /SuOuVHgLX0CjvzOJ4AhnMvy9nG4ZFI6Lu4WE6glDkx8FmAnGfnDPSa3Y1bpcrcRp+xk +o5w== X-Gm-Message-State: AOAM532BrOkzbXxaL4+Co3jn8w9CACbUWrTSBg+l4d3FW44v1je07SnR tkevEKooO+MRW2KA1g8g+4dQSr8kuGI= X-Google-Smtp-Source: ABdhPJyQQl3B63WBVfYOCs+659W+f/IrZeHS7aj1+OJik2i8Mav4tejIw7ehEdyKBigZk+SWhRPlSw== X-Received: by 2002:a17:902:bd4b:b0:148:fdda:e85b with SMTP id b11-20020a170902bd4b00b00148fddae85bmr72549620plx.116.1641693856846; Sat, 08 Jan 2022 18:04:16 -0800 (PST) Received: from garuda-lan (c-24-18-44-142.hsd1.wa.comcast.net. [24.18.44.142]) by smtp.gmail.com with ESMTPSA id m9sm3000349pjk.51.2022.01.08.18.04.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Jan 2022 18:04:16 -0800 (PST) From: Chris Marusich To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#52940: [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. References: <87zgofaw2g.fsf@gmail.com> <87a6gaknml.fsf@gnu.org> Date: Sat, 08 Jan 2022 18:04:11 -0800 In-Reply-To: <87a6gaknml.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 05 Jan 2022 20:07:14 +0100") Message-ID: <87czl1hdgk.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 52940-close Cc: 52940-close@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, Ludovic Court=C3=A8s writes: > Good catch! We could go fancy and have =E2=80=98loop=E2=80=99 in > =E2=80=98file-needed/recursive=E2=80=99 thread a map of device/inode numb= er pairs to > file names; when calling =E2=80=98search-path=E2=80=99, we=E2=80=99d chec= k whether the file we > found already is in the set, possibly under a different name, and we=E2= =80=99d > use that name instead of introducing a new one. That=E2=80=99d be more > efficient that calling =E2=80=98canonicalize-path=E2=80=99, especially O(= n=C2=B3) times > roughly. > > But=E2=80=A6 given that this is a corner case, that modifying (guix build > gremlin) entails a full rebuild, and that there just should be that > =E2=80=9C/lib/../lib=E2=80=9D entry in the first place, I=E2=80=99d lean = towards leaving > gremlin.scm unchanged. > > WDYT? > > However=E2=80=A6 > >> diff --git a/tests/gremlin.scm b/tests/gremlin.scm >> index 9af899c89a..86757e62b4 100644 >> --- a/tests/gremlin.scm >> +++ b/tests/gremlin.scm >> @@ -1,5 +1,6 @@ >> ;;; GNU Guix --- Functional package management for GNU >> ;;; Copyright =C2=A9 2015, 2018, 2020 Ludovic Court=C3=A8s >> +;;; Copyright =C2=A9 2022 Chris Marusich >> ;;; >> ;;; This file is part of GNU Guix. >> ;;; >> @@ -92,7 +93,9 @@ >> (loop result)))))) >>=20=20 >> (define ground-truth >> - (remove (cut string-prefix? "linux-vdso.so" <>) >> + (remove (lambda (entry) >> + (or (string-prefix? "linux-vdso.so" entry) >> + (string-prefix? "linux-vdso64.so" entry))) >> (read-ldd-output pipe))) >>=20=20 >> (and (zero? (close-pipe pipe)) > > =E2=80=A6 I think this part should definitely be committed (=E2=80=98mast= er=E2=80=99 is fine). I agree that the existing behavior is probably fine. It makes me feel better to know that you also think so. With that in mind, I've committed a simpler fix in 6a2050b on master that just changes the test. I've also updated the guix package two times. Now, after running "guix pull", I can once again successfully build the guix package on powerpc64le-linux. Hooray! Thank you for the review! I'll now close this bug. =2D-=20 Chris PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=3D106836 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJJBAEBCAAzFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAmHaQpsVHGNtbWFydXNp Y2hAZ21haWwuY29tAAoJEN1AmhXYIkadL8oQAKAkKMSQXZ2ZcoJH8i1wnEjsA63V ebcbMaX/6K0g93YzlW28hQ0pTsBrla5WS+Jb1cgL1HaQMOybGhXiTQTbM1c9TlOj Bao98VdxqM7WzLV/BGwh85dUKKCdnzU4gjbfZ3imstFPUsHhc6GgujZatQXXQ5p4 Xdg4AFgpLPCZB4Cw8HK6J/MOvppwQN0jqe1Q27uv/Rkhu780fDYWA+Ft4+4zThDV 6x95wvhaDoPnM21+H1RX/6qH6Vxzkr/prFhTND85ZXBFYHhAy85Ng+ZUg88bo7yR jLhdW0oKQAYLEZBRWvn1s4QiM/ALq3eeOZ9dUP7pFlP5XTTj8qpRpoB+gouJ78u3 VMjTDYzlZ5Shqpt2WG9WPVV9HHN5oGOcSeZlSsq+vgqV7VpK2BCv+dy1lDG1Pw1r nbQ+o5eqs+pMsELam2gvFHqO8NXUBoOvW+eOaPuVoqwiYM9Ouc+Irq1TjAIIkUzu pntOkS7/tRnP84m2k73o0eNomWqWNP5lhdUkUgI0VPfi2QhG7HhXL6m0A8IhW02e YlSN2rjyB7n76VI+321bc5KF751Mk9EighIiBE7bsMqj3AXfkq//Kz2aPYiFwq/+ OYnIhvtVvY57bRP0M6blMQnGT80KGxrG66zzD7aYGjWypJjKij9GW6qyHgCaSNPg Nr8+hAhI3zz7qonC =XwjQ -----END PGP SIGNATURE----- --=-=-=-- From unknown Sat Jun 21 03:27:23 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 06 Feb 2022 12:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator