From unknown Fri Jun 13 10:12:54 2025 X-Loop: help-debbugs@gnu.org Subject: bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Resent-From: Ricardo Wurmus Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 20 Nov 2021 15:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 52001 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 52001@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16374228375256 (code B ref -1); Sat, 20 Nov 2021 15:41:02 +0000 Received: (at submit) by debbugs.gnu.org; 20 Nov 2021 15:40:37 +0000 Received: from localhost ([127.0.0.1]:43529 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moSTk-0001Mh-Ml for submit@debbugs.gnu.org; Sat, 20 Nov 2021 10:40:36 -0500 Received: from lists.gnu.org ([209.51.188.17]:49034) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moSTi-0001Ma-VF for submit@debbugs.gnu.org; Sat, 20 Nov 2021 10:40:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42476) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1moSTi-00071h-N6 for bug-guix@gnu.org; Sat, 20 Nov 2021 10:40:34 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21134) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1moSTg-0001o1-KW for bug-guix@gnu.org; Sat, 20 Nov 2021 10:40:34 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1637422828; cv=none; d=zohomail.com; s=zohoarc; b=fJcwzjuqw066f9SlXDPVIzgaNbYNCDAiJzIHKcy6f5r/H4BYikEjBC0963csu7ymI9N8qWqypTWj3xQhA+vBDkOQcvQwD0uVGxmhPhW83Yh0T2JwuAa8DkXwVkp8iTzzZf3X7qgIurWzKhRwQOGMwvV2IaZ6xexxBMClOfg9kd0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1637422828; h=Content-Type:Content-Transfer-Encoding:Date:From:MIME-Version:Message-ID:Subject:To; bh=OcWpYtrqVOyEXTSpl2Cv1ORWhRA7emZ78FRxbfEVAVI=; b=R2Odx8U99vCg+dfe1RjIZ3ANs9bdVPVr8eYXv5tMjE8lDYzxOxffDVr/TiIQTBIn+ncSZbIeRun9bLIwQA54iOJnyQRJrwSXt+E7d9LRL/fI8d48mszZaIfRgIYjXT5a7N2qzGZak93T2oncN6bjvGQHXnYOWVAcy/eoM++gfrE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1637422828; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=OcWpYtrqVOyEXTSpl2Cv1ORWhRA7emZ78FRxbfEVAVI=; b=g4GTaUh1tqexaxMGniGYqTvf4qNWx5xhb7r4JaZa99sBNk+RhcC79kyIFqFGJlnr 3QzaaOqHZy3UzgLl08BmyUiMrSBfzFGmMklxvHaIycIvMnrRZsvpQPYwtBOXm4NA7cZ +en2U9t0XEu5pFTe8UikhFyEF2qo7jRSEUWda2gA= Received: from localhost (p4fd5aaa8.dip0.t-ipconnect.de [79.213.170.168]) by mx.zohomail.com with SMTPS id 1637422825345923.176776713093; Sat, 20 Nov 2021 07:40:25 -0800 (PST) User-agent: mu4e 1.6.6; emacs 27.2 From: Ricardo Wurmus Date: Sat, 20 Nov 2021 15:36:57 +0000 X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <87y25ihm3e.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Numpy accelerates operations with native code. Early in the build=20 it probes the CPU / compiler for supported instruction extensions=20 and then proceeds to build extensions for different features. I=E2=80=99m= =20 guessing that this won=E2=80=99t work when building on older CPUs, so the=20 results of two builds on different CPUs may differ. It would be helpful if someone could confirm this by building=20 numpy on an older CPU and then compare the build result with that=20 provided by ci.guix.gnu.org. --=20 Ricardo From unknown Fri Jun 13 10:12:54 2025 X-Loop: help-debbugs@gnu.org Subject: bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 20 Nov 2021 16:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52001 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ricardo Wurmus , 52001@debbugs.gnu.org Received: via spool by 52001-submit@debbugs.gnu.org id=B52001.163742561710625 (code B ref 52001); Sat, 20 Nov 2021 16:27:01 +0000 Received: (at 52001) by debbugs.gnu.org; 20 Nov 2021 16:26:57 +0000 Received: from localhost ([127.0.0.1]:43556 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moTCb-0002lJ-5M for submit@debbugs.gnu.org; Sat, 20 Nov 2021 11:26:57 -0500 Received: from xavier.telenet-ops.be ([195.130.132.52]:42886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moTCX-0002l4-GO for 52001@debbugs.gnu.org; Sat, 20 Nov 2021 11:26:55 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by xavier.telenet-ops.be with bizsmtp id LgSn260094UW6Th01gSn9D; Sat, 20 Nov 2021 17:26:50 +0100 Message-ID: From: Maxime Devos Date: Sat, 20 Nov 2021 16:26:47 +0000 In-Reply-To: <87y25ihm3e.fsf@elephly.net> References: <87y25ihm3e.fsf@elephly.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r21; t=1637425610; bh=9WF5zrHSERbYcaD+9AJGK7qmF3wRXwg+C5cPBySkjM0=; h=Subject:From:To:Date:In-Reply-To:References; b=FDyEqpdjGRQrIPyp4HnLurtmMfI0fUy7vJTWYTCMWrHpdjvyxur7h+YpGg+9rmmhX 7YJTUb07WXVd3yFNoEDh5111FNQyS9hAHzsMzZmKPWoJIQt7kLs7SGrEe4HIV30Qyr d4aI8cvVIphMeek/p6Xl+OWn2DdLBQN75oZS6X0a56PIMEvaj/dlGt79aNSFoZY7q6 r/vV1U9oeYx6mImy9RDvvYBjHeqVRMfsAyYa/u9KERWS/Zze5AGpRkO9aGi0A9rfSw b0zVDJKpqmer+2NayLp/mQIDNvPTQTHOq+8nuMSAhglgnVVLuV/O0fj58OAjF9wA4c UFYNcJ1gcMyww== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Ricardo Wurmus schreef op za 20-11-2021 om 15:36 [+0000]: > Numpy accelerates operations with native code.  Early in the build > it probes the CPU / compiler for supported instruction extensions > and then proceeds to build extensions for different features.  I’m > guessing that this won’t work when building on older CPUs, so the > results of two builds on different CPUs may differ. > > It would be helpful if someone could confirm this by building > numpy on an older CPU and then compare the build result with that > provided by ci.guix.gnu.org. > I built it locally, and have the same result as ci.guix.gnu.org. However, the bordeaux substitute is different: $ LANGUAGE=en guix challenge python-numpy /gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3 contents differ: local hash: 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7 https://ci.guix.gnu.org/nar/lzip/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3 : 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7 https://bordeaux.guix.gnu.org/nar/lzip/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3 : 0cbl3q19bshb6ddz8xkcrjzkcmillsqii4z852ybzixyp7rg40qa bordeaux.guix.gnu.org 3.9MiB 6.7MiB/s 00:01 [##################] 100.0% 1 store items were analyzed: - 0 (0.0%) were identical - 1 (100.0%) differed - 0 (0.0%) were inconclusive Diffoscope output: bordeaux.guix.gnu.org 3.9MiB 4.2MiB/s 00:01 [##################] 100.0% [...] │ │ │ │ │ │ │ │ --- /tmp/guix-directory.6ivbVt/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc │ │ │ │ │ │ │ ├── +++ /gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc │ │ │ │ │ │ │ │ @@ -106,15 +106,15 @@ │ │ │ │ │ │ │ │ 00000690: 0104 0106 0104 0104 0106 0104 f906 0904 ................ │ │ │ │ │ │ │ │ 000006a0: 0104 0208 0308 06da 085f 5f6d 6169 6e5f .........__main_ │ │ │ │ │ │ │ │ 000006b0: 5f29 01da 036c 6f67 e902 0000 0029 01da _)...log.....).. │ │ │ │ │ │ │ │ 000006c0: 1463 7573 746f 6d69 7a65 645f 6663 6f6d .customized_fcom │ │ │ │ │ │ │ │ 000006d0: 7069 6c65 7272 0700 0000 2901 da08 636f pilerr....)...co │ │ │ │ │ │ │ │ 000006e0: 6d70 696c 6572 2911 da0a 5f5f 6675 7475 mpiler)...__futu │ │ │ │ │ │ │ │ 000006f0: 7265 5f5f 7202 0000 0072 0300 0000 7204 re__r....r....r. │ │ │ │ │ │ │ │ -00000700: 0000 0072 1900 0000 da1d 6e75 6d70 792e ...r......numpy. │ │ │ │ │ │ │ │ +00000700: 0000 0072 1900 0000 5a1d 6e75 6d70 792e ...r....Z.numpy. │ │ │ │ │ │ │ │ 00000710: 6469 7374 7574 696c 732e 6663 6f6d 7069 distutils.fcompi │ │ │ │ │ │ │ │ 00000720: 6c65 722e 676e 7572 0500 0000 da09 636f ler.gnur......co │ │ │ │ │ │ │ │ 00000730: 6d70 696c 6572 7372 0600 0000 7227 0000 mpilersr....r'.. │ │ │ │ │ │ │ │ 00000740: 00da 0964 6973 7475 7469 6c73 7233 0000 ...distutilsr3.. │ │ │ │ │ │ │ │ 00000750: 00da 0d73 6574 5f76 6572 626f 7369 7479 ...set_verbosity │ │ │ │ │ │ │ │ 00000760: da0f 6e75 6d70 792e 6469 7374 7574 696c ..numpy.distutil │ │ │ │ │ │ │ │ 00000770: 7372 3500 0000 da05 7072 696e 7472 2000 sr5.....printr . 1 bit is different. Greetings, Maxime. From unknown Fri Jun 13 10:12:54 2025 X-Loop: help-debbugs@gnu.org Subject: bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 10 Dec 2021 22:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52001 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ricardo Wurmus Cc: 52001@debbugs.gnu.org, Maxime Devos Received: via spool by 52001-submit@debbugs.gnu.org id=B52001.163917617223479 (code B ref 52001); Fri, 10 Dec 2021 22:43:02 +0000 Received: (at 52001) by debbugs.gnu.org; 10 Dec 2021 22:42:52 +0000 Received: from localhost ([127.0.0.1]:48142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mvobL-00066d-SP for submit@debbugs.gnu.org; Fri, 10 Dec 2021 17:42:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:60006) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mvobK-00066P-2K for 52001@debbugs.gnu.org; Fri, 10 Dec 2021 17:42:50 -0500 Received: from [2001:470:142:3::e] (port=51922 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvobE-0004i0-E3; Fri, 10 Dec 2021 17:42:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=BbdwOCUcRQp5p3tLzq7/AB4svYMD5TqZxIspD5zf2vA=; b=kpPOuZhTXmmV/dQHovd4 yyod/xA4zvyiFhtBfwFfHLvFEob4d59P3slA5pYrZK/TRBqMbhrfaYVhckkwcKvIpI+OsV+AJO2Pd 9NKbGoBA9N50lOJknZVuqfzN27wer+8ishuZFLdoYyF8cuqjaUlvkgEG7WcQcr+0oW8ukfXcuAV1H NDJtWTDmiEFNsuB3vVYfcXQ1o/3U50R9rMW40gFmt1LCAgmkxojTPLjQHPTW/6dhLVh9QOU5Ld8+l b6mB2WZfCEiwuSduhH0jLf0hgML3RC4Rzo75aXnTrFdoqNwhH+mklDRNs35mWWBLBVcd3gF9vAMo+ tvo2rpUgykPx+w==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:49556 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvobE-0000pm-CD; Fri, 10 Dec 2021 17:42:44 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87y25ihm3e.fsf@elephly.net> Date: Fri, 10 Dec 2021 23:42:41 +0100 In-Reply-To: <87y25ihm3e.fsf@elephly.net> (Ricardo Wurmus's message of "Sat, 20 Nov 2021 15:36:57 +0000") Message-ID: <87y24s5bfi.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Ricardo Wurmus skribis: > Numpy accelerates operations with native code. Early in the build it > probes the CPU / compiler for supported instruction extensions and > then proceeds to build extensions for different features. I=E2=80=99m > guessing that this won=E2=80=99t work when building on older CPUs, so the= =20 > results of two builds on different CPUs may differ. I don=E2=80=99t see anything in the build log that suggests it might be bui= lding for a specific ISA extension (on x86_64). On the contrary, it seems to do the right thing by dispatching to the right optimized implementation at run time, in numpy/core/src/umath. Apparently this directory contains templates for home-made function multi-versioning, with variants and dispatch code generated by numpy/core/code_generators/generate_umath.py. Besides, I see the same .pyc difference as Maxime, but I don=E2=80=99t thin= k it relates to ISA extensions. Ludo=E2=80=99.