From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 27 09:26:28 2020 Received: (at submit) by debbugs.gnu.org; 27 Oct 2020 13:26:28 +0000 Received: from localhost ([127.0.0.1]:43303 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXOzc-00020e-3A for submit@debbugs.gnu.org; Tue, 27 Oct 2020 09:26:28 -0400 Received: from lists.gnu.org ([209.51.188.17]:37288) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXOza-00020X-JB for submit@debbugs.gnu.org; Tue, 27 Oct 2020 09:26:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kXOza-0000RJ-CZ for bug-guix@gnu.org; Tue, 27 Oct 2020 09:26:26 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:50204) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kXOzV-00077Z-EG for bug-guix@gnu.org; Tue, 27 Oct 2020 09:26:24 -0400 Received: by mail-wm1-x32b.google.com with SMTP id 13so1382140wmf.0 for ; Tue, 27 Oct 2020 06:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=ZjVh0eJCUzXdE2IxvJX1OarM1aaN5YM9z/OeTxE2HUo=; b=HBBwsc4d+EPGzE7bY4qyemwqEOST+LS+Hr3kHT8pvuh5Hd5C9266RXZNph1dai0OqR /iqqxJ5zJnXg0cXjKEVlg3pOEKj7YMvCgqzmiGYiHjMPwjCSqbtYduZgjWzp8mpEXB3W XiMLfor9MIxp5SbFpEclSDdhSw6bHhpJiZ/73Uk9UHL9K/jYxwNSSa7JzRLriIRzeZJA s33F+6sQlikhxuRLLomnjkyhhrSzPdFmbZi1QHZXV2zHXRcG4xFllNAMimAXHiOO4Aob 2U2ZRZ4J9EM9DsR6nF9QGtZA4FreC1KkR/NxktqXg727gw+QQzQKOnZ4UlfFB4T9R+ty 9/Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=ZjVh0eJCUzXdE2IxvJX1OarM1aaN5YM9z/OeTxE2HUo=; b=UuiSz1eKQsGpZElRLYs2uuu6iA17N4tw8nzAfqHgW5iTbbasEFmIhboVDjvQj6255O QzUA9LYn49bz2tZFN5vljXXMR2b3/gOOh4FSVtv8Zw2SbAuwYf+UmUXP5GBzFV2W8E2o q7DIuoJGD6isElhyMkmcj8RvmIMA07ULVNLdFtLzuSahKFe5tBKQMeaLOqoMfPcYClK7 5RpBvNUG5ccF0WJeRdx5WZhHqBHtDt9nE8JNVwdEK5r3UvH46yT7X8WQB5jfVeXvDmmJ HsqVh6qD35P4puO6fFl6OAjPn1DLRFMVvFKmhO9ckcE2COr9a3IpPWjZTBLj8bKMbxXN Uthg== X-Gm-Message-State: AOAM532kUrYFSNtV7Y5e7w/4X08c1kD32mPkd4s++t+32LReQVYppEuI G9Wy5QYtkeSVHCAtyecSdQ9XoN87dTwo9pUDBW/Xjk/O6qLR/1hIXR2IGOLcnlgarGthnoqZxfz nEb0yWQ+SAwtOnwwcbkp5SyKEcgZwQwx2ze8SDCIoGGs2X/K0+Xusz7y0YquIhZS9uO7FabSfSd JCE2kr X-Google-Smtp-Source: ABdhPJyjj4F9yKW76jPyjQKTMUmBLEtI4zZDePrGvA2cEnzNotwDCH4pBkWghiakXPXHA3XZXZxFuA== X-Received: by 2002:a7b:c418:: with SMTP id k24mr2901498wmi.118.1603805175533; Tue, 27 Oct 2020 06:26:15 -0700 (PDT) Received: from localhost (dynamic-2a01-0c23-7c00-b000-7592-f1eb-27a8-a555.c23.pool.telefonica.de. [2a01:c23:7c00:b000:7592:f1eb:27a8:a555]) by smtp.gmail.com with ESMTPSA id y201sm1904264wmd.27.2020.10.27.06.26.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Oct 2020 06:26:14 -0700 (PDT) Date: Tue, 27 Oct 2020 14:26:14 +0100 From: Lars-Dominik Braun To: bug-guix@gnu.org Subject: Performance of package input rewriting Message-ID: <20201027132614.GB3081@zpidnp36> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="5/uDoXvLw7AC5HRs" Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) Received-SPF: none client-ip=2a00:1450:4864:20::32b; envelope-from=ldb@leibniz-psychology.org; helo=mail-wm1-x32b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --5/uDoXvLw7AC5HRs Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, this issue is similar to https://issues.guix.gnu.org/41702, but I=E2=80=99m= not sure it=E2=80=99s exactly the same. For guix-science I=E2=80=99m trying to provi= de some packages like python-jupyterlab, which depend on a mix of packages from guix proper = and newer versions of packages already included in guix proper. Thus I need to rewrite inputs of the former to the latter. (Because Python only propagates dependencies and thus collisions would occur.) Previously I have been doing this using package-input-rewriting, but starti= ng an environment containing python-jupyterlab alone took about 20s (warm cach= es, all derivations in the store). Manually rewriting inputs by inheriting and alist-delete=E2=80=99ing brings this down to 3s, which is pretty significan= t. --no-grafts has not much of an impact (15s vs 2s) here. See https://github.com/guix-science/guix-science/commit/972795a23cc9eb5a0bb1a2f= fb5681d151fc4d4b0 for the exact changes. My expectation would be that package-input-rewriting is the preferred, beca= use easier, solution to this problem and thus should have minimal impact on performance. Cheers, Lars --5/uDoXvLw7AC5HRs Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl+YH/IACgkQQhN3ARo3 hEYtWgv/ZeeJRbS39QGqBHhDiCkMrRAP2BxrkQeiIms/Vtp2JtOWEelrmdWj6ZNe +46a4EnqRX/kv6z+ZKZPHz5KBkEPvwnNoRQbdFOlKSFPCe5PrvNHbyhpscS+TMj/ u/hGS8NKt1di24SfnyqJHeWYeK9QTQnxNwycsCnQx1SUB5BrWQx49jEVS1DmmwkY zcSnB0ztlhOu+Fa+D6CrkppC2BILPdQZc3sDYQ0Cj74VCp3FwSNJJ1spLeKXIMCr cLbzdL8mAMhr5b43+ePz0pBJ9FUz+ly5nV9Ml+d1oo+wpA6dn0REqLn+7Hu4uJJF vzkCIVJ8UiG8/7HCKPiX5YZd7ka5wgyGVbE9nIAFGJ58oB22dRaAcVWho0TzTITK oJOyQ1xuCN+RbjkpWSq89NqeNC5ajoa90p8yaW9ANRn4Z6jDMH26cEbCl/mNIolH KdF1Fsbcvyk55GYmPEGEMdzgTmSDOg31quMENSP2aYrMUntSu4LI0yQbnlhPnbzj z3oMKxDF =X55m -----END PGP SIGNATURE----- --5/uDoXvLw7AC5HRs-- From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 27 10:14:33 2020 Received: (at 44254) by debbugs.gnu.org; 27 Oct 2020 14:14:33 +0000 Received: from localhost ([127.0.0.1]:45359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXPk9-0003im-FE for submit@debbugs.gnu.org; Tue, 27 Oct 2020 10:14:33 -0400 Received: from mail-wr1-f48.google.com ([209.85.221.48]:36665) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXPk6-0003iW-Lv for 44254@debbugs.gnu.org; Tue, 27 Oct 2020 10:14:31 -0400 Received: by mail-wr1-f48.google.com with SMTP id x7so2106130wrl.3 for <44254@debbugs.gnu.org>; Tue, 27 Oct 2020 07:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=Qre+zqMlPuphUr5CnS5AkI09bJ0BDSYnuMid+w7XY4A=; b=tBUi4NkVPWAGlB5r2g+ACKEK0pAIPKPeTeodwjQFJz1OETq0sl9oJh/z063D8SAWl/ YGwC8Pnz0Nyse92lBmKAE+sCQ7NBHABGw5CJZ+ZDGlATInlufGWwgDRPHOZXyvuQOxma Il6rpBWfGY0DK5UCn7xRjm75QxMBalXcm8ap3SWW2+2TbbAKB0R2EX4ZaJOu6zw++MeD gKAqbENmHUkrnlCH2aQvf7eucYd62gYt3zB0k22Z7tz1jlrjTa/voHCJKhbsBgiuiurF acu3dWXob5eF/ZRFblfEErmQbBBn8pqO7YCEQbWg2oKsjhCa7OMtTl9hqtOXjhiUxjFU Q9yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=Qre+zqMlPuphUr5CnS5AkI09bJ0BDSYnuMid+w7XY4A=; b=ouDFtRr7ciP7bVGuyoWzSpi37gC9gD+mpHrUjKdY68jD7F4yAiNOYVhnATyGnihbG4 SjMcQ21tyhElN+k43bzlW/O6p2OewUq6O4+7nkDUpdGimFsb5FgVoBLs6hjs7pkTWn+2 j/ppXoMetz7Bzq2cBwkdweFFuKjbsGDHuTH74LFGAVyg7zb2A5SLjWUkLyPt9MnOmuev p4zBDwkoWloTu7eos50vKgqONudRUL72ZSKkwSHbsqO1MnYmf5K8YO1IG196V7CxGvSu PJnSNS44GlJFnVjz1cm3ZjVcpCHNvd8pWcLn8Wx5V6+RkLEEWQ5Eg6Tj7jWemjU6zIHN M28Q== X-Gm-Message-State: AOAM533QAZGsBR0eSfnWlg4ymLrJjJOsL+SwYIqIJSt95c7N1XD/VD0c ZZ679mDv0qb1PVW6dqgmBkt6X0zkMus= X-Google-Smtp-Source: ABdhPJw7r2UeFrkjV9tB05Tna5sBvv1UeSwJjxFfUtZ47OKUDeAo2QU81cjjc5qIlirn2LvZxoy8PA== X-Received: by 2002:adf:ee83:: with SMTP id b3mr2749069wro.139.1603808064706; Tue, 27 Oct 2020 07:14:24 -0700 (PDT) Received: from lili ([88.126.110.68]) by smtp.gmail.com with ESMTPSA id u3sm2069478wme.0.2020.10.27.07.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Oct 2020 07:14:24 -0700 (PDT) From: zimoun To: Lars-Dominik Braun , 44254@debbugs.gnu.org Subject: Re: bug#44254: Performance of package input rewriting In-Reply-To: <20201027132614.GB3081@zpidnp36> References: <20201027132614.GB3081@zpidnp36> Date: Tue, 27 Oct 2020 15:14:22 +0100 Message-ID: <86361z4uyp.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 44254 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Lars, On Tue, 27 Oct 2020 at 14:26, Lars-Dominik Braun wrote: > Previously I have been doing this using package-input-rewriting, but star= ting > an environment containing python-jupyterlab alone took about 20s (warm ca= ches, > all derivations in the store). Manually rewriting inputs by inheriting and > alist-delete=E2=80=99ing brings this down to 3s, which is pretty signific= ant. > --no-grafts has not much of an impact (15s vs 2s) here. See > https://github.com/guix-science/guix-science/commit/972795a23cc9eb5a0bb1a= 2ffb5681d151fc4d4b0 > for the exact changes. Is it not related to =E2=80=9C#:deep? #t=E2=80=9C by default? The default = was #f. Well, using =E2=80=99inherit=E2=80=99 only rewrites the direct explicit dep= endencies. However, =E2=80=99package-input-rewriting=E2=80=99 traverse all the graph of dependencies and replaces accordingly. Maybe I misunderstand. All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 27 16:01:18 2020 Received: (at submit) by debbugs.gnu.org; 27 Oct 2020 20:01:18 +0000 Received: from localhost ([127.0.0.1]:45964 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXV9i-0006Ei-3O for submit@debbugs.gnu.org; Tue, 27 Oct 2020 16:01:18 -0400 Received: from lists.gnu.org ([209.51.188.17]:48522) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXV9g-0006EX-NE for submit@debbugs.gnu.org; Tue, 27 Oct 2020 16:01:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kXV9g-0003Js-6E for bug-guix@gnu.org; Tue, 27 Oct 2020 16:01:16 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21153) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kXV9Z-0000rC-Oy for bug-guix@gnu.org; Tue, 27 Oct 2020 16:01:15 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1603828588; cv=none; d=zohomail.com; s=zohoarc; b=dq4lcCRXUdKFlRawbQxJJgqtjUsCy8YfMZn75jHpx/05kaNNDPuI0FdfP1sXNPdMM1bBNiPcf2HMi/Ok3a/I5Fv5ytBOthlQmgL1mtZpnWUHoHb9ljGEK6pQCxF5D7mhJaQVaW/YMRuxMqVu6k1nP7mlB9saUeprJkzhBtELAIQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603828588; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=SV5OcI/AzP82xPPenQ0FMSSkzZXd4LpsUEpu8feu7YE=; b=B+nxWBUtaTjWrcfjhy/AlY/YGKDfghomyiliapUhuZDLDwyfNNb5YDwq2JkHW6Xoh67MxMNFx2DUaXnjv5aSugkJLXWhjbNW7snu9gPo8ocEB1QFR0z36eTrZ/ZI7bTjPLQqSP6vyuN3tvL3Bvwl1pcqKTA7TbdlMyn47ttXI28= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1603828588; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=SV5OcI/AzP82xPPenQ0FMSSkzZXd4LpsUEpu8feu7YE=; b=G3gBMhUngfKPi/iwMk8rlrae+oVydneMF80PouHzDc/GVaJ31P2gXbiSJPmdbxCx oQxoRIawVJ1pq79NlmC/fflxjhovZZttWCGxu1OZAh5ih/e6BCNApmPN93+T5sJsml3 B5ODRx8zcqAHdqTcHaD9gYH836e/oNzSkF8jMOq0= Received: from localhost (p54ad4da6.dip0.t-ipconnect.de [84.173.77.166]) by mx.zohomail.com with SMTPS id 1603828586108191.86356554257964; Tue, 27 Oct 2020 12:56:26 -0700 (PDT) References: <20201027132614.GB3081@zpidnp36> User-agent: mu4e 1.4.13; emacs 27.1 From: Ricardo Wurmus To: Lars-Dominik Braun Subject: Re: bug#44254: Performance of package input rewriting In-reply-to: <20201027132614.GB3081@zpidnp36> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Tue, 27 Oct 2020 20:58:14 +0100 Message-ID: <87imav1lwp.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/27 16:01:05 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: 44254@debbugs.gnu.org, bug-guix@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Lars-Dominik Braun writes: > this issue is similar to https://issues.guix.gnu.org/41702, but I=E2=80= =99m not sure > it=E2=80=99s exactly the same. For guix-science I=E2=80=99m trying to pro= vide some packages > like python-jupyterlab, which depend on a mix of packages from guix prope= r and > newer versions of packages already included in guix proper. Thus I need to > rewrite inputs of the former to the latter. (Because Python only propagat= es > dependencies and thus collisions would occur.) > > Previously I have been doing this using package-input-rewriting, but star= ting > an environment containing python-jupyterlab alone took about 20s (warm ca= ches, > all derivations in the store). Manually rewriting inputs by inheriting and > alist-delete=E2=80=99ing brings this down to 3s, which is pretty signific= ant. Could you show us a concrete example? Input rewriting is recursive and will traverse the whole package graph by default, even if you *know* that, say, GCC doesn=E2=80=99t need to be rewritten. For the more generic =E2=80=9Cpackage-mapping=E2=80=9D you can provide a = =E2=80=9Ccut?=E2=80=9D procedure to determine when to stop recursion. Perhaps this would make things faster in your case? --=20 Ricardo From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 28 10:20:06 2020 Received: (at 44254) by debbugs.gnu.org; 28 Oct 2020 14:20:06 +0000 Received: from localhost ([127.0.0.1]:50002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXmJ3-0000vH-Oz for submit@debbugs.gnu.org; Wed, 28 Oct 2020 10:20:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58434) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXmJ2-0000uh-6s for 44254@debbugs.gnu.org; Wed, 28 Oct 2020 10:20:04 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45039) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kXmIw-0006no-Pc; Wed, 28 Oct 2020 10:19:58 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41206 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kXmIw-0007iu-AN; Wed, 28 Oct 2020 10:19:58 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zimoun Subject: Re: bug#44254: Performance of package input rewriting References: <20201027132614.GB3081@zpidnp36> <86361z4uyp.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 28 Oct 2020 15:19:56 +0100 In-Reply-To: <86361z4uyp.fsf@gmail.com> (zimoun's message of "Tue, 27 Oct 2020 15:14:22 +0100") Message-ID: <87o8km301f.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44254 Cc: 44254@debbugs.gnu.org, Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, zimoun skribis: > On Tue, 27 Oct 2020 at 14:26, Lars-Dominik Braun wrote: > >> Previously I have been doing this using package-input-rewriting, but sta= rting >> an environment containing python-jupyterlab alone took about 20s (warm c= aches, >> all derivations in the store). Manually rewriting inputs by inheriting a= nd >> alist-delete=E2=80=99ing brings this down to 3s, which is pretty signifi= cant. >> --no-grafts has not much of an impact (15s vs 2s) here. See >> https://github.com/guix-science/guix-science/commit/972795a23cc9eb5a0bb1= a2ffb5681d151fc4d4b0 >> for the exact changes. > > Is it not related to =E2=80=9C#:deep? #t=E2=80=9C by default? The defaul= t was #f. Yes, that=E2=80=99s a possible culprit. Try passing #:deep? #f if it works= for your use case. Another thing to look at is the object graph (as show by =E2=80= =98guix graph=E2=80=99). Input rewriting can duplicate parts of the graph, which in turn defeats package->derivation memoization. Just looking at the number of nodes in the graph can give hints. Like Ricardo wrote, it=E2=80=99d be great it you could share a short reprod= ucer. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 30 04:42:54 2020 Received: (at 44254) by debbugs.gnu.org; 30 Oct 2020 08:42:54 +0000 Received: from localhost ([127.0.0.1]:56015 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kYPzq-0002SB-3p for submit@debbugs.gnu.org; Fri, 30 Oct 2020 04:42:54 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:33819) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kYPzo-0002Rz-M2 for 44254@debbugs.gnu.org; Fri, 30 Oct 2020 04:42:52 -0400 Received: by mail-wr1-f47.google.com with SMTP id i16so108822wrv.1 for <44254@debbugs.gnu.org>; Fri, 30 Oct 2020 01:42:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :in-reply-to; bh=IHKRB6j6c3w9bX1I5IGfRbFo/bZCO5whWqO394koJWU=; b=UlZFmWQMdWwGIQ4AfqGonyqNumeqEABou73gsSoIq8X364zNy5S/X2zwVt/k2YKYH5 O9TOQy14gDz5yQ+MJAnLL/0VDJJdF1lAloRuyKbTqV6GpMy4rOWm04Peq+FLma+RKeC+ j+dL64vyIfLcAltMW93Iq2SsNnMaCRgd5eAiNzZfypOHmsS9oph7D9aZeDlKtHMi2HRJ EDAvvmCMEDaRFgTWtqa1G3SZF9jlQ0x1saK7oRAQ30mGyuNFEylausgkR+s6qmZSUVXD sZ7eaeqsP2AftZz6JxFMJKV27UeoJnXYoEbeq+3dvgQs7ljtV8a9NXRuwRMRBNxjo1wb Cd5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=IHKRB6j6c3w9bX1I5IGfRbFo/bZCO5whWqO394koJWU=; b=o0SO69ELM78t268rVBkjxDFSJiw42ymIQ2ZWRu51vE43y8r6Xh177Z66i7g0eQY5j/ OxA2gagzw973ao3ZWgw/PjSHag7jX8MQAfNZrEu6EX3cT26cOXWZedjFZae1BHD2JFtE DbhhnxpsKdecJ/oDwMhYRW0eGPFkDxTOQm0++EVA5y/vhXG452FNaC4iHrRvVmMFaX/r 7CNPEO4MJlRJ6G6Uc47auGQwsuw7v9RLmesQo4BaTx5A46BQbAiNeAw1Kb+p5Ykl81L0 xmb/3s6sDo+MO79i8f+2Pc/QrNYHClQgY0LNDeKI9vmBTaMBdobo9oVEYXlySNuU/V52 OKyg== X-Gm-Message-State: AOAM5331QmZRHzytcXr/WIG2jJS84A7OXsGR1UtK0Qxdi5GqN+Erqe/v jI/pVFfsRGzo6BJeR7Jil7u8G0a/Q2h5SHwruU+8NST+WUYEI9Q6BDMAsWQem2vMCOnsbvEuvWW IcucUx4PRoOlwSST+i1eJNK8Dc2rt1ACxQx4+cFDlkY0YLNg0+lYbtJrkte/dqaRF/6XlhTwFTp 48ntk= X-Google-Smtp-Source: ABdhPJwKtiL8B+26nu6kjB5gvXri41Kbjjb1GJgJlIVrzZvQk/KjM1iUmjcVZgH/warE5NcP98rOfw== X-Received: by 2002:a5d:54c1:: with SMTP id x1mr1749583wrv.172.1604047366454; Fri, 30 Oct 2020 01:42:46 -0700 (PDT) Received: from localhost (dynamic-2a01-0c22-c82e-e900-3cb0-1527-c3b4-f9df.c22.pool.telefonica.de. [2a01:c22:c82e:e900:3cb0:1527:c3b4:f9df]) by smtp.gmail.com with ESMTPSA id y7sm3837614wmg.40.2020.10.30.01.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Oct 2020 01:42:45 -0700 (PDT) Date: Fri, 30 Oct 2020 09:42:45 +0100 From: Lars-Dominik Braun To: Ricardo Wurmus , Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: bug#44254: Performance of package input rewriting Message-ID: <20201030084245.GB3128@zpidnp36> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="f2QGlHpHGjS2mn6Y" Content-Disposition: inline In-Reply-To: <87o8km301f.fsf@gnu.org> <87imav1lwp.fsf@elephly.net> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 44254 Cc: 44254@debbugs.gnu.org, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --f2QGlHpHGjS2mn6Y Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, > Yes, that=E2=80=99s a possible culprit. Try passing #:deep? #f if it wor= ks for > your use case. Yeah, that brings it down to ~8s, which is still alot. > Another thing to look at is the object graph (as show by =E2=80= =98guix > graph=E2=80=99). Input rewriting can duplicate parts of the graph, which= in > turn defeats package->derivation memoization. Just looking at the > number of nodes in the graph can give hints. Aha, it=E2=80=99s 913 nodes without rewriting, 13916 with rewriting (#:deep= ? #t) and 4286 with rewriting (#:deep? #f) as determined by a rather ad-hoc `guix gra= ph -L . -t package python-jupyterlab | grep 'shape =3D box' | wc -l`. That see= ms way too much. Does that mean I=E2=80=99m using package rewriting in the wrong w= ay or is that a bug? Unfortunately I don=E2=80=99t have a short reproducer right now. I=E2=80=99= ll look at the graph more closely to figure out which parts are actually duplicated. Maybe I can create a reproducing testcase with more information. Cheers, Lars --f2QGlHpHGjS2mn6Y Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl+b0gEACgkQQhN3ARo3 hEZKNgv/XsWkY+fy4zRIPx/SK5XuAOjLzvKI9w4ahuvN8ewSGzpuuhRTyzU6TIN6 AWgTJJjSkHkeFY8iDgHXwgJ9KIOBtNOz7Pwvg6yUG6+nEIVjxAmmL6T2q43FPHxz gLou6pWJjsNOx10gNsKduWDcwUdeYW9KDLN2HdY3RQHnzf93cK13YuF6DiCp63Hb sq454Q7MFHo7EMrGaNx6QwuSG4tqHW8wjs9er2ioECJ4DT9XJseUXEvNKjcfuUkW TpJWuMraO1Rzb4JZD5GkqzKzX32Dgf355b8MxR//XUwfgxI4hD57CS+xedrqCUFN XhJoQgHds3cAqO54Mv2b9004kDlqHc4tb/PFnsjVEjtBwG19ARTOwA+v+avHlGop hYT8tnlrl9B+mufMqILVdq9eU/KvOHmv6wtxQycb/yX30gxecc7Bqqk0MaZ//B23 hSSnReGdqmNtY/9hKnSva90iwLvZXoq9nBJo/BuBhcWcSEr2MrTwu7GIO3NAhnuH dYdKQhzq =Of5M -----END PGP SIGNATURE----- --f2QGlHpHGjS2mn6Y-- From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 31 06:27:19 2020 Received: (at 44254) by debbugs.gnu.org; 31 Oct 2020 10:27:19 +0000 Received: from localhost ([127.0.0.1]:60589 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kYo6R-0006rT-9K for submit@debbugs.gnu.org; Sat, 31 Oct 2020 06:27:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50826) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kYo6Q-0006rD-6n for 44254@debbugs.gnu.org; Sat, 31 Oct 2020 06:27:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36287) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kYo6I-0005Jw-T3; Sat, 31 Oct 2020 06:27:12 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56806 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kYo6G-0007xd-2E; Sat, 31 Oct 2020 06:27:08 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Lars-Dominik Braun Subject: Re: bug#44254: Performance of package input rewriting References: <20201030084245.GB3128@zpidnp36> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 10 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 31 Oct 2020 11:27:06 +0100 In-Reply-To: <20201030084245.GB3128@zpidnp36> (Lars-Dominik Braun's message of "Fri, 30 Oct 2020 09:42:45 +0100") Message-ID: <87tuuavgg5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44254 Cc: Ricardo Wurmus , 44254@debbugs.gnu.org, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Lars, Lars-Dominik Braun skribis: >> Another thing to look at is the object graph (as show by =E2= =80=98guix >> graph=E2=80=99). Input rewriting can duplicate parts of the graph, whic= h in >> turn defeats package->derivation memoization. Just looking at the >> number of nodes in the graph can give hints. > Aha, it=E2=80=99s 913 nodes without rewriting, 13916 with rewriting (#:de= ep? #t) and > 4286 with rewriting (#:deep? #f) as determined by a rather ad-hoc `guix g= raph > -L . -t package python-jupyterlab | grep 'shape =3D box' | wc -l`. That s= eems way > too much. Does that mean I=E2=80=99m using package rewriting in the wrong= way or is > that a bug? It could be a mixture thereof. :-) I guess it=E2=80=99s easy to end up creating huge object graphs. Here=E2= =80=99s an example of an anti-pattern: (define a ((package-input-rewriting x) ((package-input-rewriting y) p1)))=20 (define b ((package-input-rewriting x) ((package-input-rewriting y) p2))) The correct use is: (define transform (package-input-rewriting (append x y))) (define a (transform p1)) (define b (transform p2)) That guarantees that =E2=80=98a=E2=80=99 and =E2=80=98b=E2=80=99 share most= of the nodes of their object graph. >From a quick look, the code in Guix-Science seemed to be following the pattern above. For example, there=E2=80=99s: --8<---------------cut here---------------start------------->8--- (define python-ipykernel-5.3-bootstrap (let ((rewritten ((package-input-rewriting `((,python-jupyter-client . ,python-jupyter-client-6.1-bootstrap) ;; Indirect through IPython. (,python-testpath . ,python-testpath-0.4) (,python-nbformat . ,python-nbformat-5.0))) python-ipykernel-5.3-proper))) (package (inherit rewritten) (name "python-ipykernel-bootstrap")))) (define-public python-jupyter-client-6.1 ((package-input-rewriting `((,python-ipykernel . ,python-ipykernel-5.3-bootstrap) (,python-jupyter-core . ,python-jupyter-core-4.6) ;; Indirect through IPython. (,python-testpath . ,python-testpath-0.4) (,python-nbformat . ,python-nbformat-5.0))) python-jupyter-client-6.1-proper)) (define-public python-ipykernel-5.3 ((package-input-rewriting `((,python-jupyter-client . ,python-jupyter-client-6.1) ;; Indirect through IPython. (,python-testpath . ,python-testpath-0.4) (,python-nbformat . ,python-nbformat-5.0))) python-ipykernel-5.3-proper)) --8<---------------cut here---------------end--------------->8--- It seems to me that you=E2=80=99re redefining a dependency graph, node by n= ode. Thus, you probably don=E2=80=99t need =E2=80=98package-input-rewriting=E2= =80=99 here. What you did in Guix-Science commit 972795a23cc9eb5a0bb1a2ffb5681d151fc4d4b0 looks more appropriate to me, in terms of style and semantics. Does that make sense? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 03 03:23:52 2020 Received: (at 44254) by debbugs.gnu.org; 3 Nov 2020 08:23:52 +0000 Received: from localhost ([127.0.0.1]:43264 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kZrbc-0006sH-CG for submit@debbugs.gnu.org; Tue, 03 Nov 2020 03:23:52 -0500 Received: from mail-wr1-f52.google.com ([209.85.221.52]:36706) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kZrbb-0006s4-7Q for 44254@debbugs.gnu.org; Tue, 03 Nov 2020 03:23:51 -0500 Received: by mail-wr1-f52.google.com with SMTP id x7so17489777wrl.3 for <44254@debbugs.gnu.org>; Tue, 03 Nov 2020 00:23:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=L1beBSc/H4MF13ftalbYuwlrusr2p0oQ5kcHvtZeGbA=; b=XaG84lF5UWAfeTmnueoxG+84Gz8iiGvrTH+G4IChyvRwAKnnYHoSBCRUT2st3dMS/A yFGWZ3elIgHDcwPALEUvcRMCS0ANYxGo/HL33e5qT2NtPTlMgb6hbdhM8gwtIFW8vH59 YXjIACiYN3p0PVgjeRWyLugeFVeuE3e47CSCO3sufPyS07tadRce+ofWENbxAiSor43F I7vV23pRracIKXKeBidKmxbIyvZGf0dcgPn3V04Eb9hYHVx5k4RhxElsJy2736bOHjwW ecmLFxzO9WCc7mzwR2Jr1rdihL3tr1/NavZWCh1oLc8smIL9cct7hw6ghezaQ+kdoY5e upNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=L1beBSc/H4MF13ftalbYuwlrusr2p0oQ5kcHvtZeGbA=; b=C5gHZeiU2g+/lvZKhEVUfWnYzy7TYl/+smj+EzJ5KKgOoKIZfu33/oO46XmYcKae5T sdn5bjsm1XPDBgND/LOH2oe2KsS4b5s8fhR8IIlK4PXzaN9hp8/dwVr+uRrWWt/6YSHU VwwydrxU7GCguRRD9D6oyntTalP7APRgfTBfuI4ONCo5Ob2Yq4DUi3RS2Iit+3tVnbMZ BKzmHh8qo7SX8/Mqi8RVEM+TtRwd4fIadxyGphGiUc0KU3IpKSa/lM4VpnllyEnbx45N Fq4vDUi+sbcG+uFIc8HFDeKHekzual3XvhAldeTfbBF+91yqM9jEdD4YaEcDlQpKCjn0 yWIg== X-Gm-Message-State: AOAM531XhGYv46/mUU87ZmQntOVAszVYajJDtOs1hxg+uC9wNTZRNfUF 3bcrDMv1urSe1guWZMLX3G1c5RyEMFARwqNEoAOdDw6VTFCSa27js+FO6ubzLN92PuzdrY3Au6/ Phwpw+hqfv7sCNyXKI2X3nuxGCSdfFyYVxhTOPU2OjArLtmdc7d3olNLefy7nKa96s6BQx98vtf AY6jo= X-Google-Smtp-Source: ABdhPJzUOrM2HFoASV0O/1oUhWjhh4z7ATfsFFUov8ZJJLGvka5C8McEP4ChakSSCj8hXwAyEz/9Rw== X-Received: by 2002:adf:de89:: with SMTP id w9mr24522866wrl.212.1604391825321; Tue, 03 Nov 2020 00:23:45 -0800 (PST) Received: from localhost (dynamic-2a01-0c23-7840-f200-1dfb-7920-a3a3-0bf8.c23.pool.telefonica.de. [2a01:c23:7840:f200:1dfb:7920:a3a3:bf8]) by smtp.gmail.com with ESMTPSA id x10sm24419895wrp.62.2020.11.03.00.23.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Nov 2020 00:23:44 -0800 (PST) Date: Tue, 3 Nov 2020 09:23:43 +0100 From: Lars-Dominik Braun To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: bug#44254: Performance of package input rewriting Message-ID: <20201103082343.GC3127@zpidnp36> References: <20201030084245.GB3128@zpidnp36> <87tuuavgg5.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mvpLiMfbWzRoNl4x" Content-Disposition: inline In-Reply-To: <87tuuavgg5.fsf@gnu.org> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 44254 Cc: 44254@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --mvpLiMfbWzRoNl4x Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Ludo, > I guess it=E2=80=99s easy to end up creating huge object graphs. Here=E2= =80=99s an > example of an anti-pattern: >=20 > (define a > ((package-input-rewriting x) ((package-input-rewriting y) p1)))=20 >=20 > (define b > ((package-input-rewriting x) ((package-input-rewriting y) p2))) >=20 > The correct use is: >=20 > (define transform > (package-input-rewriting (append x y))) >=20 > (define a (transform p1)) > (define b (transform p2)) that sounds like a section for the cookbook :) > It seems to me that you=E2=80=99re redefining a dependency graph, node by= node. > Thus, you probably don=E2=80=99t need =E2=80=98package-input-rewriting=E2= =80=99 here. What you > did in Guix-Science commit 972795a23cc9eb5a0bb1a2ffb5681d151fc4d4b0 > looks more appropriate to me, in terms of style and semantics. Okay, got it. My initial concern was that rewriting the graph =E2=80=9Cby h= and=E2=80=9D (i.e. alist-delete) would be tedious and error-prone. Thank you very much, Lars --=20 Lars-Dominik Braun Wissenschaftlicher Mitarbeiter/Research Associate www.leibniz-psychology.org ZPID - Leibniz-Institut f=C3=BCr Psychologie / ZPID - Leibniz Institute for Psychology Universit=C3=A4tsring 15 D-54296 Trier - Germany Tel.: +49=E2=80=93651=E2=80=93201-4964 --mvpLiMfbWzRoNl4x Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl+hE4sACgkQQhN3ARo3 hEZxzgwA0HnQCSUKVxjo9Q0EPYBRZR5OBV1AADcFQ+aLy48K2F/5oS6+Dm5fsK3o Wpog7EbunDAub8ry3oAfqnAMDfmLL0mLFiWpIZpLp5UEMnRpnc9PxifWvJRA3SAq DAhtMhBkEjuJ48+y4IxzFK7BzWTdTTuwUy6fv0Um7Bzy6uh515D9r0pJZ33EJunU GufM1llTjre+9rDo+orXWy52TwlqbiNaZm58tJ5DmrLeoOS5l8DoFcCA4Y076KPD 91O/TXQMwU2a9pRwrBu+p1i+q9mjbHhJJVXZNJjvKQPMo6tiuS7z9+1LD2+0I4t6 SC3rkEyXOpx49nV/yBzo6+K902MzqPEy97SfQKmeNLkO5rTuIMK7Z8auu4mFP8Do o+cyPRC/tl2UqJPDAK11tXrDj+C5TGsDpGsVNvM6MkoeXoMQdjQzr/YdFT/uASjA 1/Wf2UINl/Rlfu0tZtI4UY2csHlJjy21fDyx9X4w+arBQVFh5YyC+fBv4TtIikJ6 jluKAbfR =jZax -----END PGP SIGNATURE----- --mvpLiMfbWzRoNl4x-- From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 03 04:32:25 2020 Received: (at 44254) by debbugs.gnu.org; 3 Nov 2020 09:32:26 +0000 Received: from localhost ([127.0.0.1]:43435 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kZsfx-0000CI-Jq for submit@debbugs.gnu.org; Tue, 03 Nov 2020 04:32:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:34664) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kZsfv-0000Bz-TP for 44254@debbugs.gnu.org; Tue, 03 Nov 2020 04:32:24 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40433) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZsfq-0006yJ-BO; Tue, 03 Nov 2020 04:32:18 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=58010 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kZsfp-0001aI-GX; Tue, 03 Nov 2020 04:32:17 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Lars-Dominik Braun Subject: Re: bug#44254: Performance of package input rewriting References: <20201030084245.GB3128@zpidnp36> <87tuuavgg5.fsf@gnu.org> <20201103082343.GC3127@zpidnp36> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 13 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 03 Nov 2020 10:32:16 +0100 In-Reply-To: <20201103082343.GC3127@zpidnp36> (Lars-Dominik Braun's message of "Tue, 3 Nov 2020 09:23:43 +0100") Message-ID: <87tuu6g50f.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44254 Cc: 44254@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Lars-Dominik Braun skribis: >> I guess it=E2=80=99s easy to end up creating huge object graphs. Here= =E2=80=99s an >> example of an anti-pattern: >>=20 >> (define a >> ((package-input-rewriting x) ((package-input-rewriting y) p1)))=20 >>=20 >> (define b >> ((package-input-rewriting x) ((package-input-rewriting y) p2))) >>=20 >> The correct use is: >>=20 >> (define transform >> (package-input-rewriting (append x y))) >>=20 >> (define a (transform p1)) >> (define b (transform p2)) > that sounds like a section for the cookbook :) Note that there=E2=80=99s a new section in the manual on this topic: https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.= html >> It seems to me that you=E2=80=99re redefining a dependency graph, node b= y node. >> Thus, you probably don=E2=80=99t need =E2=80=98package-input-rewriting= =E2=80=99 here. What you >> did in Guix-Science commit 972795a23cc9eb5a0bb1a2ffb5681d151fc4d4b0 >> looks more appropriate to me, in terms of style and semantics. > Okay, got it. My initial concern was that rewriting the graph =E2=80=9Cby= hand=E2=80=9D (i.e. > alist-delete) would be tedious and error-prone. I haven=E2=80=99t looked closely enough. If you can define a single proced= ure that rewrites the graph, that=E2=80=99s of course better than rewriting nod= es one by one. Maybe that=E2=80=99s possible, but you need to be careful about factorizing the transformation procedure as I shown above. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 16 10:07:17 2020 Received: (at control) by debbugs.gnu.org; 16 Nov 2020 15:07:17 +0000 Received: from localhost ([127.0.0.1]:56743 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keg68-0006FC-NI for submit@debbugs.gnu.org; Mon, 16 Nov 2020 10:07:16 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39064) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keg66-0006Ex-GQ for control@debbugs.gnu.org; Mon, 16 Nov 2020 10:07:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34880) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1keg5z-0002il-LN for control@debbugs.gnu.org; Mon, 16 Nov 2020 10:07:07 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48068 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1keg5z-0005qs-1p for control@debbugs.gnu.org; Mon, 16 Nov 2020 10:07:07 -0500 Date: Mon, 16 Nov 2020 16:07:03 +0100 Message-Id: <87y2j1uyqw.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #44254 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 44254 notabug close 44254 quit From unknown Fri Aug 15 16:19:14 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 15 Dec 2020 12:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator