From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 02 12:22:41 2023 Received: (at submit) by debbugs.gnu.org; 2 Jun 2023 16:22:41 +0000 Received: from localhost ([127.0.0.1]:40758 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q57Y1-0005h0-DX for submit@debbugs.gnu.org; Fri, 02 Jun 2023 12:22:41 -0400 Received: from lists.gnu.org ([209.51.188.17]:40748) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q57Xx-0005gr-Rr for submit@debbugs.gnu.org; Fri, 02 Jun 2023 12:22:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q57Xw-0000Lz-76 for bug-guix@gnu.org; Fri, 02 Jun 2023 12:22:36 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q57Xt-0000Kb-TM for bug-guix@gnu.org; Fri, 02 Jun 2023 12:22:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=W+1Whuz6JmkOP1cZP6PK7qliZjinbgTG2Xa66hyDC/g=; b=Dz7kfwX8t5ca4QgPox8k5OrHwKwD1E/Sng3izGJIT89AY4Bsw0PVXji0 mqvEuje20dFPflFkg26FrE/6JWmJNMLkpFr+Z99RFf+wyZUmtYN8oTFKk O3R6pVdrQcprBmY+PGxy0KF69LaHdRJmg9Olk6mUQiJ4oVvceVcWIibQL 8=; Authentication-Results: mail2-relais-roc.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludovic.courtes@inria.fr; dmarc=fail (p=none dis=none) d=inria.fr X-IronPort-AV: E=Sophos;i="6.00,213,1681164000"; d="scan'208";a="110998696" Received: from unknown (HELO ribbon) ([193.50.110.155]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2023 18:22:29 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: bug-guix@gnu.org Subject: =?utf-8?Q?=E2=80=98guix-package-cache=2Edrv=E2=80=99?= eats all your memory with =?utf-8?B?4oCYZ3VpeC1jcmFu4oCZ?= X-Debbugs-Cc: Lars-Dominik Braun X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Quartidi 14 Prairial an 231 de la =?utf-8?Q?R=C3=A9v?= =?utf-8?Q?olution=2C?= jour de l'Acacia X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 02 Jun 2023 18:22:29 +0200 Message-ID: <87sfb93kca.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Consider this: --8<---------------cut here---------------start------------->8--- $ cat /tmp/cran.scm (cons (channel (name 'guix-cran) (url "https://github.com/guix-science/guix-cran.git")) %default-channels) $ guix time-machine -C /tmp/cran.scm -- describe Updating channel 'guix-cran' from Git repository at 'https://github.com/gui= x-science/guix-cran.git'... Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.or= g/git/guix.git'... [...] The following derivations will be built: /gnu/store/90qiphp9048zgvi96lj8zqg8zhwi4js0-profile.drv /gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.drv /gnu/store/vnjf8gf6yf5gjm1syvz7yyfn71mmvy27-profile.drv /gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv /gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv /gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv /gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules.drv /gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.drv /gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv /gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv /gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv /gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv /gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv /gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.drv building /gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.dr= v... building /gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv... building /gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv= ... building /gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv... building /gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.dr= v... building /gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules= .drv... building /gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv.= .. building /gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv... building /gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv... building /gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv... building /gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv... building CA certificate bundle... listing Emacs sub-directories... building fonts directory... building directory of Info manuals... building profile with 2 packages... building /gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.dr= v... building package cache... --8<---------------cut here---------------end--------------->8--- It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-= package-cache.drv, which consumes memory until it gets OOM-killed. It used to work fine a few months ago. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 02 16:55:19 2023 Received: (at control) by debbugs.gnu.org; 2 Jun 2023 20:55:19 +0000 Received: from localhost ([127.0.0.1]:40919 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5Bnr-00056A-Ed for submit@debbugs.gnu.org; Fri, 02 Jun 2023 16:55:19 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:17395) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5Bno-00055r-Qt for control@debbugs.gnu.org; Fri, 02 Jun 2023 16:55:18 -0400 Authentication-Results: mail2-relais-roc.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludo@gnu.org; dmarc=fail (p=none dis=none) d=gnu.org X-IronPort-AV: E=Sophos;i="6.00,214,1681164000"; d="scan'208";a="111019219" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2023 22:55:10 +0200 Date: Fri, 02 Jun 2023 22:55:09 +0200 Message-Id: <87r0qt37pu.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #63852 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) severity 63852 important quit From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 03 10:05:35 2023 Received: (at 63852) by debbugs.gnu.org; 3 Jun 2023 14:05:35 +0000 Received: from localhost ([127.0.0.1]:44017 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5Rst-00044X-9o for submit@debbugs.gnu.org; Sat, 03 Jun 2023 10:05:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5Rso-000449-Gm for 63852@debbugs.gnu.org; Sat, 03 Jun 2023 10:05:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q5Rsi-0004rz-V4; Sat, 03 Jun 2023 10:05:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=7xuf+T9Wcjp2S7mUtylyERRPOZ5SxelTlHkSSL7xzpA=; b=I31onZnZz6zEYGk7jE+2 hWHGmwoczI40jpH3kjEd0J0Zp1OBCDDvCGHp0sBQqTxvuoZsnpWp6iuEd60OvStH5OooE70VSUs1D StVOEnrR5V1kHnngh0/5qDkl06aOFT9ri0JTpz3WFjGK9Wu0U3Tycl4TvPBGGj8CpM38iu1IRzWZR 0muxNXnGngE2zG+jrOc3bZXOWOleR653QrVZ/P4lU8Uld/gNLMD0ZPH8YO+q6A3Ir5kEgWJ/Itm0p dADTcSkmNCP4ppREtWL2vAOvT/DyVt7V4kudrf33zZZQ/11jzBLiPgJ5E3RVyZPz/cmSKlUcdTij/ bSQ8P8gYD3A6ug==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q5RsV-0003xI-Be; Sat, 03 Jun 2023 10:05:24 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 63852@debbugs.gnu.org Subject: Re: bug#63852: =?utf-8?Q?=E2=80=98guix-package-cache=2Edrv?= =?utf-8?Q?=E2=80=99?= eats all your memory with =?utf-8?B?4oCYZ3VpeC1jcmFu4oCZ?= References: <87sfb93kca.fsf@inria.fr> Date: Sat, 03 Jun 2023 16:05:08 +0200 In-Reply-To: <87sfb93kca.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Fri, 02 Jun 2023 18:22:29 +0200") Message-ID: <875y843aln.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 63852 Cc: Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s skribis: > It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-gui= x-package-cache.drv, which > consumes memory until it gets OOM-killed. The culprit is easily found (here on just the 22K packages of =E2=80=98guix= =E2=80=99, so less than half of what you get with =E2=80=98guix-cran=E2=80=99): --8<---------------cut here---------------start------------->8--- $ guix repl GNU Guile 3.0.9 Copyright (C) 1995-2023 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guix-user)> ,m(gnu packages) scheme@(gnu packages)> ,use(statprof) scheme@(gnu packages)> (gcprof (lambda () (generate-package-cache "/tmp/t")= )) % cumulative self=20=20=20=20=20=20=20=20=20=20=20=20=20 time seconds seconds procedure 50.00 13.77 13.77 ice-9/boot-9.scm:2498:4 6.94 1.91 1.91 display 5.56 1.53 1.53 object->string 2.78 0.76 0.76 reverse 2.78 0.76 0.76 gensym 2.78 0.76 0.76 ice-9/boot-9.scm:2217:0:%load-announce 1.39 6.88 0.38 gnu/packages.scm:388:4:expand-cache 1.39 0.76 0.38 srfi/srfi-1.scm:1028:0:lset-intersection 1.39 0.38 0.38 number->string 1.39 0.38 0.38 gnu/packages/crates-io.scm:17783:2:arguments 1.39 0.38 0.38 open-output-string 1.39 0.38 0.38 gnu/packages/crates-io.scm:18450:2:arguments 1.39 0.38 0.38 gnu/packages/crates-io.scm:36208:2:arguments 1.39 0.38 0.38 guix/packages.scm:1320:8:mproc 1.39 0.38 0.38 gnu/packages/crates-io.scm:25101:2:arguments 1.39 0.38 0.38 gnu/packages/crates-io.scm:29972:2:arguments 1.39 0.38 0.38 gnu/packages/crates-io.scm:883:2:arguments 1.39 0.38 0.38 ice-9/vlist.scm:449:0:vhash-cons 1.39 0.38 0.38 append 1.39 0.38 0.38 string-append 1.39 0.38 0.38 system/vm/assembler.scm:2175:8 1.39 0.38 0.38 guix/build/syscalls.scm:1168:0:read-dirent-head= er/linux 1.39 0.38 0.38 srfi/srfi-1.scm:1033:17 1.39 0.38 0.38 gnu/packages/crates-io.scm:9336:2:arguments 1.39 0.38 0.38 gnu/packages/crates-io.scm:64009:2:arguments 1.39 0.38 0.38 guix/build-system/cargo.scm:229:4 1.39 0.38 0.38 ice-9/boot-9.scm:2759:0:module-make-local-var! 0.00 7328.64 0.00 system/vm/assembler.scm:1258:0:intern-constant 0.00 1423.97 0.00 ice-9/boot-9.scm:220:5:map1 0.00 201.90 0.00 ice-9/threads.scm:388:4 [...] --- Sample count: 72 Total time: 27.531145665 seconds (14.056589389 seconds in GC) --8<---------------cut here---------------end--------------->8--- The assembler allocates a huge constant table in =E2=80=98intern-constant= =E2=80=99 (=E2=80=98generate-package-cache=E2=80=99 produces one literal list contain= ing one small vector per package, each of which contains strings and other constants). The attached file emulates what =E2=80=98generate-package-cache=E2=80=99 do= es. Each vector in the list looks like this: #(0 "xyzxyz-0" "1.0.0" (gnu packages xyz0) xyzxyz-0 ("out") #t #f "gnu/pa= ckages/xyz0" 0 1) The program terminates with roughly a 250=C2=A0MiB heap for 22K entries=E2= =80=94about 10=C2=A0MiB per vector. The heap profile on completion looks like this: --8<---------------cut here---------------start------------->8--- % type self avg obj size 57.5 bytevector 8,205,632 455868.4 23.9 vector 3,413,857 549.3 7.7 struct 1,093,840 40.8 5.3 pair 753,376 16.0 2.7 stringbuf 385,856 51.1 1.1 symbol 151,424 32.0 0.4 program 59,424 47.3 0.3 string 49,152 32.0 0.2 heap-number 23,520 32.0 0.2 variable 21,696 30.3 0.1 smob 21,056 32.0 0.1 vm-continuation 20,864 32.0 0.1 atomic-box 20,736 32.0 0.1 weak-table 19,584 31.8 0.1 unknown 19,040 32.0 0.0 pointer 2,000 16.0 0.0 hash-table 896 32.0 0.0 primitive 144 16.0 0.0 weak-vector 96 16.0 0.0 frame 96 32.0 0.0 primitive-generic 64 32.0 0.0 keyword 16 16.0 sampled heap: 13.60166 MiB (heap size: 233.16016 MiB) --8<---------------cut here---------------end--------------->8--- It=E2=80=99s likely that the =E2=80=98buf=E2=80=99 field of is what= =E2=80=99s taking space On that topic, see also: https://lists.gnu.org/archive/html/guile-devel/2023-01/msg00013.html Ludo=E2=80=99. --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=assembler-memory-consumption.scm Content-Description: the reproducer ;; https://issues.guix.gnu.org/63852 (use-modules (system base compile) (language tree-il) (srfi srfi-1) (srfi srfi-26) (ice-9 match) (ice-9 time) (statprof)) (define lst (unfold (cut > <> 22000) (lambda (i) ;; Mimic 'generate-package-cache'. (define name (string-append "xyzxyz-" (number->string i 8))) (vector i name (string-append "1.0." (number->string (modulo i 300))) (list 'gnu 'packages (string->symbol (string-append "xyz" (number->string i)))) (string->symbol name) '("out") #t #f (string-append "gnu/packages/xyz" (number->string (modulo i 300))) (modulo i 1000) 1)) 1+ 0)) (pk 'go! (gc-stats)) (let ((exp (match (or (getenv "VARIANT") "0") ("0" `',lst) ("1" `(list ,@lst)) ("2" (make-const #f lst)) ;avoid psyntax overhead ("3" (make-primcall #f 'list (map (lambda (vector) (make-const #f vector)) lst)))))) (compile exp #:to 'bytecode #:from (if (pair? exp) 'scheme 'tree-il) #:optimization-level 0 #:opts '(#:to-file? #t))) (pk (gc-stats)) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 05 12:35:21 2023 Received: (at 63852) by debbugs.gnu.org; 5 Jun 2023 16:35:21 +0000 Received: from localhost ([127.0.0.1]:50244 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6DAu-0004hd-SZ for submit@debbugs.gnu.org; Mon, 05 Jun 2023 12:35:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44696) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6DAr-0004hP-QP for 63852@debbugs.gnu.org; Mon, 05 Jun 2023 12:35:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q6DAl-0003Pb-CQ; Mon, 05 Jun 2023 12:35:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=BVecSV7HqiBVXxwJFvWzGiaY9mRflH3z3F7/CQANNo4=; b=YvFwZcvJ1Bwzvr9ml8SP 7SDLBbh7BKEQxSOayu/Tnr41XDmikyK9c95f4UasLBGqBKxieWTpKiKfzA+PGSlg1x6GBub5jHi4y O0nS7ukUeFHMMpkAIacL+o5BwtS58GK2TBQm9WPjjYMcGAl1IGQsguk4zfqAv9PSa90ogbFSxLMVW bjIJ+CLU87s83lr3ApQEBgBOC6JPgNBHN031kvfRUtayAEsMyJjI5ol0XPMdAjxiIFzs+gR6h3VjZ QILxA3EXvyz6JUiYWVJozX9O2cGmdAmDP24qJaiJZNLq85x63DJialeD01I1l6JfIionLyHFLmUsO A6ih0TD2TJe1pw==; Received: from [193.50.110.155] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q6DAj-0003XT-KK; Mon, 05 Jun 2023 12:35:10 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 63852@debbugs.gnu.org Subject: Re: bug#63852: =?utf-8?Q?=E2=80=98guix-package-cache=2Edrv?= =?utf-8?Q?=E2=80=99?= eats all your memory with =?utf-8?B?4oCYZ3VpeC1jcmFu4oCZ?= References: <87sfb93kca.fsf@inria.fr> <875y843aln.fsf@gnu.org> Date: Mon, 05 Jun 2023 18:34:55 +0200 In-Reply-To: <875y843aln.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Sat, 03 Jun 2023 16:05:08 +0200") Message-ID: <87ttvlhnps.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 63852 Cc: Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > Ludovic Court=C3=A8s skribis: > >> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-gu= ix-package-cache.drv, which >> consumes memory until it gets OOM-killed. > > The culprit is easily found (here on just the 22K packages of =E2=80=98gu= ix=E2=80=99, so > less than half of what you get with =E2=80=98guix-cran=E2=80=99): (Beware of =E2=80=9Ceasily=E2=80=9D when the guy doesn=E2=80=99t know what = he=E2=80=99s talking about.) Turns out the problem is more pedestrian: =E2=80=98r-future-tests=E2=80=99 = depends on itself. --8<---------------cut here---------------start------------->8--- $ guix import cran future.tests Starting download of /tmp/guix-file.TOBjcf >From http://cran.r-project.org/src/contrib/future.tests_0.7.0.tar.gz... =E2=80=A6_0.7.0.tar.gz 186KiB 1.2MiB/s 00:00 =E2=96=95=E2= =96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96= =88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88= =E2=96=88=E2=96=8F 100.0% (package (name "r-future-tests") (version "0.7.0") (source (origin (method url-fetch) (uri (cran-uri "future.tests" version)) (sha256 (base32 "10g1w99xqr7l0cn27642aphqcvfidgpas38f84r815yy6k1ryrlx")))) (properties `((upstream-name . "future.tests"))) (build-system r-build-system) (propagated-inputs (list r-cli r-crayon r-future r-prettyunits r-sessioni= nfo)) (native-inputs (list r-future-tests)) (home-page "https://future.tests.futureverse.org") (synopsis "Test Suite for 'Future API' Backends") (description "Backends implementing the Future API, as defined by the future package,= should use the tests provided by this package to validate that they meet the minim= al requirements of the Future API. The tests can be performed easily from with= in R or from outside of R from the command line making it straightforward to inc= lude them in package tests and in Continuous Integration (CI) pipelines.") $ guix describe Generation 265 Jun 04 2023 23:48:31 (current) guix eed55a6 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea --8<---------------cut here---------------end--------------->8--- To be fixed in the importer. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 05 17:58:13 2023 Received: (at 63852) by debbugs.gnu.org; 5 Jun 2023 21:58:13 +0000 Received: from localhost ([127.0.0.1]:50464 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6IDM-0004fK-QQ for submit@debbugs.gnu.org; Mon, 05 Jun 2023 17:58:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49138) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6IDK-0004f7-BY for 63852@debbugs.gnu.org; Mon, 05 Jun 2023 17:58:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q6IDE-00007e-M7; Mon, 05 Jun 2023 17:58:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=AWMRNh/YLFSa/M2LV5NjDtUJPIKFLnyQb8dIT0PHHlA=; b=olz1rReRy9RU7x66eVD1 YTXiQexKbOMLsXJG61qGEIibXRAyc3Finma4oe/KdMsCrZFYIfPDWLEcIhKYXVaAAzBN62U44HhG+ 31Ag1EPYtJHrHTlhgar5ixR5CnQzoyBUUElOjNF1k4nDkDZUjX5UZRHeUzEAn5pw+64sD2Y1aHEU8 ugobnqyFpvbM3b6+ZGI8T8bsPEfC2UckjsdYI514wEcz5klZgNoWElDKoFwBj7tLxGmuUk+GBitfB +vXAMSvweXfdON5dsUUWo1uCPX4Y4z9/Ul+V1QiqSN/K9a22o1xNnyVMmV+q00USt74eu+grEJFa3 +URjZc5Or0oATg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q6IDE-0002pZ-96; Mon, 05 Jun 2023 17:58:04 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 63852@debbugs.gnu.org Subject: Re: bug#63852: =?utf-8?Q?=E2=80=98guix-package-cache=2Edrv?= =?utf-8?Q?=E2=80=99?= eats all your memory with =?utf-8?B?4oCYZ3VpeC1jcmFu4oCZ?= References: <87sfb93kca.fsf@inria.fr> <875y843aln.fsf@gnu.org> <87ttvlhnps.fsf@gnu.org> Date: Mon, 05 Jun 2023 23:58:02 +0200 In-Reply-To: <87ttvlhnps.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 05 Jun 2023 18:34:55 +0200") Message-ID: <87pm69h8r9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 63852 Cc: Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > Ludovic Court=C3=A8s skribis: > >> Ludovic Court=C3=A8s skribis: >> >>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-g= uix-package-cache.drv, which >>> consumes memory until it gets OOM-killed. >> >> The culprit is easily found (here on just the 22K packages of =E2=80=98g= uix=E2=80=99, so >> less than half of what you get with =E2=80=98guix-cran=E2=80=99): > > (Beware of =E2=80=9Ceasily=E2=80=9D when the guy doesn=E2=80=99t know wha= t he=E2=80=99s talking about.) > > Turns out the problem is more pedestrian: =E2=80=98r-future-tests=E2=80= =99 depends on > itself. Here=E2=80=99s a patch to prevent that from happening again: https://issues.guix.gnu.org/63917 (We also need to fix the importer.) Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 15 09:13:47 2023 Received: (at 63852-done) by debbugs.gnu.org; 15 Jun 2023 13:13:48 +0000 Received: from localhost ([127.0.0.1]:46427 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q9mnL-0004LY-J4 for submit@debbugs.gnu.org; Thu, 15 Jun 2023 09:13:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q9mnJ-0004LJ-IW for 63852-done@debbugs.gnu.org; Thu, 15 Jun 2023 09:13:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q9mnE-0003D1-1i; Thu, 15 Jun 2023 09:13:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=s36SQzz/YDsvyo8oO/S3+albQPIwBmBAGoVcCsD+VFg=; b=FyA5xulKYSB0t9+v44dx Ypwm1PwU3uMBK/tqc8WqaOKJwnzZuBUQL//BcOC7uqjBnlm9+mSbqrdpWtK8P2QKvYDFsz2wXcqmt E5PVSuklftktH7PiZZ8TryItSNyO9ITFw1IIAQqqMw+gSyfStacv/TwnqNxEXKuxD2iElBNmwmeg4 5TbLD2Snd3EfxT6bPWYC4M9MZBA1JM3YHW/U/RBurkjMyYOCsZdNF5B08KcUU0u9qDzq87F4DPrjX g2MVbfjGXP/IUNIPM+2PTWIXb/ASZebMUJDg8mupHaMv44Krl828dfPWz4+WDc25XkGMDKbUVKDgp HkLRv22jexJOIQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q9mnD-0002w5-Kx; Thu, 15 Jun 2023 09:13:39 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 63852-done@debbugs.gnu.org Subject: Re: bug#63852: =?utf-8?Q?=E2=80=98guix-package-cache=2Edrv?= =?utf-8?Q?=E2=80=99?= eats all your memory with =?utf-8?B?4oCYZ3VpeC1jcmFu4oCZ?= References: <87sfb93kca.fsf@inria.fr> <875y843aln.fsf@gnu.org> <87ttvlhnps.fsf@gnu.org> Date: Thu, 15 Jun 2023 15:13:37 +0200 In-Reply-To: <87ttvlhnps.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 05 Jun 2023 18:34:55 +0200") Message-ID: <87edmcx3zy.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 63852-done Cc: Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > Ludovic Court=C3=A8s skribis: > >> Ludovic Court=C3=A8s skribis: >> >>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-g= uix-package-cache.drv, which >>> consumes memory until it gets OOM-killed. >> >> The culprit is easily found (here on just the 22K packages of =E2=80=98g= uix=E2=80=99, so >> less than half of what you get with =E2=80=98guix-cran=E2=80=99): > > (Beware of =E2=80=9Ceasily=E2=80=9D when the guy doesn=E2=80=99t know wha= t he=E2=80=99s talking about.) > > Turns out the problem is more pedestrian: =E2=80=98r-future-tests=E2=80= =99 depends on > itself. With commit e4259d4e9e3251e4c4b45d1cce4008ac32b504c8, such cycles are detected early on. Closing this bug. We should still improve the memory footprint of the assembler though. Ludo=E2=80=99. From unknown Sat Jun 21 03:24:01 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 14 Jul 2023 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator