From unknown Sat Jul 26 11:04:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23666: guix download fails for large files Resent-From: Andreas Enge Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 31 May 2016 19:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 23666 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 23666@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.146472461217122 (code B ref -1); Tue, 31 May 2016 19:57:02 +0000 Received: (at submit) by debbugs.gnu.org; 31 May 2016 19:56:52 +0000 Received: from localhost ([127.0.0.1]:48675 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7pmV-0004S5-Si for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:52 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45727) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7pmU-0004Rt-CT for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b7pmO-0003T5-DZ for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:45 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:50378) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmO-0003Ss-AM for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36685) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmM-00037g-4y for bug-guix@gnu.org; Tue, 31 May 2016 15:56:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b7pmH-0003Rg-PI for bug-guix@gnu.org; Tue, 31 May 2016 15:56:41 -0400 Received: from mailrelay2.public.one.com ([91.198.169.125]:44634) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmH-0003R5-Bz for bug-guix@gnu.org; Tue, 31 May 2016 15:56:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=enge.fr; s=20140924; h=from:subject:date:message-id:to:mime-version:content-type; bh=kBj5xaM7LmW8CzWvEwNV+gW0dYczbOtrQZVi7jjxVeg=; b=qmbIZyYtZI4U6TdaBsKC9i3cPCx6CTlH3O5zHsZK3koOnsz/VHO23oGmKZqiIhlU/G5xehIaszwLt Jyc12Tjfn0os/GyDwFEeVVIDkG5Xa0chJihwtT08S7+G5FXIcOP8ez/MkZH/ZMNF8bQDMLaLZ/3aYN H857ZgjDuGCtqVhI= X-HalOne-Cookie: af39bd020affca61a32a26ee296b8dfa93ba0fea X-HalOne-ID: c69c5c69-2769-11e6-8278-b82a72d03b9b Received: from solar (unknown [92.89.73.211]) by smtpfilter2.public.one.com (Halon Mail Gateway) with ESMTPSA; Tue, 31 May 2016 19:56:33 +0000 (UTC) Date: Tue, 31 May 2016 21:56:31 +0200 From: Andreas Enge Message-ID: <20160531195631.GA22897@solar> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.6.0 (2016-04-01) X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello, since I am regularly rebuilding texlive (who is not!), I would like to keep the source tarball on my machine, and whenever I garbage collect it from the store, add it back via guix download file:texlive-20150523-texmf.tar.xz On my Novena with a 32 bit processor and 4 GB of main memory, this fails with guix download: error: sendfile: Broken pipe The file itself has 1,8 GB. Downloading the file via http as part of "guix build" works, and "guix download file:" also works on my x86_64 machine with 8 GB of RAM. Andreas From unknown Sat Jul 26 11:04:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23666: guix download fails for large files Resent-From: Leo Famulari Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 31 May 2016 20:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23666 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Andreas Enge Cc: 23666@debbugs.gnu.org Received: via spool by 23666-submit@debbugs.gnu.org id=B23666.146472641926193 (code B ref 23666); Tue, 31 May 2016 20:27:02 +0000 Received: (at 23666) by debbugs.gnu.org; 31 May 2016 20:26:59 +0000 Received: from localhost ([127.0.0.1]:48719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7qFe-0006oP-MV for submit@debbugs.gnu.org; Tue, 31 May 2016 16:26:58 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:46228) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7qFc-0006oH-DW for 23666@debbugs.gnu.org; Tue, 31 May 2016 16:26:56 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id A266C20B35; Tue, 31 May 2016 16:26:55 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute6.internal (MEProxy); Tue, 31 May 2016 16:26:55 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=famulari.name; h= cc:content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=ci7ZH kxoAqJcZgbCXEPfzAzbadw=; b=wBvlh2Dy/5+9fcPBbj2ahrFoQrol/AxImXINo KdgW/ts/nBriQQzCBHdx/a+5sudyCzsO44mMLRoTjMgk+PKdrbsw8jxl4t4PF1iU dwAX9pEvTlfTeV1OLG6B/50iDwFopS9SliEApBGEciR5yDz6gaB1VYtIoIvzo2Ev kLMK+g= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=ci7ZHkxoAqJcZgbCXEPfzAzbadw=; b=uyaVq shwc57CjZkEA0q11njZK1iv/7jw9T/vWVQjZZtMjNwQenCVfUBp/FS2lxi3gI1y6 ZAwNTwJ5Si8ublCnx5WAaY7RwfwKP9sAeiGcfKFmtKHpl62vCJyx0FiWeg7riSww V0FM3Ttq/WvOFSc/+weG3eSBFw/Nm3NsAd3mbI= X-Sasl-enc: UYl0CvVMMJrAQfmdwx9sRWz48BcYt0/yDMmVarjj/FJ5 1464726415 Received: from localhost (c-73-188-17-148.hsd1.pa.comcast.net [73.188.17.148]) by mail.messagingengine.com (Postfix) with ESMTPA id 57301F2A4D; Tue, 31 May 2016 16:26:55 -0400 (EDT) Date: Tue, 31 May 2016 16:26:51 -0400 From: Leo Famulari Message-ID: <20160531202651.GA19070@jasmine> References: <20160531195631.GA22897@solar> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160531195631.GA22897@solar> User-Agent: Mutt/1.6.0 (2016-04-01) X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Tue, May 31, 2016 at 09:56:31PM +0200, Andreas Enge wrote: > since I am regularly rebuilding texlive (who is not!), I would like to keep > the source tarball on my machine, and whenever I garbage collect it from the > store, add it back via > guix download file:texlive-20150523-texmf.tar.xz > > On my Novena with a 32 bit processor and 4 GB of main memory, this fails with > guix download: error: sendfile: Broken pipe > > The file itself has 1,8 GB. > > Downloading the file via http as part of "guix build" works, and > "guix download file:" also works on my x86_64 machine with 8 GB of RAM. As a temporary workaround, you could make the tarball a gcroot: # ln -sf /gnu/store/...-texlive-20150523-texmf.tar.xz \ /var/guix/gcroots/texlive-texmf.tar.xz From unknown Sat Jul 26 11:04:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23666: guix download fails for large files Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 01 Jun 2016 12:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23666 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Andreas Enge Cc: 23666@debbugs.gnu.org Received: via spool by 23666-submit@debbugs.gnu.org id=B23666.14647848094206 (code B ref 23666); Wed, 01 Jun 2016 12:41:01 +0000 Received: (at 23666) by debbugs.gnu.org; 1 Jun 2016 12:40:09 +0000 Received: from localhost ([127.0.0.1]:49171 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b85RQ-00015m-T3 for submit@debbugs.gnu.org; Wed, 01 Jun 2016 08:40:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:58685) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b85RP-00015a-8f for 23666@debbugs.gnu.org; Wed, 01 Jun 2016 08:40:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b85RG-0003Mw-S9 for 23666@debbugs.gnu.org; Wed, 01 Jun 2016 08:40:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41997) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b85RG-0003Mg-Pc; Wed, 01 Jun 2016 08:39:58 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:39376 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b85RF-0002o3-0i; Wed, 01 Jun 2016 08:39:57 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <20160531195631.GA22897@solar> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 14 Prairial an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Wed, 01 Jun 2016 14:39:54 +0200 In-Reply-To: <20160531195631.GA22897@solar> (Andreas Enge's message of "Tue, 31 May 2016 21:56:31 +0200") Message-ID: <878typkrqd.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) Andreas Enge skribis: > On my Novena with a 32 bit processor and 4 GB of main memory, this fails = with > guix download: error: sendfile: Broken pipe > > The file itself has 1,8 GB. This is implemented using the =E2=80=98add-to-store=E2=80=99 RPC, which, af= ter all these years, is still implemented like this (nix/libstore/local-store.cc): --8<---------------cut here---------------start------------->8--- Path LocalStore::addToStore(const string & name, const Path & _srcPath, bool recursive, HashType hashAlgo, PathFilter & filter, bool repair) { Path srcPath(absPath(_srcPath)); debug(format("adding `%1%' to the store") % srcPath); /* Read the whole path into memory. This is not a very scalable method for very large paths, but `copyPath' is mainly used for small files. */ --8<---------------cut here---------------end--------------->8--- This is undoubtedly what=E2=80=99s causing the failure. Until this is fixed, you could wrap the file in a nar and import the nar (nar import does not have this limitation, see =E2=80=99LocalStore::importP= ath=E2=80=99 and co.). To do that, you could do: guix archive --export $(guix build -S texlive-texmf-whatever) > t.nar and later: guix archive --import < t.nar HTH, Ludo=E2=80=99. From unknown Sat Jul 26 11:04:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23666: guix download fails for large files Resent-From: Andreas Enge Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 05 Jun 2016 20:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23666 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 23666@debbugs.gnu.org Received: via spool by 23666-submit@debbugs.gnu.org id=B23666.146516033030696 (code B ref 23666); Sun, 05 Jun 2016 20:59:02 +0000 Received: (at 23666) by debbugs.gnu.org; 5 Jun 2016 20:58:50 +0000 Received: from localhost ([127.0.0.1]:55929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b9f8E-0007z2-8L for submit@debbugs.gnu.org; Sun, 05 Jun 2016 16:58:50 -0400 Received: from mailrelay7.public.one.com ([91.198.169.215]:59680) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b9f8C-0007yo-WB for 23666@debbugs.gnu.org; Sun, 05 Jun 2016 16:58:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=enge.fr; s=20140924; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=4wcLuDq4Cuys5+EXVjAy4+p92GPUvvZ2E4MgjEgpMy0=; b=nCvhLO8OIo9fS7+vGLEcguOh7ECQKZL0Z15BjTk+VDFKpOD7/kzwYaUfACJD+2G9GXm+ynIbjk/Wv j2BwJLJ9gPe1EWnHrHoJxqBEz7dNs/omkass+LW5x1Gar/XCLb4Ra0VDnf9yv6rMkjtzboVPssjUAy SCmKK80yPM3PTGFU= X-HalOne-Cookie: 3db6237ac8803e7fa0dd8ad0f131e55fc05d0f3c X-HalOne-ID: 488d750b-2b60-11e6-a0c6-b82a72cffc46 Received: from solar (unknown [92.89.73.211]) by smtpfilter4.public.one.com (Halon Mail Gateway) with ESMTPSA; Sun, 5 Jun 2016 20:58:41 +0000 (UTC) Date: Sun, 5 Jun 2016 22:58:39 +0200 From: Andreas Enge Message-ID: <20160605205839.GB20831@solar> References: <20160531195631.GA22897@solar> <878typkrqd.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <878typkrqd.fsf@gnu.org> User-Agent: Mutt/1.6.0 (2016-04-01) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Thanks for your suggestions, Leo and Ludovic! I still see this as a bug; should I report it upstream to Nix? On Wed, Jun 01, 2016 at 02:39:54PM +0200, Ludovic Courtès wrote: > This is implemented using the ‘add-to-store’ RPC, which, after all these > years, is still implemented like this (nix/libstore/local-store.cc): > > --8<---------------cut here---------------start------------->8--- > Path LocalStore::addToStore(const string & name, const Path & _srcPath, > bool recursive, HashType hashAlgo, PathFilter & filter, bool repair) > { > Path srcPath(absPath(_srcPath)); > debug(format("adding `%1%' to the store") % srcPath); > > /* Read the whole path into memory. This is not a very scalable > method for very large paths, but `copyPath' is mainly used for > small files. */ > --8<---------------cut here---------------end--------------->8--- Something that mainly does not fail could indeed be seen as a bug... But how come that "guix download http://" succeeds, where "guix download file://" fails? Andreas From unknown Sat Jul 26 11:04:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#23666: guix download fails for large files Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 06 Jun 2016 07:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23666 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Andreas Enge Cc: 23666@debbugs.gnu.org Received: via spool by 23666-submit@debbugs.gnu.org id=B23666.14651967807826 (code B ref 23666); Mon, 06 Jun 2016 07:07:01 +0000 Received: (at 23666) by debbugs.gnu.org; 6 Jun 2016 07:06:20 +0000 Received: from localhost ([127.0.0.1]:56155 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b9oc8-00022A-FM for submit@debbugs.gnu.org; Mon, 06 Jun 2016 03:06:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:41148) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b9oc7-00021x-4E for 23666@debbugs.gnu.org; Mon, 06 Jun 2016 03:06:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b9oby-0006Zk-W5 for 23666@debbugs.gnu.org; Mon, 06 Jun 2016 03:06:14 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45263) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b9oby-0006ZU-SI; Mon, 06 Jun 2016 03:06:10 -0400 Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:45908 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b9obx-0007yy-6I; Mon, 06 Jun 2016 03:06:09 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <20160531195631.GA22897@solar> <878typkrqd.fsf@gnu.org> <20160605205839.GB20831@solar> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Prairial an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Mon, 06 Jun 2016 09:06:07 +0200 In-Reply-To: <20160605205839.GB20831@solar> (Andreas Enge's message of "Sun, 5 Jun 2016 22:58:39 +0200") Message-ID: <87k2i27q5c.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) Andreas Enge skribis: > On Wed, Jun 01, 2016 at 02:39:54PM +0200, Ludovic Court=C3=A8s wrote: >> This is implemented using the =E2=80=98add-to-store=E2=80=99 RPC, which,= after all these >> years, is still implemented like this (nix/libstore/local-store.cc): >>=20 >> --8<---------------cut here---------------start------------->8--- >> Path LocalStore::addToStore(const string & name, const Path & _srcPath, >> bool recursive, HashType hashAlgo, PathFilter & filter, bool repair) >> { >> Path srcPath(absPath(_srcPath)); >> debug(format("adding `%1%' to the store") % srcPath); >>=20 >> /* Read the whole path into memory. This is not a very scalable >> method for very large paths, but `copyPath' is mainly used for >> small files. */ >> --8<---------------cut here---------------end--------------->8--- > > Something that mainly does not fail could indeed be seen as a bug... > But how come that "guix download http://" succeeds, where > "guix download file://" fails? Both use the =E2=80=98add-to-store=E2=80=99 RPC, so both have equal chances= of failing. Note that, fortunately, fixed-output derivations (=E2=80=98guix build -S fo= o=E2=80=99) do not use =E2=80=98add-to-store=E2=80=99. In general, except in the case = of =E2=80=98guix download=E2=80=99, =E2=80=98add-to-store=E2=80=99 is only used with small f= iles like when adding .scm files to the store, etc. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 24 10:03:27 2016 Received: (at control) by debbugs.gnu.org; 24 Jul 2016 14:03:28 +0000 Received: from localhost ([127.0.0.1]:35927 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bRK07-0003ZP-OB for submit@debbugs.gnu.org; Sun, 24 Jul 2016 10:03:27 -0400 Received: from eggs.gnu.org ([208.118.235.92]:34410) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bRK06-0003ZC-6e for control@debbugs.gnu.org; Sun, 24 Jul 2016 10:03:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bRK00-0004mW-D3 for control@debbugs.gnu.org; Sun, 24 Jul 2016 10:03:21 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36841) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bRK00-0004mG-AA for control@debbugs.gnu.org; Sun, 24 Jul 2016 10:03:20 -0400 Received: from reverse-83.fdn.fr ([80.67.176.83]:43776 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bRJzy-0007pb-Gp for control@debbugs.gnu.org; Sun, 24 Jul 2016 10:03:19 -0400 Date: Sun, 24 Jul 2016 16:03:16 +0200 Message-Id: <87oa5ncevv.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #24053 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.3 (------) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.3 (------) merge 24053 23666 From unknown Sat Jul 26 11:04:12 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Andreas Enge Subject: bug#23666: closed (Close) Message-ID: References: <20160531195631.GA22897@solar> X-Gnu-PR-Message: they-closed 23666 X-Gnu-PR-Package: guix Reply-To: 23666@debbugs.gnu.org Date: Sun, 06 Jul 2025 20:21:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1751833262-27516-1" This is a multi-part message in MIME format... ------------=_1751833262-27516-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #23666: 'add-to-store' RPC loads files in memory (daemon-side) which was filed against the guix package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 23666@debbugs.gnu.org. --=20 23666: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D23666 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1751833262-27516-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 23666-done) by debbugs.gnu.org; 6 Jul 2025 20:20:21 +0000 Received: from localhost ([127.0.0.1]:59013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uYVqX-00073F-5C for submit@debbugs.gnu.org; Sun, 06 Jul 2025 16:20:21 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:59160) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uYVqT-00071D-IG; Sun, 06 Jul 2025 16:20:18 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 4178618D; Sun, 6 Jul 2025 22:20:11 +0200 (CEST) Authentication-Results: hera.aquilenet.fr; none X-Virus-Scanned: Debian amavis at hera.aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavis, port 10024) with ESMTP id NiX3Yy4yx8of; Sun, 6 Jul 2025 22:20:10 +0200 (CEST) Received: from jurong (86.56.141.77.rev.sfr.net [77.141.56.86]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 665C813E; Sun, 6 Jul 2025 22:20:10 +0200 (CEST) Date: Sun, 6 Jul 2025 22:20:09 +0200 From: Andreas Enge To: 23666-done@debbugs.gnu.org, 30706-done@debbugs.gnu.org, 43274-done@debbugs.gnu.org, 67048-done@debbugs.gnu.org Subject: Close Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spamd-Result: default: False [-3.22 / 15.00]; NEURAL_HAM(-3.00)[-1.000]; BAYES_HAM(-0.62)[82.05%]; MID_RHS_NOT_FQDN(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[] X-Rspamd-Action: no action X-Rspamd-Queue-Id: 4178618D X-Rspamd-Server: hera X-Spamd-Bar: --- X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 23666-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Closing old issues. Andreas ------------=_1751833262-27516-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 31 May 2016 19:56:52 +0000 Received: from localhost ([127.0.0.1]:48675 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7pmV-0004S5-Si for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:52 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45727) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7pmU-0004Rt-CT for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b7pmO-0003T5-DZ for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:45 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:50378) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmO-0003Ss-AM for submit@debbugs.gnu.org; Tue, 31 May 2016 15:56:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36685) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmM-00037g-4y for bug-guix@gnu.org; Tue, 31 May 2016 15:56:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b7pmH-0003Rg-PI for bug-guix@gnu.org; Tue, 31 May 2016 15:56:41 -0400 Received: from mailrelay2.public.one.com ([91.198.169.125]:44634) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b7pmH-0003R5-Bz for bug-guix@gnu.org; Tue, 31 May 2016 15:56:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=enge.fr; s=20140924; h=from:subject:date:message-id:to:mime-version:content-type; bh=kBj5xaM7LmW8CzWvEwNV+gW0dYczbOtrQZVi7jjxVeg=; b=qmbIZyYtZI4U6TdaBsKC9i3cPCx6CTlH3O5zHsZK3koOnsz/VHO23oGmKZqiIhlU/G5xehIaszwLt Jyc12Tjfn0os/GyDwFEeVVIDkG5Xa0chJihwtT08S7+G5FXIcOP8ez/MkZH/ZMNF8bQDMLaLZ/3aYN H857ZgjDuGCtqVhI= X-HalOne-Cookie: af39bd020affca61a32a26ee296b8dfa93ba0fea X-HalOne-ID: c69c5c69-2769-11e6-8278-b82a72d03b9b Received: from solar (unknown [92.89.73.211]) by smtpfilter2.public.one.com (Halon Mail Gateway) with ESMTPSA; Tue, 31 May 2016 19:56:33 +0000 (UTC) Date: Tue, 31 May 2016 21:56:31 +0200 From: Andreas Enge To: bug-guix@gnu.org Subject: guix download fails for large files Message-ID: <20160531195631.GA22897@solar> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.6.0 (2016-04-01) X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello, since I am regularly rebuilding texlive (who is not!), I would like to keep the source tarball on my machine, and whenever I garbage collect it from the store, add it back via guix download file:texlive-20150523-texmf.tar.xz On my Novena with a 32 bit processor and 4 GB of main memory, this fails with guix download: error: sendfile: Broken pipe The file itself has 1,8 GB. Downloading the file via http as part of "guix build" works, and "guix download file:" also works on my x86_64 machine with 8 GB of RAM. Andreas ------------=_1751833262-27516-1-- From unknown Sat Jul 26 11:04:12 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Mark H Weaver Subject: bug#24053: closed (Close) Message-ID: References: <87h9bhmvz3.fsf@netris.org> X-Gnu-PR-Message: they-closed 24053 X-Gnu-PR-Package: guix Reply-To: 24053@debbugs.gnu.org Date: Sun, 06 Jul 2025 20:21:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1751833262-27516-3" This is a multi-part message in MIME format... ------------=_1751833262-27516-3 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #23666: guix download file:// fails with a ~2 GB file on i686 which was filed against the guix package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 24053@debbugs.gnu.org. --=20 23666: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D23666 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1751833262-27516-3 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 23666-done) by debbugs.gnu.org; 6 Jul 2025 20:20:21 +0000 Received: from localhost ([127.0.0.1]:59013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uYVqX-00073F-5C for submit@debbugs.gnu.org; Sun, 06 Jul 2025 16:20:21 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:59160) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uYVqT-00071D-IG; Sun, 06 Jul 2025 16:20:18 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 4178618D; Sun, 6 Jul 2025 22:20:11 +0200 (CEST) Authentication-Results: hera.aquilenet.fr; none X-Virus-Scanned: Debian amavis at hera.aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavis, port 10024) with ESMTP id NiX3Yy4yx8of; Sun, 6 Jul 2025 22:20:10 +0200 (CEST) Received: from jurong (86.56.141.77.rev.sfr.net [77.141.56.86]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 665C813E; Sun, 6 Jul 2025 22:20:10 +0200 (CEST) Date: Sun, 6 Jul 2025 22:20:09 +0200 From: Andreas Enge To: 23666-done@debbugs.gnu.org, 30706-done@debbugs.gnu.org, 43274-done@debbugs.gnu.org, 67048-done@debbugs.gnu.org Subject: Close Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spamd-Result: default: False [-3.22 / 15.00]; NEURAL_HAM(-3.00)[-1.000]; BAYES_HAM(-0.62)[82.05%]; MID_RHS_NOT_FQDN(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[] X-Rspamd-Action: no action X-Rspamd-Queue-Id: 4178618D X-Rspamd-Server: hera X-Spamd-Bar: --- X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 23666-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Closing old issues. Andreas ------------=_1751833262-27516-3 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 22 Jul 2016 17:19:53 +0000 Received: from localhost ([127.0.0.1]:33968 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bQe76-0000zs-RH for submit@debbugs.gnu.org; Fri, 22 Jul 2016 13:19:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45484) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bQe75-0000zh-Kh for submit@debbugs.gnu.org; Fri, 22 Jul 2016 13:19:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bQe6z-0003CN-NP for submit@debbugs.gnu.org; Fri, 22 Jul 2016 13:19:46 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:40146) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQe6z-0003CE-K7 for submit@debbugs.gnu.org; Fri, 22 Jul 2016 13:19:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36450) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQe6x-0005TZ-IK for bug-guix@gnu.org; Fri, 22 Jul 2016 13:19:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bQe6u-0003Bl-2u for bug-guix@gnu.org; Fri, 22 Jul 2016 13:19:43 -0400 Received: from world.peace.net ([50.252.239.5]:40854) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQe6t-0003Bh-VF for bug-guix@gnu.org; Fri, 22 Jul 2016 13:19:40 -0400 Received: from pool-71-174-37-60.bstnma.east.verizon.net ([71.174.37.60] helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bQe6t-0003GU-Bh; Fri, 22 Jul 2016 13:19:39 -0400 From: Mark H Weaver To: bug-guix@gnu.org Subject: guix download file:// fails with a ~2 GB file on i686 Date: Fri, 22 Jul 2016 13:19:28 -0400 Message-ID: <87h9bhmvz3.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On my i686 GuixSD machine, the following command consistently fails: mhw@jojen:~$ guix download file:///home/mhw/texlive-20160523b-texmf.tar.xz guix download: error: sendfile: Broken pipe The file is 2071190200 bytes, which is about 73 MiB less than 2 GiB. I have 2 GiB of RAM and 4 GiB of swap. My guix client is v0.10.0-1899-g0f971a0. My guix-daemon is 0.10.0. Mark ------------=_1751833262-27516-3--