GNU bug report logs - #23666
'add-to-store' RPC loads files in memory (daemon-side)

Previous Next

Full log

Message #19 received at 23666 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andreas Enge <andreas <at> enge.fr>
Cc: 23666 <at> debbugs.gnu.org
Subject: Re: bug#23666: guix download fails for large files
Date: Mon, 06 Jun 2016 09:06:07 +0200

Andreas Enge <andreas <at> enge.fr> skribis:

> On Wed, Jun 01, 2016 at 02:39:54PM +0200, Ludovic Courtès wrote:
>> This is implemented using the ‘add-to-store’ RPC, which, after all these
>> years, is still implemented like this (nix/libstore/local-store.cc):
>> 
>> --8<---------------cut here---------------start------------->8---
>> Path LocalStore::addToStore(const string & name, const Path & _srcPath,
>>     bool recursive, HashType hashAlgo, PathFilter & filter, bool repair)
>> {
>>     Path srcPath(absPath(_srcPath));
>>     debug(format("adding `%1%' to the store") % srcPath);
>> 
>>     /* Read the whole path into memory. This is not a very scalable
>>        method for very large paths, but `copyPath' is mainly used for
>>        small files. */
>> --8<---------------cut here---------------end--------------->8---
>
> Something that mainly does not fail could indeed be seen as a bug...
> But how come that "guix download http://" succeeds, where
> "guix download file://" fails?

Both use the ‘add-to-store’ RPC, so both have equal chances of failing.

Note that, fortunately, fixed-output derivations (‘guix build -S foo’)
do not use ‘add-to-store’.  In general, except in the case of ‘guix
download’, ‘add-to-store’ is only used with small files like when adding
.scm files to the store, etc.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.