Subject: Re: bug#23666: guix download fails for large files

From: [email-masked] (Ludovic Courtès)
Date: Mon, 06 Jun 2016 09:06:07 +0200

Andreas Enge <[email-masked]> skribis:

> On Wed, Jun 01, 2016 at 02:39:54PM +0200, Ludovic Courtès wrote:
>> This is implemented using the ‘add-to-store’ RPC, which, after all these
>> years, is still implemented like this (nix/libstore/local-store.cc):
>> 
>> --8<---------------cut here---------------start------------->8---
>> Path LocalStore::addToStore(const string & name, const Path & _srcPath,
>>     bool recursive, HashType hashAlgo, PathFilter & filter, bool repair)
>> {
>>     Path srcPath(absPath(_srcPath));
>>     debug(format("adding `%1%' to the store") % srcPath);
>> 
>>     /* Read the whole path into memory. This is not a very scalable
>>        method for very large paths, but `copyPath' is mainly used for
>>        small files. */
>> --8<---------------cut here---------------end--------------->8---
>
> Something that mainly does not fail could indeed be seen as a bug...
> But how come that "guix download http://" succeeds, where
> "guix download file://" fails?

Both use the ‘add-to-store’ RPC, so both have equal chances of failing.

Note that, fortunately, fixed-output derivations (‘guix build -S foo’)
do not use ‘add-to-store’.  In general, except in the case of ‘guix
download’, ‘add-to-store’ is only used with small files like when adding
.scm files to the store, etc.

Ludo’.