GNU bug report logs - #70877
guix-daemon fails to copy 4+GB file to store

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#70877: closed (guix-daemon fails to copy 4+GB file to store)
Date: Mon, 13 May 2024 16:25:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 13 May 2024 18:24:22 +0200
with message-id <87ikzh7k95.fsf <at> gnu.org>
and subject line Re: bug#70877: guix-daemon fails to copy 4+GB file to store
has caused the debbugs.gnu.org bug report #70877,
regarding guix-daemon fails to copy 4+GB file to store
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
70877: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70877
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Ricardo Wurmus <rekado <at> elephly.net>
To: bug-guix <at> gnu.org
Subject: guix-daemon fails to copy 4+GB file to store
Date: Sat, 11 May 2024 12:52:53 +0200

[Message part 3 (text/plain, inline)]

The guix-daemon's libutil/util.cc uses copy_file_range to copy a
downloaded file into the store.  copy_file_range fails on files larger
than 4GB with an error like this:

    guix build: error: short write in copy_file_range `15' to `16': No such file or directory

The man page for copy_file_range says that it could return EFBIG when
the range exceeds the maximum range.  The daemon code does not check any
limits and will attempt to copy the whole file.

I believe our code ought to check the value of st.size and fall back to
a boring copy if it exceeds some "reasonable" value.

This is where copy_file_range is used:
https://git.savannah.gnu.org/cgit/guix.git/tree/nix/libutil/util.cc#n382

Here is a little reproducer:

[bug.scm (text/plain, inline)]

(use-modules (guix download)
             (guix packages)
             (guix build-system trivial))

(package
  (name "chungus")
  (version "1")
  (source
   (origin
     (method url-fetch)
     (uri "http://localhost:1111/chungus")
     (sha256
      (base32 "0nx67d4ls2nfwcfdmg81vf240z6lpwpdqypssr1wzn3hyz4szci4"))))
  (build-system trivial-build-system)
  (home-page "")
  (synopsis "")
  (description "")
  (license #f))

[Message part 5 (text/plain, inline)]

--8<---------------cut here---------------start------------->8---
# generate a big file
dd bs=1M count=4096 if=/dev/zero of=/tmp/chungus
# serve it
guix shell woof -- woof -i 127.0.0.1 -p 1111 -c 1 /tmp/chungus
# build the source derivation
guix build --no-grafts -Sf bug.scm
# observe the error
# guix build: error: short write in copy_file_range `15' to `16': No such file or directory
--8<---------------cut here---------------end--------------->8---

-- 
Ricardo

[Message part 6 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 70877-done <at> debbugs.gnu.org
Subject: Re: bug#70877: guix-daemon fails to copy 4+GB file to store
Date: Mon, 13 May 2024 18:24:22 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Pushed as 7757fdd491862fa5c33f1f894503346b89898a01.
>
> I’ll update the ‘guix’ package to make the fix available.

Done in 58be9a79e2862d5fa9842d73f498ce2e5442b9ce.

Ludo'.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.