Ludovic Courtès writes: > Hey Chris, > > Christopher Baines skribis: > >> The Guix derivation graph isn't that deep, so I don't think this generally >> opens lots of files, but I think it's still unnecessary to keep more files >> than needed open. >> >> * guix/derivations.scm (read-derivation-from-file): Read each derivation to a >> string, which is passed as a port to read-derivation. > > [...] > >> - (let ((drv (call-with-input-file file read-derivation))) >> + (let ((drv >> + ;; read-derivation can call read-derivation-from-file, so to >> + ;; avoid having multiple open files when reading a derivation >> + ;; with inputs, read it in to a string first. >> + (call-with-input-string >> + (call-with-input-file file >> + get-string-all) >> + read-derivation))) > > How real is the risk of having too many open files due to a > ‘read-derivation’ call? (Where too many is >= 100.) > > You might think it’s likely because: > > $ guix gc -R $(guix build -d --no-grafts emacs) |grep drv$ | wc -l > 2234 > > > But in fact, due to the shape of the graph + memoization (I suppose¹), > there are at most 27 open file descriptors in this case: > > $ strace -o /tmp/log.strace -e openat guile -c "(use-modules (guix)) (read-derivation-from-file \"$(guix build -d --no-grafts emacs)\")" > $ cut -d '=' -f 2- < /tmp/log.strace |sort -un | tail > 18 > 19 > 20 > 21 > 22 > 23 > 24 > 25 > 26 > 27 > > > So to me the status quo is probably okay. > > The reason I’m paying attention to this is that allocating a string port > plus a string for the whole contents every time would put pressure on > memory, which is worth avoiding if we can. > > WDYT? I guess there should be a way of arranging the code so that it doesn't keep unnecessary ports, but also doesn't use strings, but that will require some rearranging. I think I just got thinking about this as the build coordinator was using excessive file descriptors, but this isn't the cause.