GNU bug report logs - #41607
Deleted store items are not actually deleted

Previous Next

Full log

View this message in rfc822 format

From: Stephen Scheck <singularsyntax <at> gmail.com>
To: Christopher Marusich <cmmarusich <at> gmail.com>
Cc: 41607 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>, zimoun <zimon.toutoune <at> gmail.com>, Leo Famulari <leo <at> famulari.name>
Subject: bug#41607: Deleted store items are not actually deleted
Date: Fri, 5 Jun 2020 12:05:24 -0400

[Message part 1 (text/plain, inline)]

Very cool - thanks Chris!

In the meantime, I've updated my build script to externalize the Guix
environment from the Docker container.
So far the daily builds are staying nice and small at ~197MB and one layer.
The images and updated script are
here if anybody is curious:

https://hub.docker.com/r/singularsyntax/guix/tags
https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml

GitLab allows you to cache files between job stages and even full pipeline
runs. I first tried to cache /var/guix
and /gnu/store and mount them inside a container in which to perform `guix
pull` etc. However, it seems
that handling hard links on externally mounted file systems from within a
container is problematic. I think
passing `--disable-deduplication` to guix-daemon might resolve it, but I
couldn't figure out where/how to
change the Shepherd configuration to do that. So instead, I just copy the
directories into and out of the
container at start and end of the process. It seems to work. Mounting would
probably speed up the process
quite a bit if I could make it work.

Cheers,
-Steve

On Fri, Jun 5, 2020 at 5:32 AM Christopher Marusich <cmmarusich <at> gmail.com>
wrote:

> Chris Marusich <cmmarusich <at> gmail.com> writes:
>
> > Ludovic Courtès <ludo <at> gnu.org> writes:
> >
> >>> Should Guix do anything about this?  We could change guix-daemon to
> take
> >>> correct action in the face of an XDEV error.  We could also improve the
> >>> logging, since currently it silently swallows the XDEV error.
> >>
> >> I guess we could delete recursively right away upon EXDEV.  It should be
> >> just two lines of code, right?
> >
> > I'll try making the change and report back.  Yes, there are other cases
> > where we immediately delete without moving into the trash directory
> > (e.g., when the trash directory fails to be created), so it seems OK.
>
> Here is a patch.  Turns out it's was just a one line change!  If nobody
> has any further feedback on it, I'll go ahead and merge it to the master
> branch in the next couple days.
>
> I tested it in one of the Docker containers provided by Stephen which
> exhibited the problem.  I built the new Guix inside the Docker container
> and verified that a path which was previously unable to be GC'd due to
> the EXDEV error, was now able to be successfully GC'd.
>
> My understanding is that the only reason why the guix-daemon attempts to
> move dead directories to the trash directory is to save time on
> deleting, since large directories could take a while to fully delete.
> If there is any reason why it might be unsafe to delete the directories
> directly in case of EXDEV (I cannot think of any), please let me know.
>
> --
> Chris
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.