GNU bug report logs - #51787
GC takes more than 9 hours on berlin

Previous Next

Full log

Message #55 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christian Thäter <ct.guix <at> pipapo.org>
To: bug-guix <at> gnu.org
Subject: Re: bug#51787: GC takes more than 9 hours on berlin
Date: Mon, 13 Dec 2021 17:13:33 +0100

On 2021-12-12 18:09, Mathieu Othacehe wrote:

>Hey,
>
>> OK, thanks. Looks it just finished removing the trash directory
>> content. I started a GC process from my session to monitor it
>> closely.  
>
>Daily GC recap:
>
>* The GC process I started yesterday, did collect 5.5TiB in
>  approximately 24 hours, that are now in the /gnu/store/trash
>  directory.
>  
>* The /gnu/store/trash directory contains 288910 entries. If those
>items
>  are removed at the same rate than on the previous days, it will take
>  days/months to delete them all.
>
>* I noticed that the upstream Nix GC process can now operate without
>  locking. I think it shouldn't be too hard to port it to our fork or
>  maybe rewrite the process in Guile while we are at it.
>
>  That will not fix the slow hard-drives issues though.

While discussing this issue on IRC I came up with some idea:

'rmrfd' a system daemon that deletes huge trees in the background where
'-rf' stands for --really --fast :)

Actually this is an use case that happens for on my backup system too.
With that idea I just started coding and ran some experiments. For me
this looks quite feasible now and I will continue next days on this
small project. Any feedback or help would be welcomed!

The initial ideas and experiments are at https://github.com/cehteh/rmrfd

Note that the important part is that it will put some efforts into
freeing as much space as possible at begin of the freeing process,
Unlike just 'rm -rf' where space may only freed really late when the
last link count of the data goes to zero.

	Cheers
		Christian

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.