GNU bug report logs - #44053
Poor profile generation performance on spinning disks

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: bug#44053: closed (Re: bug#44053: Poor profile generation
 performance on spinning disks)
Date: Thu, 03 Apr 2025 07:27:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#44053: Poor profile generation performance on spinning disks

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 44053 <at> debbugs.gnu.org.

-- 
44053: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44053
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: jbranso--- via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: 44053-done <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>,
 Luis Felipe <luis.felipe.la <at> protonmail.com>,
 Ludovic Courtès <ludo <at> gnu.org>, jbranso <at> dismail.de
Subject: Re: bug#44053: Poor profile generation performance on spinning disks
Date: Thu, 03 Apr 2025 16:25:36 +0900

Hi,

jbranso--- via Bug reports for GNU Guix <bug-guix <at> gnu.org> writes:

> April 2, 2025 at 6:16 AM, "Andreas Enge" <andreas <at> enge.fr
> mailto:andreas <at> enge.fr?to=%22Andreas%20Enge%22%20%3Candreas%40enge.fr%3E
>> wrote:
>
>
>
>> 
>> Hello,
>> 
>> as "there is little room for optimization" and nobody has taken the time
>> to look at it more closely, can we close this issue?
>>
>> Andreas
>
> Perhaps less people use spinning disks at this point.  SSDs/NVMes are fairly
> popular.  Also I don't use guix at the moment, so I am unable to be a tester
> if this issue was fixed.  :(

I've moved to NVMe myself and things have been zippy enough.  I guess we
could consider this issue having been obsoleted by advances in
technology.

Closing.

-- 
Thanks,
Maxim

[Message part 3 (message/rfc822, inline)]

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: bug-guix <bug-guix <at> gnu.org>
Subject: Poor profile generation performance on spinning disks
Date: Sat, 17 Oct 2020 23:35:45 -0400

Hello!

I've noticed on multiple occasions that using Guix on traditional
spinning drives can be quite slow.

On my home machine, will is still relying on 2 x 1 TB spinning drives in
RAID1, rebuilding my user profile, which contains 182 entries, takes on
average about 20 minutes, even when there are no packages to be built:

--8<---------------cut here---------------start------------->8--- $ time
guix package -i perl --max-jobs=1 The following package will be
upgraded: perl (dependencies or package changed)

The following derivation will be built:
   /gnu/store/lhywla1z2zcz16df4hbvvvngr9zmswr7-profile.drv

building CA certificate bundle...
building fonts directory...
generating GLib schema cache...
creating GTK+ icon theme cache...
building cache files for GTK+ input methods...
building directory of Info manuals...
building database for manual pages...
building XDG desktop file cache...
building XDG MIME database...
building profile with 182 packages...

real    19m0.126s
user    0m5.648s
sys     0m0.333s
--8<---------------cut here---------------end--------------->8---

Most of the time remains spent after the message 'building profile with
182 package...'.  That part seems IO-bound, with the spinning disks
grinding heavily and the CPU mostly idling.  The rest of the time (3
minutes), was used by the profile hooks.

The same operation on a second, more modern machine equipped with M2
SSDs does much better and takes about 1 minute to accomplish the same,
so it seems the bad performance can be mostly attributed to the much
slower disk seek times of the spinning disks.

On the older machine, two profile hooks are also sticking out w.r.t. the
time they take (they take more than one minute opposed to a few
seconds):

--8<---------------cut here---------------start------------->8---
The following profile hook will be built:
   /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
building database for manual pages...
Creating manual page database...
[2139/2139] building list of man-db entries...
175322 entries processed in 95.1 s
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
/gnu/store/wzp4mk2r7r4ysciw74gqbfkyai0zmrcc-manual-database

real    1m36.378s
user    0m1.674s
sys     0m0.108s

The following profile hook will be built:
   /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
/gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database

real    1m7.344s
user    0m1.331s
sys     0m0.053s
--8<---------------cut here---------------end--------------->8---

So we should profile what's going on while generating the profile (no
pun intended) and try to improve this at first since this is where most
of the time is spent on spinning drives (17 minutes out of the 20 in the
above example).

After that we could look into the two above profile hooks.

Thanks,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.