GNU bug report logs - #74748
Constructor created by define-configuration is slow (>25s) when many fields are defined

Previous Next

To reply to this bug, email your comments to 74748 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: bug-guix <at> gnu.org
Subject: Constructor created by define-configuration is slow (>25s) when
 many fields are defined
Date: Mon, 09 Dec 2024 16:13:16 +0100

[Message part 1 (text/plain, inline)]

Hello,

I am writing a configuration for mpv and hit an issue of the generated
constructor (`home-mpv-configuration') is extremely slow.  While the
almost 800 fields is unusually high, initializing the instances of the
record should *not* take almost 30 seconds.  The full file is attached
to this email (as `mpv.scm').

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use (gnu home services mpv)
scheme@(guix-user)> ,t (and (home-mpv-configuration (global (mpv-profile-configuration (fullscreen #t)))) #t)                                                                                  
$1 = #t
;; 29.615315s real time, 50.896973s run time.  24.031723s spent in GC.
--8<---------------cut here---------------end--------------->8---

Interestingly, profiling does not give us anything useful:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,pr (and (home-mpv-configuration (global (mpv-profile-configuration (fullscreen #t)))) #t)                                                                                 
No samples recorded.
--8<---------------cut here---------------end--------------->8---

I also tried to remove all my custom sanitizers, to make sure the issue
is not in my code, but that did not help.

I took a short look, but code of both the define-configuration and of
Guix records is complex, full of syntax magic and I did not have any
luck trying to debug it myself.  Hence this report.

Have a nice day,
Tomas

[mpv.scm (text/x-guile, attachment)]

[Message part 3 (text/plain, inline)]

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 74748 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: 74748 <at> debbugs.gnu.org
Subject: Re: bug#74748: Constructor created by define-configuration is slow
 (>25s) when many fields are defined
Date: Tue, 24 Dec 2024 15:38:30 +0100

[Message part 1 (text/plain, inline)]

Hi,

Tomas Volf <~@wolfsden.cz> skribis:

> I am writing a configuration for mpv and hit an issue of the generated
> constructor (`home-mpv-configuration') is extremely slow.  While the
> almost 800 fields is unusually high, initializing the instances of the
> record should *not* take almost 30 seconds.  The full file is attached
> to this email (as `mpv.scm').

AFAICS, time is spent in one of the compilation steps:

--8<---------------cut here---------------start------------->8---
scheme@(gnu home services mpv)> ,o interp #t
scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
$13 = #<procedure 7f6ed3796700 at ice-9/eval.scm:330:13 ()>
;; 0.376347s real time, 0.375402s run time.  0.000000s spent in GC.
scheme@(gnu home services mpv)> ,o interp #f
scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
$14 = #<procedure 7f6ec32088f8 at <unknown port>:40:3 ()>
;; 11.149828s real time, 12.052915s run time.  1.581736s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (macroexpand '(mpv-profile-configuration)))
$15 = #t
;; 0.373865s real time, 0.372698s run time.  0.000000s spent in GC.
--8<---------------cut here---------------end--------------->8---

If we look more closely, it’s the optimizing compiler that takes time;
the baseline compile (-O1) is doing okay:

--8<---------------cut here---------------start------------->8---
scheme@(gnu home services mpv)> ,use(system base compile)
scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
$20 = #t
;; 0.378741s real time, 0.377043s run time.  0.000000s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'bytecode #:env (current-module)))
$21 = #t
;; 11.946879s real time, 12.961704s run time.  1.777478s spent in GC.
scheme@(gnu home services mpv)> (define til (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode))
$22 = #t
;; 11.403420s real time, 12.317183s run time.  1.581972s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode #:optimization-level 1))
$23 = #t
;; 0.225455s real time, 0.156137s run time.  0.000000s spent in GC.
--8<---------------cut here---------------end--------------->8---

Currently (guix build compile) applies these options to services:

        ((string-contains file "gnu/services/")
         ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
         ;; large files like gnu/services/mail.scm.
         (override-option #:letrectify? #f
                          (optimizations-for-level 2)))

I think we should at least apply this patch:

[Message part 2 (text/x-patch, inline)]

diff --git a/guix/build/compile.scm b/guix/build/compile.scm
index 5b27b55d02..f90016b9ae 100644
--- a/guix/build/compile.scm
+++ b/guix/build/compile.scm
@@ -127,7 +127,8 @@ (define (optimization-options file)
          ;; Use '-O1' to have partial evaluation and primitive inlining so we
          ;; can honor the "macro writer's bill of rights".
          (optimizations-for-level 1))
-        ((string-contains file "gnu/services/")
+        ((or (string-contains file "gnu/services/")
+             (string-contains file "gnu/home/services/"))
          ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
          ;; large files like gnu/services/mail.scm.
          (override-option #:letrectify? #f

[Message part 3 (text/plain, inline)]

Would that be enough for the home-mpv configuration records you wrote?

Thanks,
Ludo’.

Message #11 received at 74748 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 74748 <at> debbugs.gnu.org
Subject: Re: bug#74748: Constructor created by define-configuration is slow
 (>25s) when many fields are defined
Date: Fri, 27 Dec 2024 23:50:06 +0100

[Message part 1 (text/plain, inline)]

Hello,

thank you for finding the time to look into this issue.  Replies below.

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Tomas Volf <~@wolfsden.cz> skribis:
>
>> I am writing a configuration for mpv and hit an issue of the generated
>> constructor (`home-mpv-configuration') is extremely slow.  While the
>> almost 800 fields is unusually high, initializing the instances of the
>> record should *not* take almost 30 seconds.  The full file is attached
>> to this email (as `mpv.scm').
>
> AFAICS, time is spent in one of the compilation steps:
>
> scheme@(gnu home services mpv)> ,o interp #t
> scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
> $13 = #<procedure 7f6ed3796700 at ice-9/eval.scm:330:13 ()>
> ;; 0.376347s real time, 0.375402s run time.  0.000000s spent in GC.
> scheme@(gnu home services mpv)> ,o interp #f
> scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
> $14 = #<procedure 7f6ec32088f8 at <unknown port>:40:3 ()>
> ;; 11.149828s real time, 12.052915s run time.  1.581736s spent in GC.
> scheme@(gnu home services mpv)> ,t (->bool (macroexpand '(mpv-profile-configuration)))
> $15 = #t
> ;; 0.373865s real time, 0.372698s run time.  0.000000s spent in GC.
>
>
> If we look more closely, it’s the optimizing compiler that takes time;
> the baseline compile (-O1) is doing okay:
>
> scheme@(gnu home services mpv)> ,use(system base compile)
> scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
> $20 = #t
> ;; 0.378741s real time, 0.377043s run time.  0.000000s spent in GC.
> scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'bytecode #:env (current-module)))
> $21 = #t
> ;; 11.946879s real time, 12.961704s run time.  1.777478s spent in GC.
> scheme@(gnu home services mpv)> (define til (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
> scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode))
> $22 = #t
> ;; 11.403420s real time, 12.317183s run time.  1.581972s spent in GC.
> scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode #:optimization-level 1))
> $23 = #t
> ;; 0.225455s real time, 0.156137s run time.  0.000000s spent in GC.
>
> Currently (guix build compile) applies these options to services:
>
>         ((string-contains file "gnu/services/")
>          ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
>          ;; large files like gnu/services/mail.scm.
>          (override-option #:letrectify? #f
>                           (optimizations-for-level 2)))
>
> I think we should at least apply this patch:
>
> diff --git a/guix/build/compile.scm b/guix/build/compile.scm
> index 5b27b55d02..f90016b9ae 100644
> --- a/guix/build/compile.scm
> +++ b/guix/build/compile.scm
> @@ -127,7 +127,8 @@ (define (optimization-options file)
>           ;; Use '-O1' to have partial evaluation and primitive inlining so we
>           ;; can honor the "macro writer's bill of rights".
>           (optimizations-for-level 1))
> -        ((string-contains file "gnu/services/")
> +        ((or (string-contains file "gnu/services/")
> +             (string-contains file "gnu/home/services/"))
>           ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
>           ;; large files like gnu/services/mail.scm.
>           (override-option #:letrectify? #f
>
>
> Would that be enough for the home-mpv configuration records you wrote?

I admit I did not test the patch attached, but my understanding is that
even if the speed up was sufficient, it would take effect only for
compilation via the make command.  Assuming that is correct, it would
not change much.  I am (luckily) former C++ developer, so I am used to
compilations taking ~forever.  So, yes, while the compilation of mpv.scm
(via `make') took roughly 40s on my machine, it was annoying, but not
worthy of a bug report.

The actual problem for me was the slowness in REPL.  Would that be
addressed by the patch above?  If not, do we have any way to do that?

)tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 74748 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> chbouib.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 74748 <at> debbugs.gnu.org
Subject: Re: bug#74748: Constructor created by define-configuration is slow
 (>25s) when many fields are defined
Date: Tue, 15 Apr 2025 10:25:39 +0200

Hi,

Tomas Volf <~@wolfsden.cz> writes:

> I admit I did not test the patch attached, but my understanding is that
> even if the speed up was sufficient, it would take effect only for
> compilation via the make command.

No, (guix build compile) is also used by (guix self), which is what
‘guix pull’ & co. use.

I pushed the patch as 5b158ddca9425d79ea4ceb374003fe0f7e6bd336, it can
only be beneficial.

> The actual problem for me was the slowness in REPL.  Would that be
> addressed by the patch above?  If not, do we have any way to do that?

At the REPL you could type:

  ,o optimization-level 1

You could use ‘repl-option-set!’ from ~/.guile somehow I guess.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.