On Fri, Oct 07, 2022 at 06:04:24PM -0400, Mark H Weaver wrote:
> Hi Efraim,
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > On Tue, Sep 13, 2022 at 07:56:46PM -0400, Mark H Weaver wrote:
> >> The following patch, which I've tested only on my private branch of
> >> Guix, dramatically reduces the amount of memory and time required to
> >> build Rust.
> >> 
> >> With this patch applied, I'm able to build Rust on a Thinkpad X200 with
> >> only 4 GB of RAM using both cores.  Without it, I'm unable to build Rust
> >> at all, even with parallel builds disabled, running in a bare text-only
> >> terminal, and with every nonessential daemon stopped.  When I tried, a
> >> single compiler process used more than 9 GB of virtual memory with heavy
> >> thrashing before the OOM killer finally put it out of its misery.
> >
> > Your x200 is an x86_64?
> 
> Yes, with only 4 GB of RAM.
> 
> > How much RAM+swap did you end up using with this patch?
> 
> During the times when I was monitoring it, the build process with
> parallel build enabled (2 parallel jobs) fit entirely within my 4 GB of
> RAM, while also running a modest X session with Emacs.
> 
> >> Moreover, with this patch the builds are dramatically faster than I've
> >> previously observed.  In my earlier bootstraps of Rust (up to
> >> rust-1.51), it took about 5 hours to build each compiler on my machine.
> >> Now it takes only about 2 hours per compiler, except for the initial
> >> bootstrap (1.54) which takes about 4 hours.
> >
> > I tested it out on my machine, it brought the build time down from 101
> > minutes to 71 minutes. Not the same improvement, but still noticeable.
> 
> I'm glad to hear it!  I guess the improvement was more dramatic on my
> system because it was previously thrashing (lots of swapping due to
> insufficient RAM), whereas your system probably has enough RAM :-)
> 
> >> The following patch is meant to be applied to the 'staging' branch, but
> >> I've only tested it on my own private branch, which is based on very old
> >> 'master' (April 2021) and is thus quite different by now.
> >> 
> >> I offer it here in case it might be of interest, although it hasn't been
> >> tested on contemporary Guix.  I'm not subscribed to this mailing list,
> >> so please make sure to CC me on any replies you'd like me to see.
> >> 
> >>      Regards,
> >>        Mark
> >
> > I'm building out to librsvg on staging right now with this patch
> > applied, so hopefully we'll see that it's good.
> >
> > I'm assuming that setting CARGO_PROFILE_DEV_DEBUG to false won't cause
> > any problems.
> 
> After writing my previous message, I ended up adding another hunk to my
> patch, to disable debugging in the final 'rust' compiler:
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
> index 25295a283e..cf2c66f334 100644
> --- a/gnu/packages/rust.scm
> +++ b/gnu/packages/rust.scm
> @@ -704,6 +704,13 @@ (define-public rust
>                       (("^python =.*" all)
>                        (string-append all
>                                       "gdb = \"" gdb "/bin/gdb\"\n"))))))
> +             (add-before 'build 'disable-debug
> +               (lambda _
> +                 ;; reduce memory requirements during the build
> +                 (setenv "CARGO_PROFILE_DEV_DEBUG" "false")
> +                 (substitute* "config.toml"
> +                   (("\\[rust\\].*" all)
> +                    (string-append all "debug = false\n")))))
>               (replace 'build
>                 ;; Phase overridden to also build rustfmt.
>                 (lambda* (#:key parallel-build? #:allow-other-keys)
> --8<---------------cut here---------------end--------------->8---
> 
> In retrospect, for the post-mrustc compilers, it might have been
> sufficient to add "debug = false" to the [rust] section of config.toml
> in the 'rust-1.55' variable.  This might also eliminate the need to set
> CARGO_PROFILE_DEV_DEBUG.  However, I haven't tried it.

In my testing I didn't see a significant difference between using
CARGO_PROFILE_DEV_DEBUG, debug = false or both. My personal preference
would be to use CARGO_PROFILE_DEV_DEBUG for rust-bootstrap and then use
debug = false for the rest, since it keeps the config options together
in the config.toml file.

I did also experiment with parallel-compiler = true to build a
multithreaded rustc but I wasn't able to prevent the test suite for
rust-1.60 to not error out. I also tried skipping the tests for rust
itself and compiling librsvg with it, but they were serious when they
said it wasn't ready.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted