GNU bug report logs - #41077
[feature/native-comp] Segfaults when compiling ELC+ELN

Previous Next

Full log

View this message in rfc822 format

From: Kévin Le Gouguec <kevin.legouguec <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 41077-done <at> debbugs.gnu.org
Subject: bug#41077: [feature/native-comp] virtual memory exhausted
Date: Sun, 10 May 2020 16:26:21 +0200

[Message part 1 (text/plain, inline)]

Hi Andrea!

Thank you for implementing this blacklist; it turns out that
char-fold.el was the only file my laptop could not handle[1], though as
you'll see org.el was a strong contender.

It took 3 days, but make -j1 successfully ran to completion on commit
92cf4bb.  In comparison, I've just compiled commit 9d8fc3a on master
from scratch, and it took exactly 1 hour 35 minutes.

As soon as I've figured out how to use the elisp-benchmarks package,
I'll post some figures; is there a specific place (bug number,
emacs-devel thread) where you usually collect such feedback?

I've attached some more graphs on the compilation process, as well as
the source material[2] (measurement script, measurements, and plotting
script).

The overall picture (only files which took more than 10 minutes to
compile are labeled):

[monitor.pdf (application/pdf, attachment)]

[Message part 3 (text/plain, inline)]

Some "areas of detail" (only files which took more than 5 minutes to
compile are labeled):

[monitor-swap.pdf (application/pdf, attachment)]

[monitor-gnus.pdf (application/pdf, attachment)]

[monitor-org.pdf (application/pdf, attachment)]

[Message part 7 (text/plain, inline)]

Some comments:

- As could be predicted from your previous measurements, org.el was a
  beast, but unlike with char-fold.el the little guy pulled through 🙌

- It's hard to be sure since my measurements were so imprecise[3], but
  AFAICT the compilation process for a single file seems to follow a
  memory usage pattern of "slow rise - spike - drop - spike".  See
  e.g. files.el, isearch.el, simple.el, subr.el, window.el, info.el,
  package.el, erc.el, gnus-sum.el, org.el, python.el.


Thank you for taking the time to guide me through compiling this branch.
I know that reducing the memory footprint of native compilation is
probably not your main focus right now, but I figured it would be
interesting to provide some orders of magnitude.


[1] $ git diff
    diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
    index 60b41f95bd..ff3c42a178 100644
    --- a/lisp/emacs-lisp/comp.el
    +++ b/lisp/emacs-lisp/comp.el
    @@ -85,7 +85,7 @@ comp-always-compile
       :group 'comp)

     (defcustom comp-bootstrap-black-list
    -  '("^leim/")
    +  '("^leim/" "^char-fold")
       "List of regexps to exclude files from native compilation during bootstrap.
     Skip if any is matching."
       :type 'list

[2] Measurement script:

[monitor.sh (application/x-shellscript, attachment)]

[Message part 9 (text/plain, inline)]

    Measurements:

[monitor.log.tgz (application/x-compressed-tar, attachment)]

[Message part 11 (text/plain, inline)]

    Plotting script (requires matplotlib):

[plot.py (text/x-python, attachment)]

[Message part 13 (text/plain, inline)]

[3] I didn't instrument the build process; as can be seen in the
    scripts, I just relied on:

        ps --sort=-vsz p $pids | head 1

    $pids include the emacs session in which I was running make; this
    session was masking basically everything other than the ELC+ELN
    processes.

    Also, the sampling step being 30 seconds, a lot of interesting
    patterns may not have been recorded.

    Overall these scripts are an exercise in How Not To Collect Data 😒.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.