GNU bug report logs - #79001
Retain profile image files generated during build

Previous Next

To reply to this bug, email your comments to 79001 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Retain profile image files generated during build
Date: Sat, 12 Jul 2025 18:07:03 -0400

[Message part 1 (text/plain, inline)]

I want to use low-level profiling information generated during the
build process for evaluating changes to C primitives, particularly the
reader.

The attached patch sets up the environment variable GMON_OUT_PREFIX
when running emacs during the build so that the output incorporates
the target in the name and generates unique names during a parallel
build.

However, in a parallel build, I am now seeing the cus-load.el target
get run twice simultaneously, so that one instance will exit with a
file lock error and stop the build.

I also get random segmentation faults from emacs during the build, but
they do not repeat when I reinvoke make, and the file being
byte-compiled when the segfault occurred is not rerun.

In addition to the patches to commit
d63746d5191fb6e4ee6e39addfc1a05e0e0214b1, I have attached a list of
the names of the generated profile images for review.

I would like to have profiling start before the dump file is loaded,
but my simple minded attempt caused the shell to terminate emacs with
a message like "profiling timer expired".  I assume monstartup
(src/emacs.c:2557) does something to undo the signal handling set up
during emacs initialization, but I can't find any documentation for
that function on linux, or determine what else would have to change to
move that call up to the start of main.

You can get the new behavior with
./configure --enable-profiling=all
make PROFILE_BUILD=yes

./configure --enable-profiling=build
should have the effect of 'all' plus making PROFILE_BUILD=yes the
default in the Makefiles, but I have not tested it.

Also, I'm not sure if there are any canned analyses that should be
incorporated in the Makefiles as targets, or sets of files that should
be summarized for such analyses.

Lynn

[0002-The-does-not-work-with-the-newline-escaping.patch (text/x-patch, attachment)]

[0003-The-recipe-code-for-summarizing-recursive-targets-ne.patch (text/x-patch, attachment)]

[0001-Support-for-retaining-profiling-from-build.patch (text/x-patch, attachment)]

[0004-configure.ac-Updated-help-for-enable-profiling-optio.patch (text/x-patch, attachment)]

[profile-images (application/octet-stream, attachment)]

Message #8 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: 79001 <at> debbugs.gnu.org
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Sat, 12 Jul 2025 23:31:18 -0400

[Message part 1 (text/plain, inline)]

On Sat, Jul 12, 2025 at 6:08 PM Lynn Winebarger <owinebar <at> gmail.com> wrote:
> However, in a parallel build, I am now seeing the cus-load.el target
> get run twice simultaneously, so that one instance will exit with a
> file lock error and stop the build.

I am not sure why the generate-ja-dic target in lisp/Makefile.in
depends on custom-deps, but that seems to be the cause of the race
condition.  At least the attached patch appears to eliminate the
problem (and the random segfaults for that matter) in a parallel make.

Lynn

[0005-Removed-race-from-generate-ja-dic-dependency-on-cust.patch (text/x-patch, attachment)]

Message #11 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: 79001 <at> debbugs.gnu.org
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Sat, 12 Jul 2025 23:37:19 -0400

[Message part 1 (text/plain, inline)]

On Sat, Jul 12, 2025 at 11:31 PM Lynn Winebarger <owinebar <at> gmail.com> wrote:
>
> On Sat, Jul 12, 2025 at 6:08 PM Lynn Winebarger <owinebar <at> gmail.com> wrote:
> > However, in a parallel build, I am now seeing the cus-load.el target
> > get run twice simultaneously, so that one instance will exit with a
> > file lock error and stop the build.
>
> I am not sure why the generate-ja-dic target in lisp/Makefile.in
> depends on custom-deps, but that seems to be the cause of the race
> condition.  At least the attached patch appears to eliminate the
> problem (and the random segfaults for that matter) in a parallel make.

Also, I've attached an updated list of generated profile image files.
It appears there are 1684 now.

Lynn

[profile-images (application/octet-stream, attachment)]

Message #14 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lynn Winebarger <owinebar <at> gmail.com>
Cc: 79001 <at> debbugs.gnu.org
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Sun, 13 Jul 2025 08:02:25 +0300

> From: Lynn Winebarger <owinebar <at> gmail.com>
> Date: Sat, 12 Jul 2025 23:31:18 -0400
> 
> On Sat, Jul 12, 2025 at 6:08 PM Lynn Winebarger <owinebar <at> gmail.com> wrote:
> > However, in a parallel build, I am now seeing the cus-load.el target
> > get run twice simultaneously, so that one instance will exit with a
> > file lock error and stop the build.
> 
> I am not sure why the generate-ja-dic target in lisp/Makefile.in
> depends on custom-deps

It doesn't.  It is probably some local snafu on your system, because
lisp/Makefile.in I have here says

  # This is the OKURO-NASI compilation trigger.
  generate-ja-dic: main-first
	  $(AM_V_at)$(MAKE) -C ../leim generate-ja-dic EMACS="$(EMACS)"
	  $(AM_V_at)$(MAKE) compile-targets TARGETS="./leim/ja-dic/ja-dic.elc"

without custom-deps dependency anywhere in sight.

Maybe try re-running the configure script, after syncing
lisp/Makefile.in with the upstream repository?

Message #17 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79001 <at> debbugs.gnu.org
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Mon, 14 Jul 2025 22:00:13 -0400

[Message part 1 (text/plain, inline)]

On Sun, Jul 13, 2025 at 1:02 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > I am not sure why the generate-ja-dic target in lisp/Makefile.in
> > depends on custom-deps
>
> It doesn't.  It is probably some local snafu on your system, because
> lisp/Makefile.in I have here says

Ugh, you are right - it was in the first patch, though I do not recall
adding it.
I made a fresh clone of the emacs repo and applied all the changes
before committing.  I'm attaching the combined patch which does not
contain that non-sequitur.

The profiles of the bootstrap dump and the dump with the loadup
libraries byte-compiled is in read0 for 0.2 and 0.1 seconds
respectively (barely enough to measure).  When I summarize profiles of
all the ELC jobs, though, the total time is about 5 seconds.  It
should be at least enough for a meaningful comparison, even though it
is a tiny part of the overall computation time.  If you're interested
in measuring just read0, anyway. and not the cost of the entire
activation of the various entry points to read0.

Lynn

[0001-Support-for-retaining-profiling-from-build.patch (text/x-patch, attachment)]

Message #20 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79001 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Thu, 17 Jul 2025 09:21:19 -0400

[Message part 1 (text/plain, inline)]

I just recalled Paul is listed as a maintainer for the configure/make
files, so adding him.  This patch is almost entirely for the build
process.  I always see requests for measurement of a proposed change's
impact on performance, but no standard benchmarking that I'm aware of.  The
build is one process that stresses the lisp implementation, so this patch
should at least provide one useful benchmark. If nothing else, it will
allow others to replicate the measurements I'm making for
comparison/validation.

On Mon, Jul 14, 2025, 10:00 PM Lynn Winebarger <owinebar <at> gmail.com> wrote:

> On Sun, Jul 13, 2025 at 1:02 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > > I am not sure why the generate-ja-dic target in lisp/Makefile.in
> > > depends on custom-deps
> >
> > It doesn't.  It is probably some local snafu on your system, because
> > lisp/Makefile.in I have here says
>
> Ugh, you are right - it was in the first patch, though I do not recall
> adding it.
> I made a fresh clone of the emacs repo and applied all the changes
> before committing.  I'm attaching the combined patch which does not
> contain that non-sequitur.
>
> The profiles of the bootstrap dump and the dump with the loadup
> libraries byte-compiled is in read0 for 0.2 and 0.1 seconds
> respectively (barely enough to measure).  When I summarize profiles of
> all the ELC jobs, though, the total time is about 5 seconds.  It
> should be at least enough for a meaningful comparison, even though it
> is a tiny part of the overall computation time.  If you're interested
> in measuring just read0, anyway. and not the cost of the entire
> activation of the various entry points to read0.
>
> Lynn
>

[Message part 2 (text/html, inline)]

Message #23 received at 79001 <at> debbugs.gnu.org (full text, mbox):

From: Lynn Winebarger <owinebar <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79001 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#79001: Retain profile image files generated during build
Date: Sat, 19 Jul 2025 11:27:07 -0400

[Message part 1 (text/plain, inline)]

I revised the patch to fix a bug causing profiling to always be
enabled.   The commit message is much more detailed.

I have also noticed a bug in the current build process that occurs
when enable-profiling is specified and with a parallel build.  I
randomly see compile jobs reporting segmentation faults.  The compiled
files appear to be correctly output, so my best guess (and that is all
it is) is that the race condition for 32 compile processes all
competing to write "gmon.out" triggers some bug in the profiling code
inserted by gcc with the pg flag.  I don't see these issues with this
patch, but a more targeted fix could be to have emacs specify the
environment variable "GMON_OUT_PREFIX" at startup if it is not already
set, since the profiling code will then use that name suffixed by the
process id instead of just "gmon.out".  That should at least address
the race condition, both with other instances of emacs and with the
helper executables which will also race for the file "gmon.out" if
executed from the same directory as the emacs process that invokes
them.

I don't know whether the segfaults should be considered a bug in "gcc
-pg", or a violation of the requirements imposed on using the
instrumented code generated by "gcc -pg".

Lynn

[0001-Support-for-retaining-profile-images-from-build.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.