GNU bug report logs - #75356
impossible to benchmark byte-compiled code in a native-comp build

Previous Next

Full log

View this message in rfc822 format

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: pipcet <at> protonmail.com, 75356 <at> debbugs.gnu.org
Subject: bug#75356: impossible to benchmark byte-compiled code in a native-comp build
Date: Mon, 06 Jan 2025 16:24:24 +0200

> Cc: pipcet <at> protonmail.com
> From: Andrea Corallo <acorallo <at> gnu.org>
> Date: Mon, 06 Jan 2025 04:24:44 -0500
> 
> Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
> editors" <bug-gnu-emacs <at> gnu.org> writes:
> 
> > elisp-benchmarks.el unconditionally forces use of native compilation
> > if run on an Emacs which is compiled with native-comp enabled.  This
> > is a minor issue, but it's still important to benchmark this
> > configuration once in a while.
> 
> Why is this important?  On a native compiled Emacs all code is supposed
> to run native compiled, not only the test, but runtime libraries as
> well, further more even Emacs primitives are different in order to
> accomodate native code execution.  My opinion is that results of such a
> mixed tests would make little no sense in general.

I think the idea is that we don't want to force someone who wants to
benchmark bytecode to build a separate Emacs configured without native
compilation.

It is okay to default to native code, but it would be nice to have a
knob to run the benchmarks without compiling them to native code.

> Anyway if you really want to run bytecode on a native compiled Emacs
> elisp-benchamerks let you do this with:
> 
> emacs -batch -l ./elisp-benchmarks.el --eval '(progn (setq elb-speed -1) (elisp-benchmarks-run))'

Would this refrain from loading *.eln files if they are already there?
Or does the benchmark suite remove all the *.eln files after it
finishes?  (Apologies for not taking a closer look at the code.)

> > In general, the compilation logic of elisp-benchmarks.el is fragile
> > and will lead to unreliabe test results,
> 
> Why do you think so?

I suggest to avoid such general remarks, and instead talk about
specific issues where you think the logic could easily break.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.