GNU bug report logs - #44674
28.0.50; Adding current-cpu-time for performance tests

Previous Next

Full log

Message #86 received at 44674 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 44674 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#44674: 28.0.50; Adding current-cpu-time for performance tests
Date: Mon, 16 Nov 2020 21:12:46 +0200

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Mon, 16 Nov 2020 19:31:47 +0100
> Cc: 44674 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
> 
> Am Mo., 16. Nov. 2020 um 18:17 Uhr schrieb Eli Zaretskii <eliz <at> gnu.org>:
> >
> > > From: Philipp Stephani <p.stephani2 <at> gmail.com>
> > > Date: Mon, 16 Nov 2020 12:46:07 +0100
> > > Cc: 44674 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
> > >
> > > > Btw, once this goes in, how about making benchmark-run use it?
> > >
> > > benchmark-run measures walltime, not CPU time
> >
> > It does now, but we may want to change that.
> 
> That would be a breaking change.

I could argue that it's a bugfix.

> > Does it really make
> > sense to "benchmark" something using elapsed time?
> 
> Yes, the benchmarks I know all measure wall time. After all, that's
> what the user cares about.

??? On a system that is heavily loaded, and Emacs doesn't get an
execution unit close to 100% of the time, the wallclock time can be so
skewed as to defeat any meaningful measurements.  E.g., imagine that
you are running a benchmark while (another version of) Emacs is being
built with "make -jN", with N large enough to make all the cores busy.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.