Subject: Re: bug#44674: 28.0.50; Adding current-cpu-time for performance tests

From: Eli Zaretskii <[email-masked]>
Date: Mon, 16 Nov 2020 21:12:46 +0200

> From: Philipp Stephani <[email-masked]>
> Date: Mon, 16 Nov 2020 19:31:47 +0100
> Cc: [email-masked], Stefan Monnier <[email-masked]>
> 
> Am Mo., 16. Nov. 2020 um 18:17 Uhr schrieb Eli Zaretskii <[email-masked]>:
> >
> > > From: Philipp Stephani <[email-masked]>
> > > Date: Mon, 16 Nov 2020 12:46:07 +0100
> > > Cc: [email-masked], Stefan Monnier <[email-masked]>
> > >
> > > > Btw, once this goes in, how about making benchmark-run use it?
> > >
> > > benchmark-run measures walltime, not CPU time
> >
> > It does now, but we may want to change that.
> 
> That would be a breaking change.

I could argue that it's a bugfix.

> > Does it really make
> > sense to "benchmark" something using elapsed time?
> 
> Yes, the benchmarks I know all measure wall time. After all, that's
> what the user cares about.

??? On a system that is heavily loaded, and Emacs doesn't get an
execution unit close to 100% of the time, the wallclock time can be so
skewed as to defeat any meaningful measurements.  E.g., imagine that
you are running a benchmark while (another version of) Emacs is being
built with "make -jN", with N large enough to make all the cores busy.