Subject: [scratch/elisp-benchmarks] test dependencies

From: Pip Cet <[email-masked]>
Date: Sat, 04 Jan 2025 16:37:26 +0000

There are dependencies between emacs-lisp-benchmarks tests, meaning we
can't always modify one test without affecting others.  The ones I'm
aware of are:

- elb-bytecomp.el uses elb-smie.el as compilation input (in this
  context, elb-smie.el is a "resource", not source code).  If we
  modify elb-smie.el, elb-bytecomp.el results will change and become
  uncomparable.

- elb-scroll.el uses elb-smie.el as a prerequisite.  If we modify
  elb-smie.el, elb-scroll.el results will change and become
  uncomparable.  Furthermore, elb-scroll.el relies on elb-smie.el
  being compiled in the appropriate fashion by some external
  mechanism.  (I think the best solution here is to merge elb-smie.el
  and elb-scroll.el into a single file declaring several tests, using
  the established ERT test framework)

- bubble.el and bubble-no-cons.el both "provide" the same feature.
  I'm not sure whether this causes any acute problems, but it
  demonstrates that the inconsistent use of (provide) in the
  benchmarks is problematic.  If we ever load tests using Frequire, we
  will load the wrong files.  If we don't, provide shouldn't be
  necessary.

- xmenu.c is used as a resource; this is fine.  But we must make sure
  we never copy the "current" xmenu.c to the benchmarks directory
  without clearly indicating that the benchmark results will become
  uncomparable.