GNU bug report logs - #75359
[scratch/elisp-benchmarks] test dependencies

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> protonmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [scratch/elisp-benchmarks] test dependencies
Date: Sat, 04 Jan 2025 16:37:26 +0000

There are dependencies between emacs-lisp-benchmarks tests, meaning we
can't always modify one test without affecting others.  The ones I'm
aware of are:

- elb-bytecomp.el uses elb-smie.el as compilation input (in this
  context, elb-smie.el is a "resource", not source code).  If we
  modify elb-smie.el, elb-bytecomp.el results will change and become
  uncomparable.

- elb-scroll.el uses elb-smie.el as a prerequisite.  If we modify
  elb-smie.el, elb-scroll.el results will change and become
  uncomparable.  Furthermore, elb-scroll.el relies on elb-smie.el
  being compiled in the appropriate fashion by some external
  mechanism.  (I think the best solution here is to merge elb-smie.el
  and elb-scroll.el into a single file declaring several tests, using
  the established ERT test framework)

- bubble.el and bubble-no-cons.el both "provide" the same feature.
  I'm not sure whether this causes any acute problems, but it
  demonstrates that the inconsistent use of (provide) in the
  benchmarks is problematic.  If we ever load tests using Frequire, we
  will load the wrong files.  If we don't, provide shouldn't be
  necessary.

- xmenu.c is used as a resource; this is fine.  But we must make sure
  we never copy the "current" xmenu.c to the benchmarks directory
  without clearly indicating that the benchmark results will become
  uncomparable.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.