Subject: bug#76446: [Patch Debbugs v3] Infrastructure Improvements

From: Michael Albinus <[email-masked]>
Date: Tue, 25 Feb 2025 20:29:19 +0100

Morgan Smith <[email-masked]> writes:

Hi Morgan,

> I started looking at what emacs does.  It looks complicated.  I don't
> entirely understand what they are doing.  However, I would argue that
> what is currently in the debbugs makefile is quite powerful and I would
> be interested in what use cases your solution provides.
>
> Are you trying to improve performance of running the tests?  Here are the
> performance numbers for running the current tests on my laptop:

No. It isn't performance. What I'm trying to do is to run only one test
file by a given make call. This wasn't possible; your version of the
Makefile has always loaded everything.

> Are you trying to selectively run tests so we can debug tests one at a time?
> That is already entirely doable with the current setup.  Running 'make check
> TESTS=get-status' only runs a singular test.

This is exactly what is called SELECTOR in the Emacs make. And this is
the same name used in the ERT manual.

I said already in my last message that this is missing, and I will add this.

> The patch you have provided actually removes the ability to run one test at a
> time as we'd have to run entire files.

No, with adding SELECTOR to the Makefile we'll have it. Read test/README
from the Emacs source tree for a better understanding how Emacs does it.

Disclaimer: I have written a large part of that README.

> I'm not certain what the SELECTORs do but perhaps you're thinking that
> in the future we would have groups of tests (like "expensive" or
> "network") that we could select.  In the interest of not
> over-engineering on hypotheticals, I would say that is a future problem.
> One that I would likely solve by putting those test files in
> sub-directories labeled "expensive" and "network".

A SELECTOR can be anything. A regexp, a test name a tag, you name it.
See (info "(ert) Test Selectors")

Best regards, Michael.