Subject: bug#76446: [Patch Debbugs v3] Infrastructure Improvements

From: Michael Albinus <[email-masked]>
Date: Fri, 28 Feb 2025 13:04:23 +0100

Michael Albinus <[email-masked]> writes:

Hi Morgan,

>> Are you trying to selectively run tests so we can debug tests one at a time?
>> That is already entirely doable with the current setup.  Running 'make check
>> TESTS=get-status' only runs a singular test.
>
> This is exactly what is called SELECTOR in the Emacs make. And this is
> the same name used in the ERT manual.
>
> I said already in my last message that this is missing, and I will add this.
>
>> The patch you have provided actually removes the ability to run one test at a
>> time as we'd have to run entire files.
>
> No, with adding SELECTOR to the Makefile we'll have it. Read test/README
> from the Emacs source tree for a better understanding how Emacs does it.
>
> Disclaimer: I have written a large part of that README.
>
>> I'm not certain what the SELECTORs do but perhaps you're thinking that
>> in the future we would have groups of tests (like "expensive" or
>> "network") that we could select.  In the interest of not
>> over-engineering on hypotheticals, I would say that is a future problem.
>> One that I would likely solve by putting those test files in
>> sub-directories labeled "expensive" and "network".
>
> A SELECTOR can be anything. A regexp, a test name a tag, you name it.
> See (info "(ert) Test Selectors")

I have extended the Makefile to support also SELECTOR, see
appended. Comments?

Best regards, Michael.

diff --git a/Makefile b/Makefile
index da0764ea36..258d553ae1 100644
--- a/Makefile
+++ b/Makefile
@@ -1,15 +1,48 @@
+# Copyright (C) 2024-2025 Free Software Foundation, Inc.
+
+# Author: Morgan Smith <[email-masked]>
+# Package: debbugs
+# Keywords: comm, hypermedia, maint
+
+# This file is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This file is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+### Commentary:
+
+## Some  test targets:
+##
+## check: re-run all tests.
+## filename-tests: re-run tests from test/filename-tests.el(c).
+
+## SELECTOR discrimination (see ERT manual for more possibilities):
+##
+## SELECTOR='"regexp"': Run all tests which name match "regexp"
+## SELECTOR='test-name': Run test with name test-name
+
+### Code:
+
 EMACS ?= emacs
 MAKEINFO ?= makeinfo

-# regex of tests to run
-TESTS=.*
+SOURCE = $(wildcard *.el)
+TESTSOURCE = $(wildcard test/*.el)
+TARGET = $(filter-out debbugs-pkg.elc,$(patsubst %.el,%.elc,$(SOURCE)))
+TESTTARGET = $(patsubst %.el,%.elc,$(TESTSOURCE))

-SOURCE=$(wildcard *.el)
-TESTSOURCE=$(wildcard test/*.el)
-TARGET=$(filter-out debbugs-pkg.elc,$(patsubst %.el,%.elc,$(SOURCE)))
-TESTTARGET=$(patsubst %.el,%.elc,$(TESTSOURCE))
+TESTS = $(patsubst test/%.el,%,$(wildcard test/*-tests.el))
+SELECTOR ?= (not (tag :unstable))

-INFOMANUALS=debbugs.info debbugs-ug.info
+INFOMANUALS = debbugs.info debbugs-ug.info

 .PHONY: all build check clean checkdoc
 .PRECIOUS: %.elc
@@ -30,10 +63,14 @@ doc: $(INFOMANUALS)
 build: $(TARGET)

 checkdoc: $(SOURCE) $(TESTSOURCE)
-	@$(EMACS) -Q --batch -l resources/debbugs-checkdoc-config.el $(foreach file,$^,"--eval=(checkdoc-file \"$(file)\")")
+	@$(EMACS) -Q --batch -l resources/debbugs-checkdoc-config.el \
+	  $(foreach file,$^,"--eval=(checkdoc-file \"$(file)\")")
+
+check: $(TESTS)

-check: build $(TESTTARGET)
-	@$(EMACS) -Q --batch -L . -L ./test $(foreach file,$(TESTSOURCE), -l $(file)) --eval '(ert-run-tests-batch-and-exit "$(TESTS)")'
+%-tests: build $(TESTTARGET)
+	@$(EMACS) -Q --batch -L . -L ./test -l $@ \
+	  --eval '(ert-run-tests-batch-and-exit (quote ${SELECTOR}))'

 clean:
 	-rm -f $(TARGET) $(TESTTARGET) $(INFOMANUALS)