GNU bug report logs - #47320
Improve failure reporting in test/lisp/electrict-tests.el

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: bug-gnu-emacs <at> gnu.org
Cc: João Távora <joaotavora <at> gmail.com>
Subject: Improve failure reporting in test/lisp/electrict-tests.el
Date: Mon, 22 Mar 2021 14:24:37 +0000

Hello, Emacs.

When running make check, and a failure occurs in electric-tests.elc, the
resulting output in electric-tests.log is frustratingly cryptic.
Although one can fairly easily get back to the source code for the test
in electric-tests.el, it is difficult to reconstruct the snippet of text
on which the test was performed.  This is because the tests are
(necessarily) generated by fairly inscrutable macros.

This is a shame, since each generated test has its own exceptionally
clear generated doc-string.  An example of such a doc string is:

#########################################################################
Electricity test in a `c-mode' buffer.

Start with point at 7 in a 7-char-long buffer
like this one:

  |"foo \"|   (buffer start and end are denoted by `|')

Now call this:

#'electric-quote-local-mode

Ensure the following bindings:

'((electric-quote-replace-double . t)
  (electric-quote-comment . t)
  (electric-quote-string . t))

Now press the key for: "

The buffer's contents should become:

  |"foo \""|

, and point should be at 7.
#########################################################################

My proposal is that on a test failure, this generated doc-string should
be output along with the other failure stuff.  It doesn't actually add
all that much bulk to the .log file.

Here is patch which does this.  If there are no objections, I will
commit it in a day or two.



diff --git a/test/lisp/electric-tests.el b/test/lisp/electric-tests.el
index 62a42b7fe4..44b3d8b672 100644
--- a/test/lisp/electric-tests.el
+++ b/test/lisp/electric-tests.el
@@ -50,7 +50,8 @@ save-electric-modes
   `(call-with-saved-electric-modes #'(lambda () ,@body)))
 
 (defun electric-pair-test-for (fixture where char expected-string
-                                       expected-point mode bindings fixture-fn)
+                                       expected-point mode bindings
+                                       fixture-fn &optional doc-string)
   (with-temp-buffer
     (funcall mode)
     (insert fixture)
@@ -63,6 +64,14 @@ electric-pair-test-for
             (mapcar #'car bindings)
             (mapcar #'cdr bindings)
           (call-interactively (key-binding `[,last-command-event])))))
+    (when
+        (and doc-string
+             (not
+              (and
+               (equal (buffer-substring-no-properties (point-min) (point-max))
+                      expected-string)
+               (equal (point) expected-point))))
+      (message "\n%s\n" doc-string))
     (should (equal (buffer-substring-no-properties (point-min) (point-max))
                    expected-string))
     (should (equal (point)
@@ -109,14 +118,9 @@ electric-pair-test-for
            (fixture (format "%s%s%s" prefix fixture suffix))
            (expected-string (format "%s%s%s" prefix expected-string suffix))
            (expected-point (+ (length prefix) expected-point))
-           (pos (+ (length prefix) pos)))
-      `(ert-deftest ,(intern (format "electric-pair-%s-at-point-%s-in-%s%s"
-                                     name
-                                     (1+ pos)
-                                     mode
-                                     extra-desc))
-           ()
-         ,(format "Electricity test in a `%s' buffer.\n
+           (pos (+ (length prefix) pos))
+           (doc-string
+            (format "Electricity test in a `%s' buffer.\n
 Start with point at %d in a %d-char-long buffer
 like this one:
 
@@ -143,7 +147,14 @@ electric-pair-test-for
                   char
                   (if (string= fixture expected-string) "stay" "become")
                   (replace-regexp-in-string "\n" "\\\\n" expected-string)
-                  expected-point)
+                  expected-point)))
+      `(ert-deftest ,(intern (format "electric-pair-%s-at-point-%s-in-%s%s"
+                                     name
+                                     (1+ pos)
+                                     mode
+                                     extra-desc))
+           ()
+         ,doc-string
          (electric-pair-test-for ,fixture
                                  ,(1+ pos)
                                  ,char
@@ -151,7 +162,8 @@ electric-pair-test-for
                                  ,expected-point
                                  ',mode
                                  ,bindings
-                                 ,fixture-fn)))))
+                                 ,fixture-fn
+                                 ,doc-string)))))
 
 (cl-defmacro define-electric-pair-test
     (name fixture


-- 
Alan Mackenzie (Nuremberg, Germany).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.