GNU bug report logs - #76549
30.1; Intermittent proced-refine test failires

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76549 in the body.
You can then email your comments to 76549 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Oliver <p.d.oliver <at> mavit.org.uk>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1; Intermittent proced-refine test failires
Date: Tue, 25 Feb 2025 10:48:46 +0000 (GMT)

I intermittently see the following test failure (and a similar one for proced-refine-with-update-test) for Emacs 30.1 on Fedora 42.

Full logs are at https://kojipkgs.fedoraproject.org//work/tasks/4052/129594052/build.log and https://kojipkgs.fedoraproject.org//work/tasks/4051/129594051/build.log

make[3]: Entering directory '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
/usr/bin/mkdir -p lisp/
HOME=/nonexistent LANG=C EMACS_TEST_DIRECTORY=/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test   "../src/emacs" --module-assertions --no-init-file --no-site-file --no-site-lisp -L ":."  \
  -l ert --eval "(setq treesit-extra-load-path '(\"/builddir/.emacs.d/tree-sitter\"))" -l lisp/proced-tests \
  --batch --eval '(ert-run-tests-batch-and-exit (quote (not (or (tag :expensive-test) (tag :unstable)))))' > lisp/proced-tests.log 2>&1 || { STAT=$?; cat lisp/proced-tests.log; exit $STAT; }
Running 6 tests (2025-02-24 22:08:34+0000, selector `(not (or (tag :expensive-test) (tag :unstable)))')
   passed  1/6  proced-color-test (0.063154 sec)
   passed  2/6  proced-format-test (0.272138 sec)
Test proced-refine-test backtrace:
  proced-<(nil 88.88888888888889)
  proced-update()
  proced-refine()
  #f(compiled-function () #<bytecode 0x90a95dc86d9c8dd>)()
  #f(compiled-function () #<bytecode 0x81d18e5e5e509e9>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x81d18e5e5e509e9>
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name proced-refine-test :documentation nil
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test proced-refine-test condition:
    (wrong-type-argument number-or-marker-p nil)
   FAILED  3/6  proced-refine-test (0.083490 sec) at lisp/proced-tests.el:117
   passed  4/6  proced-refine-with-update-test (0.089915 sec)
   passed  5/6  proced-revert-test (0.101340 sec)
   passed  6/6  proced-update-test (0.071953 sec)
Ran 6 tests, 5 results as expected, 1 unexpected (2025-02-24 22:08:35+0000, 0.735331 sec)
1 unexpected results:
   FAILED  proced-refine-test
make[3]: Leaving directory '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
make[3]: *** [Makefile:184: lisp/proced-tests.log] Error 1

-- 
Peter Oliver

Message #10 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Oliver <p.d.oliver <at> mavit.org.uk>,
 Laurence Warne <laurencewarne <at> gmail.com>
Cc: 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Wed, 26 Feb 2025 14:45:02 +0200

> Date: Tue, 25 Feb 2025 10:48:46 +0000 (GMT)
> From: Peter Oliver <p.d.oliver <at> mavit.org.uk>
> 
> I intermittently see the following test failure (and a similar one for proced-refine-with-update-test) for Emacs 30.1 on Fedora 42.
> 
> Full logs are at https://kojipkgs.fedoraproject.org//work/tasks/4052/129594052/build.log and https://kojipkgs.fedoraproject.org//work/tasks/4051/129594051/build.log
> 
> make[3]: Entering directory '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
> /usr/bin/mkdir -p lisp/
> HOME=/nonexistent LANG=C EMACS_TEST_DIRECTORY=/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test   "../src/emacs" --module-assertions --no-init-file --no-site-file --no-site-lisp -L ":."  \
>    -l ert --eval "(setq treesit-extra-load-path '(\"/builddir/.emacs.d/tree-sitter\"))" -l lisp/proced-tests \
>    --batch --eval '(ert-run-tests-batch-and-exit (quote (not (or (tag :expensive-test) (tag :unstable)))))' > lisp/proced-tests.log 2>&1 || { STAT=$?; cat lisp/proced-tests.log; exit $STAT; }
> Running 6 tests (2025-02-24 22:08:34+0000, selector `(not (or (tag :expensive-test) (tag :unstable)))')
>     passed  1/6  proced-color-test (0.063154 sec)
>     passed  2/6  proced-format-test (0.272138 sec)
> Test proced-refine-test backtrace:
>    proced-<(nil 88.88888888888889)
>    proced-update()
>    proced-refine()
>    #f(compiled-function () #<bytecode 0x90a95dc86d9c8dd>)()
>    #f(compiled-function () #<bytecode 0x81d18e5e5e509e9>)()
>    handler-bind-1(#f(compiled-function () #<bytecode 0x81d18e5e5e509e9>
>    ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>    ert-run-test(#s(ert-test :name proced-refine-test :documentation nil
>    ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
>    ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
>    ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
>    ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
>    eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
>    command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
>    command-line()
>    normal-top-level()
> Test proced-refine-test condition:
>      (wrong-type-argument number-or-marker-p nil)
>     FAILED  3/6  proced-refine-test (0.083490 sec) at lisp/proced-tests.el:117
>     passed  4/6  proced-refine-with-update-test (0.089915 sec)
>     passed  5/6  proced-revert-test (0.101340 sec)
>     passed  6/6  proced-update-test (0.071953 sec)
> Ran 6 tests, 5 results as expected, 1 unexpected (2025-02-24 22:08:35+0000, 0.735331 sec)
> 1 unexpected results:
>     FAILED  proced-refine-test
> make[3]: Leaving directory '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
> make[3]: *** [Makefile:184: lisp/proced-tests.log] Error 1

Laurence, could you please look into fixing this?

Message #13 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Laurence Warne <laurencewarne <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Peter Oliver <p.d.oliver <at> mavit.org.uk>, 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Wed, 26 Feb 2025 22:03:49 +0000

[Message part 1 (text/plain, inline)]

Hi, I don't seem to be able to see the ERT explainer output in the logs
added in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73441 for
proced-refine-test.

Has that patch been applied here or am I missing it?  If not it would
possibly explain the flakiness fixed by that patch too.

Otherwise I can't reproduce the test failing with no ERT explainer output
copying the flags in the logs (my hope would be the ERT explainer output
would show the underlying cause as before).

Thanks, Laurence

On Wed, Feb 26, 2025 at 12:45 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Date: Tue, 25 Feb 2025 10:48:46 +0000 (GMT)
> > From: Peter Oliver <p.d.oliver <at> mavit.org.uk>
> >
> > I intermittently see the following test failure (and a similar one for
> proced-refine-with-update-test) for Emacs 30.1 on Fedora 42.
> >
> > Full logs are at
> https://kojipkgs.fedoraproject.org//work/tasks/4052/129594052/build.log
> and
> https://kojipkgs.fedoraproject.org//work/tasks/4051/129594051/build.log
> >
> > make[3]: Entering directory
> '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
> > /usr/bin/mkdir -p lisp/
> > HOME=/nonexistent LANG=C
> EMACS_TEST_DIRECTORY=/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test
>  "../src/emacs" --module-assertions --no-init-file --no-site-file
> --no-site-lisp -L ":."  \
> >    -l ert --eval "(setq treesit-extra-load-path
> '(\"/builddir/.emacs.d/tree-sitter\"))" -l lisp/proced-tests \
> >    --batch --eval '(ert-run-tests-batch-and-exit (quote (not (or (tag
> :expensive-test) (tag :unstable)))))' > lisp/proced-tests.log 2>&1 || {
> STAT=$?; cat lisp/proced-tests.log; exit $STAT; }
> > Running 6 tests (2025-02-24 22:08:34+0000, selector `(not (or (tag
> :expensive-test) (tag :unstable)))')
> >     passed  1/6  proced-color-test (0.063154 sec)
> >     passed  2/6  proced-format-test (0.272138 sec)
> > Test proced-refine-test backtrace:
> >    proced-<(nil 88.88888888888889)
> >    proced-update()
> >    proced-refine()
> >    #f(compiled-function () #<bytecode 0x90a95dc86d9c8dd>)()
> >    #f(compiled-function () #<bytecode 0x81d18e5e5e509e9>)()
> >    handler-bind-1(#f(compiled-function () #<bytecode 0x81d18e5e5e509e9>
> >    ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
> >    ert-run-test(#s(ert-test :name proced-refine-test :documentation nil
> >    ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
> >    ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
> >    ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
> >    ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
> >    eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
> >    command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
> >    command-line()
> >    normal-top-level()
> > Test proced-refine-test condition:
> >      (wrong-type-argument number-or-marker-p nil)
> >     FAILED  3/6  proced-refine-test (0.083490 sec) at
> lisp/proced-tests.el:117
> >     passed  4/6  proced-refine-with-update-test (0.089915 sec)
> >     passed  5/6  proced-revert-test (0.101340 sec)
> >     passed  6/6  proced-update-test (0.071953 sec)
> > Ran 6 tests, 5 results as expected, 1 unexpected (2025-02-24
> 22:08:35+0000, 0.735331 sec)
> > 1 unexpected results:
> >     FAILED  proced-refine-test
> > make[3]: Leaving directory
> '/builddir/build/BUILD/emacs-30.1-build/emacs-30.1/build-lucid/test'
> > make[3]: *** [Makefile:184: lisp/proced-tests.log] Error 1
>
> Laurence, could you please look into fixing this?
>

[Message part 2 (text/html, inline)]

Message #16 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Laurence Warne <laurencewarne <at> gmail.com>
Cc: p.d.oliver <at> mavit.org.uk, 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Thu, 27 Feb 2025 10:09:10 +0200

> From: Laurence Warne <laurencewarne <at> gmail.com>
> Date: Wed, 26 Feb 2025 22:03:49 +0000
> Cc: Peter Oliver <p.d.oliver <at> mavit.org.uk>, 76549 <at> debbugs.gnu.org
> 
> Hi, I don't seem to be able to see the ERT explainer output in the logs added in
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73441 for proced-refine-test.
> 
> Has that patch been applied here or am I missing it?  If not it would possibly explain the flakiness fixed by
> that patch too.

Which patch are you referring to?  Bug#73441 has more than one.

Message #19 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Laurence Warne <laurencewarne <at> gmail.com>, p.d.oliver <at> mavit.org.uk,
 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Fri, 28 Feb 2025 12:22:15 +0100

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

Hi,

>> From: Laurence Warne <laurencewarne <at> gmail.com>
>> Date: Wed, 26 Feb 2025 22:03:49 +0000
>> Cc: Peter Oliver <p.d.oliver <at> mavit.org.uk>, 76549 <at> debbugs.gnu.org
>>
>> Hi, I don't seem to be able to see the ERT explainer output in the logs added in
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73441 for proced-refine-test.
>>
>> Has that patch been applied here or am I missing it?  If not it would possibly explain the flakiness fixed by
>> that patch too.
>
> Which patch are you referring to?  Bug#73441 has more than one.

I don't believe this belongs to Bug#73441. Patches there have added an
ert explainer, which isn't triggered here. And FWIW, proced-tests.el is
the same in the meacs-30 and master branches.

The backtrace Peter has provided us is

--8<---------------cut here---------------start------------->8---
Test proced-refine-test backtrace:
  proced-<(nil 88.88888888888889)
  proced-update()
  proced-refine()
  #f(compiled-function () #<bytecode 0x90a95dc86d9c8dd>)()
  #f(compiled-function () #<bytecode 0x81d18e5e5e509e9>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x81d18e5e5e509e9>
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name proced-refine-test :documentation nil
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test proced-refine-test condition:
    (wrong-type-argument number-or-marker-p nil)
   FAILED  3/6  proced-refine-test (0.083490 sec) at lisp/proced-tests.el:117
   passed  4/6  proced-refine-with-update-test (0.089915 sec)
   passed  5/6  proced-revert-test (0.101340 sec)
   passed  6/6  proced-update-test (0.071953 sec)
Ran 6 tests, 5 results as expected, 1 unexpected (2025-02-24 22:08:35+0000, 0.735331 sec)
1 unexpected results:
   FAILED  proced-refine-test
--8<---------------cut here---------------end--------------->8---

The error has appeared in proced-refine, before any ert-specific macro,
carrying the explainer, could run.

Likely, we must improve proced-<. If either NUM1 or NUM2 is not a
number, this shouldn't raise an arror.

Peter, are you able to reproduce the problem? Does the appended patch
help?

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #24 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Peter Oliver <p.d.oliver <at> mavit.org.uk>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Laurence Warne <laurencewarne <at> gmail.com>,
 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Fri, 28 Feb 2025 12:29:43 +0000 (GMT)

[Message part 1 (text/plain, inline)]

On Fri, 28 Feb 2025, Michael Albinus wrote:

> Peter, are you able to reproduce the problem?

I’m afraid that I’ve stopped running tests as a part of Fedora builds for now, because different intermittent failures were disrupting them so often.

-- 
Peter Oliver

Message #29 received at 76549-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Peter Oliver <p.d.oliver <at> mavit.org.uk>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Laurence Warne <laurencewarne <at> gmail.com>,
 76549-done <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Fri, 28 Feb 2025 15:35:05 +0100

Version: 30.2

Peter Oliver <p.d.oliver <at> mavit.org.uk> writes:

Hi Peter,

> I’m afraid that I’ve stopped running tests as a part of Fedora builds
> for now, because different intermittent failures were disrupting them
> so often.

OK. I've pushed the change to the emacs-30 branch, closing the bug. Pls
report, if there are still problems.

Best regards, Michael.

Message #32 received at 76549 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Peter Oliver <p.d.oliver <at> mavit.org.uk>,
 Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Laurence Warne <laurencewarne <at> gmail.com>,
 76549 <at> debbugs.gnu.org
Subject: Re: bug#76549: 30.1; Intermittent proced-refine test failires
Date: Sat, 1 Mar 2025 18:47:42 -0800

Peter Oliver <p.d.oliver <at> mavit.org.uk> writes:

> On Fri, 28 Feb 2025, Michael Albinus wrote:
>
>> Peter, are you able to reproduce the problem?
>
> I’m afraid that I’ve stopped running tests as a part of Fedora builds for now, because different intermittent failures were disrupting them so often.

That's unfortunate.  We really should do better here.

I will try to do my best to report intermittent test failures more
often, and on the release branch I think an acceptable fix is often to
just mark them as unstable, fix them on master, and then move on.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.