GNU bug report logs - #66782
29.1; ERT tests reports test redefined depending on loading sequence

Previous Next

Full log

Message #41 received at 66782 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 66782 <at> debbugs.gnu.org
Subject: Re: bug#66782: 29.1; ERT tests report test redefined depending on
 loading sequence
Date: Sun, 29 Oct 2023 14:53:23 -0700

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> 28 okt. 2023 kl. 23.19 skrev Xiyue Deng <manphiz <at> gmail.com>:
>
>> To clarify one of my
>> original proposals: if it makes no sense for a `ert-deftest' to depend
>> on other `ert-deftest', I would say there is no reason for such a
>> dependency, and people should move reusable components to a helper
>> module instead.
>
> There is no reason for one `ert-deftest` to make use of another, but was that
> the case here? I interpreted it as a test suite that spread its `ert-deftest`
> definitions over multiple .el files which isn't necessarily a bad way of
> organising them.
>

As they just use eask to run all tests under folder test in their
Makefile[1], I don't think spreading definitions is what they are doing.
To clarify, I think instead they are requiring lsp-integration-test just
because they want use the `lsp-test-wait' macro[2] in
lsp-clangd-test[3][4][5], which could have been done by putting those
macros in a separate file and requiring that from both tests.

> As said before, there is little sense in including the same individual tests from multiple 'root' .el files since that would run them more than once.
>
> Perhaps that is what you meant? If so, please forgive me for the pedantry.
>

Exactly, and no worries.

As far as I know, there is no practical use for a test case to depend on
another test case in any language I worked with as a test case provides
no reusable component.  On the other hand, there are alternatives that
makes more sense to do what they (e.g. lsp-mode) are doing.  Again, IMHO
I think it makes sense for ERT not to support it or to even forbid it.
Doing this also avoids such misleading error message I was reporting,
which got triggered depending on the loading sequence which may make
test suites flaky (as the loading sequence is not deterministic).

[1] https://github.com/emacs-lsp/lsp-mode/blob/master/Makefile#L42
[2] https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-integration-test.el#L51
[3] https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L109
[4] https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L118
[5] https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L141

-- 
Xiyue Deng

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.