Subject: Re: bug#24940: [PATCH] Add should-call, should-not-call, and their tests

From: Gemini Lasswell <[email-masked]>
Date: Mon, 14 Nov 2016 21:52:23 -0800

Eli Zaretskii <[email-masked]> writes:

> I don't think our test suite should in general test whether some
> functions are or aren't called during execution of the
> command/function under test.  That's because we are not interested in
> the fine details of the implementation of the features being tested,
> we are interested in whether it does what it's supposed to do.  "What
> it's supposed to do" means we should test the return values and the
> side effects, a.k.a. "the behavior", and not how the code works
> internally.

I'll modify the tests I've written to focus on "the behavior".

> might do the latter.  A better way, though perhaps much harder, would
> be to test something much more basic that is common to _any_ prompt,
> like look in the echo-area or the minibuffer buffers for suitable
> text, or maybe even C-level infrastructure that catches the situation
> where Emacs waits for user input.

I agree that infrastructure to allow a test to set up input to be
supplied to the next prompt and to read the prompt message would be a
better solution.

Keyboard macros work to simulate user input for any test code that isn't
trying to test kmacro-step-edit-macro. If there was a hook that the C
code could run just prior to taking input from a keyboard macro, tests
could use it to check that the correct prompt appears at the correct
point in the macro.

Such a hook would also be helpful in the attempt to fix
kmacro-step-edit-macro's bugs. Mostly where it messes up now is when
several input events happen during one command, and a pre-input hook
would allow it to inspect keyboard macro execution event by event rather
than command by command.

The remaining problem that I see is that lots of code in lots of places
suppresses messages depending on called-interactively-p,
executing-kbd-macro and/or noninteractive, and although that doesn't
just affect tests that use keyboard macros, it does make it hard to test
how Emacs behaves interactively.

> If we can agree that the above use cases are the only ones where such
> macros should be used, then these macros should be much leaner than
> what you propose, because there should be no need to provide many of
> the features in your proposal, like testing the arguments and the
> return value, or the number of times the subroutines are called.

The leanest version I can think of would be:

(with-added-advice BINDINGS
   BODY)

where BINDINGS is a list of argument lists for advice-add. It would call
advice-add once for each element of BINDINGS, execute BODY, and then
remove all the added advice. Then it looks like something that belongs
in nadvice.el instead of ert.el.