Subject: bug#46722: test-map-into fails

From: Stefan Monnier <[email-masked]>
Date: Wed, 05 Jan 2022 14:17:19 -0500

>> So the underlying bug only appears if either the hash-table is rehashed
>> or the new `sxhash-equal` value happens to the same as the old one.
> I can't say I understand this, but I'm contented if map-tests.el stops flapping.

Basically, here's what happens:

On one call, `dispatch` is of the form (1 #<eq-generalizer> #<t-generalizer>)
because the arg n°1 has methods that use `eql` and `t`.
So an entry for it is placed in bucket 7 of the hash-table (say) with
the corresponding "eq + t dispatch function".

Now, this very `dispatch` object is also stored in the generic
function metadata.  And later on, some new `cl-defmethod` is executed
which means the arg n°1 now also has methods that use `head`, so this
object is modified (by setcdr)
to (1 #<eq-generalizer> #<head-generalizer> #<t-generalizer>).
Since it's the very same object, it means that the hash table now has
a mapping from (1 #<eq-generalizer> #<head-generalizer> #<t-generalizer>)
to the "eq + t dispatch function", which is wrong and the core of problem.

Luckily for us, this problem has not bitten us very often because, the
hash of (1 #<eq-generalizer> #<head-generalizer> #<t-generalizer>) is
not the same as the hash of (1 #<eq-generalizer> #<t-generalizer>) so
our (now incorrect) entry (still placed in bucket 7) will simply never be
found by `gethash` because gethash will look for it in another bucket.

>> The benefit of the `method-cache` OTOH should be very significant
>> since without it we'd have to recompute the effective method at every
>> call.
>
> I don't get it.  This assumes `method-cache` persists between successive
> calls to the byte-compiled closure... but `method-cache` is clearly let'ted.

It does persist: the byte-compiled closure is a "dispatcher builder".
The function we store in the `symbol-function` slot of generic
functions, is the (lambda (,@fixedargs &rest args) ...) function
*returned* by this function.  This inner function has the method cache
in its captured environment which is why it persists between calls.

        Stefan