Subject: Re: bug#78872: Byte compiler: warning message lacks line and column numbers.

From: Stefan Monnier <[email-masked]>
Date: Mon, 30 Jun 2025 00:17:35 -0400

>> Why is it that the fallback doesn't work here?
> I think it's because all the symbols up to a car/cdr/combination of 10
> are lacking positions.  These symbols are generated by the
> compiler or macro expander (more the latter, I think).

See below.

>> [ I don't understand where the above warning comes from, so I'm probably
>>   missing something.  AFAICT the only place where
>>   `font-lock-keyword-face` appears is within a `quote` so there's no
>>   chance it will elicit such a warning (although it also deserves such
>>   a warning, admittedly 🙂).  ]
>
> By the time '(1 font-lock-keyword-place t) gets used by the byte-compile
> inside c-make-font-lock-search-function, that quote has been evaluated
> away.

Aaahhhh!

I think that's the culprit: the code with the `font-lock-keyword-face`
reference is generated by a function and passed to `byte-compile`.
There can't be any sympos in it for that reason (we need to strip
sympos before we pass any sexp to `eval`).

If we want good warnings with good location information, we need to
replace the `c-make-font-lock-search-function` function (which
internally calls `byte-compile`) with a macro (which doesn't call
`byte-compile` but instead returns the corresponding code so it gets
byte-compiled by "the outer compiler").
[ I have a trivial patch to do that, but sadly it doesn't quite fit in
  the margin so I'll leave it as an exercise for the reader.  ]

>> > I seems to me that counting the depth of search is here sub-optimal.
>> No doubt.  I thought it was OK because of the fallback to
>> `byte-compile-form-stack` and also because returning nil occasionally is
>> just a fact of life: the form may fail to contain any sympos, after all.
> But it happens for a very shallowly nested form, here.

Actually, I think it happens here because the stack is empty in the
inner `byte-compile` (it would have to consult the stack of the outer
compiler, but the two compiler invocations don't know each other, so
it's not that simple).

> What's happening is that eval-when-compile is wrongly compiling its
> argument, then evaluating it.  This compilation removes the symbols'
> positions, so they are no longer available when
> c-make-font-lock-search-function's byte-compile runs.

The symbols' positions have to be removed before we can run the code,
regardless if we compile the code or not, otherwise sympos can "leak"
outside their compilation.

> Interestingly enough, eval-AND-compile _doesn't_ compile its argument.
> It just macro-expands it, then both evaluates the result of the
> expansion, and returns it.

But the inner `byte-compile` would give a similarly poor location info
in the warning because `eval-and-compile` also strips the sympos before
evaluating the code.

        Stefan