Subject: 28.0.50; Support project-wide diagnostics reports in flymake.el

From: João Távora <[email-masked]>
Date: Sun, 29 Aug 2021 01:53:08 +0100

This is coming from:

   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34418
   https://github.com/joaotavora/eglot/pull/640

In the first, Philip wonders why the BUFFER argument to
flymake-make-diagnostic is mostly ignored.

In the second, Theodor requests that the Eglot LSP client manage and
display the diagnostics reported by the LSP server not only for the
current buffer being edited, but for other buffers or files.

The connection between the two will hopefully become apparent soon.

Normally, diagnostics reports are emitted by diagnostic backend for a
single file, usually as a result of a change performed shortly before in
the buffer visiting that file.  This is true for most LSP servers when
acting as Flymake backends.  But some LSP servers seem to take this
opportunity to also "piggy-back" diagnostics for other files, in a kind
of "side-band signal".  Other servers yet will sometimes report
diagnostics upfront for all files in a given project, whether visited by
a buffer or not.

Theodor's pull request handles this side band signal for LSP specifically
in the Eglot client, but it contains some twists and turns that should
not be taken in Eglot.  This functionality should be supported in
Flymake.

* What kind of new API support for Flymake backends is needed for this?
  This is the good news: I think not much.

  The helper function flymake-make-diagnostic, used by Flymake backends
  such as Eglot's, already supports a BUFFER first argument, but, as the
  manual states.
   
     Currently, it is unspecified behavior to make diagnostics for
     buffers other than the buffer that the Flymake backend is
     responsible for.
   
  My idea is to make it specified.  As floated in bug#34418 I think the
  idea of generalizing the argument to mean BUFFER-OR-FILE is decent and
  sufficient.

* What kind of infrastructure changes in Flymake are needed for this?

  That's more complicated: substantial, but not insane.  Here's a
  top-level plan: the current buffer-local variable
  flymake--backend-state should probably first become a global hash
  table indexed by buffer, which should be mostly indistinguishable from
  a buffer-local var.

  If that works, then it should be possible to store the piggy-backed
  diagnostics in (1) file-indexed entries to that global hash table, for
  files which are not being visited by buffers.  It should also be
  possible to add (2) buffer-indexed entries to it for buffers which,
  though live, don't have Flymake properly activated or don't have that
  specific piggy-backing backend in their running backend list.

  These new types of entries will likely not contain the full-blown
  flymake--backend-state structs, i.e. they will be missing the
  'running', 'reported-p' and 'disabled' slots.  All they will have is a
  non-nil list 'diags'.

  If, for situations (1) or (2), we discover that a "proper"
  buffer-indexed entry already exists in the global hash table, we must
  decide if the information in them should take priority.  I'm not sure
  of the answer to this yet.

  Anyway, in flymake--handle-report is where the recording of
  information should happen.  We can have a look at the current
  aforementioned "unspecified" behavior:

         (setq new-diags
              (cl-remove-if-not
               (lambda (diag) (eq (flymake--diag-buffer diag) (current-buffer)))
               report-action))

  (As we can see, the current "unspecification" is just ignoring reports
  for BUFFERs other than the current.)

* In what new UI parts is the augmented information to be useful?

  Currently, I see only one place, the diagnostic listing buffer
  obtained by M-x flymake-show-diagnostics-buffer.  That buffer is
  usually associated with only one source buffer
  (flymake--diagnostics-buffer-source).  Now it should start listing all
  the diagnostics for buffers or files known to belong to the same
  project, using 'project.el' functionality for that.

João