Subject: Re: bug#13400: 23.4; overlapping process filter calls

From: Noam Postavsky <[email-masked]>
Date: Wed, 07 Aug 2019 21:15:49 -0400

Hendrik Tews <[email-masked]> writes:

> thanks for addressing this quite old bug report. I do have some
> comments:

Thanks for still following up even though the reply was so delayed.

>>> - Section "37.9 Receiving Output from Processes" does not list
>>>   process-send-string. How about other blocking I/O functions?
>>
>> In the attached patch, I've added a mention/xref for functions which send
>> data to processes.
>
> How about other blocking I/O functions?  When output is accepted
> inside process-send-string, then it probably is in all potentially
> blocking I/O functions?

As far as I know, all the blocking functions are listed in the Input to
Processes node which I xref'd.

>>> - Same in "37.9.2. Process Filter Functions"
>>
>> This section is repeated twice (I addressed the second instance below).
>
> I mentioned this section twice, because there are _two_
> documentation problems.

Aha, missed that, thanks.

>>> - Same in "37.4 Creating an Asynchronous Process" ,
>>>   process-send-string is neither waiting for input not time
>>>   delay.
>>
>> I don't see any mention of process-send-string in that section, nor how
>> it's relevant to the rest of this report.
>
> Come on, please read that section carefully. "Emacs accepts data
> from the process only while waiting for input or for a time
> delay" in there implies that Emacs is _not_ accepting data during
> process-send-string, because, as I wrote, 

Thanks, it's hard to pick out such details on the Nth time reading the
manual.

>>> - "37.7 Sending Input to Processes" says that filters can run
>>>   inside process-send-string, but it could be clearer about the
>>>   point that this can also happen inside the same filter for the
>>>   same process.
>>
>> I'm not really convinced that is necessary.
>
> It is about a few words making the documentation more precise,
> potentially saving somebody a painful debugging session of
> several hours

I'm just not sure anyone is going to notice such details when it really
matters.  In my experience the manual is more about having something
authoritative to point to when folks ask "why does X happen?".  But I've
added a parenthetical to the patch.

> Adding to the original bug report, I would suggest to restructure
> the documentation, such that there is only one section
> documenting all the functions in which process output could
> arrive and such that all the other sections only refer to that
> section. It should really not be the case that different sections
> make different and sometimes inconsistent statements about the
> same feature.

Yes, hence the xrefs.  See attached updated patch.

>From 895b3e135828006732bde40d54f74b1d5d3957f2 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <[email-masked]>
Date: Fri, 26 Jul 2019 23:20:37 -0400
Subject: [PATCH] Note that process filter can be called recursively
 (Bug#13400)

* doc/lispref/processes.texi (Asynchronous Processes): Note that input
may read when sending data as well.
(Output from Processes): Note that functions which send data may also
trigger reading from processes.
(Input to Processes, Filter Functions): Note that filter functions may
be called recursively.
---
 doc/lispref/processes.texi | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/doc/lispref/processes.texi b/doc/lispref/processes.texi
index a93f4db428..bd807cdcee 100644
--- a/doc/lispref/processes.texi
+++ b/doc/lispref/processes.texi
@@ -588,9 +588,8 @@ Asynchronous Processes
 parallel with Emacs, and Emacs can communicate with it using the
 functions described in the following sections (@pxref{Input to
 Processes}, and @pxref{Output from Processes}).  Note that process
-communication is only partially asynchronous: Emacs sends data to the
-process only when certain functions are called, and Emacs accepts data
-from the process only while waiting for input or for a time delay.
+communication is only partially asynchronous: Emacs sends and receives
+data to and from a process only when those functions are called.
 
 @cindex pty, when to use for subprocess communications
 @cindex pipe, when to use for subprocess communications
@@ -1200,8 +1199,9 @@ Input to Processes
 because the input buffer is full.  When this happens, the send functions
 wait a short while, accepting output from subprocesses, and then try
 again.  This gives the subprocess a chance to read more of its pending
-input and make space in the buffer.  It also allows filters, sentinels
-and timers to run---so take account of that in writing your code.
+input and make space in the buffer.  It also allows filters (including
+the one currently running), sentinels and timers to run---so take
+account of that in writing your code.
 
   In these functions, the @var{process} argument can be a process or
 the name of a process, or a buffer or buffer name (which stands
@@ -1416,9 +1416,10 @@ Output from Processes
 
   Output from a subprocess can arrive only while Emacs is waiting: when
 reading terminal input (see the function @code{waiting-for-user-input-p}),
-in @code{sit-for} and @code{sleep-for} (@pxref{Waiting}), and in
-@code{accept-process-output} (@pxref{Accepting Output}).  This
-minimizes the problem of timing errors that usually plague parallel
+in @code{sit-for} and @code{sleep-for} (@pxref{Waiting}), in
+@code{accept-process-output} (@pxref{Accepting Output}), and in
+functions which send data to processes (@pxref{Input to Processes}).
+This minimizes the problem of timing errors that usually plague parallel
 programming.  For example, you can safely create a process and only
 then specify its buffer or filter function; no output can arrive
 before you finish, if the code in between does not call any primitive
@@ -1594,14 +1595,10 @@ Filter Functions
   By default, the error output from the process, if any, is also
 passed to the filter function, unless the destination for the standard
 error stream of the process was separated from the standard output
-when the process was created (@pxref{Output from Processes}).
-
-  The filter function can only be called when Emacs is waiting for
-something, because process output arrives only at such times.  Emacs
-waits when reading terminal input (see the function
-@code{waiting-for-user-input-p}), in @code{sit-for} and
-@code{sleep-for} (@pxref{Waiting}), and in
-@code{accept-process-output} (@pxref{Accepting Output}).
+when the process was created.  Emacs will only call the filter
+function during certain function calls.  @xref{Output from Processes}.
+Note that if any of those functions are called by the filter, the
+filter may be called recursively.
 
   A filter function must accept two arguments: the associated process
 and a string, which is output just received from it.  The function is
-- 
2.11.0