Subject: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at commit bbc1803

From: Dmitry Gutov <[email-masked]>
Date: Mon, 10 Jun 2024 07:09:52 +0300

On 10/06/2024 00:30, [email-masked] wrote:
> Dmitry Gutov<[email-masked]>  writes:
> 
>> OP, could you retest with the latest master and read-process-output-fast set
>> to its default value (t)?
>>
> As of 12d44fe6420e84eab8f750f9a0f8cd73c3e70bb2, still hangs with
> read-process-output-fast t, works with it nil.

Okay, thank you. I can reproduce it too.

FWIW, it hangs here:

Lisp Backtrace:
"accept-process-output" (0xf19ff6a8)
"nnheader-accept-process-output" (0xf19ff610)
"nnimap-wait-for-response" (0xf19ff5a8)
"nnimap-retrieve-headers" (0xf19ff530)
"gnus-retrieve-headers" (0xf19ff4b8)
"gnus-cache-retrieve-headers" (0xf19ff450)
"gnus-retrieve-headers" (0xf19ff3d0)
"gnus-fetch-headers" (0xf19ff360)
"gnus-select-newsgroup" (0xf19ff2c0)
"gnus-summary-read-group-1" (0xf19ff228)
"gnus-summary-read-group" (0xf19ff188)
"gnus-group-read-group" (0xf19ff100)
--Type <RET> for more, q to quit, c to continue without paging--
"gnus-group-select-group" (0xffffdb10)
"funcall-interactively" (0xffffdb08)
"call-interactively" (0xf19ff070)
"command-execute" (0xffffddf8)

Commands can be different, but nnimap-wait-for-response is where it 
stops. And it's not edebug-able: instrumenting the function makes the 
bug go away.

Print-debugging seems to say that there might be something wrong with 
what (forward-line -1) does: output has the ^M chars, and apparently the 
new logic makes it skip over too many characters.

The only thing relevant I found is this bit, which for now I simply 
copied over from read_and_dispose_of_process_output. But the latter 
calls it after "decoding to string" yet before inserting the string into 
the buffer. Can we do this after the decode_coding_c_string call where 
dst_object is the target buffer?

diff --git a/src/process.c b/src/process.c
index 9670be64279..420b4533cc6 100644
--- a/src/process.c
+++ b/src/process.c
@@ -6372,6 +6372,5 @@
     {			/* We have to decode the input.  */
       Lisp_Object curbuf;
-      int carryover = 0;
       specpdl_ref count1 = SPECPDL_INDEX ();
 
       XSETBUFFER (curbuf, current_buffer);
@@ -6387,12 +6389,45 @@
 
       TEMP_SET_PT_BOTH (PT + process_coding->produced_char,
 			PT_BYTE + process_coding->produced);
       signal_after_change (PT - process_coding->produced_char,
 			   0, process_coding->produced_char);
-      carryover = process_coding->carryover_bytes;
-      if (carryover > 0)
-	memcpy (buf, process_coding->carryover,
-		process_coding->carryover_bytes);
+
+      Vlast_coding_system_used = CODING_ID_NAME (process_coding->id);
+      /* A new coding system might be found.  */
+      if (!EQ (p->decode_coding_system, Vlast_coding_system_used))
+	{
+	  pset_decode_coding_system (p, Vlast_coding_system_used);
+
+	  /* Don't call setup_coding_system for
+	     proc_decode_coding_system[channel] here.  It is done in
+	     detect_coding called via decode_coding above.  */
+
+	  /* If a coding system for encoding is not yet decided, we set
+	     it as the same as coding-system for decoding.
+
+	     But, before doing that we must check if
+	     proc_encode_coding_system[p->outfd] surely points to a
+	     valid memory because p->outfd will be changed once EOF is
+	     sent to the process.  */
+	  eassert (p->outfd < FD_SETSIZE);
+	  if (NILP (p->encode_coding_system) && p->outfd >= 0
+	      && proc_encode_coding_system[p->outfd])
+	    {
+	      pset_encode_coding_system
+		(p, coding_inherit_eol_type (Vlast_coding_system_used, Qnil));
+	      setup_coding_system (p->encode_coding_system,
+				   proc_encode_coding_system[p->outfd]);
+	    }
+	}
+
+      if (process_coding->carryover_bytes > 0)
+	{
+	  if (SCHARS (p->decoding_buf) < process_coding->carryover_bytes)
+	    pset_decoding_buf (p, make_uninit_string (process_coding->carryover_bytes));
+	  memcpy (SDATA (p->decoding_buf), process_coding->carryover,
+		  process_coding->carryover_bytes);
+	  p->decoding_carryover = process_coding->carryover_bytes;
+	}
     }
 
   read_process_output_after_insert (p, &old_read_only, old_begv, old_zv,