GNU bug report logs - #71452
30.0.50; GNUS hangs on IMAP/s reads starting at commit bbc1803

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 71452 in the body.
You can then email your comments to 71452 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: epg <at> pretzelnet.org
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; GNUS hangs on IMAP/s reads starting at commit bbc1803
Date: Sun, 09 Jun 2024 11:36:30 -0500

1. Configure GNUS like so:

(setq gnus-select-method '(nnnil "")
      gnus-secondary-select-methods
      '(
        ;; machine imap.fastmail.com login epg <at> gillespies.us
        (nnimap "imap.fastmail.com"
                (nnimap-stream ssl)
                (nnimap-record-commands t)
                )
        )
      )

2. Start GNUS, open some folders and messages.

It's not clear to me exactly what's going on; sometimes the hang
happens upon opening a folder, sometimes opening a message.

git bisect shows that commit
bbc18031aff6f22a1f2b63355f18f294fbdeb797 introduced the bug.



In GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, X toolkit,
 cairo version 1.18.0) of 2024-06-08 built on dogato
Repository revision: 4468216f17aca811c02b49d18c5bee4fb9fd6943
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101012
System Description: openSUSE Tumbleweed

Configured using:
 'configure -C --prefix=/opt/emacs-30.0.50.2.4468216f17a
 --disable-silent-rules --with-x-toolkit=athena --without-sqlite3
 --without-lcms2 --without-libsystemd --without-imagemagick
 --with-json --with-tree-sitter --without-xft --without-libotf
 --without-toolkit-scroll-bars --without-xaw3d --without-gpm
 --without-dbus --without-gconf --without-gsettings
 --without-selinux --without-compress-install --with-x'

Configured features:
CAIRO FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG LIBXML2 MODULES
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  gnus-undo-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  line-number-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug mule-util cl-extra sort gnus-cite shr-color
color mm-archive mail-extr textsec uni-scripts idna-mapping
ucs-normalize uni-confusable textsec-check gnus-async gnus-bcklg
qp gnus-ml disp-table nndraft nnmh utf-7 network-stream nsm
nnfolder nnnil gnus-agent gnus-srvr gnus-score score-mode
nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime
gnutls dig nntp gnus-cache gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source
utf7 nnoo parse-time iso8601 gnus-spec gnus-int gnus-range
gnus-win gnus nnheader range elfeed-show elfeed-search wid-edit
bookmark message sendmail yank-media dired dired-loaddefs rfc822
mml mml-sec epa epg rfc6068 epg-config gnus-util mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
shr pixel-fill kinsoku puny svg dom elfeed-csv elfeed elfeed-curl
elfeed-log elfeed-db elfeed-lib time-date avl-tree generator
url-queue url-file browse-url xml-query xml rust-utils rust-mode
derived rust-rustfmt rust-playpen rust-compile rust-cargo eglot
external-completion jsonrpc xref flymake thingatpt project compat
diff ert pp ewoc debug backtrace help-mode find-func filenotify
warnings compile text-property-search imenu ob-sql ob ob-tangle
ol rx org-src ob-ref ob-lob ob-table ob-exp ob-comint comint
ansi-osc ansi-color ring ob-core org-cycle org-fold org-fold-core
ob-eval org-keys oc org-compat org-macs format-spec pcase vc-git
diff-mode track-changes easy-mmode vc-dispatcher url-cache url
url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf mailcap url-util
url-parse auth-source cl-seq eieio eieio-core cl-macs icons
cl-loaddefs cl-lib password-cache json subr-x map byte-opt gv
bytecomp byte-compile url-vars rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd
touch-screen tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer
select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face
macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads inotify
dynamic-setting font-render-setting cairo x-toolkit xinput2 x
multi-tty move-toolbar make-network-process emacs)

Memory information:
((conses 16 356352 45636) (symbols 48 22166 7) (strings 32 71942 3853)
 (string-bytes 1 1769693) (vectors 16 60022) (vector-slots 8 675472 56298)
 (floats 8 406 13148) (intervals 56 451 94) (buffers 992 21))

Message #8 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: epg <at> pretzelnet.org
Cc: 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50;
 GNUS hangs on IMAP/s reads starting at commit bbc1803
Date: Sun, 09 Jun 2024 21:13:18 +0300

> From: epg <at> pretzelnet.org
> Date: Sun, 09 Jun 2024 11:36:30 -0500
> 
> 
> 1. Configure GNUS like so:
> 
> (setq gnus-select-method '(nnnil "")
>       gnus-secondary-select-methods
>       '(
>         ;; machine imap.fastmail.com login epg <at> gillespies.us
>         (nnimap "imap.fastmail.com"
>                 (nnimap-stream ssl)
>                 (nnimap-record-commands t)
>                 )
>         )
>       )
> 
> 2. Start GNUS, open some folders and messages.
> 
> It's not clear to me exactly what's going on; sometimes the hang
> happens upon opening a folder, sometimes opening a message.
> 
> git bisect shows that commit
> bbc18031aff6f22a1f2b63355f18f294fbdeb797 introduced the bug.

Commit 1f407e286c was supposed to fix that.  Did you try updating from
Git?

Message #11 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: epg <at> pretzelnet.org
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at
 commit bbc1803
Date: Sun, 09 Jun 2024 13:56:28 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

> Commit 1f407e286c was supposed to fix that.  Did you try updating from
> Git?

I'm sorry I wasn't clear:  the problem is present in master as of
0aed6f5f0d9319dc616424e2c7df80580a04c739.  I was reporting the
first commit to show the bug, not the latest.

Thanks!

-- 
Eric Gillespie  <*>  epg <at> pretzelnet.org

Message #14 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: epg <at> pretzelnet.org
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at
 commit bbc1803
Date: Sun, 09 Jun 2024 14:37:30 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

> Does it help to set read-process-output-fast to a nil value?

As far as I can tell, yes that does help!  I opened quite a few
folders and messages, way more than it took to cause a hang
earlier.  No hang!

Tested with 'emacs -Q' built from commit
0aed6f5f0d9319dc616424e2c7df80580a04c739 and with the following
configuration:

(setq
      read-process-output-fast nil
      gnus-select-method '(nnnil "")
      gnus-secondary-select-methods
      '(
        (nnimap "imap.fastmail.com"
                (nnimap-stream ssl)
                (nnimap-record-commands t)
                 ;; TODO expiry?!  (nnmail-expiry-target "nnimap+gmail:[Gmail]/Trash")
                )
        )
      )

Thanks!

-- 
Eric Gillespie  <*>  epg <at> pretzelnet.org

Message #17 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: epg <at> pretzelnet.org, Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50;
 GNUS hangs on IMAP/s reads starting at commit bbc1803
Date: Sun, 9 Jun 2024 16:43:02 -0400

epg <at> pretzelnet.org writes:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>> Does it help to set read-process-output-fast to a nil value?
>
> As far as I can tell, yes that does help!  I opened quite a few
> folders and messages, way more than it took to cause a hang
> earlier.  No hang!
>
> Tested with 'emacs -Q' built from commit
> 0aed6f5f0d9319dc616424e2c7df80580a04c739 and with the following
> configuration:
>
> (setq
>       read-process-output-fast nil
>       gnus-select-method '(nnnil "")
>       gnus-secondary-select-methods
>       '(
>         (nnimap "imap.fastmail.com"
>                 (nnimap-stream ssl)
>                 (nnimap-record-commands t)
>                  ;; TODO expiry?!  (nnmail-expiry-target "nnimap+gmail:[Gmail]/Trash")
>                 )
>         )
>       )
>
> Thanks!

Copying in Dmitry.

Not sure if it's relevant, but note also my recent commit
a1a4c6dccbe7aeeac97635df54797ebf6c0ce789.

Message #20 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Stefan Kangas <stefankangas <at> gmail.com>, epg <at> pretzelnet.org,
 Eli Zaretskii <eliz <at> gnu.org>
Cc: 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at commit
 bbc1803
Date: Sun, 9 Jun 2024 23:46:40 +0300

On 09/06/2024 23:43, Stefan Kangas wrote:
> epg <at> pretzelnet.org  writes:
> 
>> Eli Zaretskii<eliz <at> gnu.org>  writes:
>>
>>> Does it help to set read-process-output-fast to a nil value?
>> As far as I can tell, yes that does help!  I opened quite a few
>> folders and messages, way more than it took to cause a hang
>> earlier.  No hang!
>>
>> Tested with 'emacs -Q' built from commit
>> 0aed6f5f0d9319dc616424e2c7df80580a04c739 and with the following
>> configuration:
>>
>> (setq
>>        read-process-output-fast nil
>>        gnus-select-method '(nnnil "")
>>        gnus-secondary-select-methods
>>        '(
>>          (nnimap "imap.fastmail.com"
>>                  (nnimap-stream ssl)
>>                  (nnimap-record-commands t)
>>                   ;; TODO expiry?!  (nnmail-expiry-target "nnimap+gmail:[Gmail]/Trash")
>>                  )
>>          )
>>        )
>>
>> Thanks!
> Copying in Dmitry.
> 
> Not sure if it's relevant, but note also my recent commit
> a1a4c6dccbe7aeeac97635df54797ebf6c0ce789.

Thank you.

OP, could you retest with the latest master and read-process-output-fast 
set to its default value (t)?

Message #23 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: epg <at> pretzelnet.org
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at
 commit bbc1803
Date: Sun, 09 Jun 2024 16:30:40 -0500

Dmitry Gutov <dmitry <at> gutov.dev> writes:

> OP, could you retest with the latest master and read-process-output-fast set
> to its default value (t)?
>

As of 12d44fe6420e84eab8f750f9a0f8cd73c3e70bb2, still hangs with
read-process-output-fast t, works with it nil.

Thanks!

-- 
Eric Gillespie  <*>  epg <at> pretzelnet.org

Message #26 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: epg <at> pretzelnet.org
Cc: 71452 <at> debbugs.gnu.org
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at
 commit bbc1803
Date: Sun, 09 Jun 2024 22:18:13 +0300

> From: epg <at> pretzelnet.org
> Cc: 71452 <at> debbugs.gnu.org
> Date: Sun, 09 Jun 2024 13:56:28 -0500
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Commit 1f407e286c was supposed to fix that.  Did you try updating from
> > Git?
> 
> I'm sorry I wasn't clear:  the problem is present in master as of
> 0aed6f5f0d9319dc616424e2c7df80580a04c739.  I was reporting the
> first commit to show the bug, not the latest.

Does it help to set read-process-output-fast to a nil value?

Message #29 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: epg <at> pretzelnet.org
Cc: Eli Zaretskii <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at commit
 bbc1803
Date: Mon, 10 Jun 2024 07:09:52 +0300

[Message part 1 (text/plain, inline)]

On 10/06/2024 00:30, epg <at> pretzelnet.org wrote:
> Dmitry Gutov<dmitry <at> gutov.dev>  writes:
> 
>> OP, could you retest with the latest master and read-process-output-fast set
>> to its default value (t)?
>>
> As of 12d44fe6420e84eab8f750f9a0f8cd73c3e70bb2, still hangs with
> read-process-output-fast t, works with it nil.

Okay, thank you. I can reproduce it too.

FWIW, it hangs here:

Lisp Backtrace:
"accept-process-output" (0xf19ff6a8)
"nnheader-accept-process-output" (0xf19ff610)
"nnimap-wait-for-response" (0xf19ff5a8)
"nnimap-retrieve-headers" (0xf19ff530)
"gnus-retrieve-headers" (0xf19ff4b8)
"gnus-cache-retrieve-headers" (0xf19ff450)
"gnus-retrieve-headers" (0xf19ff3d0)
"gnus-fetch-headers" (0xf19ff360)
"gnus-select-newsgroup" (0xf19ff2c0)
"gnus-summary-read-group-1" (0xf19ff228)
"gnus-summary-read-group" (0xf19ff188)
"gnus-group-read-group" (0xf19ff100)
--Type <RET> for more, q to quit, c to continue without paging--
"gnus-group-select-group" (0xffffdb10)
"funcall-interactively" (0xffffdb08)
"call-interactively" (0xf19ff070)
"command-execute" (0xffffddf8)

Commands can be different, but nnimap-wait-for-response is where it 
stops. And it's not edebug-able: instrumenting the function makes the 
bug go away.

Print-debugging seems to say that there might be something wrong with 
what (forward-line -1) does: output has the ^M chars, and apparently the 
new logic makes it skip over too many characters.

The only thing relevant I found is this bit, which for now I simply 
copied over from read_and_dispose_of_process_output. But the latter 
calls it after "decoding to string" yet before inserting the string into 
the buffer. Can we do this after the decode_coding_c_string call where 
dst_object is the target buffer?

[Vlast_coding_system_used.diff (text/x-patch, attachment)]

Message #32 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: epg <at> pretzelnet.org
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at
 commit bbc1803
Date: Mon, 10 Jun 2024 22:39:08 -0500

Dmitry Gutov <dmitry <at> gutov.dev> writes:

> I've pushed a fix, commit 5a576069fcd. This fixes Gnus in my testing.

Seems to work for me too.

Thanks!

-- 
Eric Gillespie  <*>  epg <at> pretzelnet.org

Message #35 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: epg <at> pretzelnet.org
Cc: Eli Zaretskii <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#71452: 30.0.50; GNUS hangs on IMAP/s reads starting at commit
 bbc1803
Date: Tue, 11 Jun 2024 04:55:25 +0300

On 10/06/2024 07:09, Dmitry Gutov wrote:
> Print-debugging seems to say that there might be something wrong with 
> what (forward-line -1) does: output has the ^M chars, and apparently the 
> new logic makes it skip over too many characters.

All right, this was the newline cache.

I've pushed a fix, commit 5a576069fcd. This fixes Gnus in my testing.

Also heads up: the variable to disable optimization had been renamed to 
'fast-read-process-output'. Just in case you or somebody else needs to 
disable the fast path for this or some newer reason.

Message #38 received at 71452 <at> debbugs.gnu.org (full text, mbox):

From: "Eric Gillespie" <epg <at> pretzelnet.org>
To: "Dmitry Gutov" <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#71452: 30.0.50;
 GNUS hangs on IMAP/s reads starting at commit bbc1803
Date: Wed, 12 Jun 2024 09:22:46 -0500

That seems to have fixed it. Thanks!

Message #43 received at 71452-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Eric Gillespie" <epg <at> pretzelnet.org>
Cc: dmitry <at> gutov.dev, 71452-done <at> debbugs.gnu.org, stefankangas <at> gmail.com
Subject: Re: bug#71452: 30.0.50;
 GNUS hangs on IMAP/s reads starting at commit bbc1803
Date: Wed, 12 Jun 2024 19:47:55 +0300

> Date: Wed, 12 Jun 2024 09:22:46 -0500
> From: "Eric Gillespie" <epg <at> pretzelnet.org>
> Cc: "Eli Zaretskii" <eliz <at> gnu.org>, 71452 <at> debbugs.gnu.org,
>  "Stefan Kangas" <stefankangas <at> gmail.com>
> 
> That seems to have fixed it. Thanks!

Thanks, closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.