Subject: bug#67704: 30.0.50; Emacs hangs indefinitely when writing GPG-encrypted file

From: Tassilo Horn <[email-masked]>
Date: Fri, 08 Dec 2023 07:49:13 +0100

I have several GPG-encrypted files such as ~/.authinfo.gpg which I use
as auth-source.  I can still open them fine where I get a pinentry
dialog the first time until the password is cached in the gpg-agent.
However, I cannot save/write such files from emacs anymore which used to
work flawlessly for many years.

When I try to save, emacs just seems to hang indefinitely.  Setting
debug-on-quit and then C-g-ing reveals it hangs in there (emacs -Q):

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (quit)
  epg-wait-for-status(#s(epg-context :protocol OpenPGP :program "/usr/bin/gpg2" :home-directory nil :armor nil :textmode nil :include-certs nil :cipher-algorithm nil :digest-algorithm nil :compress-algorithm nil :passphrase-callback (epa-file-passphrase-callback-function . "/home/horn/Repos/configs/dot-authinfo.gpg") :progress-callback (epa-progress-callback-function . "Encrypting /home/horn/Repos/configs/dot-authinfo.gpg") :edit-callback nil :signers nil :sender nil :sig-notations nil :process #<process epg> :output-file "/tmp/epg-outputxL3w1m" :result nil :operation encrypt :pinentry-mode nil :error-output "" :error-buffer #<buffer  *epg-error*>) ("BEGIN_ENCRYPTION"))
  epg-start-encrypt(#s(epg-context :protocol OpenPGP :program "/usr/bin/gpg2" :home-directory nil :armor nil :textmode nil :include-certs nil :cipher-algorithm nil :digest-algorithm nil :compress-algorithm nil :passphrase-callback (epa-file-passphrase-callback-function . "/home/horn/Repos/configs/dot-authinfo.gpg") :progress-callback (epa-progress-callback-function . "Encrypting /home/horn/Repos/configs/dot-authinfo.gpg") :edit-callback nil :signers nil :sender nil :sig-notations nil :process #<process epg> :output-file "/tmp/epg-outputxL3w1m" :result nil :operation encrypt :pinentry-mode nil :error-output "" :error-buffer #<buffer  *epg-error*>) #s(epg-data :file nil :string "...") (#s(epg-key :owner-trust ultimate :sub-key-list (#s(epg-sub-key :validity ultimate :capability (sign certify) :secret-p nil :algorithm 1 :length 4096 :id "..." :creation-time 1474387910 :expiration-time nil :fingerprint "...") #s(epg-sub-key :validity ultimate :capability (encrypt) :secret-p nil :algorithm 1 :length 4096 :id "..." :creation-time 1474387910 :expiration-time nil :fingerprint "...")) :user-id-list (#s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil)))) nil nil)
  epg-encrypt-string(#s(epg-context :protocol OpenPGP :program "/usr/bin/gpg2" :home-directory nil :armor nil :textmode nil :include-certs nil :cipher-algorithm nil :digest-algorithm nil :compress-algorithm nil :passphrase-callback (epa-file-passphrase-callback-function . "/home/horn/Repos/configs/dot-authinfo.gpg") :progress-callback (epa-progress-callback-function . "Encrypting /home/horn/Repos/configs/dot-authinfo.gpg") :edit-callback nil :signers nil :sender nil :sig-notations nil :process #<process epg> :output-file "/tmp/epg-outputxL3w1m" :result nil :operation encrypt :pinentry-mode nil :error-output "" :error-buffer #<buffer  *epg-error*>) "..." (#s(epg-key :owner-trust ultimate :sub-key-list (#s(epg-sub-key :validity ultimate :capability (sign certify) :secret-p nil :algorithm 1 :length 4096 :id "..." :creation-time 1474387910 :expiration-time nil :fingerprint "...") #s(epg-sub-key :validity ultimate :capability (encrypt) :secret-p nil :algorithm 1 :length 4096 :id "..." :creation-time 1474387910 :expiration-time nil :fingerprint "...")) :user-id-list (#s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil) #s(epg-user-id :validity ultimate :string "Tassilo Horn <[email-masked]>" :signature-list nil)))))
  epa-file-write-region(nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg" nil)
  epa-file-handler(write-region nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg" nil)
  write-region(nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg")
  basic-save-buffer-2()
  basic-save-buffer-1()
  basic-save-buffer(t)
  save-buffer(1)
  funcall-interactively(save-buffer 1)
  command-execute(save-buffer)
--8<---------------cut here---------------end--------------->8---

There's a gpg2 process started like that:

  /usr/bin/gpg2 --no-tty --status-fd 1 --yes --enable-progress-filter \
                --command-fd 0 --output /tmp/epg-outputxL3w1m \
                --encrypt -r <MY_GPG_KEY_ID>

When I perform that command (with a different --output file) on the
command line, it says

--8<---------------cut here---------------start------------->8---
[GNUPG:] KEY_CONSIDERED 0F1D0793E6C7D8A5DA026F1373292E404EDE1CF4 0

--8<---------------cut here---------------end--------------->8---

and I can type text and then hit C-d where it will say

--8<---------------cut here---------------start------------->8---
[GNUPG:] PROGRESS stdin ? 0 0 B
[GNUPG:] BEGIN_ENCRYPTION 2 9
[GNUPG:] PROGRESS stdin ? 21 0 B
[GNUPG:] END_ENCRYPTION
--8<---------------cut here---------------end--------------->8---

and write the text I entered to the encrypted --output file.  So it
looks like emacs doesn't do the equivalent of sending the "I am finished
entering text" code which the key C-d on the shell does.

The above was with the current emacs master.  With the released version
29.1 as provided by my distro, I have the same issue, however the
backtrace looks a bit different (a accept-process-output on top):

Debugger entered--Lisp error: (quit)
  accept-process-output(#<process epg> 1)
  epg-wait-for-status(#s(...) ("BEGIN_ENCRYPTION"))
  epg-start-encrypt(...)
  epg-encrypt-string(...)
  epa-file-write-region(nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg" nil)
  apply(epa-file-write-region (nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg" nil))
  epa-file-handler(write-region nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg" nil)
  write-region(nil nil "/home/horn/Repos/configs/dot-authinfo.gpg" nil t "~/Repos/configs/dot-authinfo.gpg")
  basic-save-buffer-2()
  basic-save-buffer-1()
  basic-save-buffer(t)
  save-buffer(1)
  funcall-interactively(save-buffer 1)
  call-interactively(save-buffer nil nil)
  command-execute(save-buffer)

The GnuPG version is 2.4.3.

I cannot say when the issue started appearing.  I frequently read those
files (which still works) but seldomly need to change them.  So it's
possible it's broken since several weeks.  Or maybe it's not caused by
emacs but the gpg2 interface changed?  Oh yeah, that could be likely.
My package manager log says

  [2023-12-04T08:32:29+0100] [ALPM] upgraded gnupg (2.2.41-2 -> 2.4.3-2)

so there was a minor version upgrade on December, 4th.  Maybe emacs is
not yet ready for GnuPG 2.4.x?

In GNU Emacs 30.0.50 (build 24, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.18.0) of 2023-12-08 built on thinkpad-t440p
Repository revision: 3cf7d32b90ad32321c141bb2a34a670d2d6df5b7
Repository branch: master
System Description: Arch Linux

Configured using:
 'configure --with-tree-sitter --with-pgtk --with-modules'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LC_MONETARY: de_DE.utf8
  value of $LC_NUMERIC: de_DE.utf8
  value of $LC_TIME: de_DE.utf8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e:main

Minor modes in effect:
  hexl-follow-ascii: t
  breadcrumb-mode: t
  editorconfig-mode: t
  global-aggressive-indent-mode: t
  pdf-occur-global-minor-mode: t
  diredfl-global-mode: t
  mu4e-search-minor-mode: t
  mu4e-update-minor-mode: t
  mu4e-context-minor-mode: t
  mu4e-modeline-mode: t
  which-key-mode: t
  highlight-parentheses-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  server-mode: t
  corfu-history-mode: t
  global-corfu-mode: t
  corfu-mode: t
  vertico-mode: t
  marginalia-mode: t
  minibuffer-depth-indicate-mode: t
  switchy-window-minor-mode: t
  electric-pair-mode: t
  recentf-mode: t
  override-global-mode: t
  repeat-mode: t
  save-place-mode: t
  savehist-mode: t
  puni-global-mode: t
  puni-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  column-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  overwrite-mode: overwrite-mode-binary

Load-path shadows:
~/Repos/el/mu/mu4e/mu4e hides ~/Repos/el/mu/build/mu4e/mu4e
~/Repos/el/mu/mu4e/mu4e-modeline hides ~/Repos/el/mu/build/mu4e/mu4e-modeline
~/Repos/el/mu/mu4e/mu4e-context hides ~/Repos/el/mu/build/mu4e/mu4e-context
~/Repos/el/mu/mu4e/mu4e-main hides ~/Repos/el/mu/build/mu4e/mu4e-main
~/Repos/el/mu/mu4e/mu4e-vars hides ~/Repos/el/mu/build/mu4e/mu4e-vars
~/Repos/el/mu/mu4e/mu4e-window hides ~/Repos/el/mu/build/mu4e/mu4e-window
~/Repos/el/mu/mu4e/mu4e-speedbar hides ~/Repos/el/mu/build/mu4e/mu4e-speedbar
~/Repos/el/mu/mu4e/mu4e-view hides ~/Repos/el/mu/build/mu4e/mu4e-view
~/Repos/el/mu/mu4e/mu4e-thread hides ~/Repos/el/mu/build/mu4e/mu4e-thread
~/Repos/el/mu/mu4e/mu4e-bookmarks hides ~/Repos/el/mu/build/mu4e/mu4e-bookmarks
~/Repos/el/mu/mu4e/mu4e-org hides ~/Repos/el/mu/build/mu4e/mu4e-org
~/Repos/el/mu/mu4e/mu4e-lists hides ~/Repos/el/mu/build/mu4e/mu4e-lists
~/Repos/el/mu/mu4e/mu4e-actions hides ~/Repos/el/mu/build/mu4e/mu4e-actions
~/Repos/el/mu/mu4e/mu4e-helpers hides ~/Repos/el/mu/build/mu4e/mu4e-helpers
~/Repos/el/mu/mu4e/mu4e-search hides ~/Repos/el/mu/build/mu4e/mu4e-search
~/Repos/el/mu/mu4e/mu4e-server hides ~/Repos/el/mu/build/mu4e/mu4e-server
~/Repos/el/mu/mu4e/mu4e-obsolete hides ~/Repos/el/mu/build/mu4e/mu4e-obsolete
~/Repos/el/mu/mu4e/mu4e-update hides ~/Repos/el/mu/build/mu4e/mu4e-update
~/Repos/el/mu/mu4e/mu4e-message hides ~/Repos/el/mu/build/mu4e/mu4e-message
~/Repos/el/mu/mu4e/mu4e-compose hides ~/Repos/el/mu/build/mu4e/mu4e-compose
~/Repos/el/mu/mu4e/mu4e-headers hides ~/Repos/el/mu/build/mu4e/mu4e-headers
~/Repos/el/mu/mu4e/mu4e-query-items hides ~/Repos/el/mu/build/mu4e/mu4e-query-items
~/Repos/el/mu/mu4e/mu4e-notification hides ~/Repos/el/mu/build/mu4e/mu4e-notification
~/Repos/el/mu/mu4e/mu4e-contacts hides ~/Repos/el/mu/build/mu4e/mu4e-contacts
~/Repos/el/mu/mu4e/mu4e-icalendar hides ~/Repos/el/mu/build/mu4e/mu4e-icalendar
~/Repos/el/mu/mu4e/mu4e-mark hides ~/Repos/el/mu/build/mu4e/mu4e-mark
~/Repos/el/mu/mu4e/mu4e-contrib hides ~/Repos/el/mu/build/mu4e/mu4e-contrib
~/Repos/el/mu/mu4e/mu4e-folders hides ~/Repos/el/mu/build/mu4e/mu4e-folders
~/Repos/el/mu/mu4e/mu4e-mime-parts hides ~/Repos/el/mu/build/mu4e/mu4e-mime-parts
/home/horn/.emacs.d/elpa/ef-themes-1.4.0.0.20231205.64730/theme-loaddefs hides /home/horn/Repos/el/emacs/lisp/theme-loaddefs
/home/horn/.emacs.d/elpa/transient-20231205.1848/transient hides /home/horn/Repos/el/emacs/lisp/transient

Features:
(shadow sort expreg cap-words superword subword face-remap mail-extr
rdictcc emacsbug cus-start so-long view tramp-cmds puni
display-fill-column-indicator display-line-numbers ef-cherie-theme
generic totp-widget totp hexl bindat yaml-mode fish-mode cargo xref
cargo-process web-mode disp-table preview-latex auctex tex-site
breadcrumb pulse project editorconfig editorconfig-core
editorconfig-core-handle editorconfig-fnmatch elfeed-show elfeed-search
vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view debbugs-browse
elfeed-csv elfeed elfeed-curl elfeed-log elfeed-db elfeed-lib avl-tree
generator url-queue xml-query hl-todo aggressive-indent rainbow-mode
pdf-occur tablist tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet pdf-isearch
pdf-misc pdf-tools pdf-view jka-compr pdf-cache pdf-info tq pdf-util
pdf-macs image-mode exif vc-git vc-dir ewoc epa-file diredfl dired-x
mu4e-icalendar gnus-icalendar icalendar diary-lib diary-loaddefs mu4e
mu4e-org mu4e-notification notifications mu4e-main smtpmail mu4e-view
mu4e-mime-parts mu4e-headers mu4e-thread mu4e-actions org-capture
org-refile org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete org-list org-footnote org-faces
org-entities ob-emacs-lisp ob-core ob-eval org-cycle org-table ol
org-fold org-fold-core org-keys oc org-loaddefs cal-menu calendar
cal-loaddefs org-compat org-version org-macs mu4e-compose mu4e-search
mu4e-lists mu4e-bookmarks mu4e-mark mu4e-message flow-fill mule-util
hl-line mu4e-contacts mu4e-update mu4e-folders mu4e-context
mu4e-query-items mu4e-server mu4e-modeline mu4e-vars mu4e-helpers
mu4e-config mu4e-window magit-bookmark bookmark ido mu4e-obsolete
hippie-exp auto-dictionary flyspell ispell tramp-smb which-key
highlight-parentheses restclient yasnippet forge-list forge-commands
forge-semi forge-bitbucket buck forge-gogs gogs forge-gitea gtea
forge-gitlab glab forge-github ghub-graphql treepy gsexp ghub let-alist
forge-notify forge-revnote forge-pullreq forge-issue forge-topic yaml
forge-post markdown-mode noutline outline forge-repo forge forge-core
forge-db closql emacsql-sqlite-common emacsql emacsql-compiler
eieio-base magit-submodule magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff smerge-mode diff diff-mode git-commit log-edit pcvs-util
add-log magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor comp comp-cstr server
magit-mode magit-git magit-base magit-section cursor-sensor crm dash
visual-filename-abbrev rg vc vc-dispatcher rg-info-hack advice rg-menu
transient rg-ibuffer rg-result wgrep-rg wgrep rg-history rg-header
ibuf-ext ibuffer ibuffer-loaddefs grep compile debbugs soap-client
url-http url-auth url-gw nsm warnings rng-xsd rng-dt rng-util xsd-regexp
bug-reference thingatpt kind-icon svg-lib color corfu-history corfu
vertico marginalia icomplete mb-depth use-package-diminish
switchy-window compat elec-pair recentf tree-widget edmacro kmacro
use-package-bind-key bind-key diminish repeat rust-ts-mode c-ts-common
find-func treesit saveplace tramp-sh tramp trampver tramp-integration
files-x tramp-message tramp-compat xdg shell pcomplete comint ansi-osc
ring format-spec ansi-color tramp-loaddefs savehist smiley gnus-art
mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum shr pixel-fill
kinsoku url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus
comp-run comp-common rx xml gnus-cloud nnimap nnmail mail-source utf7
nnoo parse-time iso8601 gnus-spec gnus-int gnus-range message sendmail
yank-media puny dired dired-loaddefs rfc822 mml mml-sec epa derived epg
rfc6068 epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util text-property-search time-date mm-util mail-prsvr
mail-utils range ef-themes cl-extra help-mode use-package-ensure
use-package-core finder-inf cus-edit pp cus-load icons wid-edit
aggressive-indent-autoloads auto-dictionary-autoloads
breadcrumb-autoloads cape-autoloads cargo-autoloads
clojure-mode-autoloads corfu-autoloads csv-mode-autoloads
debbugs-autoloads diminish-autoloads diredfl-autoloads
editorconfig-autoloads ef-themes-autoloads elfeed-autoloads
ement-autoloads expreg-autoloads fish-mode-autoloads forge-autoloads
closql-autoloads emacsql-autoloads ghub-autoloads
highlight-parentheses-autoloads hl-todo-autoloads kind-icon-autoloads
magit-autoloads pcase git-commit-autoloads marginalia-autoloads
markdown-mode-autoloads mastodon-autoloads pdf-tools-autoloads
persist-autoloads plz-autoloads puni-autoloads easy-mmode
rainbow-mode-autoloads rcirc-color-autoloads request-autoloads
restclient-autoloads rg-autoloads svg-lib-autoloads
symbol-overlay-autoloads tablist-autoloads taxy-magit-section-autoloads
taxy-autoloads magit-section-autoloads dash-autoloads totp-autoloads
transient-autoloads treepy-autoloads vertico-autoloads
visual-filename-abbrev-autoloads web-mode-autoloads wgrep-autoloads
which-key-autoloads with-editor-autoloads info compat-autoloads
yaml-autoloads yaml-mode-autoloads yasnippet-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache
json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs
cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win
pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo gtk pgtk lcms2 multi-tty move-toolbar
make-network-process native-compile emacs)

Memory information:
((conses 16 894950 154975) (symbols 48 52000 7) (strings 32 241367 6416)
 (string-bytes 1 7370278) (vectors 16 97111) (vector-slots 8 1664749 96887)
 (floats 8 710 316) (intervals 56 10169 364) (buffers 992 19))