Subject: 29.0.91; many calls to make-process cause varying undesired behavior

From: [email-masked]
Date: Fri, 07 Jul 2023 21:57:47 +0000 (UTC)

I'm heavily using `make-process', creating around 60 processes each with
an `:stderr' pipe process. These are all SSH processes being issued in
parallel. I've had nothing but problems when forking around this many
processes or more. As a workaround I've made a process queue to limit
the maximum number of processes to around 50, and this seems to stop the
problem. This is all done in batch mode, or non-interactive use of
emacs. I'm also using threading functionality, but the issues remain
with or without the use of it.

The issues being experienced include a process made with `make-process'
that starts, `process-live-p' returns '(run open listen connect stop)
but neither the process sentinel or filter is ever called. It is
expected that at least the sentinel is called with the "run\n"
event. Secondly, sometimes emacs crashes and outputs a backtrace. The
backtrace is the following:

**
GLib:ERROR:../glib/glib/gmain.c:3536:g_main_context_acquire: assertion failed: (context->owner_count == 0)
Bail out! GLib:ERROR:../glib/glib/gmain.c:3536:g_main_context_acquire: assertion failed: (context->owner_count == 0)
Backtrace:
emacs(+0x15f50b)[0x558288e4950b]
emacs(+0x15f5ff)[0x558288e495ff]
/usr/lib/libc.so.6(+0x39ab0)[0x7f8d776fbab0]
/usr/lib/libc.so.6(+0x8926c)[0x7f8d7774b26c]
/usr/lib/libc.so.6(gsignal+0x18)[0x7f8d776fba08]
/usr/lib/libc.so.6(abort+0xd7)[0x7f8d776e4538]
/usr/lib/libglib-2.0.so.0(+0x1e05e)[0x7f8d7ab2f05e]
/usr/lib/libglib-2.0.so.0(g_assertion_message_expr+0x50)[0x7f8d7ab8fe90]
/usr/lib/libglib-2.0.so.0(g_main_context_acquire+0xa2)[0x7f8d7ab68cc2]
emacs(+0x28a241)[0x558288f74241]
emacs(+0x2394f7)[0x558288f234f7]
emacs(+0x23b622)[0x558288f25622]
emacs(+0x1d8531)[0x558288ec2531]
emacs(+0x1d8ec5)[0x558288ec2ec5]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1da57d)[0x558288ec457d]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1d9015)[0x558288ec3015]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1da175)[0x558288ec4175]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1d960d)[0x558288ec360d]
emacs(+0x22104f)[0x558288f0b04f]
emacs(+0x1d9b46)[0x558288ec3b46]
emacs(+0x1d7e47)[0x558288ec1e47]
emacs(+0x1d8459)[0x558288ec2459]
emacs(+0x1da3c1)[0x558288ec43c1]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1dab19)[0x558288ec4b19]
emacs(+0x1d8413)[0x558288ec2413]
emacs(+0x1d960d)[0x558288ec360d]
emacs(+0x1d9b46)[0x558288ec3b46]
emacs(+0x1d7e47)[0x558288ec1e47]
emacs(+0x1d960d)[0x558288ec360d]
emacs(+0x1d468e)[0x558288ebe68e]
emacs(+0x26190b)[0x558288f4b90b]
emacs(+0x1d2c57)[0x558288ebcc57]
emacs(+0x262245)[0x558288f4c245]
/usr/lib/libc.so.6(+0x8744b)[0x7f8d7774944b]
/usr/lib/libc.so.6(+0x10ae40)[0x7f8d777cce40]
...

Let me know how I can help debug and solve this issue. Thanks

In GNU Emacs 29.0.91 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.37, cairo version 1.17.8) of 2023-05-23 built on ca1
Repository revision: 117b29c6f6681e4ae0710a9f90f1b8d274afc7c0
Repository branch: makepkg
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-modules --without-libotf --without-m17n-flt --without-gconf
 --with-native-compilation=yes --with-xinput2 --with-pgtk
 --without-xaw3d --with-sound=no --with-tree-sitter --without-gpm
 --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=native -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP XIM GTK3 ZLIB

Important settings:
  value of $EMACSNATIVELOADPATH: /home/piknik/.local/var/lib/emacs/cache/eln-cache/
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  consult-org-roam-mode: t
  org-roam-db-autosync-mode: t
  shell-dirtrack-mode: t
  nameless-mode: t
  show-smartparens-mode: t
  smartparens-mode: t
  yas-minor-mode: t
  display-fill-column-indicator-mode: t
  auto-insert-mode: t
  global-corfu-mode: t
  corfu-mode: t
  gcmh-mode: t
  global-auto-revert-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  projectile-mode: t
  burly-tabs-mode: t
  server-mode: t
  savehist-mode: t
  marginalia-mode: t
  vertico-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  prettify-symbols-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/piknik/.local/var/lib/emacs/packages/transient-20230315.1520/transient hides /usr/share/emacs/29.0.91/lisp/transient

Features:
(shadow mail-extr emacsbug man vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs
vc-rcs log-view vc bug-reference magit-extras magit-bookmark
magit-submodule magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode git-commit log-edit pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process magit-mode
transient magit-git magit-base smtpmail rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid nxml-mode nxml-outln nxml-rap sgml-mode facemenu
consult-register two-column filecache embark-org embark-consult embark
semantic/symref/grep semantic/symref semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
pcmpl-linux proced network-stream pulse consult-xref data-structures
display-line-numbers plz cl-print consult-imenu calc-alg calc-ext
calc-menu calc calc-loaddefs rect calc-macs pcmpl-gnu macros conf-mode
sort latexenc obi-task-report obi org-ml obi-doc-build xlet
ucs-normalize crm vterm face-remap color vterm-module term/xterm xterm
helpful cc-langs trace edebug debug backtrace info-look help-fns
radix-tree elisp-refs dockerfile-mode json-mode json-snatcher js
c-ts-common cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs sql view tabify elfeed-link
pk-init-mode-elfeed elfeed-show elfeed-search elfeed-csv elfeed
elfeed-curl elfeed-log xml-query elfeed-db elfeed-lib ffap org-clock
org-eldoc tcl sh-script smie executable org-indent pk-roam oc-basic
ol-eww eww mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum
gnus-group gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail
mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range ol-docview doc-view jka-compr image-mode exif ol-bbdb
ol-w3m ol-doi org-link-doi org-ref org-ref-core org-ref-glossary
org-ref-bibtex doi-utils url-http url-auth url-gw nsm message sendmail
yank-media rfc822 mml mml-sec gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader org-ref-utils org-ref-export
citeproc citeproc-itemgetters citeproc-biblatex citeproc-bibtex
ol-bibtex citeproc-cite citeproc-subbibs citeproc-sort citeproc-name
citeproc-formatters citeproc-number rst citeproc-proc citeproc-disamb
citeproc-itemdata citeproc-generic-elements citeproc-macro
citeproc-choose citeproc-date citeproc-context citeproc-prange
citeproc-style citeproc-locale citeproc-term citeproc-rt citeproc-lib
citeproc-s ox-org org-ref-misc-links org-ref-label-link
org-ref-ref-links org-ref-citation-links xref org-ref-bibliography-links
hydra lv bibtex-completion biblio biblio-download biblio-dissemin
biblio-ieee biblio-hal biblio-dblp biblio-crossref biblio-arxiv timezone
biblio-doi biblio-core let-alist url-queue ido hl-line parsebib bibtex
pk-init-mode-consult-org-roam consult-org-roam consult-org-roam-buffer
pk-init-mode-org-roam org-roam-migrate org-roam-log org-roam-mode
org-roam-capture org-roam-id org-roam-node org-roam-db
emacsql-sqlite-builtin sqlite org-roam-utils org-roam-compat org-roam
org-capture org-attach doom-themes-ext-org ox-md ox-odt rng-loc rng-uri
rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns
nxml-enc xmltok nxml-util ox-latex ox-icalendar ox-html table ox-ascii
ox-publish ox pk-init-mode-org-ql org-ql peg ts org-habit
pk-init-mode-org-agenda org-agenda org-element org-persist xdg
pk-init-mode-org-id org-id org-refile org-duration org-expiry
ob-restclient restclient ob-lisp ob-org ob-plantuml ob-python
pk-init-mode-python devdocs shr pixel-fill kinsoku url-file puny svg xml
dom python treesit ob-shell ob-eshell ob-ledger org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete
org-list avl-tree org-footnote org-faces org-entities noutline outline
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs find-func cal-menu calendar
cal-loaddefs org-version org-compat org-macs emacsql-sqlite
emacsql-sqlite-common emacsql emacsql-compiler magit-section vc-git
diff-mode vc-dispatcher password-store-otp misearch multi-isearch pass f
f-shortdoc s imenu password-store auth-source-pass with-editor
ace-window tramp-cmds tramp-cache time-stamp tramp-sh tramp-container
tramp tramp-loaddefs trampver tramp-integration tramp-compat parse-time
iso8601 ls-lisp format-spec pk-init-mode-avy avy time-date mwim
pcmpl-unix pk-init-mode-eshell em-unix em-term pk-init-mode-term term
disp-table shell ehelp em-script em-prompt em-ls em-hist em-pred em-glob
em-extpipe em-cmpl em-dirs esh-var pcomplete em-basic em-banner em-alias
esh-mode eshell esh-cmd generator esh-ext esh-opt esh-proc esh-io
esh-arg esh-module esh-groups esh-util files-x doom-one-theme
doom-themes doom-themes-base pk-init-mode-consult consult-vertico
consult cus-theme mule-util orderless nameless easy-mmode
pk-init-mode-smartparens edmacro kmacro smartparens advice loadhist dash
pk-init-mode-yasnippet cus-start yasnippet display-fill-column-indicator
autoinsert corfu gcmh recentf tree-widget autorevert undo-tree diff
queue projectile project lisp-mnt wgrep pk-init-mode-grep grep ibuf-ext
ibuffer ibuffer-loaddefs burly-tabs burly frameset thingatpt bookmark
epa-file epa derived epg rfc6068 epg-config server savehist marginalia
vertico comp comp-cstr warnings cl-extra help-mode pinentry
tree-sitter-langs tree-sitter-langs-build tar-mode arc-mode archive-mode
tree-sitter-hl tree-sitter tree-sitter-load tree-sitter-cli tsc tsc-dyn
tsc-dyn-get compile text-property-search comint ansi-osc ansi-color ring
dired-aux dired dired-loaddefs tsc-obsolete pk-init-post cl-indent
pk-init-mode-elisp pk-init-mode-cape pk-init-mode-prog pk-init-mode-org
org-roam-ext-find org-roam-ext-filter org-roam-ext pk-init-kb
pk-init-config cus-edit pp cus-load icons wid-edit pk-init-pre pk-init
filenotify finder-inf yaml-mode-autoloads org-make-toc-autoloads
plz-autoloads esxql-autoloads pk-roam-autoloads
tci-features-emacs-autoloads tci-devops-emacs-autoloads treepy-autoloads
org-trello-autoloads request-deferred-autoloads request-autoloads
deferred-autoloads obi-autoloads nameless-autoloads
org-roam-ui-autoloads websocket-autoloads simple-httpd-autoloads
org-roam-ext-autoloads pk-init-autoloads cape compat
emacs-conflict-autoloads d2-mode-autoloads cape-autoloads
corfu-autoloads copilot-autoloads ob-restclient-autoloads
restclient-autoloads editorconfig-autoloads powerthesaurus-autoloads
jeison-autoloads tempel-autoloads json-reformat-autoloads
json-mode-autoloads rx json-snatcher-autoloads markdown-mode-autoloads
toml-mode-autoloads php-mode-autoloads tide-autoloads flycheck-autoloads
pkg-info-autoloads epl-autoloads web-mode-autoloads rjsx-mode-autoloads
js2-mode-autoloads nginx-mode-autoloads lua-mode-autoloads
dockerfile-mode-autoloads devdocs-autoloads mu4e-autoloads
ledger-mode-autoloads prism-autoloads doom-themes-autoloads
pcre2el-autoloads ggtags-autoloads treemacs-autoloads pfuture-autoloads
cfrs-autoloads undo-tree-autoloads tree-sitter-langs-autoloads
tree-sitter-autoloads tsc-autoloads smartparens-autoloads
visual-fill-column-autoloads mwim-autoloads sly-autoloads
elfeed-autoloads plantuml-mode-autoloads vterm-autoloads
ripgrep-autoloads projectile-autoloads alert-autoloads gntp-autoloads
log4e-autoloads company-autoloads yatemplate-autoloads
auto-yasnippet-autoloads yasnippet-autoloads pass-autoloads
password-store-otp-autoloads password-store-autoloads
frog-jump-buffer-autoloads frog-menu-autoloads posframe-autoloads
link-hint-autoloads consult-org-roam-autoloads org-roam-autoloads
emacsql-autoloads org-ref-autoloads htmlize-autoloads hydra-autoloads
lv-autoloads bibtex-completion-autoloads biblio-autoloads
biblio-core-autoloads citeproc-autoloads queue-autoloads
string-inflection-autoloads parsebib-autoloads ox-pandoc-autoloads
org-transclusion-autoloads org-ql-autoloads org-super-agenda-autoloads
ht-autoloads ov-autoloads peg-autoloads ts-autoloads
org-cliplink-autoloads org-ml-autoloads org-contrib-autoloads
burly-autoloads wgrep-autoloads highlight-sexp-autoloads
eshell-bookmark-autoloads rainbow-blocks-autoloads outorg-autoloads
helpful-autoloads f-autoloads elisp-refs-autoloads s-autoloads
rainbow-delimiters-autoloads magit-autoloads pcase git-commit-autoloads
magit-section-autoloads dash-autoloads transient-autoloads
with-editor-autoloads pinentry-autoloads which-key-autoloads
free-keys-autoloads ace-window-autoloads avy-autoloads
embark-consult-autoloads embark-autoloads marginalia-autoloads
orderless-autoloads consult-autoloads vertico-autoloads info
compat-autoloads gcmh-autoloads auto-compile-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache
json subr-x map byte-opt bytecomp byte-compile url-vars cl-loaddefs
cl-lib avoid gv rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 7432395 1992745)
 (symbols 48 270459 20)
 (strings 32 717445 408227)
 (string-bytes 1 21688060)
 (vectors 16 214582)
 (vector-slots 8 4379427 1173482)
 (floats 8 1510 5984)
 (intervals 56 305806 117086)
 (buffers 984 409))