Package: emacs;
Reported by: Alexander Miller <alexanderm <at> web.de>
Date: Tue, 12 May 2020 21:57:01 UTC
Severity: normal
Found in version 26.3
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Alexander Miller <alexanderm <at> web.de> To: bug-gnu-emacs <at> gnu.org Subject: 26.3; Repeated yielding in worker thread insufficient to process input events in the main thread Date: Tue, 12 May 2020 23:56:52 +0200
This report is based on the discussion in https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01566.html. In this thread we were thinking up ways how to make Elisp asynchronous with the help of threads - to do heavy lifting in a worker thread, but yield for user interactions. Part of that was a benchmark for how well persistently yielding in the worker thread after every finished work-unit would perform. My conclusion was that it doesn't, input was ignored most of the time. Stefan thought that it *should* work, and asked me to make a bug report, so here it is. I have modified my benchmark to work with emacs -q. It will set up a worker thread, a queue, a timer to check the queue and wake up the worker, as well as a long CPU consuming task that emulates the kind of work I would want to use threads for in a real-world scenario. The task needs close to half a minute on my system. While it runs navigation with C-n/p is barely possible. Here is the full code: (defconst worker-mutex (make-mutex "*WORKER MUTEX*")) (defconst worker-cond-var (make-condition-variable worker-mutex)) (defvar worker-queue (list)) (setf worker (make-thread (lambda () (while t (while worker-queue (let* ((work-unit (pop worker-queue)) (fn (car work-unit)) (args (cdr work-unit))) (apply fn args) (thread-yield))) (message "Worker Goes To Sleep") (with-mutex worker-mutex (condition-wait worker-cond-var)))) "*WORKER*")) (setf worker-timer (run-with-idle-timer 1 t (lambda () (when worker-queue (with-mutex worker-mutex (condition-notify worker-cond-var)))))) (defun get-the-things () (let ((run-start (float-time)) (things)) (dotimes (_ 100000) (let* ((status (prog1 (random 4) (dotimes (_ 1000) (random)))) (thing (pcase status (0 'a) (1 'b) (2 'c) (3 'd)))) (push thing things)) (thread-yield)) (message "Run Time %ss" (- (float-time) run-start)))) (push (list #'get-the-things) worker-queue) In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.10) of 2019-08-29 built on juergen Windowing system distributor 'The X.Org Foundation', version 11.0.12008000 System Description: Manjaro Linux Recent messages: worker-cond-var worker-queue #<thread *WORKER*> Worker Goes To Sleep [nil 0 1 0 t (closure (t) nil (if worker-queue (progn (let ((mutex worker-mutex)) (mutex-lock mutex) (unwind-protect (progn (condition-notify worker-cond-var)) (mutex-unlock mutex)))))) nil idle 0] get-the-things ((get-the-things)) Auto-saving... Run Time 90.80384421348572s Worker Goes To Sleep Configured using: 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD LCMS2 Important settings: value of $LC_MONETARY: de_DE.UTF-8 value of $LC_NUMERIC: de_DE.UTF-8 value of $LC_TIME: de_DE.UTF-8 value of $LANG: en_GB.utf8 locale-coding-system: utf-8 Major mode: Emacs-Lisp Minor modes in effect: ivy-posframe-mode: t diff-auto-refine-mode: t rainbow-delimiters-mode: t rainbow-mode: t display-line-numbers-mode: t helm-mode: t async-bytecomp-package-mode: t helm--remap-mouse-mode: t projectile-mode: t global-company-mode: t company-mode: t global-evil-vimish-fold-mode: t evil-vimish-fold-mode: t vimish-fold-mode: t show-smartparens-global-mode: t show-smartparens-mode: t smartparens-global-mode: t smartparens-mode: t org-super-agenda-mode: t treemacs-icons-dired-mode: t treemacs-filewatch-mode: t treemacs-follow-mode: t treemacs-git-mode: deferred treemacs-fringe-indicator-mode: t gcmh-mode: t framey-mode: t purpose-mode: t shackle-mode: t winum-mode: t eyebrowse-mode: t global-subword-mode: t subword-mode: t global-evil-surround-mode: t evil-surround-mode: t evil-lion-mode: t evil-goggles-mode: t global-undo-tree-mode: t undo-tree-mode: t shell-dirtrack-mode: t evil-mode: t evil-local-mode: t tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t prettify-symbols-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: /usr/share/emacs/26.3/lisp/emacs-lisp/let-alist hides /home/am/.emacs.d/straight/build/let-alist/let-alist /usr/share/emacs/site-lisp/mu4e/mu4e-headers hides /home/am/.emacs.d/straight/build/mu4e/mu4e-headers /usr/share/emacs/site-lisp/mu4e/mu4e-draft hides /home/am/.emacs.d/straight/build/mu4e/mu4e-draft /usr/share/emacs/site-lisp/mu4e/mu4e-lists hides /home/am/.emacs.d/straight/build/mu4e/mu4e-lists /usr/share/emacs/site-lisp/mu4e/mu4e-proc hides /home/am/.emacs.d/straight/build/mu4e/mu4e-proc /usr/share/emacs/site-lisp/mu4e/org-mu4e hides /home/am/.emacs.d/straight/build/mu4e/org-mu4e /usr/share/emacs/site-lisp/mu4e/mu4e-message hides /home/am/.emacs.d/straight/build/mu4e/mu4e-message /usr/share/emacs/site-lisp/mu4e/mu4e-compose hides /home/am/.emacs.d/straight/build/mu4e/mu4e-compose /usr/share/emacs/site-lisp/mu4e/mu4e-speedbar hides /home/am/.emacs.d/straight/build/mu4e/mu4e-speedbar /usr/share/emacs/site-lisp/mu4e/mu4e-vars hides /home/am/.emacs.d/straight/build/mu4e/mu4e-vars /usr/share/emacs/site-lisp/mu4e/mu4e-org hides /home/am/.emacs.d/straight/build/mu4e/mu4e-org /usr/share/emacs/site-lisp/mu4e/mu4e-mark hides /home/am/.emacs.d/straight/build/mu4e/mu4e-mark /usr/share/emacs/site-lisp/mu4e/mu4e-context hides /home/am/.emacs.d/straight/build/mu4e/mu4e-context /usr/share/emacs/site-lisp/mu4e/mu4e-view hides /home/am/.emacs.d/straight/build/mu4e/mu4e-view /usr/share/emacs/site-lisp/mu4e/mu4e-utils hides /home/am/.emacs.d/straight/build/mu4e/mu4e-utils /usr/share/emacs/site-lisp/mu4e/mu4e-contrib hides /home/am/.emacs.d/straight/build/mu4e/mu4e-contrib /usr/share/emacs/site-lisp/mu4e/mu4e-actions hides /home/am/.emacs.d/straight/build/mu4e/mu4e-actions /usr/share/emacs/site-lisp/mu4e/mu4e hides /home/am/.emacs.d/straight/build/mu4e/mu4e /usr/share/emacs/site-lisp/mu4e/mu4e-main hides /home/am/.emacs.d/straight/build/mu4e/mu4e-main /usr/share/emacs/site-lisp/mu4e/mu4e-icalendar hides /home/am/.emacs.d/straight/build/mu4e/mu4e-icalendar ~/Documents/git/treemacs/src/elisp/treemacs-interface hides /home/am/.emacs.d/straight/build/treemacs/treemacs-interface ~/Documents/git/treemacs/src/elisp/treemacs-rendering hides /home/am/.emacs.d/straight/build/treemacs/treemacs-rendering ~/Documents/git/treemacs/src/elisp/treemacs-dom hides /home/am/.emacs.d/straight/build/treemacs/treemacs-dom ~/Documents/git/treemacs/src/elisp/treemacs-tags hides /home/am/.emacs.d/straight/build/treemacs/treemacs-tags ~/Documents/git/treemacs/src/elisp/treemacs hides /home/am/.emacs.d/straight/build/treemacs/treemacs ~/Documents/git/treemacs/src/elisp/treemacs-workspaces hides /home/am/.emacs.d/straight/build/treemacs/treemacs-workspaces ~/Documents/git/treemacs/src/elisp/treemacs-customization hides /home/am/.emacs.d/straight/build/treemacs/treemacs-customization ~/Documents/git/treemacs/src/elisp/treemacs-faces hides /home/am/.emacs.d/straight/build/treemacs/treemacs-faces ~/Documents/git/treemacs/src/elisp/treemacs-themes hides /home/am/.emacs.d/straight/build/treemacs/treemacs-themes ~/Documents/git/treemacs/src/elisp/treemacs-mouse-interface hides /home/am/.emacs.d/straight/build/treemacs/treemacs-mouse-interface ~/Documents/git/treemacs/src/elisp/treemacs-mode hides /home/am/.emacs.d/straight/build/treemacs/treemacs-mode ~/Documents/git/treemacs/src/elisp/treemacs-icons hides /home/am/.emacs.d/straight/build/treemacs/treemacs-icons ~/Documents/git/treemacs/src/elisp/treemacs-compatibility hides /home/am/.emacs.d/straight/build/treemacs/treemacs-compatibility ~/Documents/git/treemacs/src/elisp/treemacs-follow-mode hides /home/am/.emacs.d/straight/build/treemacs/treemacs-follow-mode ~/Documents/git/treemacs/src/elisp/treemacs-visuals hides /home/am/.emacs.d/straight/build/treemacs/treemacs-visuals ~/Documents/git/treemacs/src/elisp/treemacs-core-utils hides /home/am/.emacs.d/straight/build/treemacs/treemacs-core-utils ~/Documents/git/treemacs/src/elisp/treemacs-extensions hides /home/am/.emacs.d/straight/build/treemacs/treemacs-extensions ~/Documents/git/treemacs/src/elisp/treemacs-filewatch-mode hides /home/am/.emacs.d/straight/build/treemacs/treemacs-filewatch-mode ~/Documents/git/treemacs/src/elisp/treemacs-persistence hides /home/am/.emacs.d/straight/build/treemacs/treemacs-persistence ~/Documents/git/treemacs/src/elisp/treemacs-async hides /home/am/.emacs.d/straight/build/treemacs/treemacs-async ~/Documents/git/treemacs/src/elisp/treemacs-bookmarks hides /home/am/.emacs.d/straight/build/treemacs/treemacs-bookmarks ~/Documents/git/treemacs/src/elisp/treemacs-tag-follow-mode hides /home/am/.emacs.d/straight/build/treemacs/treemacs-tag-follow-mode ~/Documents/git/treemacs/src/elisp/treemacs-logging hides /home/am/.emacs.d/straight/build/treemacs/treemacs-logging ~/Documents/git/treemacs/src/elisp/treemacs-fringe-indicator hides /home/am/.emacs.d/straight/build/treemacs/treemacs-fringe-indicator ~/Documents/git/treemacs/src/elisp/treemacs-diagnostics hides /home/am/.emacs.d/straight/build/treemacs/treemacs-diagnostics ~/Documents/git/treemacs/src/elisp/treemacs-macros hides /home/am/.emacs.d/straight/build/treemacs/treemacs-macros ~/Documents/git/treemacs/src/elisp/treemacs-scope hides /home/am/.emacs.d/straight/build/treemacs/treemacs-scope Features: (shadow sort mail-extr emacsbug cl-print debug benchmark eros company-quickhelp pos-tip ivy-posframe posframe swiper ivy delsel ivy-overlay colir help-fns radix-tree org-clock holidays hol-loaddefs diary-lib diary-loaddefs cal-iso smartparens-javascript js smartparens-html sgml-mode cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs smartparens-python python tramp-sh tramp tramp-compat tramp-loaddefs trampver ucs-normalize json map fish-mode conf-mode i3wm-config-mode writeroom-mode visual-fill-column autorevert org-indent disp-table ol-eww eww mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnir gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range gnus-win gnus nnheader ol-docview ol-bibtex bibtex ol-bbdb ol-w3m elfeed-org elfeed-show elfeed-search wid-edit treemacs-bookmarks bookmark elfeed-csv elfeed elfeed-curl url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf elfeed-log elfeed-db elfeed-lib url-queue xml-query mailcap vc-mtn vc-hg vc-git diff-mode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher rainbow-delimiters rainbow-mode xterm-color display-line-numbers elec-pair winner helm-command helm-elisp helm-eval edebug helm-info info shr-color color url-util url-parse url-vars shr svg dom face-remap pp helm-mode helm-projectile helm-files helm-tags helm-buffers helm-occur helm-grep helm-regexp helm-utils helm-locate helm-help helm-types framey-helm helm-config async-bytecomp helm helm-source helm-multi-match helm-lib async view treemacs-projectile projectile grep compile ibuf-ext ibuffer ibuffer-loaddefs mu4e-alert time alert log4e notifications dbus company-keywords company-dabbrev-code company-dabbrev company-yasnippet company-files company-capf company server evil-vimish-fold vimish-fold smartparens-config smartparens-org smartparens-text paren smartparens xml gntp org-mu4e mu4e desktop frameset mu4e-org german-holidays org-super-agenda ts org-habit org-element avl-tree generator org-agenda org-refile org-bullets org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete org-list org-faces org-entities noutline outline org-version ob-emacs-lisp ob-core ob-eval org-table ol org-keys org-compat org-macs org-loaddefs find-func mu4e-main mu4e-view cal-menu calendar cal-loaddefs browse-url mu4e-headers mu4e-compose mu4e-context mu4e-draft mu4e-actions ido rfc2368 smtpmail auth-source sendmail mu4e-mark mu4e-message flow-fill mu4e-proc mu4e-utils doc-view jka-compr mu4e-lists mu4e-vars message rmc puny seq treemacs-icons-dired treemacs-evil treemacs treemacs-header-line treemacs-compatibility treemacs-mode treemacs-interface treemacs-extensions treemacs-persistence treemacs-mouse-interface treemacs-tag-follow-mode treemacs-filewatch-mode treemacs-tags imenu xref project filenotify treemacs-follow-mode treemacs-rendering treemacs-async treemacs-workspaces treemacs-dom treemacs-visuals treemacs-fringe-indicator treemacs-scope treemacs-faces treemacs-icons treemacs-themes treemacs-core-utils pfuture ace-window avy hl-line treemacs-macros treemacs-logging treemacs-customization easy-mmode dired+ image-dired image-mode image-file dired-x dired-aux dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader mu4e-meta gcmh hydra lv pcase doom-modeline doom-modeline-segments doom-modeline-env doom-modeline-core shrink-path rx f all-the-icons all-the-icons-faces data-material data-weathericons data-octicons data-fileicons data-faicons data-alltheicons memoize subr-x framey inline ht s window-purpose window-purpose-fixes window-purpose-prefix-overload window-purpose-switch let-alist window-purpose-layout window-purpose-core window-purpose-configuration eieio-compat eieio byte-opt bytecomp byte-compile cconv window-purpose-utils shackle trace cl-extra help-mode winum eyebrowse format-spec morning-star-theme cap-words superword subword evil-surround evil-lion evil-goggles pulse cl-seq evil evil-keybindings evil-integration undo-tree diff evil-maps evil-commands reveal flyspell ispell evil-jumps evil-command-window evil-types evil-search evil-ex shell pcomplete comint ansi-color evil-macros evil-repeat evil-states evil-core advice evil-common windmove thingatpt rect evil-digraphs evil-vars ring edmacro kmacro dash yasnippet-autoloads yaml-mode-autoloads xterm-color-autoloads wttrin-autoloads writeroom-mode-autoloads with-editor-autoloads winum-autoloads window-purpose-autoloads eieio-core cl-macs gv eieio-loaddefs cl-loaddefs cl-lib vterm-autoloads visual-fill-column-autoloads vimish-fold-autoloads undo-tree-autoloads ts-autoloads tridactylrc-mode-autoloads treepy-autoloads treemacs-autoloads transient-autoloads swiper-autoloads straight-autoloads spinner-autoloads smartparens-autoloads shrink-path-autoloads shackle-autoloads s-autoloads rainbow-mode-autoloads rainbow-delimiters-autoloads projectile-autoloads pretty-hydra-autoloads posframe-autoloads pos-tip-autoloads popup-autoloads pkg-info-autoloads pfuture-autoloads persp-mode-autoloads org-super-agenda-autoloads org-bullets-autoloads org-autoloads multi-compile-autoloads mu4e-alert-autoloads mu4e-autoloads morning-star-autoloads memoize-autoloads markdown-mode-autoloads magit-autoloads macrostep-autoloads lv-autoloads lsp-ui-autoloads lsp-treemacs-autoloads lsp-mode-autoloads loop-autoloads log4e-autoloads let-alist-autoloads ledger-mode-autoloads ivy-posframe-autoloads ivy-autoloads imenu-list-autoloads i3wm-config-mode-autoloads hydra-autoloads ht-autoloads helpful-autoloads helm-projectile-autoloads helm-org-autoloads helm-easymenu easymenu helm-core-autoloads helm-ag-autoloads helm-autoloads goto-chg-autoloads gntp-autoloads git-commit-autoloads ghub-autoloads german-holidays-autoloads gcmh-autoloads framey-autoloads forge-autoloads flycheck-pos-tip-autoloads flycheck-autoloads fish-mode-autoloads fill-column-indicator-autoloads f-autoloads eyebrowse-autoloads expand-region-autoloads evil-vimish-fold-autoloads evil-surround-autoloads evil-nerd-commenter-autoloads evil-magit-autoloads evil-lion-autoloads evil-ledger-autoloads evil-goggles-autoloads evil-exchange-autoloads evil-collection-autoloads evil-autoloads eros-autoloads epl-autoloads emacsql-sqlite-autoloads emacsql-autoloads elisp-refs-autoloads elfeed-org-autoloads elfeed-autoloads doom-modeline-autoloads doct-autoloads dired+-autoloads dash-functional-autoloads dash-autoloads company-shell-autoloads company-quickhelp-autoloads company-makefile-autoloads company-box-autoloads company-autoloads closql-autoloads buttercup-autoloads avy-autoloads async-autoloads anzu-autoloads annalist-autoloads all-the-icons-autoloads alert-autoloads ace-window-autoloads time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 1134735 336269) (symbols 48 62609 1) (miscs 40 669 2944) (strings 32 246316 89817) (string-bytes 1 7685472) (vectors 16 113372) (vector-slots 8 2276151 279318) (floats 8 6416 2552) (intervals 56 38016 2345) (buffers 992 44))
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.