GNU bug report logs - #15695
Incomplete data from subprocesses with pty connections

Previous Next

To reply to this bug, email your comments to 15695 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Richard <theonewiththeevillook <at> yahoo.fr>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3; multiple async shell commands at once run randomly
Date: Wed, 23 Oct 2013 23:15:53 +0200

Hi,

Trying to debug a problem when pressing `&' in dired-mode with multiple
files marked, I found that running the following
$ emacs -q --eval '(shell-command "echo foo&echo bar&echo baz&")'
shows a *Async Shell Command* buffer with apparent random content.

I ran elisp which does something like the above 820 times. Here's what
happened (formatted as "output: number of times it appeared") :
"baz\nfoo\n": 236
"baz\nfoo\nbar\n": 73
"baz\n": 469
"baz\nbar\n": 29
"bar\nbaz\n": 5
"baz\nbar\nfoo\n": 1
"foo\nbaz\n": 6
"foo\nbar\nbaz\n": 1
which looks like random to me.

What I expected is to only see the lines with all of "foo" "baz" and
"bar" (in any order).

I have to say that it's not only a problem of output not seen in emacs :
replacing "echo" by "touch" like so:
$ rm -f foo bar baz; emacs -q --batch --eval '(shell-command "touch foo&touch bar&touch baz&")' --eval '(sit-for 1)'; ls -la
usually gives me only one or two files, instead of three (in an otherwise
empty dir)

Browsing through other bugs, I spotted the followings bug report which might be related:
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=421

I run emacs --version : GNU Emacs 24.3.1
and GNU bash, version 4.2.45(1)-release (i686-pc-linux-gnu)

Here's the elisp used for the above statistics -- it uses
"start-process" directly instead of shell-command.

#+BEGIN_SRC emacs-lisp
  (defvar statistics nil "Hash table for keeping statistics")
  (defun yf/keep-stats (process change) "Used as sentinel"
    (with-current-buffer (process-buffer process)
      (if (equal change "finished\n")
          (puthash (buffer-string) (1+ (or (gethash (buffer-string) statistics) 0)) statistics)
        (error "Unknown change for process %s: %s" process change))))
  (defun yf/pp-hash-table (hashtable) "Pretty print hash table"
    (let ((res))
      (maphash (lambda (k v) (setq res (concat res (yf/pp-hash--line k v)))) hashtable)
      (or res "")))
  (defun yf/pp-hash--line (key value) "Pretty print one hash line"
    (format "%s: %s\n" (pp key) (pp value)))
  (progn 
    (let ((n 820))
      (setq statistics (make-hash-table :test 'equal))
      (dotimes (i n)
        (let* ((buffername (concat "Shell-test" (number-to-string i)))
               (buffer (get-buffer-create buffername)))
          (with-current-buffer buffer
            (erase-buffer))
          (set-process-sentinel
           (start-process buffername
                          buffer
                          shell-file-name
                          shell-command-switch "echo foo&echo bar&echo baz")
           #'yf/keep-stats)))
      t)
    (sit-for 2)
    (insert (yf/pp-hash-table statistics)))
#+END_SRC

In GNU Emacs 24.3.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2013-10-14 on geodiff-mac3
Windowing system distributor `The X.Org Foundation', version 11.0.11304000
Configured using:
 `configure '--with-x-toolkit=lucid' '--enable-checking''

Important settings:
  value of $LANG: fr_FR.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  yas-global-mode: t
  TeX-PDF-mode: t
  diff-auto-refine-mode: t
  projectile-global-mode: t
  rcirc-track-minor-mode: t
  desktop-save-mode: t
  dynamic-completion-mode: t
  shell-dirtrack-mode: t
  show-paren-mode: t
  recentf-mode: t
  winner-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
SPC C-e C-d C-e SPC ? C-n C-SPC <C-end> C-w <return> 
<backspace> <backspace> SPC I t ' s SPC b u i l t SPC 
<backspace> - i n SPC f o r SPC m e SPC C-u M-x e m 
a c s - v e r s i o n <return> C-x C-x M-f M-f M-f 
M-f M-f M-f M-f M-f M-f C-f C-f C-M-k SPC t o o C-k 
. <up> <C-end> <up> <end> <C-left> <left> SPC o n M-q 
<C-end> <return> <return> N i <backspace> <backspace> 
N i c o . <return> <up> - - SPC <return> <end> <C-end> 
<up> <up> <end> <up> <up> <up> <up> <up> <up> <up> 
<down> <home> C-SPC <C-home> <C-down> <C-down> C-w 
<C-end> C-c C-c / N q g <C-home> <return> <return> 
n n n SPC SPC SPC SPC SPC SPC SPC SPC SPC SPC n n n 
n k q <return> q C-n C-n C-n C-n C-n C-n <return> c 
C-g C-n <return> F <C-end> <up> <up> <up> C-SPC <C-end> 
C-w <return> D i d SPC y o u SPC t r y SPC b y SPC 
r e c o m p i l i n g SPC r c i r c . e l SPC ? M-b 
M-b M-x f i n d SPC l i b <tab> <return> <return> C-x 
K M-b M-b M-d C-d C-e C-n C-p C-n C-c C-k y e s <return> 
C-p <return> c y g <return> <return> q <return> n n 
n n n C-p <return> ^ q g <return> <return> , c y q 
g C-g q y C-c C-SPC ESC [ > 1 ; 3 2 0 1 ; 0 c ESC x 
r e p o r t SPC e m TAB SPC b u DEL DEL TAB RET

Recent messages:
Are you sure you want to quit reading news? (y or n)  y
Saving file /home/youngfrog/.newsrc...
Wrote /home/youngfrog/.newsrc
Saving /home/youngfrog/.newsrc.eld...
Saving file /home/youngfrog/.newsrc.eld...
Wrote /home/youngfrog/.newsrc.eld
Saving /home/youngfrog/.newsrc.eld...done
Cleaning up the recentf list...done (0 removed)
Contacting host: www.emacswiki.org:80 [5 times]
When done with this frame, type C-x 5 0

Load-path shadows:
/home/youngfrog/.emacs.d/elpa/ess-20130912.915/.dir-locals hides /home/youngfrog/.emacs.d/elpa/pkg-info-20130817.2334/.dir-locals
~/.emacs.d/lisp/asy-mode hides /usr/local/texlive/2012/texmf/asymptote/asy-mode
/home/youngfrog/.emacs.d/elpa/json-1.2/json hides /usr/local/share/emacs/24.3/lisp/json
~/sources/org-mode/lisp/org-footnote hides /usr/local/share/emacs/24.3/lisp/org/org-footnote
~/sources/org-mode/lisp/ob-io hides /usr/local/share/emacs/24.3/lisp/org/ob-io
~/sources/org-mode/lisp/ob-sqlite hides /usr/local/share/emacs/24.3/lisp/org/ob-sqlite
~/sources/org-mode/lisp/ob-ditaa hides /usr/local/share/emacs/24.3/lisp/org/ob-ditaa
~/sources/org-mode/lisp/ob-ocaml hides /usr/local/share/emacs/24.3/lisp/org/ob-ocaml
~/sources/org-mode/lisp/org-irc hides /usr/local/share/emacs/24.3/lisp/org/org-irc
~/sources/org-mode/lisp/ob-eval hides /usr/local/share/emacs/24.3/lisp/org/ob-eval
~/sources/org-mode/lisp/ob-scheme hides /usr/local/share/emacs/24.3/lisp/org/ob-scheme
~/sources/org-mode/lisp/org-capture hides /usr/local/share/emacs/24.3/lisp/org/org-capture
~/sources/org-mode/lisp/org-table hides /usr/local/share/emacs/24.3/lisp/org/org-table
~/sources/org-mode/lisp/ob-exp hides /usr/local/share/emacs/24.3/lisp/org/ob-exp
~/sources/org-mode/lisp/org-eshell hides /usr/local/share/emacs/24.3/lisp/org/org-eshell
~/sources/org-mode/lisp/org-inlinetask hides /usr/local/share/emacs/24.3/lisp/org/org-inlinetask
~/sources/org-mode/lisp/ob-sql hides /usr/local/share/emacs/24.3/lisp/org/ob-sql
~/sources/org-mode/lisp/ob-emacs-lisp hides /usr/local/share/emacs/24.3/lisp/org/ob-emacs-lisp
~/sources/org-mode/contrib/lisp/org-vm hides /usr/local/share/emacs/24.3/lisp/org/org-vm
~/sources/org-mode/lisp/org-list hides /usr/local/share/emacs/24.3/lisp/org/org-list
~/sources/org-mode/lisp/org-colview hides /usr/local/share/emacs/24.3/lisp/org/org-colview
~/sources/org-mode/lisp/ob-perl hides /usr/local/share/emacs/24.3/lisp/org/ob-perl
~/sources/org-mode/lisp/org-agenda hides /usr/local/share/emacs/24.3/lisp/org/org-agenda
~/sources/org-mode/lisp/ob-keys hides /usr/local/share/emacs/24.3/lisp/org/ob-keys
~/sources/org-mode/lisp/org-element hides /usr/local/share/emacs/24.3/lisp/org/org-element
~/sources/org-mode/lisp/ob-table hides /usr/local/share/emacs/24.3/lisp/org/ob-table
~/sources/org-mode/lisp/org-compat hides /usr/local/share/emacs/24.3/lisp/org/org-compat
~/sources/org-mode/lisp/org-mobile hides /usr/local/share/emacs/24.3/lisp/org/org-mobile
~/sources/org-mode/lisp/ob-comint hides /usr/local/share/emacs/24.3/lisp/org/ob-comint
~/sources/org-mode/lisp/org-src hides /usr/local/share/emacs/24.3/lisp/org/org-src
~/sources/org-mode/lisp/org-crypt hides /usr/local/share/emacs/24.3/lisp/org/org-crypt
~/sources/org-mode/lisp/org-info hides /usr/local/share/emacs/24.3/lisp/org/org-info
~/sources/org-mode/lisp/ob-maxima hides /usr/local/share/emacs/24.3/lisp/org/ob-maxima
~/sources/org-mode/lisp/org-ctags hides /usr/local/share/emacs/24.3/lisp/org/org-ctags
~/sources/org-mode/lisp/org-habit hides /usr/local/share/emacs/24.3/lisp/org/org-habit
~/sources/org-mode/lisp/ob hides /usr/local/share/emacs/24.3/lisp/org/ob
~/sources/org-mode/lisp/org-datetree hides /usr/local/share/emacs/24.3/lisp/org/org-datetree
~/sources/org-mode/lisp/org-clock hides /usr/local/share/emacs/24.3/lisp/org/org-clock
~/sources/org-mode/lisp/org-bbdb hides /usr/local/share/emacs/24.3/lisp/org/org-bbdb
~/sources/org-mode/lisp/ob-fortran hides /usr/local/share/emacs/24.3/lisp/org/ob-fortran
~/sources/org-mode/lisp/ob-picolisp hides /usr/local/share/emacs/24.3/lisp/org/ob-picolisp
~/sources/org-mode/contrib/lisp/org-wl hides /usr/local/share/emacs/24.3/lisp/org/org-wl
~/sources/org-mode/lisp/org-gnus hides /usr/local/share/emacs/24.3/lisp/org/org-gnus
~/sources/org-mode/lisp/ob-clojure hides /usr/local/share/emacs/24.3/lisp/org/ob-clojure
~/sources/org-mode/contrib/lisp/org-mew hides /usr/local/share/emacs/24.3/lisp/org/org-mew
~/sources/org-mode/lisp/org-attach hides /usr/local/share/emacs/24.3/lisp/org/org-attach
~/sources/org-mode/lisp/ob-tangle hides /usr/local/share/emacs/24.3/lisp/org/ob-tangle
~/sources/org-mode/lisp/org-protocol hides /usr/local/share/emacs/24.3/lisp/org/org-protocol
~/sources/org-mode/lisp/org-indent hides /usr/local/share/emacs/24.3/lisp/org/org-indent
~/sources/org-mode/lisp/org-install hides /usr/local/share/emacs/24.3/lisp/org/org-install
~/sources/org-mode/lisp/ob-org hides /usr/local/share/emacs/24.3/lisp/org/ob-org
~/sources/org-mode/lisp/org-docview hides /usr/local/share/emacs/24.3/lisp/org/org-docview
~/sources/org-mode/lisp/org-rmail hides /usr/local/share/emacs/24.3/lisp/org/org-rmail
~/sources/org-mode/lisp/ob-mscgen hides /usr/local/share/emacs/24.3/lisp/org/ob-mscgen
~/sources/org-mode/lisp/org-plot hides /usr/local/share/emacs/24.3/lisp/org/org-plot
~/sources/org-mode/lisp/ob-haskell hides /usr/local/share/emacs/24.3/lisp/org/ob-haskell
~/sources/org-mode/lisp/org-timer hides /usr/local/share/emacs/24.3/lisp/org/org-timer
~/sources/org-mode/lisp/ob-ledger hides /usr/local/share/emacs/24.3/lisp/org/ob-ledger
~/sources/org-mode/lisp/org-mhe hides /usr/local/share/emacs/24.3/lisp/org/org-mhe
~/sources/org-mode/lisp/org-entities hides /usr/local/share/emacs/24.3/lisp/org/org-entities
~/sources/org-mode/lisp/ob-C hides /usr/local/share/emacs/24.3/lisp/org/ob-C
~/sources/org-mode/lisp/org-mouse hides /usr/local/share/emacs/24.3/lisp/org/org-mouse
~/sources/org-mode/lisp/ob-lisp hides /usr/local/share/emacs/24.3/lisp/org/ob-lisp
~/sources/org-mode/lisp/org-archive hides /usr/local/share/emacs/24.3/lisp/org/org-archive
~/sources/org-mode/lisp/ob-python hides /usr/local/share/emacs/24.3/lisp/org/ob-python
~/sources/org-mode/lisp/ob-ref hides /usr/local/share/emacs/24.3/lisp/org/ob-ref
~/sources/org-mode/lisp/ob-gnuplot hides /usr/local/share/emacs/24.3/lisp/org/ob-gnuplot
~/sources/org-mode/lisp/ob-sass hides /usr/local/share/emacs/24.3/lisp/org/ob-sass
~/sources/org-mode/lisp/ob-sh hides /usr/local/share/emacs/24.3/lisp/org/ob-sh
~/sources/org-mode/lisp/ob-octave hides /usr/local/share/emacs/24.3/lisp/org/ob-octave
~/sources/org-mode/lisp/org-pcomplete hides /usr/local/share/emacs/24.3/lisp/org/org-pcomplete
~/sources/org-mode/lisp/org-w3m hides /usr/local/share/emacs/24.3/lisp/org/org-w3m
~/sources/org-mode/lisp/org-feed hides /usr/local/share/emacs/24.3/lisp/org/org-feed
~/sources/org-mode/lisp/ob-calc hides /usr/local/share/emacs/24.3/lisp/org/ob-calc
~/sources/org-mode/lisp/ob-shen hides /usr/local/share/emacs/24.3/lisp/org/ob-shen
~/sources/org-mode/lisp/ob-plantuml hides /usr/local/share/emacs/24.3/lisp/org/ob-plantuml
~/sources/org-mode/lisp/org-version hides /usr/local/share/emacs/24.3/lisp/org/org-version
~/sources/org-mode/lisp/ob-R hides /usr/local/share/emacs/24.3/lisp/org/ob-R
~/sources/org-mode/lisp/ob-asymptote hides /usr/local/share/emacs/24.3/lisp/org/ob-asymptote
~/sources/org-mode/lisp/ob-latex hides /usr/local/share/emacs/24.3/lisp/org/ob-latex
~/sources/org-mode/lisp/ob-dot hides /usr/local/share/emacs/24.3/lisp/org/ob-dot
~/sources/org-mode/lisp/ob-matlab hides /usr/local/share/emacs/24.3/lisp/org/ob-matlab
~/sources/org-mode/lisp/ob-ruby hides /usr/local/share/emacs/24.3/lisp/org/ob-ruby
~/sources/org-mode/lisp/org-id hides /usr/local/share/emacs/24.3/lisp/org/org-id
~/sources/org-mode/lisp/ob-java hides /usr/local/share/emacs/24.3/lisp/org/ob-java
~/sources/org-mode/lisp/org-faces hides /usr/local/share/emacs/24.3/lisp/org/org-faces
~/sources/org-mode/lisp/ob-css hides /usr/local/share/emacs/24.3/lisp/org/ob-css
~/sources/org-mode/lisp/org-bibtex hides /usr/local/share/emacs/24.3/lisp/org/org-bibtex
~/sources/org-mode/lisp/ob-screen hides /usr/local/share/emacs/24.3/lisp/org/ob-screen
~/sources/org-mode/lisp/org hides /usr/local/share/emacs/24.3/lisp/org/org
~/sources/org-mode/lisp/ob-scala hides /usr/local/share/emacs/24.3/lisp/org/ob-scala
~/sources/org-mode/lisp/ob-lob hides /usr/local/share/emacs/24.3/lisp/org/ob-lob
~/sources/org-mode/lisp/org-macs hides /usr/local/share/emacs/24.3/lisp/org/org-macs
~/sources/org-mode/lisp/ob-awk hides /usr/local/share/emacs/24.3/lisp/org/ob-awk
~/sources/org-mode/lisp/ob-lilypond hides /usr/local/share/emacs/24.3/lisp/org/ob-lilypond
~/sources/org-mode/lisp/ob-js hides /usr/local/share/emacs/24.3/lisp/org/ob-js
~/sources/org-mode/lisp/org-loaddefs hides /usr/local/share/emacs/24.3/lisp/org/org-loaddefs
/home/youngfrog/.emacs.d/elpa/ess-20130912.915/.dir-locals hides /usr/local/share/emacs/24.3/lisp/gnus/.dir-locals

Features:
(shadow emacsbug org-datetree uudecode calc-stuff calccomp calc-forms
calc-help calc-undo calc-alg calc-yank clippy pos-tip clippy-autoloads
pos-tip-autoloads skeleton helm-sys helm-utils helm haskell-font-lock
haskell-mode haskell-string haskell-sort-imports haskell-align-imports
flymake files-x ruler-mode hl-line hexl eldoc table loadhist calc-arith
calc-math calc-ext calc-misc calc-menu calc-aent calc calc-loaddefs
calc-macs ox-beamer ox-latex ox-icalendar ox-html ox-ascii ox-publish ox
dmassign hi-lock face-remap sgml-mode htmlize org-feed cal-move vc-git
org-attach org-id org-colview cal-china lunar solar cal-dst cal-bahai
cal-islam cal-hebrew holidays hol-loaddefs diary-lib diary-loaddefs
cal-iso finder yasnippet nlinum linum tutorial smerge-mode url-handlers
org-download cus-edit image-file sh-script smie executable newst-backend
enriched gnus-draft pcmpl-unix tex-buf bookmark tramp-cmds mule-util
org-element org-indent top-mode async-file dired-aux async
async-autoloads ispell tabify man js2-mode js imenu json-mode
json-mode-autoloads debbugs-gnu debbugs soap-client find-dired
descr-text two-column iso-transl smartparens mailalias smtpmail sendmail
boxquote conf-mode cus-start cus-load nyan-mode nyan-mode-autoloads
tar-mode lorem-ipsum-autoloads autoload lisp-mnt ibuf-ext ibuffer
url-queue canlock fic-ext-mode cc-langs asy-mode cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
edebug quail texmathp timezone url-http url-gw url-cache url-auth
apropos rect google-translate json reftex-dcr reftex-auc reftex
reftex-vars org-rmail org-mhe org-irc org-info org-gnus org-docview
org-bibtex bibtex org-bbdb org-w3m org-table font-latex latex tex-style
tex dbus crm dabbrev misearch multi-isearch pp shr-color color shr
browse-url sort gnus-cite flow-fill mm-archive mail-extr gnus-async
gnus-bcklg qp gnus-ml nndraft nnmh gnutls nnfolder parse-time netrc
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg nntp
gnus-cache time-stamp add-log view help-mode debug magit-cherry
magit-bisect magit-key-mode magit magit-version diff-mode
git-rebase-mode rx git-commit-mode magit-compat tramp-cache tramp-sh
tramp tramp-compat tramp-loaddefs paredit hideshow
ack-and-a-half-autoloads ampc-autoloads auto-dim-other-buffers-autoloads
boxquote-autoloads circe-autoloads color-theme-solarized-autoloads
color-theme-autoloads crosshairs-autoloads col-highlight-autoloads
csv-mode-autoloads csv-nav-autoloads debbugs-autoloads
dictionary-autoloads dropbox-autoloads dtrt-indent-autoloads
eimp-autoloads elnode-autoloads db-autoloads creole-autoloads
elpy-autoloads auto-complete-autoloads fakir-autoloads
fic-ext-mode-autoloads find-file-in-project-autoloads flx-ido-autoloads
flx-autoloads fuzzy-autoloads gist-autoloads gh-autoloads
git-commit-mode-autoloads git-rebase-mode-autoloads
golden-ratio-autoloads google-translate-autoloads hackernews-autoloads
haskell-mode-autoloads helm-projectile-autoloads helm-autoloads
highlight-indentation-autoloads hl-line+-autoloads idomenu-autoloads
iedit-autoloads jabber-autoloads js2-mode-autoloads json-autoloads
kv-autoloads lcs-autoloads litable-autoloads log4j-mode-autoloads
logito-autoloads lua-mode-autoloads lui-autoloads
lusty-explorer-autoloads mark-tools-autoloads memory-usage-autoloads
minimap-autoloads mo-git-blame-autoloads multifiles-autoloads
multiple-cursors-autoloads mwe-log-commands-autoloads nlinum-autoloads
noflet-autoloads nose-autoloads oauth-autoloads org-email-autoloads
paredit-autoloads pastebin-autoloads pcache-autoloads pcsv-autoloads
popup-autoloads powerline-autoloads powerline powerline-separators
powerline-themes pretty-mode-autoloads projectile grep compile dash s
thingatpt projectile-autoloads pkg-info-autoloads pymacs-autoloads
python-mode-autoloads pyvirtualenv-autoloads
rainbow-delimiters-autoloads rainbow-mode-autoloads rebox2-autoloads
request-autoloads s-autoloads scala-mode-autoloads
shell-switcher-autoloads sicp-autoloads slime-autoloads
smartparens-autoloads dash-autoloads finder-inf smex-autoloads
sr-speedbar-autoloads switch-window-autoloads top-mode-autoloads
tracking-autoloads shorten-autoloads typing-autoloads
undo-tree-autoloads use-package-autoloads diminish-autoloads
bind-key-autoloads virtualenv-autoloads vline-autoloads w3-autoloads
web-autoloads window-number-autoloads workspaces-autoloads wtf-autoloads
yasnippet-autoloads zen-and-art-theme-autoloads package disp-table
network-stream starttls rcirc-color rcirc twittering-mode epa derived
epg tls url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util xml desktop yf/makeadefun completion
pcase ob-R ob-sh shell windmove latexenc paren doc-view jka-compr
image-mode uniquify recentf tree-widget org-contacts org-capture
org-agenda gnus-art mm-uu mml2015 epg-config mm-view mml-smime smime dig
mailcap gnus-sum nnoo gnus-group gnus-undo nnmail mail-source gnus-start
gnus-spec gnus-int gnus-range message cl-macs gv rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader mail-utils wid-edit cl find-lisp dired org-inlinetask org
warnings org-macro org-footnote org-pcomplete pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp ob-comint comint
ansi-color ob-core ob-eval org-compat org-macs format-spec find-func
cal-menu calendar cal-loaddefs time ffap url-parse auth-source eieio
byte-opt bytecomp byte-compile cconv gnus-util time-date mm-util
mail-prsvr password-cache url-vars winner ring edmacro kmacro
magit-autoloads two-mode-mode tex-site auto-loads org-loaddefs ido-hacks
advice help-fns cl-lib advice-preload easy-mmode ido info easymenu
server tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting x-toolkit x
multi-tty emacs)

Message #8 received at 15695 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Nicolas Richard <theonewiththeevillook <at> yahoo.fr>
Cc: 15695 <at> debbugs.gnu.org
Subject: Re: bug#15695: 24.3;
 multiple async shell commands at once run randomly
Date: Thu, 24 Oct 2013 03:07:06 -0400

Nicolas Richard wrote:

> Browsing through other bugs, I spotted the followings bug report which
> might be related:
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=421

I don't think that's related; that's specific to gnome-open:
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=421#10

  I straced gnome-open and it ends with a clone statement, so I suppose
  emacs gets confused by the original process dying.

(I don't think 421 is an Emacs bug; but that's off-topic for here.)

Message #15 received at 15695 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Nicolas Richard <theonewiththeevillook <at> yahoo.fr>
Cc: 15695 <at> debbugs.gnu.org
Subject: Re: bug#15695: 24.3;
 multiple async shell commands at once run randomly
Date: Mon, 29 Jan 2018 08:52:50 -0500

retitle 15695 Incomplete data from subprocesses with pty connections
quit

Nicolas Richard <theonewiththeevillook <at> yahoo.fr> writes:

>   (defvar statistics nil "Hash table for keeping statistics")
>   (defun yf/keep-stats (process change) "Used as sentinel"
>     (with-current-buffer (process-buffer process)
>       (if (equal change "finished\n")
>           (puthash (buffer-string) (1+ (or (gethash (buffer-string) statistics) 0)) statistics)
>         (error "Unknown change for process %s: %s" process change))))
>   (defun yf/pp-hash-table (hashtable) "Pretty print hash table"
>     (let ((res))
>       (maphash (lambda (k v) (setq res (concat res (yf/pp-hash--line k v)))) hashtable)
>       (or res "")))
>   (defun yf/pp-hash--line (key value) "Pretty print one hash line"
>     (format "%s: %s\n" (pp key) (pp value)))
>   (progn 
>     (let ((n 820))
>       (setq statistics (make-hash-table :test 'equal))
>       (dotimes (i n)
>         (let* ((buffername (concat "Shell-test" (number-to-string i)))
>                (buffer (get-buffer-create buffername)))
>           (with-current-buffer buffer
>             (erase-buffer))
>           (set-process-sentinel
>            (start-process buffername
>                           buffer
>                           shell-file-name
>                           shell-command-switch "echo foo&echo bar&echo baz")
>            #'yf/keep-stats)))
>       t)
>     (sit-for 2)
>     (insert (yf/pp-hash-table statistics)))

If I set process-connection-type to nil, then I get expected results
(though I had to reduce n in order to avoid bumping into the file
descriptor limit):

"baz\nfoo\nbar\n": 58
"baz\nbar\nfoo\n": 41
"foo\nbaz\nbar\n": 1

Without setting process-connection-type (all tests with 26.0.91):

"baz\n": 86
"baz\nbar\n": 5
"baz\nfoo\n": 6
"foo\nbaz\n": 1
"baz\nfoo\nbar\n": 2

This problem is also talked about in [1] and [2].

[1]: http://nullprogram.com/blog/2018/01/17/
[2]: http://nullprogram.com/blog/2014/02/06/#pseudo-terminals

Message #20 received at 15695 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: 15695 <at> debbugs.gnu.org
Cc: Nicolas Richard <theonewiththeevillook <at> yahoo.fr>
Subject: Re: bug#15695: 24.3;
 multiple async shell commands at once run randomly
Date: Mon, 02 Apr 2018 19:49:27 -0400

merge 15695 16019
quit

> If I set process-connection-type to nil, then I get expected results

> This problem is also talked about in [1] and [2].
>
> [1]: http://nullprogram.com/blog/2018/01/17/
> [2]: http://nullprogram.com/blog/2014/02/06/#pseudo-terminals

Actually, I don't think this bug is a problem of Emacs losing data after
all, it's just that the shell does the 'echo baz' and then usually exits
before the other echo commands get to run (when running with a pty,
Emacs sends SIGHUP to all child process in the terminal).  If I add a ";
wait" to the shell command, then I get

"baz\nbar\nfoo\n": 374
"baz\nfoo\nbar\n": 428
"foo\nbaz\nbar\n": 17
"bar\nfoo\nbaz\n": 1

which is as expected.  I'm merging with #16019 which is about Emacs
sending SIGHUP to all child processes of a pty subprocess.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.