From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: me@alandmoore.com Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 26 Jan 2017 22:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 25549@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.14854708502615 (code B ref -1); Thu, 26 Jan 2017 22:48:02 +0000 Received: (at submit) by debbugs.gnu.org; 26 Jan 2017 22:47:30 +0000 Received: from localhost ([127.0.0.1]:46442 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cWspF-0000g7-Qq for submit@debbugs.gnu.org; Thu, 26 Jan 2017 17:47:30 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cWspE-0000fu-7X for submit@debbugs.gnu.org; Thu, 26 Jan 2017 17:47:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cWsp3-0001QW-FR for submit@debbugs.gnu.org; Thu, 26 Jan 2017 17:47:23 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:36828) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cWsp3-0001QP-CK for submit@debbugs.gnu.org; Thu, 26 Jan 2017 17:47:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35050) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cWsox-0001sb-8T for bug-gnu-emacs@gnu.org; Thu, 26 Jan 2017 17:47:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cWsor-0001LY-Ad for bug-gnu-emacs@gnu.org; Thu, 26 Jan 2017 17:47:11 -0500 Received: from mx1.williamson-tn.org ([72.237.174.11]:60854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cWsor-0001Jh-3N for bug-gnu-emacs@gnu.org; Thu, 26 Jan 2017 17:47:05 -0500 X-ASG-Debug-ID: 1485468447-0a345f7be62ca960001-HqB3X1 Received: from mailman.williamson-tn.org ([192.168.1.6]) by mx1.williamson-tn.org with ESMTP id r1vxMrhXwTNSIkB3 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 26 Jan 2017 16:07:27 -0600 (CST) X-Barracuda-Envelope-From: me@alandmoore.com X-Barracuda-RBL-Trusted-Forwarder: 192.168.1.6 Received: from it-alanm.williamson-tn.org ([192.168.88.40]) by mailman.williamson-tn.org with ESMTP (TLS encrypted); Thu, 26 Jan 2017 16:07:08 -0600 From: me@alandmoore.com X-Barracuda-RBL-IP: 192.168.88.40 Date: Thu, 26 Jan 2017 16:06:44 -0600 X-ASG-Orig-Subj: 25.1; eshell grep gives inconsistent output Message-ID: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> MIME-Version: 1.0 Content-Type: text/plain X-Barracuda-Connect: UNKNOWN[192.168.1.6] X-Barracuda-Start-Time: 1485468447 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://192.168.1.65:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 10940 X-Virus-Scanned: by bsmtpd at williamson-tn.org X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=NO_REAL_NAME X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.36096 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 NO_REAL_NAME From: does not include a real name X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) When using grep on a large file in eshell, the output is sometimes inconsistent. For example, when grepping for a string in my server's apache2 access log (~5 MB) I got anywhere from around 200 to around 1300 results, just running the same grep command over and over (the file itself was not changing significantly). The bug cannot be consistently reproduced, but has happened in multiple situations and others have reportedly reproduced it per discussion on reddit (https://www.reddit.com/r/emacs/comments/5qbd70/trampeshellgrep_returning_wildly_different/). In GNU Emacs 25.1.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.22.5) of 2016-12-23 built on juergen Windowing system distributor 'The X.Org Foundation', version 11.0.11901000 System Description: Arch Linux Configured using: 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --with-x-toolkit=gtk3 --with-xft 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong' CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 Important settings: value of $LANG: en_US.utf8 locale-coding-system: utf-8-unix Major mode: EShell Minor modes in effect: magit-auto-revert-mode: t global-git-commit-mode: t diff-auto-refine-mode: t global-flycheck-mode: t pyvenv-mode: t helm-mode: t shell-dirtrack-mode: t async-bytecomp-package-mode: t delete-selection-mode: t cua-mode: t electric-pair-mode: t pdf-occur-global-minor-mode: t show-paren-mode: t tabbar-mwheel-mode: t tabbar-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t column-number-mode: t line-number-mode: t transient-mark-mode: t Recent messages: Git finished [2 times] C-c p is undefined Quit [C-t] show common commands, [?] describe events, [C-h i] show manual Running git push -v gl master:refs/heads/master [2 times] Git finished [C-t] show common commands, [?] describe events, [C-h i] show manual Running git push -v production master:refs/heads/master [2 times] Git finished [2 times] Making completion list... Load-path shadows: ~/.emacs.d/elisp/org2blog/metaweblog hides /home/alanm/.emacs.d/elpa/metaweblog-20141130.605/metaweblog /home/alanm/.emacs.d/elpa/company-20170112.2005/company-xcode hides ~/.emacs.d/elisp/company-xcode /home/alanm/.emacs.d/elpa/company-20170112.2005/company hides ~/.emacs.d/elisp/company /home/alanm/.emacs.d/elpa/company-20170112.2005/company-abbrev hides ~/.emacs.d/elisp/company-abbrev /home/alanm/.emacs.d/elpa/company-20170112.2005/company-dabbrev hides ~/.emacs.d/elisp/company-dabbrev /home/alanm/.emacs.d/elpa/company-20170112.2005/company-dabbrev-code hides ~/.emacs.d/elisp/company-dabbrev-code /home/alanm/.emacs.d/elpa/company-20170112.2005/company-oddmuse hides ~/.emacs.d/elisp/company-oddmuse /home/alanm/.emacs.d/elpa/company-20170112.2005/company-etags hides ~/.emacs.d/elisp/company-etags /home/alanm/.emacs.d/elpa/company-20170112.2005/company-eclim hides ~/.emacs.d/elisp/company-eclim /home/alanm/.emacs.d/elpa/company-20170112.2005/company-nxml hides ~/.emacs.d/elisp/company-nxml /home/alanm/.emacs.d/elpa/company-20170112.2005/company-ispell hides ~/.emacs.d/elisp/company-ispell /home/alanm/.emacs.d/elpa/company-20170112.2005/company-gtags hides ~/.emacs.d/elisp/company-gtags /home/alanm/.emacs.d/elpa/company-20170112.2005/company-tempo hides ~/.emacs.d/elisp/company-tempo /home/alanm/.emacs.d/elpa/company-20170112.2005/company-elisp hides ~/.emacs.d/elisp/company-elisp /home/alanm/.emacs.d/elpa/company-20170112.2005/company-template hides ~/.emacs.d/elisp/company-template /home/alanm/.emacs.d/elpa/company-20170112.2005/company-keywords hides ~/.emacs.d/elisp/company-keywords /home/alanm/.emacs.d/elpa/company-20170112.2005/company-files hides ~/.emacs.d/elisp/company-files /home/alanm/.emacs.d/elpa/company-20170112.2005/company-semantic hides ~/.emacs.d/elisp/company-semantic /home/alanm/.emacs.d/elpa/company-20170112.2005/company-clang hides ~/.emacs.d/elisp/company-clang /home/alanm/.emacs.d/elpa/autopair-20160304.437/autopair hides ~/.emacs.d/elisp/autopair /home/alanm/.emacs.d/elpa/company-20170112.2005/company-css hides ~/.emacs.d/elisp/company-css /home/alanm/.emacs.d/elpa/cmake-mode-20160928.505/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode ~/.emacs.d/elisp/rst hides /usr/share/emacs/25.1/lisp/textmodes/rst /home/alanm/.emacs.d/elpa/css-mode-1.0/css-mode hides /usr/share/emacs/25.1/lisp/textmodes/css-mode /home/alanm/.emacs.d/elpa/scala-mode-20170118.558/ob-scala hides /usr/share/emacs/25.1/lisp/org/ob-scala /home/alanm/.emacs.d/elpa/soap-client-3.1.1/soap-inspect hides /usr/share/emacs/25.1/lisp/net/soap-inspect /home/alanm/.emacs.d/elpa/soap-client-3.1.1/soap-client hides /usr/share/emacs/25.1/lisp/net/soap-client /home/alanm/.emacs.d/elpa/ntlm-2.1.0/ntlm hides /usr/share/emacs/25.1/lisp/net/ntlm /home/alanm/.emacs.d/elpa/seq-20151121.1017/seq hides /usr/share/emacs/25.1/lisp/emacs-lisp/seq /home/alanm/.emacs.d/elpa/tabulated-list-20120406.1351/tabulated-list hides /usr/share/emacs/25.1/lisp/emacs-lisp/tabulated-list Features: (shadow sort mail-extr emacsbug sendmail tramp-cmds company-tern dash-functional url-http-ntlm ntlm calc calc-loaddefs calc-macs hmac-md5 hex-util md4 ac-js2 skewer-mode cache-table simple-httpd js2-highlight-vars tern-auto-complete tern cursor-sensor tabify conf-mode tramp-cache winner diff network-stream nsm starttls warnings company-oddmuse company-keywords company-etags company-gtags company-dabbrev-code company-dabbrev company-files company-cmake company-xcode company-clang company-eclim company-template company-css company-nxml company-bbdb company-jedi jedi jedi-core python-environment epc ctable concurrent deferred auto-complete popup vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher ibuffer-vc em-unix em-term term ehelp em-script em-prompt em-ls em-hist em-pred em-glob em-dirs em-cmpl em-basic em-banner em-alias esh-var esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg esh-groups eshell esh-module esh-mode esh-util magit-obsolete magit-blame magit-stash magit-bisect magit-remote magit-commit magit-sequence magit-notes magit-worktree magit-branch magit-files magit-refs magit-status magit magit-repos magit-apply magit-wip magit-log magit-diff smerge-mode magit-core magit-autorevert autorevert filenotify magit-process magit-margin magit-mode magit-git crm magit-section magit-popup git-commit magit-utils log-edit message idna rfc822 mml mml-sec epg mm-decode mm-bodies mm-encode mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log with-editor web-mode disp-table whitespace vc-git diff-mode rainbow-mode color emmet-mode css-mode misearch multi-isearch lua-mode server ob-lilypond zenburn-theme flycheck subr-x org-clock org2blog ox-wp ox-latex ox-icalendar ox-html ox-ascii ox-publish ox org-element cl metaweblog xml-rpc timezone url-http tls gnutls mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-gw org org-macro org-footnote org-pcomplete org-list org-faces org-entities noutline outline org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs org-loaddefs cal-menu calendar cal-loaddefs js2-refactor js2r-paredit js2r-conveniences js2r-conditionals js2r-wrapping js2r-functions js2r-vars multiple-cursors-core rect js2r-iife js2r-formatting js2r-helpers dash js2-mode js sgml-mode cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs django-snippets yasnippet highlight-indentation flymake company elpy pyvenv elpy-django s elpy-refactor python tramp-sh json map grep files-x etags xref project ido helm-mode helm-files rx image-dired tramp tramp-compat tramp-loaddefs trampver ucs-normalize shell pcomplete dired-x dired-aux ffap thingatpt helm-buffers helm-elscreen helm-tags helm-bookmark helm-adaptive helm-info helm-locate helm-grep helm-regexp helm-external helm-net browse-url xml url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util mailcap helm-utils helm-help helm-types helm helm-source eieio-compat helm-multi-match helm-lib helm-config helm-easymenu async-bytecomp async qml-mode delsel cua-base elec-pair pdf-occur ibuf-ext ibuffer tablist tablist-filter semantic/wisent/comp semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local find-func cedet dired pdf-isearch let-alist pdf-misc imenu pdf-tools compile comint ansi-color ring cus-edit cus-start cus-load wid-edit pdf-view bookmark pp jka-compr pdf-cache pdf-info tq pdf-util advice format-spec image-mode wc-mode printing ps-print ps-def lpr paren tabbar easy-mmode edmacro kmacro finder-inf css-mode-autoloads flappymacs-autoloads epc-autoloads ctable-autoloads nose-autoloads pep8-autoloads slime-autoloads tabulated-list-autoloads url-auth url-parse auth-source cl-seq eieio eieio-core cl-macs gnus-util time-date mm-util help-fns mail-prsvr password-cache url-vars virtualenv-autoloads info package epg-config seq byte-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify dynamic-setting system-font-setting font-render-setting move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 722368 86999) (symbols 48 63599 2) (miscs 40 866 1138) (strings 32 197238 16662) (string-bytes 1 5612576) (vectors 16 99923) (vector-slots 8 1977787 49327) (floats 8 2414 842) (intervals 56 9241 322) (buffers 976 53)) From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 29 Jan 2017 17:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: me@alandmoore.com Cc: 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.14857106883468 (code B ref 25549); Sun, 29 Jan 2017 17:25:02 +0000 Received: (at 25549) by debbugs.gnu.org; 29 Jan 2017 17:24:48 +0000 Received: from localhost ([127.0.0.1]:50392 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cXtDb-0000tn-L0 for submit@debbugs.gnu.org; Sun, 29 Jan 2017 12:24:47 -0500 Received: from mail-io0-f173.google.com ([209.85.223.173]:35892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cXtDZ-0000tU-OC; Sun, 29 Jan 2017 12:24:46 -0500 Received: by mail-io0-f173.google.com with SMTP id j13so93167765iod.3; Sun, 29 Jan 2017 09:24:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=mxxEmibdDJjPECDE19Yel34wkxV6eDXKk7/XjKLGVR8=; b=VjD3VfHrT7P6aJLD+XR9GDnh8NTtBFYfdbcGJRQrdWxLSHSoDXErUVtfYE6TJ86nUe /f+xA0aXrCNPXv3qvMUBnLZRYvlq5oL9QvV5Zl/r64OHvEtKyfZshM5qKfphuvQY2b4v AGZQsnXoOhJk6BagdURVZinl2cnWGEXlqpvbiSH+Jo78HMmbLBV/9fgQVsVKaxyQucVU k3zGHjUqtVghRgquLRlIpQKY6EeK+szyKAU/SPbRyEAtN70uyBDA+sEov7NqIeL9kflk sUBFjdRoe/Mo9GGNX7V8ZUGgGk2BxorXuMEtW+R2y36WOZvavBNfGokDLEGZE0+HwTfE gy9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=mxxEmibdDJjPECDE19Yel34wkxV6eDXKk7/XjKLGVR8=; b=T/G6ClxFhYa68n5+OgGa/wklIc0kCdTwg/EaMGd1V9z/4MOiR+fK4l7ORyMLS/Avz8 nhLgX8CTjj7vycj2Hvj3qIker5EruS1mniNMgesqtX3igt/UsFAqB6eXaHdWV34IIzoI WpcBQRAs8s4STzfdQtz+X2oIBV3i1cq55lgc0CqrM2SdQ76LNAXH2ISnCaFl8xndeZYh yv9FKLHVovU/CrhLC0NymTyTnHOcW38cWWQYWttUCX3wwjIyuPjJQ1bB52Grx9wTmWFE Q4zFDi15ZR9llieH88gWwAUHssLt5/8iDDyEb+Wngd20CvvyM7IfEmyPdc3cMzsc3X4f 3M+w== X-Gm-Message-State: AIkVDXLJ6lEcxzLpmfdq7IQ/OBbJk4ZsVkkIxMvZ8TVSNwANssCPRoaV0khCpy7MWjhqxg== X-Received: by 10.107.28.204 with SMTP id c195mr17119757ioc.223.1485710680085; Sun, 29 Jan 2017 09:24:40 -0800 (PST) Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id a128sm5004761itg.22.2017.01.29.09.24.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 29 Jan 2017 09:24:39 -0800 (PST) From: npostavs@users.sourceforge.net References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> Date: Sun, 29 Jan 2017 12:25:48 -0500 In-Reply-To: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> (me@alandmoore.com's message of "Thu, 26 Jan 2017 16:06:44 -0600") Message-ID: <874m0hsrqb.fsf@users.sourceforge.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.6 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) tags 25549 moreinfo unreproducible quit me@alandmoore.com writes: > When using grep on a large file in eshell, the output is sometimes > inconsistent. For example, when grepping for a string in my > server's apache2 access log (~5 MB) I got anywhere from around 200 to > around 1300 results, just running the same grep command over and over > (the file itself was not changing significantly). > > The bug cannot be consistently reproduced, but has happened in multiple > situations and others have reportedly reproduced it per discussion on > reddit > (https://www.reddit.com/r/emacs/comments/5qbd70/trampeshellgrep_returning_wildly_different/). Could you give some hints on what is needed to reproduce this? Is it the size of the grepped file? Number of matches? Are there some non-ascii characters in the file? Is Tramp required (the first post in the linked thread mentions it, but some other comments seem to suggest it's not required)? You mention the file was not changing "significantly", does this happen also when file stays *exactly* identical? I tried doubling up xdisp.c until it was over 5 MB, but grepping for "int" in eshell returned a consistent number of matches. From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 30 Jan 2017 15:31:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: unreproducible moreinfo To: Cc: 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.14857902054235 (code B ref 25549); Mon, 30 Jan 2017 15:31:01 +0000 Received: (at 25549) by debbugs.gnu.org; 30 Jan 2017 15:30:05 +0000 Received: from localhost ([127.0.0.1]:51288 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYDu9-00016F-79 for submit@debbugs.gnu.org; Mon, 30 Jan 2017 10:30:05 -0500 Received: from ham1-4.dnslinks.net ([74.86.15.251]:34777 helo=ham1.dnslinks.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYDu6-00015f-CA for 25549@debbugs.gnu.org; Mon, 30 Jan 2017 10:30:03 -0500 Received: (qmail 1129 invoked from network); 30 Jan 2017 15:29:51 -0000 Received: from unknown (HELO 67.228.73.20) (10.13.148.196) by localhost with SMTP; 30 Jan 2017 15:29:51 -0000 MIME-Version: 1.0 Received: from 72.237.174.2 [72.237.174.2] with HTTP/1.1 (POST); Mon, 30 Jan 2017 15:29:59 +0000 Date: Mon, 30 Jan 2017 15:29:59 +0000 From: In-Reply-To: <874m0hsrqb.fsf@users.sourceforge.net> References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> <874m0hsrqb.fsf@users.sourceforge.net> Message-ID: <67846bdbfe1187275be95e9d7a5babff@10.12.2.250> X-Sender: me@alandmoore.com User-Agent: RoundCube Webmail/0.3-stable Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 X-Spam-Score: 0.9 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.9 (/) On Sun, 29 Jan 2017 12:25:48 -0500, npostavs@users.sourceforge.net wrote: > tags 25549 moreinfo unreproducible > quit > > me@alandmoore.com writes: > >> When using grep on a large file in eshell, the output is sometimes >> inconsistent. For example, when grepping for a string in my >> server's apache2 access log (~5 MB) I got anywhere from around 200 to >> around 1300 results, just running the same grep command over and over >> (the file itself was not changing significantly). >> >> The bug cannot be consistently reproduced, but has happened in multiple >> situations and others have reportedly reproduced it per discussion on >> reddit >> (https://www.reddit.com/r/emacs/comments/5qbd70/trampeshellgrep_returning_wildly_different/). > > Could you give some hints on what is needed to reproduce this? Is it > the size of the grepped file? Number of matches? Are there some > non-ascii characters in the file? Is Tramp required (the first post in > the linked thread mentions it, but some other comments seem to suggest > it's not required)? You mention the file was not changing > "significantly", does this happen also when file stays *exactly* > identical? > > I tried doubling up xdisp.c until it was over 5 MB, but grepping for "int" > in eshell returned a consistent number of matches. It seems I and others were able to reproduce it locally, but it was much harder to do so. The problem seems much more pronounced when tramp is involved. It also seems to matter that the file is being written to. I discovered it when grepping a fairly active apache access log (being written to maybe 40-60 times a minute, on average). I am able to consistently reproduce the bug with this file, even when local to the server (though it's far less pronounced when local). I have not been successful reproducing it with a file that is not changing at all. I have asked the others who confirmed the bug on reddit to comment with their observations as well. From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: test case References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> In-Reply-To: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> Resent-From: Tijs Mallaerts Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 30 Jan 2017 20:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: unreproducible moreinfo To: 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.148580873012934 (code B ref 25549); Mon, 30 Jan 2017 20:39:02 +0000 Received: (at 25549) by debbugs.gnu.org; 30 Jan 2017 20:38:50 +0000 Received: from localhost ([127.0.0.1]:51399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYIiw-0003MY-CH for submit@debbugs.gnu.org; Mon, 30 Jan 2017 15:38:50 -0500 Received: from mail-oi0-f42.google.com ([209.85.218.42]:33416) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYIiu-0003M1-Dq for 25549@debbugs.gnu.org; Mon, 30 Jan 2017 15:38:48 -0500 Received: by mail-oi0-f42.google.com with SMTP id w204so202599510oiw.0 for <25549@debbugs.gnu.org>; Mon, 30 Jan 2017 12:38:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=1dpFK5yzJgMx4eNWLBEIsOS+cA3L7TJHe1hLyBBAJVE=; b=QgewxdRZD+wWAEgNls3HXmisJmuUPDy3wBKAvT1thkiWpKipINcJHBdtmiKLoGkk2V JiAGrQA+CdIBmq9Gdvm26DOSyLLIRwbjCdzxejnSpaCgMW0pxtBUZWnfFMQpxTIIDNzk hsHoPJTPpPQbMgjjEjHdl39pVH9E2cQhnzbEdUbrP4fJ6fnRlugsMxKtzcl+ANCIu0Lf K5BcS3yeBVGo9xM0TtZsOcrnZ6SdYB1LAxqoi/k2Us88ubgMnewucS7xezEXBm2egfS4 6RKnPW1bh9W99U3KK0PJOBvNtU9zuUDc77/rvWcBs2w5nwTQ/terjJtYZlRoYRAuRNls WrAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1dpFK5yzJgMx4eNWLBEIsOS+cA3L7TJHe1hLyBBAJVE=; b=iV7if5heCjUMo39xpqjakKSiCfDHttXZS/GVlh2i6TSG3hnso61x4s6bEHAGZl8Kyj hXGV50+chTtM56EA04EcPJpCOCSpAnbFarZKODZSE5FJ3lpWkLg3qSIZCtE3A+f1zcC1 ILIY0joRjAbLONEefkuwKQ66OMH2alsINivd7TqdUi16IVjHzqgu125ldBLmdYIjsRBg zTL0PgEqBs3/L4k7JlBeIJaHsdyRtMo5XN8Z92AzJK3/ch/uVYD7FGcTR7dvK+yCz6kc FsjlnhfA+v8ZJ9zU0PR+rWw9Qg2iZtwyXOynWTsJC2LuhZfsn6mhV9lA8ityM7gP7IIK sjzg== X-Gm-Message-State: AIkVDXLAyaXkQHcnAUFrDQ6LNcgF4L0Ds96OPuIFOmu+nGcIKsGf7Iugi/ycGeauR5hV3CbMHVWeFERPqi9kSA== X-Received: by 10.202.108.84 with SMTP id h81mr11959489oic.10.1485808722616; Mon, 30 Jan 2017 12:38:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.69.77 with HTTP; Mon, 30 Jan 2017 12:38:42 -0800 (PST) From: Tijs Mallaerts Date: Mon, 30 Jan 2017 21:38:42 +0100 Message-ID: Content-Type: multipart/alternative; boundary=001a1142eb761da9a7054755cd79 X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) --001a1142eb761da9a7054755cd79 Content-Type: text/plain; charset=UTF-8 On my machine following steps seem to be a reproducible test case: - create a file "test-file.txt" with 20.000 identical lines with content "This is a line." - open eshell and insert the command: cat test-file.txt | grep line | wc This seems to return a different result every time it's run. --001a1142eb761da9a7054755cd79 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On my machine following steps seem to be a = reproducible test case:

- create a file "test-file.txt&qu= ot; with 20.000 identical lines with content "This is a line."
- open eshell and insert the command: cat test-file.txt | grep line = | wc

This seems to return a different result every time it'= ;s run.
--001a1142eb761da9a7054755cd79-- From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 31 Jan 2017 02:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: unreproducible moreinfo To: Tijs Mallaerts Cc: me@alandmoore.com, 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.14858297319478 (code B ref 25549); Tue, 31 Jan 2017 02:29:02 +0000 Received: (at 25549) by debbugs.gnu.org; 31 Jan 2017 02:28:51 +0000 Received: from localhost ([127.0.0.1]:51533 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYOBf-0002Si-43 for submit@debbugs.gnu.org; Mon, 30 Jan 2017 21:28:51 -0500 Received: from mail-it0-f65.google.com ([209.85.214.65]:36618) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYOBd-0002SB-8O; Mon, 30 Jan 2017 21:28:49 -0500 Received: by mail-it0-f65.google.com with SMTP id f200so2563220itf.3; Mon, 30 Jan 2017 18:28:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=ytqNP+w9+OCTg9ZTnduNHgUcjJOdWiZsajEqVw3oJSg=; b=tI+56ql+v/6Kx+T8Ngzy44oHpj/VcFWIU9q93S2dAl8XCVxFc1M+aXkIhvN0kQz8l7 xA6sXAMsPw/TXAi39TxWsuiRPsCYaq8yKt4sZG+ZjapAb8WDwLo6G67n8Ff8zfYazUYH Si4xkV12JKSqL0Y4CzUFzHGuuPURnKmGwmCPKc4VN0qrHaS8q7qMn9vvnAoy7OCPKdK8 KbX12bIOAsq+57efR4sHZVW1jnduuc5endB8E7Ef4SiCx9l7+N6Rc1smfJ4Ve/GAimmX GgSV6d1equJT3Y191D3duqXCVoVYBUrGX9L7S15QUGlvF2DyAExtD82gdPAG2be8anUQ CqDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=ytqNP+w9+OCTg9ZTnduNHgUcjJOdWiZsajEqVw3oJSg=; b=OCxY98iSfCEwAXe5p3dK0LHtO3yPtMLesvVMThxjd0rOQpgqN4f8poeqgdG/RANrDV FluNT68nLD+eqO8FAfr81+Ya+4aKesQJgQpMnhFJpxcF6snBRNUIMy6UX2gtL1L0jtdb eY5OKsq3jhpvRrfOBoEYjx1Zp300XIBm1Lh7j/qJZU5fPd8DXrEmc81CCmk535Fh8WV4 mzzAFN7gTvM/dWKje21NpgIyJ1Xgt/uZ8nzS6PHwm9MVneDEoT5khx1plmAXA8Y8O6rH xFUdfwGqtyFUtSNKA9RM1dZLZ1OEgDyH7gesrHPIAsDRx2VMFY8LHj/7HqrH6nNwUYan G0Hw== X-Gm-Message-State: AIkVDXLXFGtEMWFOFmM100ggb2XHBuCDej8ITR3R3eRPV+x7YpKMzb3senwa1kN6oSARKw== X-Received: by 10.36.69.79 with SMTP id y76mr18488777ita.20.1485829723793; Mon, 30 Jan 2017 18:28:43 -0800 (PST) Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id h69sm7519942ith.16.2017.01.30.18.28.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 30 Jan 2017 18:28:42 -0800 (PST) From: npostavs@users.sourceforge.net References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> Date: Mon, 30 Jan 2017 21:29:50 -0500 In-Reply-To: (Tijs Mallaerts's message of "Mon, 30 Jan 2017 21:38:42 +0100") Message-ID: <87mve8q7vl.fsf_-_@users.sourceforge.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) tags 25549 = confirmed quit Tijs Mallaerts writes: > On my machine following steps seem to be a reproducible test case: > > - create a file "test-file.txt" with 20.000 identical lines with content "This is a line." > - open eshell and insert the command: cat test-file.txt | grep line | wc > > This seems to return a different result every time it's run. Thanks, I can reproduce with this. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 31 22:55:54 2017 Received: (at control) by debbugs.gnu.org; 1 Feb 2017 03:55:54 +0000 Received: from localhost ([127.0.0.1]:52755 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYm1S-0001Xt-LB for submit@debbugs.gnu.org; Tue, 31 Jan 2017 22:55:54 -0500 Received: from mail-it0-f42.google.com ([209.85.214.42]:36865) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cYm1R-0001Xi-U3 for control@debbugs.gnu.org; Tue, 31 Jan 2017 22:55:54 -0500 Received: by mail-it0-f42.google.com with SMTP id r185so9444458ita.0 for ; Tue, 31 Jan 2017 19:55:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:mime-version; bh=P4K9cu0zSqdymdzfwGOJJRdiiwUJ/IN91pGQCpBDyjA=; b=k7aeRs1fl/irK/uC3mBc9t6GyUCmcRKZY9UnCMYD41b4+S1+yGfGuET2HXJjuqmSA7 jE6reJ0TBQe5WnY5AHZ8tOhc+d2Biw1IbehseJNPZOCqITGtAPAPqCptQdTIc7miojs2 jpyezjth3XiRcnjTHqJhux4mfO4dZoL4BECiiElxKKBGbQpzUTnhZ02CxFwydzb2twne y4+MyDVMu3vbE4hKEQxbYTBkUJkzeJEGNKeYnkgsCkjJ+Schn9DckN8upxDUOTmbawDV SgenknCeXY8X1x4tkrl7DC0fkDcEq/3gmOt3M6BxtZkaZDRCEgOKSKI2jwGPFW8PUgoN dP4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :mime-version; bh=P4K9cu0zSqdymdzfwGOJJRdiiwUJ/IN91pGQCpBDyjA=; b=ICo0etZsLHStak0rl3dA7QiYRpYSnnE0XtTeHJij3N+mhhYgUjm2FYakfFIKhe7fQQ bFtzZLMYmXlbldYL+tJzLHD1xWuunq5BADb9x9SupY4nwoaaj7vDu8HsB2HbzuuH9T5s YjYL+dqwyGuaBP4psXN8LpTHrZGW2thr/td3ZNytLRL+GCzfeuRA3rRncJjdMQEDGWHc WkKQ34XWD1Xt8E+UCgrpEYZ3pFfhCRp/pce+gc91mDc69DPnQM/2MLSy0/NmhM87BBAO y/N+HSVw5XEAo2ePfmVLjpURsV45lTkTux2feZTBKUwPb9Z58LAY/BOvc42rQFgDURck cvSw== X-Gm-Message-State: AIkVDXIlT1bjd514UB+iOr6PVTMKkmXqn9Ye0Nj6wfOv/XrZwXWETF2/V80s+lOYAzj1XA== X-Received: by 10.36.58.71 with SMTP id m68mr926117itm.36.1485921348187; Tue, 31 Jan 2017 19:55:48 -0800 (PST) Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id y126sm9413361itf.14.2017.01.31.19.55.47 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jan 2017 19:55:47 -0800 (PST) From: npostavs@users.sourceforge.net To: control@debbugs.gnu.org Subject: control message for bug #18569 Date: Tue, 31 Jan 2017 22:56:56 -0500 Message-ID: <87y3xqpnqv.fsf@users.sourceforge.net> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) merge 15414 18569 25549 quit From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 03 Feb 2017 03:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Tijs Mallaerts Cc: me@alandmoore.com, 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.148609235732433 (code B ref 25549); Fri, 03 Feb 2017 03:26:01 +0000 Received: (at 25549) by debbugs.gnu.org; 3 Feb 2017 03:25:57 +0000 Received: from localhost ([127.0.0.1]:54367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cZUVZ-0008R0-5K for submit@debbugs.gnu.org; Thu, 02 Feb 2017 22:25:57 -0500 Received: from mail-it0-f65.google.com ([209.85.214.65]:34745) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cZUVW-0008Qf-U2; Thu, 02 Feb 2017 22:25:55 -0500 Received: by mail-it0-f65.google.com with SMTP id o185so547639itb.1; Thu, 02 Feb 2017 19:25:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=1vSf3Til1cRhNnQovrcgRceOrFtyI97L2HmzfkuR8j0=; b=YIfnDZVJzyv9jeOLco7dhOR6Y9NHBd6806M6jGFnT9SnNTs4r5wIl6g9A/bZySXzP+ 4xr/ZFWr9sQGh8lK30gCV9QOPUexTlw4JjyCy8yrXOGH7AOT7S+X+SXWgJv+acqpkG7z PWy18i3xn+KyYbCXVCBSfydcEDV1VNibJs2+Al6sPD6qGdQWSqdU8xehvn3Kf/CbW+mW TfXhEXb6QBN00naS7fn+J0pl6iS8Wezu5kPz4M4Z25ETYSEKwawwm1A+Up2S2oMmMHh0 BRxUcDgUZRcrwFShZLDbzmUnEALlpfM6EsQ5w+4BdvSQd81Yuib0pVoux80fSbHq9StN RsBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=1vSf3Til1cRhNnQovrcgRceOrFtyI97L2HmzfkuR8j0=; b=AulBaG/WNd5k9AJUiPuRxUCDY+j/M4TWpZpcnMZpFBXhODkeohriUM212oKUqmI5mq 1T+bacviqIUX8Sw6ozWzwrIV5O+nIsGxcBRTGPain2dnF5TFgjiEfwwKDIF9/kjpBJ+8 CKhg709nDA24yV9SBbPJ80PBnKj0te1NZj5bp+a+61g7wacJ4zpxazONFFYRn8+8aAgA hVhADjRG/nJXhTI1jrPl8Osxn507d9b8/k3356QCRTbV62VQkEd7lpcVOUujgVOh6odM JRKJpAwpMK1Uo6nArNRgsBu+9Jt8wgaWbzweI20GMuH5UFtMRZAzPKpZDEO5B+hNITJh 9isQ== X-Gm-Message-State: AIkVDXKelx3leIql3G7SE5L4uAugdMiZpOGWKGUCsEQ9cy4yzyuk8YOF/VxhuPPGxkJdbw== X-Received: by 10.36.112.137 with SMTP id f131mr628534itc.26.1486092349273; Thu, 02 Feb 2017 19:25:49 -0800 (PST) Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id e24sm331513itd.6.2017.02.02.19.25.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 02 Feb 2017 19:25:48 -0800 (PST) From: npostavs@users.sourceforge.net References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> <87mve8q7vl.fsf_-_@users.sourceforge.net> Date: Thu, 02 Feb 2017 22:26:57 -0500 In-Reply-To: <87mve8q7vl.fsf_-_@users.sourceforge.net> (npostavs@users.sourceforge.net's message of "Mon, 30 Jan 2017 21:29:50 -0500") Message-ID: <87mve4osxq.fsf@users.sourceforge.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) --=-=-= Content-Type: text/plain tags 25549 patch quit npostavs@users.sourceforge.net writes: > > Tijs Mallaerts writes: > >> On my machine following steps seem to be a reproducible test case: >> >> - create a file "test-file.txt" with 20.000 identical lines with content "This is a line." >> - open eshell and insert the command: cat test-file.txt | grep line | wc >> >> This seems to return a different result every time it's run. > > Thanks, I can reproduce with this. The problem can be more easily reproduced (i.e., more often, and only 3000 lines) by cat test-file.txt | sleepy-cat | wc where sleepy-cat is #!/bin/sh while read line ; do echo "$line" sleep 0.000001 done The problem happens when one of the commands in the pipeline sends its output to Emacs quickly and the next command in the pipeline is slower. On receiving data from the first command in eshell-insertion-filter we call eshell-output-object to send it to the next command, but since sending might block, Emacs can run other process filters and sentinels instead. In this case, while sending a data chunk from cmd1 to cmd2, we actually end up reading all the data from cmd1 until it terminates and we call its sentinel. The sentinel closes the pipes and sends EOF to cmd2, but we still haven't sent the data from cmd1 to cmd2 yet. Closing the pipes in a timer, as in the patch below, seems to fix it for me. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=v1-0001-Make-sure-eshell-pipelines-don-t-drop-data.patch Content-Description: patch >From a1757a7114fbc20733554d8aa44cea0fa8a991e2 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Thu, 2 Feb 2017 09:19:43 -0500 Subject: [PATCH v1] Make sure eshell pipelines don't drop data * lisp/eshell/esh-proc.el (eshell-sentinel): If called while still handling output of the process, make sure to close the pipes only later, so that the next process in the pipeline recieves EOF only after getting all its input (Bug#25549). --- lisp/eshell/esh-proc.el | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/lisp/eshell/esh-proc.el b/lisp/eshell/esh-proc.el index b0dbb22..ba5cb5c 100644 --- a/lisp/eshell/esh-proc.el +++ b/lisp/eshell/esh-proc.el @@ -393,8 +393,20 @@ eshell-sentinel (unless (string= string "run") (unless (string-match "^\\(finished\\|exited\\)" string) (eshell-insertion-filter proc string)) - (eshell-close-handles (process-exit-status proc) 'nil - (cadr entry)))) + (let ((handles (nth 1 entry)) + (str (prog1 (nth 3 entry) + (setf (nth 3 entry) nil))) + (status (process-exit-status proc))) + ;; If we're in the middle of handling output + ;; from this process then schedule the EOF for + ;; later. + (letrec ((finish-io + (lambda () + (if (nth 4 entry) + (run-at-time 0 nil finish-io) + (when str (eshell-output-object str nil handles)) + (eshell-close-handles status 'nil handles))))) + (funcall finish-io))))) (eshell-remove-process-entry entry)))) (eshell-kill-process-function proc string))))) -- 2.9.3 --=-=-=-- From unknown Mon Jun 23 23:51:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#25549: 25.1; eshell grep gives inconsistent output Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 09 Feb 2017 04:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25549 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed patch To: Tijs Mallaerts Cc: me@alandmoore.com, 25549@debbugs.gnu.org Received: via spool by 25549-submit@debbugs.gnu.org id=B25549.14866155288246 (code B ref 25549); Thu, 09 Feb 2017 04:46:01 +0000 Received: (at 25549) by debbugs.gnu.org; 9 Feb 2017 04:45:28 +0000 Received: from localhost ([127.0.0.1]:60692 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbgbo-00028q-9E for submit@debbugs.gnu.org; Wed, 08 Feb 2017 23:45:28 -0500 Received: from mail-it0-f46.google.com ([209.85.214.46]:38143) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbgbm-00028Z-PN; Wed, 08 Feb 2017 23:45:27 -0500 Received: by mail-it0-f46.google.com with SMTP id c7so8102838itd.1; Wed, 08 Feb 2017 20:45:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=A2nU+JS+LvxxAp3mkWWVVvRPm/qsolbG2h6Vt8x7IuA=; b=BfGaHDMF7G8wYZ3jzYpi/dBM/Q4Bfqj3lD9v415FeIr11P8xdE0AGn1aKf4MMpitu7 ef0pcdqRv6wIzi//Qpc+UZ18zTDNuHCbXzz40DQ2zgcqKjqL5z30Q1s2Pa7v0VyWVR98 ygFY5JJEmtJumlZX53PnjfKa9j+AsT6/gYjNM6GgT+bUVgCL9oo61Pt5iuhhhkqfQz2x RDWgZ3o2dAVY++MT3rkfe5ybiXaAOT8Ins9EJdADT5CH3ClElpr8E1SbD0n+OSKr7dw4 lksSy6YgFLeX6m5DczXZEm/yMdJFuXR4E6D4uImgahBThy1A+yhmraN+fBPI4/39epe4 LNhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=A2nU+JS+LvxxAp3mkWWVVvRPm/qsolbG2h6Vt8x7IuA=; b=HBe3dgAfSgP9QOsDk8SQ4GSvwwPhMvHTyBWAd/Jg+P2Pga3N1fqITOKA7grWN6LgMk vldhSjLxsjwulQSYvR0QhE1u9EyrASCw7s9Klc9+PeGcJLo1u/ps4T25kxmatoflDgMw lUWqHbB1AIEIKTuE3OPaGZ6O5HBttyPXWoXAA61EPmpGN4o8tpKEXnpSr93BGQVFwDNE dWe2uxV4fSq1cLwqt4Rf2NVmLcgyw69dyJ7qDc0dokEtrEyROnLYjF8WrCmYWfqLONF8 VhYhUEhZcx4LowX1q7XeOFicMctZHVvKAOfbfJAX/yIXlFMF2lpupG3sMWjzKD0dfHR1 wxAw== X-Gm-Message-State: AIkVDXIC403k6QnMpUh7BCEFUPNsNaSy92QpRufeETyMoYwjiRhD8D+v23KGy7UtRzmWSw== X-Received: by 10.36.73.211 with SMTP id e80mr19437288itd.88.1486615521156; Wed, 08 Feb 2017 20:45:21 -0800 (PST) Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id 98sm12606480iol.5.2017.02.08.20.45.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 Feb 2017 20:45:20 -0800 (PST) From: npostavs@users.sourceforge.net References: <87tw8lzdaj.fsf@it-alanm.williamson-tn.org> <87mve8q7vl.fsf_-_@users.sourceforge.net> <87mve4osxq.fsf@users.sourceforge.net> Date: Wed, 08 Feb 2017 23:46:30 -0500 In-Reply-To: <87mve4osxq.fsf@users.sourceforge.net> (npostavs@users.sourceforge.net's message of "Thu, 02 Feb 2017 22:26:57 -0500") Message-ID: <87poism0nt.fsf@users.sourceforge.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.2 (/) tags 25549 fixed close 25549 26.1 quit npostavs@users.sourceforge.net writes: > > Closing the pipes in a timer, as in the patch below, seems to fix it for > me. Pushed to master, [1: 2bfe83fcd3]. 1: 2017-02-08 23:37:57 -0500 2bfe83fcd3575d03f371a7d0e6b8c5d661bc5ac7 Make sure eshell pipelines don't drop data From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 05 06:50:18 2017 Received: (at control) by debbugs.gnu.org; 5 Nov 2017 11:50:18 +0000 Received: from localhost ([127.0.0.1]:51296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eBJRR-0006vG-S0 for submit@debbugs.gnu.org; Sun, 05 Nov 2017 06:50:17 -0500 Received: from mail-it0-f53.google.com ([209.85.214.53]:45971) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eBJRR-0006v3-0S for control@debbugs.gnu.org; Sun, 05 Nov 2017 06:50:17 -0500 Received: by mail-it0-f53.google.com with SMTP id n195so2206889itg.0 for ; Sun, 05 Nov 2017 03:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:mime-version; bh=61btssE6icvGJ1jPK3u8VzPkIxnfFVb0GfkBOgGDkMQ=; b=rRzN4zmMKzFkR9pzp9R1aRzn4uciqBY+Ykd2tuNI4SmRTR+2PT2nK7KtS682Ajn0k+ Ss3/SU51hZUHzFmtGBmvi+tVoh4nGyCzJYGql7ipcZ+vcSlZ2v81O73ujuAgMHvq8g6c U4Qth3HZ85hp9l1Y19l+i+B641T3pjKe8DLwaMJz5YRqrMWz3+ErFUnLzWNwYQNi7nM2 J8Zx4RjeTDVXfq4e6Y9MKJ5ZOJHzeSApmHS+297He0lP25lrmRRk/vVeOHAoGsymiKSj LpcKGmfZcqit2DPYi+bmJsu0wewy2VTLQDj7GSArL2wO+nANrnQKnYP9cHNImLmA+WCN angA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :mime-version; bh=61btssE6icvGJ1jPK3u8VzPkIxnfFVb0GfkBOgGDkMQ=; b=fUIo37fAsjqNZPImOghMtXCtKLtuUEf1pwGTiD3XT484WKWCpoDEIgwv3hEBKdPjkn IQQc9GMXVbkAGf2+WAxD8jU5N/DMknRFpBI15rp4g9ttnIPqsLcz8dxogL8cdOwvQDLu TtV6pUzTWB1bZbJCpyvDTzh1wgdN1gc5uajxytGYjUAsjkLVDxqlqoGA+d5o+5Tov3RE SYGfP6NCGvpzGwJPIHKT4WkcL4abQVYsbLrYcTKDqizaA19VKBjZukl9+WexNkjiYvhk dIvY2ypwcbZ4SC7YYPsFzlHfvCuJbq17QkVSd4HciDmDfdbNBFZ2rTdqtURUQ2FS5Rln Bz4Q== X-Gm-Message-State: AJaThX5dJ2z/mRW5Mlf5n+fqCG2FJaSnpNev3dW71Q87Z2IJjwJfzSun PJeflfE3GtR6GVI4GIAs87MuBA== X-Google-Smtp-Source: ABhQp+RKHj20rQsrNc9REr3a2jhdVQYErxScNPRCDlXCzTI/VOFm3Adn1gz/0wfn7DvA+w90oKJoDw== X-Received: by 10.36.47.84 with SMTP id j81mr5412286itj.37.1509882611181; Sun, 05 Nov 2017 03:50:11 -0800 (PST) Received: from zebian ([45.2.119.34]) by smtp.googlemail.com with ESMTPSA id i62sm4543511ioe.31.2017.11.05.03.50.10 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 05 Nov 2017 03:50:10 -0800 (PST) From: Noam Postavsky To: control@debbugs.gnu.org Subject: control message for bug #25549 Date: Sun, 05 Nov 2017 06:50:09 -0500 Message-ID: <87r2td6j2m.fsf@users.sourceforge.net> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) # forgot to unarchive before merging unarchive 25549 merge 25549 29154 quit