GNU bug report logs - #28500
25.3; shell mode starts echoing command

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28500 in the body.
You can then email your comments to 28500 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: barry <at> python.org (Barry A. Warsaw)
To: bug-gnu-emacs <at> gnu.org
Subject: 25.3; shell mode starts echoing command
Date: Mon, 18 Sep 2017 11:46:50 -0400

* M-x shell RET
* Go to the shell and type `ls RET`
* See the output of the `ls` but *not* the actual `ls` command
* Enter an interpreter; two choices which reproduce the problem include `sqlite3` and `python3`
  Type `sqlite3` at the shell prompt
* Type .exit at the sqlite3 prompt (notice that the `.exit` was echo'd
  before the shell prompt came back; the bug is now active
* Type `ls RET` at the shell prompt.  Notice that the `ls RET` was
  echoed before the ls output was printed.

Note that using `python3` or even `python` at the prompt will also
exhibit the bug.  (To exit Python, just type ctrl-D).  The only way I've
found to "fix" the problem is to ctrl-D to exit the shell, then `M-x
shell RET` to restart it in the same buffer.



In GNU Emacs 25.3.1 (x86_64-apple-darwin16.7.0, NS appkit-1504.83 Version 10.12.6 (Build 16G29))
 of 2017-09-14 built on resist.local
Windowing system distributor 'Apple', version 10.3.1504
Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/25.3/share/info/emacs
 --prefix=/usr/local/Cellar/emacs/25.3 --without-x --with-xml2
 --without-dbus --without-gnutls --without-imagemagick --with-ns
 --disable-ns-self-contained'

Configured features:
JPEG NOTIFY ACL LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
The mark is not set now, so there is no region
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils shell pcomplete comint
ansi-color ring time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel ns-win ucs-normalize
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 200070 5259)
 (symbols 48 19931 0)
 (miscs 40 59 194)
 (strings 32 16829 5243)
 (string-bytes 1 489691)
 (vectors 16 33562)
 (vector-slots 8 655967 4691)
 (floats 8 158 281)
 (intervals 56 252 16)
 (buffers 976 20))

Message #8 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: barry <at> python.org (Barry A. Warsaw)
Cc: 28500 <at> debbugs.gnu.org
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Mon, 18 Sep 2017 20:45:23 -0400

Barry A. Warsaw wrote:

> * M-x shell RET
> * Go to the shell and type `ls RET`
> * See the output of the `ls` but *not* the actual `ls` command
> * Enter an interpreter; two choices which reproduce the problem include `sqlite3` and `python3`
>   Type `sqlite3` at the shell prompt
> * Type .exit at the sqlite3 prompt (notice that the `.exit` was echo'd
>   before the shell prompt came back; the bug is now active
> * Type `ls RET` at the shell prompt.  Notice that the `ls RET` was
>   echoed before the ls output was printed.

FWIW, I cannot reproduce this on RHEL7.

Message #11 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Nick Helm <nick <at> tenpoint.co.nz>
To: 28500 <at> debbugs.gnu.org
Cc: "Barry A. Warsaw" <barry <at> python.org>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 12:54:15 +1200

barry <at> python.org (Barry A. Warsaw) writes:

> * M-x shell RET
> * Go to the shell and type `ls RET`
> * See the output of the `ls` but *not* the actual `ls` command
> * Enter an interpreter; two choices which reproduce the problem include `sqlite3` and `python3`
>   Type `sqlite3` at the shell prompt
> * Type .exit at the sqlite3 prompt (notice that the `.exit` was echo'd
>   before the shell prompt came back; the bug is now active
> * Type `ls RET` at the shell prompt.  Notice that the `ls RET` was
>   echoed before the ls output was printed.

I can reproduce this with Emacs 25.3 on macOS 10.12.6.

Message #14 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <mkupfer <at> alum.berkeley.edu>
To: barry <at> python.org (Barry A. Warsaw)
Cc: 28500 <at> debbugs.gnu.org
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Mon, 18 Sep 2017 18:05:53 -0700

I see the same issue after using openssl to encrypt or decrypt a file.
Just never got around to filing a bug report...

-----8<-----8<-----
alto$ openssl aes-256-cbc -in foo.txt -out foo.text.enc
enter aes-256-cbc encryption password:
Verifying - enter aes-256-cbc encryption password:
alto$ ls
ls
deb9-upgrade.log  foo.text.enc	ieee-cs-election-receipt.pdf
enchant.dummy*	  foo.txt	net-neutrality-comments.pdf
alto$ echo $SHELL
echo $SHELL
/bin/bash
alto$ 
----->8----->8-----

I see this on Debian 8 and on Solaris, with various versions of Emacs.

mike

Message #17 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Barry Warsaw <barry <at> python.org>
To: Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Cc: 28500 <at> debbugs.gnu.org
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Mon, 18 Sep 2017 22:49:14 -0400

[Message part 1 (text/plain, inline)]

On Sep 18, 2017, at 21:05, Mike Kupfer <mkupfer <at> alum.berkeley.edu> wrote:
> 
> I see the same issue after using openssl to encrypt or decrypt a file.
> Just never got around to filing a bug report…

Interesting.  I’ve only noticed it in 25.2 and 25.3 on macOS (installed from brew).  I can’t reproduce it in 25.2 on Ubuntu 17.10.

> alto$ openssl aes-256-cbc -in foo.txt -out foo.text.enc
> enter aes-256-cbc encryption password:
> Verifying - enter aes-256-cbc encryption password:
> alto$ ls
> ls
> deb9-upgrade.log  foo.text.enc	ieee-cs-election-receipt.pdf
> enchant.dummy*	  foo.txt	net-neutrality-comments.pdf
> alto$ echo $SHELL
> echo $SHELL
> /bin/bash
> alto$

This is reproducible for me too on macOS.

I can also reproduce it by running the Mailman 3 shell from git checkout.  That made me think it could be readline related, but I built a Mailman 3 without readline support and it made no difference.  I haven’t figured out what the common denominator is, but I edebug’d it down to comint-send-string and then process-send-string.  There the trail ended with C code and I haven’t gdb’d into that.  It seems like some state in process-send-string gets confused and starts echoing the input.

-Barry

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Cc: 28500 <at> debbugs.gnu.org, "Barry A. Warsaw" <barry <at> python.org>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 09:34:27 +0200

On Sep 18 2017, Mike Kupfer <mkupfer <at> alum.berkeley.edu> wrote:

> I see the same issue after using openssl to encrypt or decrypt a file.
> Just never got around to filing a bug report...
>
> -----8<-----8<-----
> alto$ openssl aes-256-cbc -in foo.txt -out foo.text.enc
> enter aes-256-cbc encryption password:
> Verifying - enter aes-256-cbc encryption password:
> alto$ ls
> ls
> deb9-upgrade.log  foo.text.enc	ieee-cs-election-receipt.pdf
> enchant.dummy*	  foo.txt	net-neutrality-comments.pdf
> alto$ echo $SHELL
> echo $SHELL
> /bin/bash
> alto$ 
> ----->8----->8-----
>
> I see this on Debian 8 and on Solaris, with various versions of Emacs.

This is a bug in openssl, it doesn't properly restore the tty settings
after reading the password.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #23 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Barry Warsaw <barry <at> python.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 09:20:06 -0400

[Message part 1 (text/plain, inline)]

On Sep 19, 2017, at 03:34, Andreas Schwab <schwab <at> suse.de> wrote:
> 
> On Sep 18 2017, Mike Kupfer <mkupfer <at> alum.berkeley.edu> wrote:
> 
>> I see the same issue after using openssl to encrypt or decrypt a file.
>> Just never got around to filing a bug report...
>> 
>> -----8<-----8<-----
>> alto$ openssl aes-256-cbc -in foo.txt -out foo.text.enc
>> enter aes-256-cbc encryption password:
>> Verifying - enter aes-256-cbc encryption password:
>> alto$ ls
>> ls
>> deb9-upgrade.log  foo.text.enc	ieee-cs-election-receipt.pdf
>> enchant.dummy*	  foo.txt	net-neutrality-comments.pdf
>> alto$ echo $SHELL
>> echo $SHELL
>> /bin/bash
>> alto$
>> ----->8----->8-----
>> 
>> I see this on Debian 8 and on Solaris, with various versions of Emacs.
> 
> This is a bug in openssl, it doesn't properly restore the tty settings
> after reading the password.

Andreas, can you provide any additional details?  What exactly is not getting restored?  Are the other two examples I posted (python3 and sqlite3) related?  Why would it occur on (for me) macOS but not Ubuntu?

[signature.asc (application/pgp-signature, attachment)]

Message #26 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Barry Warsaw <barry <at> python.org>
Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 15:36:09 +0200

On Sep 19 2017, Barry Warsaw <barry <at> python.org> wrote:

> On Sep 19, 2017, at 03:34, Andreas Schwab <schwab <at> suse.de> wrote:
>> 
>> On Sep 18 2017, Mike Kupfer <mkupfer <at> alum.berkeley.edu> wrote:
>> 
>>> I see the same issue after using openssl to encrypt or decrypt a file.
>>> Just never got around to filing a bug report...
>>> 
>>> -----8<-----8<-----
>>> alto$ openssl aes-256-cbc -in foo.txt -out foo.text.enc
>>> enter aes-256-cbc encryption password:
>>> Verifying - enter aes-256-cbc encryption password:
>>> alto$ ls
>>> ls
>>> deb9-upgrade.log  foo.text.enc	ieee-cs-election-receipt.pdf
>>> enchant.dummy*	  foo.txt	net-neutrality-comments.pdf
>>> alto$ echo $SHELL
>>> echo $SHELL
>>> /bin/bash
>>> alto$
>>> ----->8----->8-----
>>> 
>>> I see this on Debian 8 and on Solaris, with various versions of Emacs.
>> 
>> This is a bug in openssl, it doesn't properly restore the tty settings
>> after reading the password.
>
> Andreas, can you provide any additional details?  What exactly is not getting restored?  Are the other two examples I posted (python3 and sqlite3) related?  Why would it occur on (for me) macOS but not Ubuntu?

$ stty; openssl aes-256-cbc -in foo.txt -out foo.text.enc; stty

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #29 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Barry Warsaw <barry <at> python.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 10:49:26 -0400

[Message part 1 (text/plain, inline)]

On Sep 19, 2017, at 09:36, Andreas Schwab <schwab <at> suse.de> wrote:

> $ stty; openssl aes-256-cbc -in foo.txt -out foo.text.enc; stty

Thanks, that’s exactly the clue I needed.

% stty
speed 9600 baud;
lflags: -echo echoe echoke echoctl
oflags: -oxtabs
cflags: cs8 -parenb
% sqlite3
SQLite version 3.16.0 2016-11-04 19:09:39
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .exit
.exit
% stty
stty
speed 9600 baud;
lflags: echoe echoke echoctl
oflags: -oxtabs
cflags: cs8 -parenb
% stty -echo
stty -echo
|resist=|@resist[~:1020]% ls
Applications/	Library/	Sites/

Notice that -echo gets lost when exiting sqlite3.  Setting -echo explicitly fixes the problem.  So not a bug per se in Emacs, although I wonder if a coming-output-filter-function could be added to do the fix until the offending processes themselves are fixed?

[signature.asc (application/pgp-signature, attachment)]

Message #32 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Barry Warsaw <barry <at> python.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Tue, 19 Sep 2017 20:19:40 -0400

[Message part 1 (text/plain, inline)]

On Sep 19, 2017, at 09:36, Andreas Schwab <schwab <at> suse.de> wrote:
> 
> $ stty; openssl aes-256-cbc -in foo.txt -out foo.text.enc; stty

What is happening is that -echo is getting lost.  `stty -echo` after exiting Python will restore normal behavior in a shell buffer.  I’m not entirely sure why, but this is neither set nor necessary in a Terminal window.

My suspicion now falls on readline from home-brew.  Specifically, if I build Python from git master but *remove* the readline.<blah>.so file so there’s no way that readline can be invoked, then I don’t see the problem.  readline gets imported implicitly if available when entering the interactive prompt, and I’ve proven to myself that if that happens, echo gets reenabled.  It’s the common feature with sqlite3 too, which for me comes from brew.  Further, because openssl comes from Apple, it isn’t linked with brew’s readline and that does *not* exhibit the problem.

And then I found this: https://github.com/Homebrew/brew/issues/651

I will follow up on that issue, but it’s clearly now not an Emacs bug.

FWIW, this little bit of Python in your $PYTHONSTARTUP file will workaround the problem:

import sys
import atexit
import termios

# Reset `stty -echo` on exit.
def no_echo():
    flags = termios.tcgetattr(sys.stdin)
    flags[3] &= ~termios.ECHO
    termios.tcsetattr(sys.stdin, termios.TCSADRAIN, flags)

atexit.register(no_echo)

Cheers!

[signature.asc (application/pgp-signature, attachment)]

Message #35 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Barry Warsaw <barry <at> python.org>
Cc: 28500 <at> debbugs.gnu.org, schwab <at> suse.de, mkupfer <at> alum.berkeley.edu
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Wed, 20 Sep 2017 09:12:01 +0300

> From: Barry Warsaw <barry <at> python.org>
> Date: Tue, 19 Sep 2017 20:19:40 -0400
> Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
> 
> I will follow up on that issue, but it’s clearly now not an Emacs bug.

Thanks for investigating.  Does that mean we can close this bug
report?

Message #38 received at 28500 <at> debbugs.gnu.org (full text, mbox):

From: Barry Warsaw <barry <at> python.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 28500 <at> debbugs.gnu.org, Andreas Schwab <schwab <at> suse.de>,
 mkupfer <at> alum.berkeley.edu
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Wed, 20 Sep 2017 09:39:14 -0400

[Message part 1 (text/plain, inline)]

On Sep 20, 2017, at 02:12, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Barry Warsaw <barry <at> python.org>
>> Date: Tue, 19 Sep 2017 20:19:40 -0400
>> Cc: 28500 <at> debbugs.gnu.org, Mike Kupfer <mkupfer <at> alum.berkeley.edu>
>> 
>> I will follow up on that issue, but it’s clearly now not an Emacs bug.
> 
> Thanks for investigating.  Does that mean we can close this bug
> report?

Yep!  And thanks.

[signature.asc (application/pgp-signature, attachment)]

Message #43 received at 28500-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Barry Warsaw <barry <at> python.org>
Cc: 28500-done <at> debbugs.gnu.org, mkupfer <at> alum.berkeley.edu
Subject: Re: bug#28500: 25.3; shell mode starts echoing command
Date: Wed, 20 Sep 2017 16:49:12 +0300

> From: Barry Warsaw <barry <at> python.org>
> Date: Wed, 20 Sep 2017 09:39:14 -0400
> Cc: Andreas Schwab <schwab <at> suse.de>,
>  28500 <at> debbugs.gnu.org,
>  mkupfer <at> alum.berkeley.edu
> 
> > Thanks for investigating.  Does that mean we can close this bug
> > report?
> 
> Yep!  And thanks.

Closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.