Subject: bug#20686: closed (Re: Emacs 24.2 vs 24.4 on Solaris: M-x shell and "tty => not a tty")

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 11 Jun 2015 23:49:04 +0000

Your bug report

#20555: "No access to tty" in Emacs shell

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
20555: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20555
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: Emacs 24.2 vs 24.4 on Solaris: M-x shell and "tty => not a tty"
From: Paul Eggert <[email-masked]>
To: Akira Shirai <[email-masked]>
Cc: "Warren, John W CIV \(US\)" <[email-masked]>, [email-masked], Georges Ko <[email-masked]>
Date: Thu, 11 Jun 2015 16:48:10 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Akira Shirai, thanks for the info about grantpt and its internal setuid program; 
that made the problem obvious.  I installed the attached patch into the Emacs 
master sources and am (perhaps optimistically) marking this bug as done.
>From 099cb9f7b8b1d00293298d947fc77b8c96120820 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Thu, 11 Jun 2015 16:41:36 -0700
Subject: [PATCH] Fix "not a tty" bug on Solaris 10

* configure.ac (PTY_OPEN): Define to plain 'open'
on SVR4-derived hosts, so that the O_CLOEXEC flag isn't set.
* src/process.c (allocate_pty): Set the O_CLOEXEC flag after
calling PTY_TTY_NAME_SPRINTF, for the benefit of SVR4-derived
hosts that call grantpt which does its work via a setuid subcommand
(Bug#19191, Bug#19927, Bug#20555, Bug#20686).
Also, set O_CLOEXEC even if PTY_OPEN is not defined, since it
seems relevant in that case too.
---
 configure.ac  |  5 ++++-
 src/process.c | 20 +++++++++++---------
 2 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/configure.ac b/configure.ac
index 9c6a74a..070b061 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4397,14 +4397,17 @@ case $opsys in
     ;;
 
   sol2* )
-    dnl On SysVr4, grantpt(3) forks a subprocess, so keep sigchld_handler()
+    dnl On SysVr4, grantpt(3) forks a subprocess, so do not use
+    dnl O_CLOEXEC when opening the pty, and keep the SIGCHLD handler
     dnl from intercepting that death.  If any child but grantpt's should die
     dnl within, it should be caught after sigrelse(2).
+    AC_DEFINE(PTY_OPEN, [fd = open (pty_name, O_RDWR | O_NONBLOCK)])
     AC_DEFINE(PTY_TTY_NAME_SPRINTF, [{ char *ptsname (int), *ptyname; int grantpt_result; sigset_t blocked; sigemptyset (&blocked); sigaddset (&blocked, SIGCHLD); pthread_sigmask (SIG_BLOCK, &blocked, 0); grantpt_result = grantpt (fd); pthread_sigmask (SIG_UNBLOCK, &blocked, 0); if (grantpt_result == -1 || unlockpt (fd) == -1 || !(ptyname = ptsname (fd))) { emacs_close (fd); return -1; } snprintf (pty_name, PTY_NAME_SIZE, "%s", ptyname); }])
     ;;
 
   unixware )
     dnl Comments are as per sol2*.
+    AC_DEFINE(PTY_OPEN, [fd = open (pty_name, O_RDWR | O_NONBLOCK)])
     AC_DEFINE(PTY_TTY_NAME_SPRINTF, [{ char *ptsname (int), *ptyname; int grantpt_result; sigset_t blocked; sigemptyset (&blocked); sigaddset (&blocked, SIGCHLD); pthread_sigmask (SIG_BLOCK, &blocked, 0); grantpt_result = grantpt (fd); pthread_sigmask (SIG_UNBLOCK, &blocked, 0); if (grantpt_result == -1) fatal("could not grant slave pty"); if (unlockpt(fd) == -1) fatal("could not unlock slave pty"); if (!(ptyname = ptsname(fd))) fatal ("could not enable slave pty"); snprintf (pty_name, PTY_NAME_SIZE, "%s", ptyname); }])
     ;;
 esac
diff --git a/src/process.c b/src/process.c
index 17fe708..b4f979f 100644
--- a/src/process.c
+++ b/src/process.c
@@ -658,22 +658,24 @@ allocate_pty (char pty_name[PTY_NAME_SIZE])
 
 	if (fd >= 0)
 	  {
-#ifdef PTY_OPEN
+#ifdef PTY_TTY_NAME_SPRINTF
+	    PTY_TTY_NAME_SPRINTF
+#else
+	    sprintf (pty_name, "/dev/tty%c%x", c, i);
+#endif /* no PTY_TTY_NAME_SPRINTF */
+
 	    /* Set FD's close-on-exec flag.  This is needed even if
 	       PT_OPEN calls posix_openpt with O_CLOEXEC, since POSIX
 	       doesn't require support for that combination.
+	       Do this after PTY_TTY_NAME_SPRINTF, which on some platforms
+	       doesn't work if the close-on-exec flag is set (Bug#20555).
 	       Multithreaded platforms where posix_openpt ignores
 	       O_CLOEXEC (or where PTY_OPEN doesn't call posix_openpt)
 	       have a race condition between the PTY_OPEN and here.  */
 	    fcntl (fd, F_SETFD, FD_CLOEXEC);
-#endif
-	    /* Check to make certain that both sides are available
-	       this avoids a nasty yet stupid bug in rlogins.  */
-#ifdef PTY_TTY_NAME_SPRINTF
-	    PTY_TTY_NAME_SPRINTF
-#else
-	    sprintf (pty_name, "/dev/tty%c%x", c, i);
-#endif /* no PTY_TTY_NAME_SPRINTF */
+
+	    /* Check to make certain that both sides are available.
+	       This avoids a nasty yet stupid bug in rlogins.  */
 	    if (faccessat (AT_FDCWD, pty_name, R_OK | W_OK, AT_EACCESS) != 0)
 	      {
 		emacs_close (fd);
-- 
2.1.0

Subject: "No access to tty" in Emacs shell
From: "Warren, John W CIV (US)" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Thu, 28 May 2015 19:41:50 +0000
List-Id: <debbugs-submit.debbugs.gnu.org>

Error message in *shell* buffer on entering shell mode:
"Warning: no access to tty; thus no job control in this shell..."

Note: The affected system does not have an internet connection,
but I can sneaker-net files across.

John Warren

Output of report-emacs-bug follows:
----------------------------------------

From: [email-masked]
To: [email-masked]
Subject: 24.4; No access to tty in Emacs shell
--text follows this line--

In GNU Emacs 24.4.1 (sparc-sun-solaris2.10, X toolkit, Xaw scroll bars)
 of 2015-05-27 on lanai
Windowing system distributor `Sun Microsystems, Inc.', version 11.0.6620
Configured using:
 `configure --with-gif=no --prefix=/home/john/local'

Important settings:
  value of $LANG: C
  locale-coding-system: nil

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<escape> x s h e l l <return> <escape> x r e p o r
t - e m a c s - b u g <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils shell pcomplete comint ansi-color
ring time-date tooltip electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dynamic-setting
font-render-setting x-toolkit x multi-tty emacs)

Memory information:
((conses 8 76071 5644)
 (symbols 24 17840 0)
 (miscs 20 47 126)
 (strings 16 11244 3372)
 (string-bytes 1 310941)
 (vectors 8 10332)
 (vector-slots 4 399617 2538)
 (floats 8 65 200)
 (intervals 28 220 236)
 (buffers 512 12))

--------------------------------