Subject: jupyter terminal out of pty devices

From: Steffen Rytter Postas <[email-masked]>
Date: Mon, 1 Jul 2019 22:54:22 +0200

Hi,

I've been having a few issues with the Jupyter package. First and
foremost, this is my first time trying it out with Guix. Previously I
have had it running using Docker, but I prefer the modularity and
programmability of Guix, not to mention the rest of the amazing concepts
it brings.

However, nothing is perfect, and as we'll see here, that is indeed true.
Let it be known that I've tested this on my desktop and on my work laptop.

On my desktop, I've had no issues getting it running, but the terminal
functionality does not work at all, for several reasons.

There may very well be another issue, that I've yet to reproduce, but
the above I can reproduce easily on my machines.

Both environments use the same command to execute, and are using `guix
environment` to start the Jupyter notebook system:

`LC_ALL=C guix environment -C -N --pure -m env.scm -- jupyter-notebook`

The `env.scm` file used, is as follows:

(define-packages '(
    "glibc-utf8-locales"
    "coreutils-minimal"
    "bash"
    "python"
    "jupyter"
    ))
(use-modules (gnu packages))
(packages->manifest (map specification->package packages))

You may notice that Python is mentioned as an explicit package, along
with coreutils-minimal. These are BOTH workarounds, to even get to the
bugs, mentioned above.

Without python, running the notebook yields the following error:

pkg_resources.DistributionNotFound: The 'six' distribution was not found
and is required by traitlets

However, I have not been able to reproduce this anywhere. I do believe
this is a bug though. Setting python to be a required package gets me
past the above error, however removing python does NOT re-introduce the
error on any of my systems.

Without coreutils-minimal, opening the terminal yields the following error:

[E 18:40:08.400 NotebookApp] Uncaught exception POST /api/terminals
(127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888',
method='POST', uri='/api/terminals', version='HTTP/1.1',
remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 1590, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 3006, in wrapper
        return method(self, *args, **kwargs)
      File
"/gnu/store/zgx9l5mypyh70ywn55rjrr085pyzs61z-python-notebook-5.7.4/lib/python3.7/site-packages/notebook/terminal/api_handlers.py",
line 16, in post
        name, _ = self.terminal_manager.new_named_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 319, in new_named_terminal
        term = self.new_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 171, in new_terminal
        pty = PtyProcessUnicode.spawn(argv, env=env,
cwd=options.get('cwd', None))
      File
"/gnu/store/fj5idczwb1s908b55z6jl64xmagcygva-python-ptyprocess-0.5.2/lib/python3.7/site-packages/ptyprocess/ptyprocess.py",
line 205, in spawn
        command_with_path = which(command)
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/shutil.py",
line 1129, in which
        if os.path.dirname(cmd):
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/posixpath.py",
line 156, in dirname
        p = os.fspath(p)
    TypeError: expected str, bytes or os.PathLike object, not NoneType
[W 18:40:08.402 NotebookApp] Unhandled error

At this point I assumed this was due to missing the `which(1)` command,
or at least some command missing, causing the above codepaths to return
None.
Adding coreutils-minimal gets me a slight bit further, and removing
coreutils-minimal DOES re-introduce the error as well.

However, at this point, Jupyter notebook "works". The script editing,
the interface, all of this works. Opening the terminal however, now
yields the following error:

[E 18:56:52.376 NotebookApp] Uncaught exception POST /api/terminals
(127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888',
method='POST', uri='/api/terminals', version='HTTP/1.1',
remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 1590, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 3006, in wrapper
        return method(self, *args, **kwargs)
      File
"/gnu/store/zgx9l5mypyh70ywn55rjrr085pyzs61z-python-notebook-5.7.4/lib/python3.7/site-packages/notebook/terminal/api_handlers.py",
line 16, in post
        name, _ = self.terminal_manager.new_named_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 319, in new_named_terminal
        term = self.new_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 171, in new_terminal
        pty = PtyProcessUnicode.spawn(argv, env=env,
cwd=options.get('cwd', None))
      File
"/gnu/store/fj5idczwb1s908b55z6jl64xmagcygva-python-ptyprocess-0.5.2/lib/python3.7/site-packages/ptyprocess/ptyprocess.py",
line 222, in spawn
        pid, fd = pty.fork()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 96, in fork
        master_fd, slave_fd = openpty()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 29, in openpty
        master_fd, slave_name = _open_terminal()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 59, in _open_terminal
        raise OSError('out of pty devices')
    OSError: out of pty devices
[W 18:56:52.377 NotebookApp] Unhandled error

So it seems that there's an issue with pty devices when running these
systems as a container.

Sincerely,

Steffen Rytter Postas