GNU bug report logs - #24464
25.1; python.el virtualenv

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24464 in the body.
You can then email your comments to 24464 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pim <pim.remkes <at> upcmail.nl>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.1; python.el virtualenv
Date: Mon, 19 Sep 2016 00:52:36 +0200

After setting the variable python-shell-virtualenv-root to the path of a
virtual environment and running the run-python command the virtualenv
doesn't get used.



In GNU Emacs 25.1.1 (x86_64-w64-mingw32)
 of 2016-09-17 built on KAEL
Windowing system distributor 'Microsoft Corp.', version 10.0.14926
Configured using:
 'configure --prefix=/tmp/emacs --without-imagemagick 'CFLAGS=-O2
 -fomit-frame-pointer -g0''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
You can run the command ‘eval-buffer’ with M-x ev-b RET
You can run the command ‘run-python’ with M-x r-py RET
Shell native completion is disabled, using fallback
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec epg
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
kmacro compile warnings tramp-cache python tramp-sh tramp tramp-compat
auth-source cl-seq eieio eieio-core cl-macs gnus-util mm-util help-fns
mail-prsvr password-cache tramp-loaddefs trampver ucs-normalize shell
pcomplete format-spec advice json map seq byte-opt gv bytecomp
byte-compile cl-extra help-mode easymenu cconv comint ring cl-loaddefs
pcase cl-lib ansi-color time-date mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp
disp-table w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote w32notify dbusbind w32 multi-tty
make-network-process emacs)

Memory information:
((conses 16 225628 10804)
 (symbols 56 23171 0)
 (miscs 48 53 150)
 (strings 32 27484 7701)
 (string-bytes 1 879903)
 (vectors 16 38112)
 (vector-slots 8 711739 6301)
 (floats 8 237 36)
 (intervals 56 257 15)
 (buffers 976 20))

Message #8 received at 24464 <at> debbugs.gnu.org (full text, mbox):

From: Justin Timmons <justinmtimmons <at> gmail.com>
To: Pim <pim.remkes <at> upcmail.nl>
Cc: 24464 <at> debbugs.gnu.org
Subject: Re: bug#24464: 25.1; python.el virtualenv
Date: Sun, 11 Dec 2016 19:39:56 -0500

[Message part 1 (text/plain, inline)]

It looks like this was caused by an incorrect assumption on the
directory structure of the virtualenv directory. According to the
virtualenv docs only POSIX systems follow the structure
"/path/to/venv/bin/", while windows systems use "/path/to/venv/Scripts"
for the location of the binary files, most importantly including the
python interpreter (see:
https://virtualenv.pypa.io/en/stable/userguide/#windows-notes).

This meant that the wrong path was added to `exec-path` and so we were
starting the python shell installed on the system, rather than the one
bundled inside of the virtualenv.

This should be addressed in the attached patch file - it just adds a
special case for Windows and prepends that path to the system PATH.

I also updated the docs for the `python-shell-calculate-exec-path`
function to make it more clear that using the python interpreter bundled
in the virtualenv will automatically set the PYTHONPATH to point at the
libraries inside of the virtualenv - something that was not immediately
obvious to me.



Steps to verify that patch is working correctly:
  1. (On Windows)
  2. Create virtualenv (eg. 'virtualenv ~/venv')
  3. Install a library NOT currently installed on the your system inside
  the virtualenv. ex:
       ~/venv/Scripts/activate.bat
       pip install sphinx
       deactivate.bat
  4. Start emacs
  5. Set `python-shell-virtualenv-root` to the root directory of your
  virtualenv (eg. "/home/jtimmons/venv")
  6. M-x run-python
  7. Make sure that you can import the library installed in the
  virtualenv in step 3 (eg. 'import sphinx')

[venv.diff (text/x-patch, attachment)]

Message #13 received at 24464 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Justin Timmons <justinmtimmons <at> gmail.com>
Cc: Pim <pim.remkes <at> upcmail.nl>, 24464 <at> debbugs.gnu.org
Subject: Re: bug#24464: 25.1; python.el virtualenv
Date: Mon, 16 Oct 2017 21:24:24 -0400

retitle 24464 python virtualenv support doesn't work on w32
quit

Justin Timmons <justinmtimmons <at> gmail.com> writes:

> It looks like this was caused by an incorrect assumption on the
> directory structure of the virtualenv directory. According to the
> virtualenv docs only POSIX systems follow the structure
> "/path/to/venv/bin/", while windows systems use "/path/to/venv/Scripts"
> for the location of the binary files, most importantly including the
> python interpreter (see:
> https://virtualenv.pypa.io/en/stable/userguide/#windows-notes).
>
> This meant that the wrong path was added to `exec-path` and so we were
> starting the python shell installed on the system, rather than the one
> bundled inside of the virtualenv.
>
> This should be addressed in the attached patch file - it just adds a
> special case for Windows and prepends that path to the system PATH.
>
> I also updated the docs for the `python-shell-calculate-exec-path`
> function to make it more clear that using the python interpreter bundled
> in the virtualenv will automatically set the PYTHONPATH to point at the
> libraries inside of the virtualenv - something that was not immediately
> obvious to me.


>  (defun python-shell-calculate-exec-path ()
>    "Calculate `exec-path'.
>  Prepends `python-shell-exec-path' and adds the binary directory
> -for virtualenv if `python-shell-virtualenv-root' is set.  If
> -`default-directory' points to a remote host, the returned value
> -appends `python-shell-remote-exec-path' instead of `exec-path'."
> +for virtualenv if `python-shell-virtualenv-root' is set - this
> +will use the python interpreter from inside the virtualenv when
> +starting the shell.  If `default-directory' points to a remote host,
> +the returned value appends `python-shell-remote-exec-path' instead
> +of `exec-path'."
>    (let ((new-path (copy-sequence
>                     (if (file-remote-p default-directory)
>                         python-shell-remote-exec-path
> -                     exec-path))))
> +                     exec-path)))
> +
> +        ;; Windows and POSIX systems use different venv directory structures
> +        (virtualenv-bin-dir (if (eq system-type 'windows-nt) "Scripts" "bin")))
>      (python-shell--add-to-path-with-priority
>       new-path python-shell-exec-path)
>      (if (not python-shell-virtualenv-root)
>          new-path
>        (python-shell--add-to-path-with-priority
>         new-path
> -       (list (expand-file-name "bin" python-shell-virtualenv-root)))
> +       (list (expand-file-name virtualenv-bin-dir python-shell-virtualenv-root)))
>        new-path)))

Looks good to me.  Have you assigned copyright for Emacs?  (The patch is
small enough to go in regardless, I only ask to know if we should mark
it.)

Message #16 received at 24464 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Justin Timmons <justinmtimmons <at> gmail.com>
Cc: Pim <pim.remkes <at> upcmail.nl>, 24464 <at> debbugs.gnu.org
Subject: Re: bug#24464: 25.1; python.el virtualenv
Date: Sun, 05 Nov 2017 12:20:57 -0500

tags 24464 fixed
close 24464 26.1
quit

Justin Timmons <justinmtimmons <at> gmail.com> writes:

> This should be addressed in the attached patch file - it just adds a
> special case for Windows and prepends that path to the system PATH.

I've pushed this to emacs-26.

[1: 18af404ef3]: 2017-11-05 11:39:06 -0500
  Support python virtualenv on w32 (Bug#24464)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=18af404ef33d8efcbb9446945e543251ab33aa3c

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.