GNU bug report logs - #19736
python.el: native completion breaks ipython magic completion

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19736 in the body.
You can then email your comments to 19736 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: fabi.87 <at> gmail.com
Subject: 24.4; python.el: native completion breaks ipython magic completion
Date: Fri, 30 Jan 2015 19:29:41 -0300

Hi Fabian,

I think you should be using [1] as the ipython completer. In a sense,
when you inherit from rlcompleter.Completer and replace the current
completer you're somehow going against the goal of native completion,
aren't you? Wouldn't it be better to just monkey patch the completer
returned by readline.get_completer() with your _callable_postfix in case
the completer indeed defines a _callable_postfix (it seems ipython
completer don't do that)?

Cheers
--
Carlos

[1] http://ipython.org/ipython-doc/dev/api/generated/IPython.core.completer.html

Message #8 received at 19736 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: 19736 <at> debbugs.gnu.org
Cc: Fabian Ezequiel Gallina <galli.87 <at> gmail.com>
Date: Mon, 2 Feb 2015 14:30:49 -0300

[Message part 1 (text/plain, inline)]

Tags: patch

Hi Fabian,

I've fixed this bug. It was harder than I'd initially thought as
several issues converged to screw ipython completion up:

1) The native completion setup code was importing rlcompleter. Just
importing this overrides the current readline completer (the import
calls set_completer to set the standard python completer, which is not
quite the same as the completer ipython had previously set up).

2) The fallback completion setup code was also importing rlcompleter
unconditionally, thus replacing the ipython completer too.

3) The _callable_postfix + triple-tab workaround was not available for
the ipython completer AFAICS, because it has no _callable_postfix
method. I opted for another (IMO a bit cleaner) workaround, see below.

So, here is what I've done to fix the issues above:

3') I'm wrapping the current complete function (as returned by
readline.get_completer) so that the wrapper never returns just one
completion. In case the original completer returned one completion, a
'__dummy_completion__' is added to the completion list by the wrapper.
This avoids the need to define a _callable_postifx (unavailable in
ipython) and also avoids the need for the triple-tab hack and
duplicate filtering (obviously, you still have to filter
'__dummy_completion__'). All in all I think the approach is more
robust although still a bit hackish. IMO, it should be documented
better in the code, I spent quite a time figuring out WTF the triple
tabs and _callable_postfix were supposed to do! :)

1') Because of 3', there is no need to import rlcompleter in the
native completion setup code anymore.

2') I've done some refactoring of the fallback completion setup code
in order to import rlcompleter only in case the current shell is not
ipython. In general, I think this setup code is a bit more clear now,
but you're the one to judge that.

I'm attaching a patch against my version of python.el which is a
little outdated and has some patches of mine applied on top. But I
think you won't have any problems figuring out the needed changes as
they closely track the description above.

Cheers
--
Carlos

[complete.diff (text/plain, attachment)]

Message #13 received at 19736 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: 19736 <at> debbugs.gnu.org
Cc: Fabian Ezequiel Gallina <galli.87 <at> gmail.com>
Date: Mon, 2 Feb 2015 14:38:15 -0300

Just in case, here is the modified code for both completion setups
(I'm not listing the minor changes to
python-shell-completion-native-get-completions below):


(defcustom python-shell-completion-setup-code
  "def __PYTHON_EL_get_completions(text):
    try:
        import __builtin__
    except ImportError:  # Python 3
        import builtins as __builtin__
    builtins = dir(__builtin__)
    is_ipython = ('__IPYTHON__' in builtins or
                  '__IPYTHON__active' in builtins)
    completions = []
    try:
        if is_ipython:
            splits = text.split()
            is_module = splits and splits[0] in ('from', 'import')
            if is_module:
                from IPython.core.completerlib import module_completion
                completions = module_completion(text.strip())
            elif '__IP' in builtins:
                completions = __IP.complete(text)
            elif 'get_ipython' in builtins:
                completions = get_ipython().Completer.all_completions(text)
        else:  # Never do this in IPython as it overrides the completer!
            import readline, rlcompleter
            i = 0
            while True:
                res = readline.get_completer()(text, i)
                if not res:
                   break
                i += 1
                completions.append(res)
    except:
        pass
    return completions"
  "Code used to setup completion in inferior Python processes."
  :type 'string
  :group 'python)


(defun python-shell-completion-native-setup ()
  "Try to setup native completion, return non-nil on success."
  (let ((process (python-shell-get-process)))
    (python-shell-send-string
     (funcall
      'mapconcat
      #'identity
      (list
       "try:"
       "    import readline"
       "    def completer(text, state, c=readline.get_completer()):"
       "        completion = c(text, state)"
       "        if not completion and state == 1:"
       "            return text + '__dummy_completion__'"
       "        else:"
       "            return completion"
       "    readline.set_completer(completer)"
       "    if readline.__doc__ and 'libedit' in readline.__doc__:"
       "        readline.parse_and_bind('bind ^I rl_complete')"
       "    else:"
       "        readline.parse_and_bind('tab: complete')"
       "    print ('python.el: readline is available')"
       "    del completer, readline  # Some cleanup"
       "except:"
       "    print ('python.el: readline not available')")
      "\n")
     process)
    (python-shell-accept-process-output process)
    (when (save-excursion
            (re-search-backward
             (regexp-quote "python.el: readline is available") nil t 1))
      (python-shell-completion-native-try))))

Message #16 received at 19736 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: 19736 <at> debbugs.gnu.org
Cc: Fabian Ezequiel Gallina <galli.87 <at> gmail.com>
Date: Mon, 2 Feb 2015 16:27:01 -0300

[Message part 1 (text/plain, inline)]

Patch updated against the current master.

[complete.patch (text/x-patch, attachment)]

Message #23 received at 19736-done <at> debbugs.gnu.org (full text, mbox):

From: fgallina <at> gnu.org (Fabián Ezequiel Gallina)
To: 19736-done <at> debbugs.gnu.org
Subject: python.el: native completion breaks ipython magic completion
Date: Thu, 09 Apr 2015 00:57:18 -0300

Revno ab9252a <at> master fixes this.


Cheers,
Fabián.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.