Subject: bug#19755: closed (python.el: native completion: more problems (and solutions))

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 09 Apr 2015 03:57:02 +0000

Your bug report

#19755: python.el: native completion: more problems (and solutions)

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
19755: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19755
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: python.el: native completion: more problems (and solutions)
From: [email-masked] (Fabián Ezequiel Gallina)
To: [email-masked]
Date: Thu, 09 Apr 2015 00:55:59 -0300
List-Id: <debbugs-submit.debbugs.gnu.org>

Hola Carlos,

I pushed a fix for this at revno 911ed2e in the master branch.

I'll be slowly catching up, sorry for the long delay.

Cheers,
Fabián.

Subject: python.el: native completion: more problems (and solutions)
From: Carlos Pita <[email-masked]>
To: [email-masked]
Date: Tue, 3 Feb 2015 09:40:21 -0300
List-Id: <debbugs-submit.debbugs.gnu.org>

X-Debbugs-CC: [email-masked]
Tags: patch

Hi Fabian,

besides the ones l have already reported and patched in [1] there are
some remaining problems with the native completion approach:

1) The waiting timeout is too low for import completions, as imports
take a bit more time than simple dirs. But making the timeout larger
is bad for cases when there really is no completion at all for the
current prefix. I want to avoid this trade off.

2) When all the completions share a common prefix, readline completes
inline up to this common prefix, so the delete-line-command sequence
length is wrong. For example, say the current prefix is "x" and that
its completions are "xxy", "xxz". Readline will complete "xx" inline,
but delete-line-command will only delete one "x".

3) A pager could be enabled to show long lists of completions.

My solutions for 1 and 2 simply extend the dummy completion hack I
introduced in [1]:

1') Ensure there is always one completion at least (a fortiori, I'm
ensuring there are at least two completions, to force listing instead
of inline completion). So we can set a larger timeout *just for the
first* accept-process-output without the risk of waiting too much when
there is really no completion available.

2') Ensure there is a completion with a different prefix, by providing
a dummy randomly prefixed completion.

For 3 I just disabled paged completions for readline, but I don't know
how to do that for libedit.

Attached is a patch with all the described changes, applied on top of
my patch for [1]. Anyway, it should be easy to selectively apply the
changes manually.

All in all I find the solution pretty simple and it leverages my
previous trick. The completer is a bit more complex now as it has to
keep track of a couple of states in order to generate the correct
sequence of completions (dummies included):

    def completer(text, state,
                  real_completer=readline.get_completer(),
                  last_state=[None]):
        completion = None
        if state == 0:
            last_state[0] = None
        if last_state[0] is None:
            completion = real_completer(text, state)
            if not completion:
                last_state[0] = state
        if (state == last_state[0] or
            state - 1 == last_state[0] == 0):
            import random
            completion = '%s%d__dummy_completion__' % (
                text, random.randint(1000, 10000))
        return completion
    readline.set_completer(completer)

Cheers
--
Carlos

[1] http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19736
diff --git a/.emacs.d/lisp/python.el b/.emacs.d/lisp/python.el
index ca69b68..15041a3 100644
--- a/.emacs.d/lisp/python.el
+++ b/.emacs.d/lisp/python.el
@@ -3071,28 +3071,38 @@ When a match is found, native completion is disabled."
   "Try to setup native completion, return non-nil on success."
   (let ((process (python-shell-get-process)))
     (python-shell-send-string
-     (funcall
-      'mapconcat
-      #'identity
-      (list
-       "try:"
-       "    import readline"
-       "    def completer(text, state, c=readline.get_completer()):"
-       "        completion = c(text, state)"
-       "        if not completion and state == 1:"
-       "            return text + '__dummy_completion__'"
-       "        else:"
-       "            return completion"
-       "    readline.set_completer(completer)"
-       "    if readline.__doc__ and 'libedit' in readline.__doc__:"
-       "        readline.parse_and_bind('bind ^I rl_complete')"
-       "    else:"
-       "        readline.parse_and_bind('tab: complete')"
-       "    print ('python.el: readline is available')"
-       "    del completer, readline  # Some cleanup"
-       "except:"
-       "    print ('python.el: readline not available')")
-      "\n")
+     ;; Ensure at least two completions:
+     ;;  - Random prefix avoids common prefix inline completion
+     ;;  - Two completions avoid single inline completion
+     ;;  - One sure completion avoids waiting output timeout
+     "try:
+    import readline
+    def completer(text, state,
+                  real_completer=readline.get_completer(),
+                  last_state=[None]):
+        completion = None
+        if state == 0:
+            last_state[0] = None
+        if last_state[0] is None:
+            completion = real_completer(text, state)
+            if not completion:
+                last_state[0] = state
+        if (state == last_state[0] or
+            state - 1 == last_state[0] == 0):
+            import random
+            completion = '%s%d__dummy_completion__' % (
+                text, random.randint(1000, 10000))
+        return completion
+    readline.set_completer(completer)
+    if readline.__doc__ and 'libedit' in readline.__doc__:
+        readline.parse_and_bind('bind ^I rl_complete')
+    else:
+        readline.parse_and_bind('tab: complete')
+        readline.parse_and_bind('set page-completions off')
+    print('python.el: readline is available')
+    del completer, readline  # Env cleanup
+except:
+    print('python.el: readline not available')"
      process)
     (python-shell-accept-process-output process)
     (when (save-excursion
@@ -3206,9 +3216,7 @@ completion."
                             #'comint-redirect-filter original-filter-fn))
                 (set-process-filter process #'comint-redirect-filter))
               (process-send-string process input-to-send)
-              (accept-process-output
-               process
-               python-shell-completion-native-output-timeout)
+              (accept-process-output process 0.3)
               ;; XXX: can't use `python-shell-accept-process-output'
               ;; here because there are no guarantees on how output
               ;; ends.  The workaround here is to call
@@ -3217,13 +3225,12 @@ completion."
               (while (accept-process-output
                       process
                       python-shell-completion-native-output-timeout))
-              (cl-remove
-               (concat input "__dummy_completion__")
+              (cl-remove-if
+               (lambda (c) (string-match "__dummy_completion__" c))
                (split-string
                 (buffer-substring-no-properties
                  (point-min) (point-max))
-                separators t)
-               :test #'string=)))
+                separators t))))
         (set-process-filter process original-filter-fn)))))
 
 (defun python-shell-completion-get-completions (process import input)