GNU bug report logs - #76205
31.0.50; 31.0.50; python.el completion broken for non-builtin objects in IPython 8.30.0 & Python 3.13.2

Previous Next

Full log

View this message in rfc822 format

From: kobarity <kobarity <at> gmail.com>
To: Visuwesh <visuweshm <at> gmail.com>, Liu Hui <liuhui1610 <at> gmail.com>
Cc: 76205 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: bug#76205: 31.0.50; 31.0.50; python.el completion broken for non-builtin objects in IPython 8.30.0 & Python 3.13.2
Date: Mon, 17 Feb 2025 23:50:36 +0900

Visuwesh wrote:
> 
> [ஞாயிறு பிப்ரவரி 16, 2025] kobarity wrote:
> 
> > Visuwesh wrote:
> >> 
> >> [சனி பிப்ரவரி 15, 2025] kobarity wrote:
> >> 
> >> >> [...]
> >> >> So...what do we do from here?  Unfortunately, I know next to nothing
> >> >> about this area to file a sensible bug report myself.
> >> >
> >> > I'm not sure either.  As ipython --simple-prompt does not support
> >> > completion, I don't know if using rlcompleter with ipython
> >> > --simple-prompt is supported or not.
> >> >
> >> > The direct cause of this problem seems to be that rlcompleter.__main__
> >> > is not as expected.  So the following steps enable completion on
> >> > ipython --simple-prompt on Python 3.13.
> >> >
> >> > import readline
> >> > import rlcompleter
> >> > import __main__
> >> > rlcompleter.__main__ = __main__
> >> > readline.parse_and_bind("tab: complete")
> >> >
> >> > Attached is a test patch that does something similar to this.
> >> 
> >> I can confirm that completion works for non-builtin objects with this
> >> patch.
> >
> > Thank you for testing the patch.
> >
> > Here's what's happening.  ipython --simple-prompt on Python 3.13 is
> > indirectly importing rlcompleter, which sets its Completer().complete
> > as the completer.  Completer() without the namespace argument uses
> > __main__.__dict__ as the source of global symbols to complete.  The
> > __main__ here is the name in rlcompleter.py namespace, which is bound
> > at import time of rlcompleter.py.  However, __main__ used in the REPL
> > of ipython --simple-prompt is different from __main__ used at the
> > import time of rlcompleter.py.  Therefore, changes in globals()
> > (i.e. __main__.__dict__) are not reflected in
> > rlcompleter.__main__.__dict__ and the completion will fail.
> >
> > Prior to Python 3.12, the problem does not occur because rlcompleter
> > was not imported indirectly.  When we import rlcompleter in the native
> > completion setup code, __main__ is already the same as the one used in
> > the REPL.
> >
> > Since IPython does not use rlcompleter, it seems difficult to consider
> > this problem a bug in IPython.  It would be better if rlcompleter
> > reads sys.modules["__main__"] instead of __main__, but it would not be
> > easy to change a long standing specification.  So I think it would be
> > better for python.el to address this issue.
> >
> > To set up a completer that works correctly, it is sufficient to
> > specify the namespace argument:
> >
> > readline.set_completer(rlcompleter.Completer(globals()).complete)
> >
> > However, we need to make sure that we do not override the customized
> > completer.  The problem is that it is difficult to distinguish between
> > the completer that is set up by importing rlcompleter and the
> > customized completer.
> >
> > So it seems to me that it would be reasonable to override
> > rlcompleter.__main__ so that the default completer works properly.  I
> > think the attached revised patch is slightly better than my previous
> > patch, since it does not add "__main__" to globals.  This change
> > affects only IPython.  I added some comments and the commit message.
> 
> Thanks for the explanation.  The attached patch works on my end too.
> 
> > I would like to hear your opinions.
> 
> I do not have background knowledge on these parts so I hold no technical
> opinions.  Please go ahead with a solution that you're satisfied with.
> I am happy as long as the completion works as expected!

Thanks for testing the patch again.

Liu Hui wrote:
> 
> On Sun, Feb 16, 2025 at 6:20 PM kobarity <kobarity <at> gmail.com> wrote:
> 
> > Here's what's happening.  ipython --simple-prompt on Python 3.13 is
> > indirectly importing rlcompleter
> 
> Thanks for the investigation. It seems that pdb imports rlcompleter
> since python 3.13 and ipython always imports pdb at startup.
> 
> > So it seems to me that it would be reasonable to override
> > rlcompleter.__main__ so that the default completer works properly.  I
> > think the attached revised patch is slightly better than my previous
> > patch, since it does not add "__main__" to globals.  This change
> > affects only IPython.  I added some comments and the commit message.
> >
> > I would like to hear your opinions.
> 
> Thank you. I think the fix is good.

Thanks for your comment.

> BTW, python 3.13 introduces pyrepl and leads to the warning "can't use
> pyrepl: terminal doesn't have the required clear capability;
> TERM=dumb".
> 
> Maybe we should set PYTHON_BASIC_REPL=1 to suppress the warning when
> starting the inferior python process?

Yes, I think that is a good idea.  Could you suggest a patch as a new
bug?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.