GNU bug report logs - #10227
Python installation fails for Python 3

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10227 in the body.
You can then email your comments to 10227 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-automake <bug-automake <at> gnu.org>
Subject: Python installation fails for Python 3
Date: Mon, 5 Dec 2011 23:53:41 +0100

The code currently used to get the python package directory is wrong
for Python 3:
>>> from distutils import sysconfig; print (sysconfig.get_python_lib(0,0,'/usr/local'))/usr/local/lib/python3/dist-packages
is wrong (should be /usr/local/lib/python3.2/dist-packages). Now, in
some sense this must be right (since the path is returned by
sysconfig), but the path produced
(/usr/local/lib/python3/dist-packages) is not searched by my Ubuntu
system's installed Python. If instead of automake I install the same
files with distutils (using a suitable setup.py), then the files are
installed in the right place (/usr/local/lib/python3.2/dist-packages).

One solution would be to use the sysconfig module (not
distutils.sysconfig), but that only works in Python 2.7 and 3.2 (not
3.0 or 3.1), so isn't really a starter. Hopefully someone who knows
more about Python than I can help, as it's much nicer to have Python
modules installed from one extra line in Makefile.am than have to add
a whole extra setup.py and require a different tool (albeit one
bundled with Python) to be used. (Actually, that is another idea: how
about delegating to distutils, since it comes with every version of
Python supported by automake?)
-- 
http://rrt.sc3d.org

Message #8 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Roumen Petrov <bugtrack <at> roumenpetrov.info>
To: 10227 <at> debbugs.gnu.org
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Tue, 06 Dec 2011 01:16:37 +0200

Reuben Thomas wrote:
> The code currently used to get the python package directory is wrong
> for Python 3:
>    
>>>> from distutils import sysconfig; print (sysconfig.get_python_lib(0,0,'/usr/local'))/usr/local/lib/python3/dist-packages
>>>>          
> is wrong (should be /usr/local/lib/python3.2/dist-packages). Now, in
> some sense this must be right (since the path is returned by
> sysconfig), but the path produced
> (/usr/local/lib/python3/dist-packages) is not searched by my Ubuntu
> system's installed Python. If instead of automake I install the same
> files with distutils (using a suitable setup.py), then the files are
> installed in the right place (/usr/local/lib/python3.2/dist-packages).
>    
It seems to me this is vendor bug as the right place is site-packages, 
see PEP 370.

A cannot reproduce cut of version with "python" build from original 
sources, i.e. without extra patches :
2.6. 2.7 3.2 return .../pythonX.Y/site-packages


[SNIP]

Roumen

Message #11 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Roumen Petrov <bugtrack <at> roumenpetrov.info>
Cc: 10227 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Wed, 21 Nov 2012 14:41:34 +0100

tags 10227 + moreinfo
thanks

Hi Roumen, Reuben.

On 12/06/2011 12:16 AM, Roumen Petrov wrote:
> Reuben Thomas wrote:
>> The code currently used to get the python package directory is wrong
>> for Python 3:
>>   
>>>>> from distutils import sysconfig; print (sysconfig.get_python_lib(0,0,'/usr/local'))/usr/local/lib/python3/dist-packages
>>>>>          
>> is wrong (should be /usr/local/lib/python3.2/dist-packages). Now, in
>> some sense this must be right (since the path is returned by
>> sysconfig), but the path produced
>> (/usr/local/lib/python3/dist-packages) is not searched by my Ubuntu
>> system's installed Python. If instead of automake I install the same
>> files with distutils (using a suitable setup.py), then the files are
>> installed in the right place (/usr/local/lib/python3.2/dist-packages).
>>    
> It seems to me this is vendor bug as the right place is site-packages, see PEP 370.
> 
> A cannot reproduce cut of version with "python" build from original sources,
> i.e. without extra patches :
> 2.6. 2.7 3.2 return .../pythonX.Y/site-packages
> 
> 
> [SNIP]
> 
> Roumen
> 
I'm going through old open bugs, and I've noticed this one.  Is the
problem still present, after the recent updates to the python support?
(They should be already merged in the maint branch by now)  If yes,
could you please provide me with a minimal, clear reproducer, or at
least restart this discussion from the basics?

Thanks,
  Stefano

Message #14 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 10227 <at> debbugs.gnu.org, Roumen Petrov <bugtrack <at> roumenpetrov.info>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Wed, 21 Nov 2012 17:58:47 +0000

[Message part 1 (text/plain, inline)]

On 21 November 2012 13:41, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:

> tags 10227 + moreinfo
> thanks
>
> Hi Roumen, Reuben.
>
> I'm going through old open bugs, and I've noticed this one.  Is the
> problem still present, after the recent updates to the python support?
> (They should be already merged in the maint branch by now)  If yes,
> could you please provide me with a minimal, clear reproducer, or at
> least restart this discussion from the basics?
>

I've just checked out the maint branch of automake, and I still see exactly
the same problem: in m4/python.m4 the code used to get the Python package
directory is:

am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import
sysconfig;
sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))"

Assuming the default am_py_prefix of /usr/local, this gives:

$ python3
Python 3.2.3 (default, Oct 19 2012, 19:53:16)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import sysconfig; print
(sysconfig.get_python_lib(0,0,'/usr/local'))
/usr/local/lib/python3/dist-packages

i.e. exactly the same incorrect result as before (the correct answer is:

/usr/local/lib/python3.2/dist-packages

on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
suggests, this is an Ubuntu packaging bug, or an automake bug. It seems to
me that this is the sort of thing the pyconfigure people ought to know
about.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #17 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 10227 <at> debbugs.gnu.org, Roumen Petrov <bugtrack <at> roumenpetrov.info>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Thu, 22 Nov 2012 14:01:07 +0100

tags 10227 - moreinfo
thanks

On 11/21/2012 06:58 PM, Reuben Thomas wrote:
> On 21 November 2012 13:41, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:
> 
>> tags 10227 + moreinfo
>> thanks
>>
>> Hi Roumen, Reuben.
>>
>> I'm going through old open bugs, and I've noticed this one.  Is the
>> problem still present, after the recent updates to the python support?
>> (They should be already merged in the maint branch by now)  If yes,
>> could you please provide me with a minimal, clear reproducer, or at
>> least restart this discussion from the basics?
>>
> 
> I've just checked out the maint branch of automake, and I still see exactly
> the same problem: in m4/python.m4 the code used to get the Python package
> directory is:
> 
> am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import
> sysconfig;
> sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))"
> 
> Assuming the default am_py_prefix of /usr/local, this gives:
> 
> $ python3
> Python 3.2.3 (default, Oct 19 2012, 19:53:16)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from distutils import sysconfig; print
> (sysconfig.get_python_lib(0,0,'/usr/local'))
> /usr/local/lib/python3/dist-packages
> 
> i.e. exactly the same incorrect result as before (the correct answer is:
> 
> /usr/local/lib/python3.2/dist-packages
> 
> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
> suggests, this is an Ubuntu packaging bug, or an automake bug.
>
OK, thanks for explaining it once again.  I can now reproduce the same
issue on Debian.  I think this is something we should try to work around,
since we cannot have our installation rules broken by default on both
Debian and Ubuntu ...

> It seems to me that this is the sort of thing the pyconfigure people
> ought to know about.
> 
By peeking at their repository, I came up with the minimal patch below,
which should do the trick.  Can you verify it works for you as well?

Regards,
  Stefano

---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ----

From 30463e31e481537462713cb680279aded6a9d449 Mon Sep 17 00:00:00 2001
Message-Id: <30463e31e481537462713cb680279aded6a9d449.1353588238.git.stefano.lattarini <at> gmail.com>
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Thu, 22 Nov 2012 12:03:07 +0100
Subject: [PATCH] python: make installed modules find by default on Debian and Ubuntu

This change fixes automake bug#10227.

The code used to get the python package directory was wrong for Python 3,
at least on Debian and Ubuntu distributions.  In the case the installation
was using the default prefix "/usr/local", python modules were incorrectly
installed in the directory

    /usr/local/lib/python3/dist-packages

(which is *not* searched by default), rather than in a directory like

    /usr/local/lib/python3.x/dist-packages

which is searched by default.

* m4/python.m4 (AM_PATH_PYTHON): Try to use the 'sysconfig' module if
possible, for better interactions with python 3.x.

Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
---
 m4/python.m4 | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/m4/python.m4 b/m4/python.m4
index 50213a9..ab3dfaa 100644
--- a/m4/python.m4
+++ b/m4/python.m4
@@ -106,6 +106,25 @@ AC_DEFUN([AM_PATH_PYTHON],
     [am_cv_python_platform=`$PYTHON -c "import sys; sys.stdout.write(sys.platform)"`])
   AC_SUBST([PYTHON_PLATFORM], [$am_cv_python_platform])

+  # Just factor out some code duplication.
+  am_python_setup_sysconfig="\
+import sys
+# Prefer sysconfig over distutils.sysconfig, for better compatibility
+# with python 3.x.  See automake bug#10227.
+try:
+    import sysconfig
+except ImportError:
+    can_use_sysconfig = 0
+else:
+    can_use_sysconfig = 1
+# Can't use sysconfig in CPython 2.7, since it's broken in virtualenvs:
+# <https://github.com/pypa/virtualenv/issues/118>
+try:
+    from platform import python_implementation
+    if python_implementation() == 'CPython' and sys.version[[:3]] == '2.7':
+        can_use_sysconfig = 0
+except ImportError:
+    pass"

   dnl Set up 4 directories:

@@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
      else
        am_py_prefix=$prefix
      fi
-     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 2>/dev/null`
+     am_cv_python_pythondir=`$PYTHON -c "
+$am_python_setup_sysconfig
+if can_use_sysconfig:
+    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
+else:
+    from distutils import sysconfig
+    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
+sys.stdout.write(sitedir)"`
      case $am_cv_python_pythondir in
      $am_py_prefix*)
        am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
@@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
      else
        am_py_exec_prefix=$exec_prefix
      fi
-     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))" 2>/dev/null`
+     am_cv_python_pyexecdir=`$PYTHON -c "
+$am_python_setup_sysconfig
+if can_use_sysconfig:
+    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})
+else:
+    from distutils import sysconfig
+    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
+sys.stdout.write(sitedir)"`
      case $am_cv_python_pyexecdir in
      $am_py_exec_prefix*)
        am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`
-- 
1.8.0.209.gf3828dc

Message #20 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 10227 <at> debbugs.gnu.org, Roumen Petrov <bugtrack <at> roumenpetrov.info>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Thu, 22 Nov 2012 13:19:05 +0000

[Message part 1 (text/plain, inline)]

On 22 November 2012 13:01, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:

> OK, thanks for explaining it once again.  I can now reproduce the same
> issue on Debian.  I think this is something we should try to work around,
> since we cannot have our installation rules broken by default on both
> Debian and Ubuntu ...
>
> > It seems to me that this is the sort of thing the pyconfigure people
> > ought to know about.
> >
> By peeking at their repository, I came up with the minimal patch below,
> which should do the trick.  Can you verify it works for you as well?
>

It does seem to work for me, but I think it would be worth getting expert
advice, e.g. from the pyconfigure developers, as to whether a) this is
definitely correct and b) whether it could be simpler.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #23 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Roumen Petrov <bugtrack <at> roumenpetrov.info>
To: 10227 <at> debbugs.gnu.org
Cc: Stefano Lattarini <stefano.lattarini <at> gmail.com>,
	Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Fri, 23 Nov 2012 00:35:50 +0200

Stefano Lattarini wrote:
> tags 10227 - moreinfo
> thanks
>
> On 11/21/2012 06:58 PM, Reuben Thomas wrote:
>> On 21 November 2012 13:41, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:
>>
>>> tags 10227 + moreinfo
>>> thanks
>>>
>>> Hi Roumen, Reuben.
>>>
>>> I'm going through old open bugs, and I've noticed this one.  Is the
>>> problem still present, after the recent updates to the python support?
[SNIP]
>>> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
>>> suggests, this is an Ubuntu packaging bug, or an automake bug.
It seems to me this is a feature as I found this one:

http://patch-tracker.debian.org/patch/series/view/python2.6/2.6.6-8/distutils-install-layout.diff

> OK, thanks for explaining it once again.  I can now reproduce the same
> issue on Debian.  I think this is something we should try to work around,
> since we cannot have our installation rules broken by default on both
> Debian and Ubuntu ...
Lets say platforms that use the "/posix_local/" installation scheme 
instead wrong or broken.

>> It seems to me that this is the sort of thing the pyconfigure people
>> ought to know about.
>>
> By peeking at their repository, I came up with the minimal patch below,
> which should do the trick.  Can you verify it works for you as well?
>
> Regards,
>    Stefano
>
> ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ---- 8< ----
>
>  From 30463e31e481537462713cb680279aded6a9d449 Mon Sep 17 00:00:00 2001
> Message-Id: <30463e31e481537462713cb680279aded6a9d449.1353588238.git.stefano.lattarini <at> gmail.com>
> From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
> Date: Thu, 22 Nov 2012 12:03:07 +0100
> Subject: [PATCH] python: make installed modules find by default on Debian and Ubuntu
>
> This change fixes automake bug#10227.
>
> The code used to get the python package directory was wrong for Python 3,
> at least on Debian and Ubuntu distributions.  In the case the installation
> was using the default prefix "/usr/local", python modules were incorrectly
> installed in the directory
>
>      /usr/local/lib/python3/dist-packages
>
> (which is *not* searched by default), rather than in a directory like
>
>      /usr/local/lib/python3.x/dist-packages
>
> which is searched by default.
>
> * m4/python.m4 (AM_PATH_PYTHON): Try to use the 'sysconfig' module if
> possible, for better interactions with python 3.x.



> Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
> ---
>   [SNIP]
>
> @@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>        else
>          am_py_prefix=$prefix
>        fi
> -     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 2>/dev/null`
> +     am_cv_python_pythondir=`$PYTHON -c "
> +$am_python_setup_sysconfig
> +if can_use_sysconfig:
> +    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
> +else:
> +    from distutils import sysconfig
> +    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
> +sys.stdout.write(sitedir)"`

Tested with python build from master branch on  2012-11-13 - both paths 
(sysconfig and distutils.sysconfig based) return same results.

>        case $am_cv_python_pythondir in
>        $am_py_prefix*)
>          am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
> @@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>        else
>          am_py_exec_prefix=$exec_prefix
>        fi
> -     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))" 2>/dev/null`
> +     am_cv_python_pyexecdir=`$PYTHON -c "
> +$am_python_setup_sysconfig
> +if can_use_sysconfig:
> +    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})

Stefano, please use platbase as variable - see Lib/sysconfig.py :
....
INSTALL_SCHEMES = {
    'posix_prefix': {
...
        'purelib': '{base}/lib/python{py_version_short}/site-packages',
        'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
...

> +else:
> +    from distutils import sysconfig
> +    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
> +sys.stdout.write(sitedir)"`
>        case $am_cv_python_pyexecdir in
>        $am_py_exec_prefix*)
>          am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`

Also I think that python experts should confirm if patch is correct.

Roumen

Message #26 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Roumen Petrov <bugtrack <at> roumenpetrov.info>
Cc: 10227 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Fri, 23 Nov 2012 10:26:54 +0100

On 11/22/2012 11:35 PM, Roumen Petrov wrote:
> Stefano Lattarini wrote:
>> tags 10227 - moreinfo
>> thanks
>>
>> On 11/21/2012 06:58 PM, Reuben Thomas wrote:
>>> On 21 November 2012 13:41, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:
>>>
>>>> tags 10227 + moreinfo
>>>> thanks
>>>>
>>>> Hi Roumen, Reuben.
>>>>
>>>> I'm going through old open bugs, and I've noticed this one.  Is the
>>>> problem still present, after the recent updates to the python support?
> [SNIP]
>>>> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
>>>> suggests, this is an Ubuntu packaging bug, or an automake bug.
> It seems to me this is a feature as I found this one:
> 
> http://patch-tracker.debian.org/patch/series/view/python2.6/2.6.6-8/distutils-install-layout.diff
>
The problem didn't lie in the Debian's use of 'dist-packages' instead
of 'site-packages' (as you noticed, that was present in python 2.6
already, and Automake still worked nicely); the issue is that, with
Python 3 on Debian, the call to

    distutils.sysconfig.get_python_lib(0,0,prefix='/usr/local')

returned:

    /usr/local/lib/python3/dist-packages

that is *not* searched by default by the 'import' statement.

>> OK, thanks for explaining it once again.  I can now reproduce the same
>> issue on Debian.  I think this is something we should try to work around,
>> since we cannot have our installation rules broken by default on both
>> Debian and Ubuntu ...
>
> Lets say platforms that use the "/posix_local/" installation scheme
> instead wrong or broken.
>
I failed to parse this, sorry.  Care to rephrase?

>>> It seems to me that this is the sort of thing the pyconfigure people
>>> ought to know about.
>>>
>> By peeking at their repository, I came up with the minimal patch below,
>> which should do the trick.  Can you verify it works for you as well?
>>

>> Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
>> ---
>>   [SNIP]
>>
>> @@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>        else
>>          am_py_prefix=$prefix
>>        fi
>> -     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 2>/dev/null`
>> +     am_cv_python_pythondir=`$PYTHON -c "
>> +$am_python_setup_sysconfig
>> +if can_use_sysconfig:
>> +    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
>> +else:
>> +    from distutils import sysconfig
>> +    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
>> +sys.stdout.write(sitedir)"`
> 
> Tested with python build from master branch on  2012-11-13 - both paths
> (sysconfig and distutils.sysconfig based) return same results.
>
Thanks for pointing that out.  I've given the code a try with a python 3.2
installed from sources, and I can confirm this happens for me as well.  So
no regression for that kind of setup, at least.

On the other hand, for a python 3.2 installed from Debian packages, the
results are different:

    # Code with distutils.sysconfig
    ${prefix}/lib/python3/dist-packages

    # Code with sysconfig
    ${prefix}/lib/python3.2/site-packages

The second result is the correct one; in fact, the 'dist-packages'
directory is meant as the location of python modules installed from
Debian packages (reference: <http://wiki.debian.org/Python>), so we
shouldn't need to install into such a directory; and when ${prefix}
is /usr/local, the "${prefix}/lib/python3.2/site-packages" directory
is searched by default by the 'import' statement.

At this point, I'm convinced my patch is quite correct.

>>        case $am_cv_python_pythondir in
>>        $am_py_prefix*)
>>          am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
>> @@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>        else
>>          am_py_exec_prefix=$exec_prefix
>>        fi
>> -     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))" 2>/dev/null`
>> +     am_cv_python_pyexecdir=`$PYTHON -c "
>> +$am_python_setup_sysconfig
>> +if can_use_sysconfig:
>> +    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})
> 
> Stefano, please use platbase as variable - see Lib/sysconfig.py:
> ....
> INSTALL_SCHEMES = {
>     'posix_prefix': {
> ...
>         'purelib': '{base}/lib/python{py_version_short}/site-packages',
>         'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
> ...
>
Good catch!  I will fix this.

>> +else:
>> +    from distutils import sysconfig
>> +    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
>> +sys.stdout.write(sitedir)"`
>>        case $am_cv_python_pyexecdir in
>>        $am_py_exec_prefix*)
>>          am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`
> 
> Also I think that python experts should confirm if patch is correct.
>
I'd love that.  Feel free to ping any GNU python expert about giving
a look at this patch.

Thanks,
  Stefano

Message #29 received at 10227 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Roumen Petrov <bugtrack <at> roumenpetrov.info>
Cc: 10227 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#10227: Python installation fails for Python 3
Date: Mon, 26 Nov 2012 13:00:28 +0100

tags 10227 + patch
close 10227
thanks

On 11/23/2012 10:26 AM, Stefano Lattarini wrote:
> On 11/22/2012 11:35 PM, Roumen Petrov wrote:
>> Stefano Lattarini wrote:
>>> tags 10227 - moreinfo
>>> thanks
>>>
>>> On 11/21/2012 06:58 PM, Reuben Thomas wrote:
>>>> On 21 November 2012 13:41, Stefano Lattarini <stefano.lattarini <at> gmail.com>wrote:
>>>>
>>>>> tags 10227 + moreinfo
>>>>> thanks
>>>>>
>>>>> Hi Roumen, Reuben.
>>>>>
>>>>> I'm going through old open bugs, and I've noticed this one.  Is the
>>>>> problem still present, after the recent updates to the python support?
>> [SNIP]
>>>>> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
>>>>> suggests, this is an Ubuntu packaging bug, or an automake bug.
>> It seems to me this is a feature as I found this one:
>>
>> http://patch-tracker.debian.org/patch/series/view/python2.6/2.6.6-8/distutils-install-layout.diff
>>
> The problem didn't lie in the Debian's use of 'dist-packages' instead
> of 'site-packages' (as you noticed, that was present in python 2.6
> already, and Automake still worked nicely); the issue is that, with
> Python 3 on Debian, the call to
> 
>     distutils.sysconfig.get_python_lib(0,0,prefix='/usr/local')
> 
> returned:
> 
>     /usr/local/lib/python3/dist-packages
> 
> that is *not* searched by default by the 'import' statement.
> 
>>> OK, thanks for explaining it once again.  I can now reproduce the same
>>> issue on Debian.  I think this is something we should try to work around,
>>> since we cannot have our installation rules broken by default on both
>>> Debian and Ubuntu ...
>>
>> Lets say platforms that use the "/posix_local/" installation scheme
>> instead wrong or broken.
>>
> I failed to parse this, sorry.  Care to rephrase?
> 
>>>> It seems to me that this is the sort of thing the pyconfigure people
>>>> ought to know about.
>>>>
>>> By peeking at their repository, I came up with the minimal patch below,
>>> which should do the trick.  Can you verify it works for you as well?
>>>
> 
>>> Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
>>> ---
>>>   [SNIP]
>>>
>>> @@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>>        else
>>>          am_py_prefix=$prefix
>>>        fi
>>> -     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 2>/dev/null`
>>> +     am_cv_python_pythondir=`$PYTHON -c "
>>> +$am_python_setup_sysconfig
>>> +if can_use_sysconfig:
>>> +    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
>>> +else:
>>> +    from distutils import sysconfig
>>> +    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
>>> +sys.stdout.write(sitedir)"`
>>
>> Tested with python build from master branch on  2012-11-13 - both paths
>> (sysconfig and distutils.sysconfig based) return same results.
>>
> Thanks for pointing that out.  I've given the code a try with a python 3.2
> installed from sources, and I can confirm this happens for me as well.  So
> no regression for that kind of setup, at least.
> 
> On the other hand, for a python 3.2 installed from Debian packages, the
> results are different:
> 
>     # Code with distutils.sysconfig
>     ${prefix}/lib/python3/dist-packages
> 
>     # Code with sysconfig
>     ${prefix}/lib/python3.2/site-packages
> 
> The second result is the correct one; in fact, the 'dist-packages'
> directory is meant as the location of python modules installed from
> Debian packages (reference: <http://wiki.debian.org/Python>), so we
> shouldn't need to install into such a directory; and when ${prefix}
> is /usr/local, the "${prefix}/lib/python3.2/site-packages" directory
> is searched by default by the 'import' statement.
> 
> At this point, I'm convinced my patch is quite correct.
> 
>>>        case $am_cv_python_pythondir in
>>>        $am_py_prefix*)
>>>          am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
>>> @@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>>        else
>>>          am_py_exec_prefix=$exec_prefix
>>>        fi
>>> -     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))" 2>/dev/null`
>>> +     am_cv_python_pyexecdir=`$PYTHON -c "
>>> +$am_python_setup_sysconfig
>>> +if can_use_sysconfig:
>>> +    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})
>>
>> Stefano, please use platbase as variable - see Lib/sysconfig.py:
>> ....
>> INSTALL_SCHEMES = {
>>     'posix_prefix': {
>> ...
>>         'purelib': '{base}/lib/python{py_version_short}/site-packages',
>>         'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
>> ...
>>
> Good catch!  I will fix this.
> 
>>> +else:
>>> +    from distutils import sysconfig
>>> +    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
>>> +sys.stdout.write(sitedir)"`
>>>        case $am_cv_python_pyexecdir in
>>>        $am_py_exec_prefix*)
>>>          am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`
>>
>> Also I think that python experts should confirm if patch is correct.
>>
> I'd love that.  Feel free to ping any GNU python expert about giving
> a look at this patch.
> 
I've pushed this patch, since it seems quite correct to me (and I
think we have correctly understood the underlying issue).  I'm ready
to revisit this change if anyone complains.

I'm marking this issue as "done".

Thanks,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.