GNU bug report logs - #74715
Request for merging "python-team" branch

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 74715 in the body.
You can then email your comments to 74715 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: guix-patches <at> gnu.org
Subject: Request for merging "python-team" branch
Date: Fri, 06 Dec 2024 21:34:33 +0000

[Message part 1 (text/plain, inline)]

Hi Guix!

I think it's a time to review and merge python-team branch.

It will bring refactored pyproject-build-system, refreshed collection of
packages from python-build module, refreshed python-numpy (1.24),
default python-pytest (8+), fresh python-flask, some clean up from
python-nose which is deprecated for few years.

Please test and report for any issues of the package you
love/need/demand.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 74715 <at> debbugs.gnu.org
Cc: sharlatanus <at> gmail.com, me <at> bonfacemunyoki.com, lars <at> 6xq.net,
 jgart <at> dismail.de, marius <at> gnu.org, guix-devel <at> gnu.org, tanguy <at> bioneland.org
Subject: Request for merging "python-team" branch
Date: Tue, 10 Dec 2024 19:31:38 +0000

[Message part 1 (text/plain, inline)]

CC Python team for wider spread.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Cc: tanguy <at> bioneland.org, me <at> bonfacemunyoki.com, lars <at> 6xq.net, jgart <at> dismail.de,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Fri, 13 Dec 2024 16:55:37 +0100

Hey there,

https://qa.guix.gnu.org/ says it’s your turn to merge!  :-)

Ludo’.

Message #14 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: "jgart" <jgart <at> dismail.de>
To: "Ludovic Courtès" <ludo <at> gnu.org>, "Sharlatan
 Hellseher" <sharlatanus <at> gmail.com>
Cc: tanguy <at> bioneland.org, me <at> bonfacemunyoki.com, lars <at> 6xq.net,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Fri, 13 Dec 2024 17:00:07 +0000

[Message part 1 (text/plain, inline)]

Hi Sharlatan,

Guix cheerleader here. Go for it! Merge ahoy!
🦜🦆
 LGTM

[Message part 2 (text/html, inline)]

Message #19 received at 74715-done <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 74715-done <at> debbugs.gnu.org
Subject: Request for merging "python-team" branch
Date: Sat, 14 Dec 2024 10:03:07 +0000

[Message part 1 (text/plain, inline)]

Hi,

python-team branch has been rebased, conflicts resolved (python-redis
and python-pydevd) and merged to master.

Long lists of rebuild are in CI right now:

- <https://ci.guix.gnu.org/eval/1903757>
- <https://ci.guix.gnu.org/eval/1903820>

Post merge fix-up might be required like adding missing build time
dependencies (python-setuptools, python-wheel) for
pyproject-build-system packages not refreshed for a while.

--
Thanks,
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: "Ian Eure" <ian <at> retrospec.tv>
To: jgart <jgart <at> dismail.de>, Ludovic Courtès <ludo <at> gnu.org>, 
 "Sharlatan Hellseher" <sharlatanus <at> gmail.com>
Cc: tanguy <at> bioneland.org, me <at> bonfacemunyoki.com, lars <at> 6xq.net,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 01:00:43 +0000

[Message part 1 (text/plain, inline)]

Hi all,

Since this merge landed, the builds for several Python packages in my personal channel broke.  Any package using pyproject-build-system for a Python project using setuptools seems to be affected.  This python-manhole package[1] is an example.  It's  about as simple as they can get, with no inputs or custom build steps.  It's failing with:

------------------------------------------------------------
starting phase `build'
Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.
Prepending '[]' to sys.path, auto-detected '#f', override '#f'.
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/gnu/store/jjcka1g6sk2cvwx8nm4fdwpdq3vll0v0-python-3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'setuptools'
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `build' failed after 0.0 seconds
command "python" "-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}" failed with status 1
build process 10 exited with status 256
------------------------------------------------------------

Since it's complaining about setuptools, I thought I might need to add that to the native-inputs.  Alas, that fails with a different error:

------------------------------------------------------------
starting phase `build'
Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.
Prepending '[]' to sys.path, auto-detected '#f', override '#f'.
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: -c --help [cmd1 cmd2 ...]
   or: -c --help-commands
   or: -c cmd --help

error: invalid command 'bdist_wheel'
------------------------------------------------------------

Is there somewhere I can find out how to fix these packages for the updated pyproject-build-system?  Should they be getting switched to python-build-system?

Noting also that the PyPI module for `guix import' hardcodes the pyproject-build-system, so it will generate unbuildable definitions for any Python project which uses setuptools.

Thanks,

  -- Ian

[1]: https://codeberg.org/ieure/atomized-guix/src/branch/main/atomized/packages/python-xyz.scm#L22

On Fri, Dec 13, 2024, at 5:00 PM, jgart wrote:
> Hi Sharlatan,
> 
> Guix cheerleader here. Go for it! Merge ahoy!
> 🦜🦆
> LGTM
> 
> 

[Message part 2 (text/html, inline)]

Message #25 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Kaelyn <kaelyn.alexi <at> protonmail.com>
To: Ian Eure <ian <at> retrospec.tv>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>, lars <at> 6xq.net,
 jgart <jgart <at> dismail.de>, 74715 <at> debbugs.gnu.org, marius <at> gnu.org,
 guix-devel <at> gnu.org, tanguy <at> bioneland.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 06:24:26 +0000

Hi Ian,

On Saturday, December 14th, 2024 at 5:00 PM, Ian Eure <ian <at> retrospec.tv> wrote:

> Hi all,
> 
> Since this merge landed, the builds for several Python packages in my personal channel broke.  Any package using pyproject-build-system for a Python project using setuptools seems to be affected.  This python-manhole package[1] is an example.  It's  about as simple as they can get, with no inputs or custom build steps.  It's failing with:
> 
> ------------------------------------------------------------
> starting phase `build'
> Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.
> Prepending '[]' to sys.path, auto-detected '#f', override '#f'.
> Traceback (most recent call last):
>   File "<string>", line 6, in <module>
>   File "/gnu/store/jjcka1g6sk2cvwx8nm4fdwpdq3vll0v0-python-3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
> ModuleNotFoundError: No module named 'setuptools'
> error: in phase 'build': uncaught exception:
> %exception #<&invoke-error program: "python" arguments: ("-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}") exit-status: 1 term-signal: #f stop-signal: #f> 
> phase `build' failed after 0.0 seconds
> command "python" "-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}" failed with status 1
> build process 10 exited with status 256
> ------------------------------------------------------------
> 
> Since it's complaining about setuptools, I thought I might need to add that to the native-inputs.  Alas, that fails with a different error:
> 
> ------------------------------------------------------------
> starting phase `build'
> Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.
> Prepending '[]' to sys.path, auto-detected '#f', override '#f'.
> usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
>    or: -c --help [cmd1 cmd2 ...]
>    or: -c --help-commands
>    or: -c cmd --help
> 
> error: invalid command 'bdist_wheel'
> ------------------------------------------------------------

I don't have any comment about the pyproject-build-system issue in general, but I've done enough maintenance of Python packages in general to be familiar with this error. It should be as simple as adding `python-wheel` to the native-inputs--it is the module that allows setuptools to generate .whl files, and I'd be surprised if additional packages were needed (I've encountered having to add explicit setuptools and wheel dependencies to requirements.txt files for pip when changing the build environment of packages such as changing distros or Python versions). HTH!

Cheers,
Kaelyn

> Is there somewhere I can find out how to fix these packages for the updated pyproject-build-system?  Should they be getting switched to python-build-system?
> 
> Noting also that the PyPI module for `guix import' hardcodes the pyproject-build-system, so it will generate unbuildable definitions for any Python project which uses setuptools.
> 
> Thanks,
> 
>   -- Ian
> 
> [1]: https://codeberg.org/ieure/atomized-guix/src/branch/main/atomized/packages/python-xyz.scm#L22
> 
> On Fri, Dec 13, 2024, at 5:00 PM, jgart wrote:
> 
> > Hi Sharlatan,
> > 
> > Guix cheerleader here. Go for it! Merge ahoy!
> > 🦜🦆
> > LGTM
> > 
> > 
> 
>

Message #28 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Ian Eure <ian <at> retrospec.tv>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 07:28:49 +0100

Hi Ian,

> Since this merge landed, the builds for several Python packages in my personal channel broke.  Any package using pyproject-build-system for a Python project using setuptools seems to be affected.

as Sharlatan Hellseher wrote in https://issues.guix.gnu.org/issue/74715#4,
you need to add python-setuptools and python-wheel to your
setuptools-based packages. The default python toolchain used by
pyproject-build-system (python-sans-pip-wrapper from
gnu/packages/python.scm) does not include these packages any more,
since they are technically not required and declaring them as *real*
inputs allows using different versions of these packages more easily
for packages, which require them. Plus there are quite a few packages,
which build using different build systems nowadays.

The python importer should probably be updated to read pyproject.toml
and parse the [build-system] table (there is a toml parser in Guix now,
so this should be easy).

Lars

Message #31 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>, Ian Eure <ian <at> retrospec.tv>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>,
 Ludovic Courtès <ludo <at> gnu.org>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>,
 guix-devel <guix-devel <at> gnu.org>
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 10:29:03 +0000

[Message part 1 (text/plain, inline)]

Hi,

Maybe we may put some news about changes in pyproject build system?

Thanks
Oleg

[Message part 2 (text/html, inline)]

Message #34 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: "Ian Eure" <ian <at> retrospec.tv>
To: "Lars-Dominik Braun" <lars <at> 6xq.net>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 17:53:07 +0000

Hi Lars-Dominik,

On Sun, Dec 15, 2024, at 6:28 AM, Lars-Dominik Braun wrote:
> Hi Ian,
>
>> Since this merge landed, the builds for several Python packages in my personal channel broke.  Any package using pyproject-build-system for a Python project using setuptools seems to be affected.
>
> as Sharlatan Hellseher wrote in https://issues.guix.gnu.org/issue/74715#4,
> you need to add python-setuptools and python-wheel to your
> setuptools-based packages. The default python toolchain used by
> pyproject-build-system (python-sans-pip-wrapper from
> gnu/packages/python.scm) does not include these packages any more,
> since they are technically not required and declaring them as *real*
> inputs allows using different versions of these packages more easily
> for packages, which require them. Plus there are quite a few packages,
> which build using different build systems nowadays.
>

Thanks, this worked for me.  I skimmed the related bug, but missed this comment.  I think the docs for pyproject-build-system are likely the best place for this, as they already mention some of the setuptools/pyproject interaction.  I sent a patch (#74899) with some draft language, let me know what you think.

> The python importer should probably be updated to read pyproject.toml
> and parse the [build-system] table (there is a toml parser in Guix now,
> so this should be easy).
>

Would it be helpful to open a bug about this?

Thanks,

  -- Ian

Message #37 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Cayetano Santos <csantosb <at> inventati.org>
To: "Ian Eure" <ian <at> retrospec.tv>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>,
 Lars-Dominik Braun <lars <at> 6xq.net>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 19:05:22 +0100

[Message part 1 (text/plain, inline)]

>dim. 15 déc. 2024 at 17:53, "Ian Eure" <ian <at> retrospec.tv> wrote:

> Would it be helpful to open a bug about this?

I confirm, this line

    (inputs (list python-setuptools python-wheel))

fixes the issue for me.

However, ‘guix lint’ complaints about python-setuptools not necessary.

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>, Ian Eure <ian <at> retrospec.tv>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>,
 Ludovic Courtès <ludo <at> gnu.org>,
 Lars-Dominik Braun <lars <at> 6xq.net>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>,
 guix-devel <guix-devel <at> gnu.org>
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 13:12:27 -0500

On Sun, Dec 15, 2024 at 10:29:03AM +0000, Sharlatan Hellseher wrote:
> Hi,
> 
> Maybe we may put some news about changes in pyproject build system?

I think that's a great idea.

Message #43 received at 74715 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Ian Eure <ian <at> retrospec.tv>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>, jgart <jgart <at> dismail.de>,
 74715 <at> debbugs.gnu.org, marius <at> gnu.org, guix-devel <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: bug#74715: Request for merging "python-team" branch
Date: Sun, 15 Dec 2024 22:01:49 +0100

Hi,

> Would it be helpful to open a bug about this?

there is https://issues.guix.gnu.org/69997, which includes patches to
add initial pyproject.toml support to `guix import pypi`.

Lars

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.