GNU bug report logs - #76211
Incomplete python-django-compressor package

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76211 in the body.
You can then email your comments to 76211 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Pisemsky <mail <at> pisemsky.site>
To: bug-guix <at> gnu.org
Subject: Incomplete python-django-compressor package
Date: Tue, 11 Feb 2025 22:12:28 +0300

For some reason installed package lacks templates and other media
files listed in the source's MANIFEST.in, which makes it unusable.

Tests can't prove it because they are run in the source directory.

Manual install from source into venv with =pip install .= works as
expected, so this could be a bug in python/pyproject build system.

Message #10 received at 76211 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> friendly-machines.com>
To: Evgeny Pisemsky <mail <at> pisemsky.site>
Cc: 76211 <at> debbugs.gnu.org
Subject: Re: bug#76211: Incomplete python-django-compressor package
Date: Wed, 26 Mar 2025 03:06:17 +0100

Hi,

MANIFEST.in contains a list of files that are not temp files.
It's basically a poor-man's version control staging.  When you invoke
"python3 setup.py sdist" (which creates a *source* distribution),
that will include all the files specified in MANIFEST.in .

As for what happens on installation (which is a different thing),
whether something is in MANIFEST.in or not shouldn't really change
anything by default (except when you set include_package_data--and
even then only in some cases).
It's interesting that for pip install, that does install templates.
What setup.py command does that use?
Which directory exactly does it put the templates in?

Are the templates and other media required for using the package as part
of a program as an end user--or only if you want to *develop* a new
package that is using python-django-compressor?

Depending on the answer to that, setup.py's include_package_data=True
could be useful.  See also <https://github.com/pypa/setuptools/issues/3732>.

Alternatively, setup.py's data_files could be set.  It would also be a
lot less opaque that way.

See <https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html>,
especially footnote 2 there.

Message #13 received at 76211 <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Pisemsky <mail <at> pisemsky.site>
To: Danny Milosavljevic <dannym <at> friendly-machines.com>
Cc: 76211 <at> debbugs.gnu.org
Subject: Re: bug#76211: Incomplete python-django-compressor package
Date: Thu, 03 Apr 2025 01:40:28 +0300

Hi!

> It's interesting that for pip install, that does install templates.
> What setup.py command does that use?

This is what pip3 in virtualenv says:

#+begin_example
Using legacy 'setup.py install' for django-compressor, since package 'wheel' is not installed.
Installing collected packages: rjsmin, rcssmin, typing-extensions, sqlparse, asgiref, Django, django-appconf, django-compressor
  Running setup.py install for django-compressor ... done
#+end_example

Everything with pure python, i.e. =guix shell python --pure=.

> Which directory exactly does it put the templates in?

After =pip3 install .= from local source in virtualenv:

#+begin_example
env/lib/python3.10/site-packages/compressor/templates/
env/lib/python3.10/site-packages/compressor/templates/compressor
env/lib/python3.10/site-packages/compressor/templates/compressor/css_file.html
env/lib/python3.10/site-packages/compressor/templates/compressor/js_preload.html
env/lib/python3.10/site-packages/compressor/templates/compressor/css_preload.html
env/lib/python3.10/site-packages/compressor/templates/compressor/js_inline.html
env/lib/python3.10/site-packages/compressor/templates/compressor/js_file.html
env/lib/python3.10/site-packages/compressor/templates/compressor/css_inline.html
#+end_example

> Are the templates and other media required for using the package as part
> of a program as an end user--or only if you want to *develop* a new
> package that is using python-django-compressor?

Templates are definitely required in runtime. I have a guixified
django project that started throwing error after upgrade - that's how
I spotted this problem. I fixed it with copy-recursively in phases.

As for suggested solutions, the main problem here is that everything
works fine in "normal" python ecosystem, but not in guix build systems.

Message #18 received at 76211-done <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 76211-done <at> debbugs.gnu.org
Subject: Incomplete python-django-compressor package
Date: Thu, 17 Apr 2025 23:34:59 +0100

[Message part 1 (text/plain, inline)]

Hi,

Thanks for reporting this.

I could investigate it and find the root of that problem, and it was a
custom function detecting which files need to be copied to "build"

--8<---------------cut here---------------start------------->8---
def find_package_data(
--8<---------------cut here---------------end--------------->8---
<https://github.com/django-compressor/django-compressor/blob/4.5.1/setup.py#L51C1-L52C1>

As package has no many dependent I could push it to master, after local build:

--8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix build -P1 build python-django-compressor
guix build: computing dependents of package build <at> 0.3.10...
/gnu/store/r5k54f35gdqrkwzdg5xp4433wwpg0ps4-build-0.3.10
/gnu/store/vimdx5lvf453591rc8d2f802nz32nx4j-cli-1.1.0
/gnu/store/mcp3d7jiljl6s63vwf0dqn99mv4sg0g4-xsd-4.0.0-doc
/gnu/store/lgpqmp9hjlva9cgxvm2x420jd8nwl4h5-xsd-4.0.0
/gnu/store/wrh3k5cwvp8dcx40bkd6583zrhd5w0w8-libxsd-frontend-2.0.0
/gnu/store/v0hdy25fxh6kpmjx90p0zsarncm5hq0n-python-django-compressor-4.5.1
--8<---------------cut here---------------end--------------->8---

Templates dirrectory is present in the built package now:
--8<---------------cut here---------------start------------->8---
> tree -d $(./pre-inst-env guix build python-django-compressor)/lib/
/gnu/store/v0hdy25fxh6kpmjx90p0zsarncm5hq0n-python-django-compressor-4.5.1/lib/
└── python3.10
    └── site-packages
        ├── compressor
        │   ├── contrib
        │   │   └── __pycache__
        │   ├── filters
        │   │   ├── cssmin
        │   │   │   └── __pycache__
        │   │   ├── jsmin
        │   │   │   └── __pycache__
        │   │   └── __pycache__
        │   ├── management
        │   │   ├── commands
        │   │   │   └── __pycache__
        │   │   └── __pycache__
        │   ├── offline
        │   │   └── __pycache__
        │   ├── parser
        │   │   └── __pycache__
        │   ├── __pycache__
        │   ├── templates
        │   │   ├── compressor
        │   │   │   └── __pycache__
        │   │   └── __pycache__
        │   ├── templatetags
        │   │   └── __pycache__
        │   ├── tests
        │   │   └── __pycache__
        │   └── utils
        │       └── __pycache__
        └── django_compressor-4.5.1.dist-info

32 directories
--8<---------------cut here---------------end--------------->8---

Pushed fixes to master as:
0879cb6f66 * master gnu: python-django-compressor: Fix Python package data.
8335f7b441 * gnu: python-django-compressor: Fix indentation.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 76211 <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Pisemsky <mail <at> pisemsky.site>
To: 76211 <at> debbugs.gnu.org
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: Re: bug#76211: closed (Incomplete python-django-compressor package)
Date: Fri, 18 Apr 2025 20:58:10 +0300

> Templates dirrectory is present in the built package now:

Hi, thanks, but there are no templates in it:

#+begin_example
|           |   |-- templates
|           |   |   |-- __init__.py
|           |   |   |-- __pycache__
|           |   |   |   `-- __init__.cpython-310.pyc
|           |   |   `-- compressor
|           |   |       |-- __init__.py
|           |   |       `-- __pycache__
|           |   |           `-- __init__.cpython-310.pyc
#+end_example

Also making those dirs python packages may create problems on its own.

Django looks for templates by naming convetion and does not need this.

Message #28 received at 76211 <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Pisemsky <mail <at> pisemsky.site>
To: 76211 <at> debbugs.gnu.org
Subject: [PATCH] gnu: python-django-compressor: Add missing files.
Date: Sat, 19 Apr 2025 14:34:32 +0300

* gnu/packages/django.scm (python-django-compressor): Add missing files.
[arguments] <phases>: Remove 'add-missing-init-files, add 'add-missing-files
to recursively copy all necessary non-python files. In 'check phase change
directory to run tests against installed package.

Change-Id: Iec7139607510ae09972241f459376750b3d7e461
---
 gnu/packages/django.scm | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/django.scm b/gnu/packages/django.scm
index 461affdef9..aedd83a52e 100644
--- a/gnu/packages/django.scm
+++ b/gnu/packages/django.scm
@@ -1468,15 +1468,19 @@ (define-public python-django-compressor
             (substitute* "compressor/tests/test_filters.py"
               (("test_calmjs_filter") "_test_calmjs_filter"))))
         ;; TODO: Report upstream.
-        (add-after 'unpack 'add-missing-init-files
-          (lambda _
-            (call-with-output-file "compressor/templates/__init__.py"
-              (const #t))
-            (call-with-output-file "compressor/templates/compressor/__init__.py"
-              (const #t))))
+        (add-after 'install 'add-missing-files
+          (lambda* (#:key inputs outputs #:allow-other-keys)
+            (let ((sitedir (site-packages inputs outputs)))
+              (for-each (lambda (dir)
+                          (copy-recursively dir (string-append sitedir "/" dir)))
+                        '("compressor/templates/compressor"
+                          "compressor/tests/static"
+                          "compressor/tests/test_templates"
+                          "compressor/tests/test_templates_jinja2")))))
         (replace 'check
-          (lambda* (#:key tests? #:allow-other-keys)
+          (lambda* (#:key tests? inputs outputs #:allow-other-keys)
             (when tests?
+              (chdir (site-packages inputs outputs))
               (setenv "DJANGO_SETTINGS_MODULE" "compressor.test_settings")
               (invoke "django-admin" "test"
                       "--pythonpath=.")))))))

base-commit: b276cf2f9785c71cc37688c3e2b1db38e2c710a1
-- 
2.49.0

Message #31 received at 76211 <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Pisemsky <mail <at> pisemsky.site>
To: 76211 <at> debbugs.gnu.org
Subject: [PATCH v1] gnu: python-django-compressor: Run tests against installed
 package.
Date: Sat, 19 Apr 2025 15:37:22 +0300

* gnu/packages/django.scm (python-django-compressor): Run tests against installed package.

Change-Id: Idd810376921d1802adb9355c52edcd1a8584eef7
---
 gnu/packages/django.scm | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/django.scm b/gnu/packages/django.scm
index 8175e0ad6c..d152e05d87 100644
--- a/gnu/packages/django.scm
+++ b/gnu/packages/django.scm
@@ -1471,14 +1471,11 @@ (define-public python-django-compressor
         (add-after 'unpack 'add-missing-init-files
           (lambda _
             (substitute* "setup.py"
-              (("package_data=.*,") "include_package_data=True,"))
-            (call-with-output-file "compressor/templates/__init__.py"
-              (const #t))
-            (call-with-output-file "compressor/templates/compressor/__init__.py"
-              (const #t))))
+              (("package_data=.*,") "include_package_data=True,"))))
         (replace 'check
-          (lambda* (#:key tests? #:allow-other-keys)
+          (lambda* (#:key tests? inputs outputs #:allow-other-keys)
             (when tests?
+              (chdir (site-packages inputs outputs))
               (setenv "DJANGO_SETTINGS_MODULE" "compressor.test_settings")
               (invoke "django-admin" "test"
                       "--pythonpath=.")))))))

base-commit: d5fc8668f461e6f2246a2861ad53df725c7dd766
-- 
2.49.0

Message #36 received at 76211-done <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 76211-done <at> debbugs.gnu.org
Subject: Incomplete python-django-compressor package
Date: Fri, 16 May 2025 21:59:43 +0100

[Message part 1 (text/plain, inline)]

Hi,

I've slightly adjust the style and open PR in upstream.

Pushed to master as 40cd5e511cf96dc4036d256e5093dccf74a48dc4.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.