GNU bug report logs - #49086
[PATCH] gnu: linkchecker: Update to 10.0.1.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49086 in the body.
You can then email your comments to 49086 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: guix-patches <at> gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH] gnu: linkchecker: Update to 10.0.1.
Date: Fri, 18 Jun 2021 17:30:22 +0530

* gnu/packages/web.scm (linkchecker): Update to 10.0.1.
[source]: Remove patches.
[inputs]: Add python-beautifulsoup4. Replace python2-dnspython-1.16,
python2-pyxdg and python2-requests with python-dnspython, python-pyxdg and
python-requests respectively.
[native-inputs]: Replace python2-pytest, python2-miniboa and
python2-parameterized with python-pytest, python-miniboa and
python-parameterized respectively.
[arguments]: Use python 3. Replace check phase instead of deleting the
standard phase and adding a custom one. Use add-installed-pythonpath instead
of setting PYTHONPATH directly.
[home-page]: Update URI.
* gnu/packages/patches/linkchecker-tests-require-network.patch: Delete file.
* gnu/local.mk (dist_patch_DATA): Unregister it.
---
 gnu/local.mk                                  |   1 -
 .../linkchecker-tests-require-network.patch   | 182 ------------------
 gnu/packages/web.scm                          |  47 ++---
 3 files changed, 16 insertions(+), 214 deletions(-)
 delete mode 100644 gnu/packages/patches/linkchecker-tests-require-network.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 918c8515ac..b2b36d860a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1383,7 +1383,6 @@ dist_patch_DATA =						\
   %D%/packages/patches/lierolibre-remove-arch-warning.patch	\
   %D%/packages/patches/lierolibre-try-building-other-arch.patch	\
   %D%/packages/patches/linbox-fix-pkgconfig.patch		\
-  %D%/packages/patches/linkchecker-tests-require-network.patch	\
   %D%/packages/patches/linphone-desktop-without-sdk.patch           \
   %D%/packages/patches/linux-libre-support-for-Pinebook-Pro.patch \
   %D%/packages/patches/linux-libre-arm64-generic-pinebook-lcd.patch \
diff --git a/gnu/packages/patches/linkchecker-tests-require-network.patch b/gnu/packages/patches/linkchecker-tests-require-network.patch
deleted file mode 100644
index f3e488cec2..0000000000
--- a/gnu/packages/patches/linkchecker-tests-require-network.patch
+++ /dev/null
@@ -1,182 +0,0 @@
-From f24c88a0732024028fffe0372039a847e91722ea Mon Sep 17 00:00:00 2001
-From: Christopher Baines <mail <at> cbaines.net>
-Date: Tue, 1 Jan 2019 22:36:29 +0000
-Subject: [PATCH] Mark more tests that require the network
-
-I believe all these tests require the network, at least they seem to
-fail if it's I run them without connecting my computer to the web.
-
-I'm looking at this as part of packaging linkchecker for GNU Guix,
-where the package is build and the tests are run in a isolated
-environment, intentionally without network access, to avoid issues
-with non-reproducible package builds.
----
- tests/checker/test_http.py          | 2 ++
- tests/checker/test_http_misc.py     | 2 ++
- tests/checker/test_http_redirect.py | 2 ++
- tests/checker/test_httpbin.py       | 5 +++++
- tests/checker/test_misc.py          | 4 ++++
- tests/checker/test_whitespace.py    | 3 +++
- 6 files changed, 18 insertions(+)
-
-diff --git a/tests/checker/test_http.py b/tests/checker/test_http.py
-index e4c1e097..8a8af567 100644
---- a/tests/checker/test_http.py
-+++ b/tests/checker/test_http.py
-@@ -20,6 +20,7 @@
- 
- import pytest
- 
-+from tests import need_network
- from .httpserver import HttpServerTest, CookieRedirectHttpRequestHandler
- 
- class TestHttp (HttpServerTest):
-@@ -29,6 +30,7 @@ def __init__(self, methodName='runTest'):
-         super(TestHttp, self).__init__(methodName=methodName)
-         self.handler = CookieRedirectHttpRequestHandler
- 
-+    @need_network
-     def test_html (self):
-         confargs = dict(recursionlevel=1)
-         self.file_test("http.html", confargs=confargs)
-diff --git a/tests/checker/test_http_misc.py b/tests/checker/test_http_misc.py
-index 9922d85f..c6b6afdb 100644
---- a/tests/checker/test_http_misc.py
-+++ b/tests/checker/test_http_misc.py
-@@ -20,11 +20,13 @@
- import os
- import sys
- from .httpserver import HttpServerTest
-+from tests import need_network
- from linkcheck.network import iputil
- 
- class TestHttpMisc (HttpServerTest):
-     """Test http:// misc link checking."""
- 
-+    @need_network
-     def test_html (self):
-         self.swf_test()
-         self.obfuscate_test()
-diff --git a/tests/checker/test_http_redirect.py b/tests/checker/test_http_redirect.py
-index f212d98e..2253a70d 100644
---- a/tests/checker/test_http_redirect.py
-+++ b/tests/checker/test_http_redirect.py
-@@ -17,6 +17,7 @@
- """
- Test http checking.
- """
-+from tests import need_network
- from .httpserver import HttpServerTest, CookieRedirectHttpRequestHandler
- 
- class TestHttpRedirect (HttpServerTest):
-@@ -26,6 +27,7 @@ def __init__(self, methodName='runTest'):
-         super(TestHttpRedirect, self).__init__(methodName=methodName)
-         self.handler = CookieRedirectHttpRequestHandler
- 
-+    @need_network
-     def test_redirect (self):
-         self.redirect1()
-         self.redirect2()
-diff --git a/tests/checker/test_httpbin.py b/tests/checker/test_httpbin.py
-index 0319c2f6..4c8fa846 100644
---- a/tests/checker/test_httpbin.py
-+++ b/tests/checker/test_httpbin.py
-@@ -18,6 +18,7 @@
- Test http stuff with httpbin.org.
- """
- import re
-+from tests import need_network
- from . import LinkCheckTest
- 
- 
-@@ -30,6 +31,7 @@ def get_httpbin_url(path):
- class TestHttpbin(LinkCheckTest):
-     """Test http:// link redirection checking."""
- 
-+    @need_network
-     def test_http_link(self):
-         linkurl = u"http://www.example.com"
-         nlinkurl = self.norm(linkurl)
-@@ -48,6 +50,7 @@ def test_http_link(self):
-         ]
-         self.direct(url, resultlines, recursionlevel=1)
- 
-+    @need_network
-     def test_basic_auth(self):
-         user = u"testuser"
-         password = u"testpassword"
-@@ -67,6 +70,7 @@ def test_basic_auth(self):
-         ]
-         self.direct(url, resultlines, confargs=confargs)
- 
-+    @need_network
-     def test_http_refresh_header(self):
-         linkurl = u"http://www.example.com"
-         nlinkurl = self.norm(linkurl)
-@@ -85,6 +89,7 @@ def test_http_refresh_header(self):
-         ]
-         self.direct(url, resultlines, recursionlevel=1)
- 
-+    @need_network
-     def test_http_content_location_header(self):
-         linkurl = u"http://www.example.com"
-         nlinkurl = self.norm(linkurl)
-diff --git a/tests/checker/test_misc.py b/tests/checker/test_misc.py
-index 2e4cfd07..f9591f9d 100644
---- a/tests/checker/test_misc.py
-+++ b/tests/checker/test_misc.py
-@@ -17,6 +17,7 @@
- """
- Test miscellaneous html tag parsing and URL types
- """
-+from tests import need_network
- from . import LinkCheckTest
- 
- 
-@@ -25,15 +26,18 @@ class TestMisc (LinkCheckTest):
-     Test misc link types.
-     """
- 
-+    @need_network
-     def test_misc (self):
-         self.file_test("misc.html")
- 
-     def test_html5 (self):
-         self.file_test("html5.html")
- 
-+    @need_network
-     def test_archive (self):
-         self.file_test("archive.html")
- 
-+    @need_network
-     def test_itms_services(self):
-         url = u"itms-services:?action=download-manifest&url=http://www.example.com/"
-         resultlines = [
-diff --git a/tests/checker/test_whitespace.py b/tests/checker/test_whitespace.py
-index 609c108a..fc2727d6 100644
---- a/tests/checker/test_whitespace.py
-+++ b/tests/checker/test_whitespace.py
-@@ -17,6 +17,7 @@
- """
- Test whitespace handling.
- """
-+from tests import need_network
- from . import LinkCheckTest
- 
- 
-@@ -25,6 +26,7 @@ class TestWhitespace (LinkCheckTest):
-     Test whitespace in URLs.
-     """
- 
-+    @need_network
-     def test_leading_whitespace (self):
-         # Leading whitespace
-         url = u" http://www.example.org/"
-@@ -50,6 +52,7 @@ def test_leading_whitespace (self):
-         ]
-         self.direct(url, resultlines)
- 
-+    @need_network
-     def test_trailing_whitespace (self):
-         # Trailing whitespace
-         url = u"http://www.example.org/ "
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index d1f1d0f755..0fc2d7e441 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -15,7 +15,7 @@
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben <at> gmail.com>
 ;;; Copyright © 2016 Clément Lassieur <clement <at> lassieur.org>
 ;;; Copyright © 2016, 2017 Nikita <nikita <at> n0.is>
-;;; Copyright © 2016, 2017, 2018, 2019 Arun Isaac <arunisaac <at> systemreboot.net>
+;;; Copyright © 2016, 2017, 2018, 2019, 2021 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2016–2021 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;; Copyright © 2016 Bake Timmons <b3timmons <at> speedymail.org>
 ;;; Copyright © 2017 Thomas Danckaert <post <at> thomasdanckaert.be>
@@ -6578,51 +6578,36 @@ Instagram and YouTube.")
 (define-public linkchecker
   (package
     (name "linkchecker")
-    (version "9.4.0")
+    (version "10.0.1")
     (source
      (origin
        (method git-fetch)
        (uri (git-reference
              (url "https://github.com/linkchecker/linkchecker")
              (commit (string-append "v" version))))
-       (patches
-        (search-patches "linkchecker-tests-require-network.patch"))
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "03ihjmc4bqxxqv71bb43r2f23sx0xnbq1k2fsg9fw05qa5s9x187"))))
+         "1j97dc9a4yhpscwadhv5dxp7036pnrxiaky18l8ddr3pvxdjvkxs"))))
     (build-system python-build-system)
     (inputs
-     `(("python2-dnspython" ,python2-dnspython-1.16)
-       ("python2-pyxdg" ,python2-pyxdg)
-       ("python2-requests" ,python2-requests)))
+     `(("python-beautifulsoup4" ,python-beautifulsoup4)
+       ("python-dnspython" ,python-dnspython)
+       ("python-pyxdg" ,python-pyxdg)
+       ("python-requests" ,python-requests)))
     (native-inputs
      `(("gettext" ,gettext-minimal)
-       ("python2-pytest" ,python2-pytest)
-       ("python2-miniboa" ,python2-miniboa)
-       ("python2-parameterized" ,python2-parameterized)))
+       ("python-pytest" ,python-pytest)
+       ("python-miniboa" ,python-miniboa)
+       ("python-parameterized" ,python-parameterized)))
     (arguments
-     `(#:python ,python-2
-       #:phases
+     `(#:phases
        (modify-phases %standard-phases
-         ;; Move the 'check phase to after 'install, so that the installed
-         ;; library can be used
-         (delete 'check)
-         (add-after 'install 'check
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let ((out (assoc-ref outputs "out")))
-               ;; Set PYTHONPATH so that the installed linkchecker is used
-               (setenv "PYTHONPATH"
-                       (string-append out "/lib/python2.7/site-packages"
-                                      ":"
-                                      (getenv "PYTHONPATH")))
-               ;; Remove this directory to avoid it being used when running
-               ;; the tests
-               (delete-file-recursively "linkcheck")
-
-               (invoke "py.test" "tests"))
-             #t)))))
-    (home-page "https://linkcheck.github.io/linkchecker")
+         (replace 'check
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (add-installed-pythonpath inputs outputs)
+             (invoke "py.test" "tests"))))))
+    (home-page "https://linkchecker.github.io/linkchecker/")
     (synopsis "Check websites for broken links")
     (description "LinkChecker is a website validator.  It checks for broken
 links in websites.  It is recursive and multithreaded providing output in
-- 
2.32.0

Message #8 received at 49086 <at> debbugs.gnu.org (full text, mbox):

From: Xinglu Chen <public <at> yoctocell.xyz>
To: Arun Isaac <arunisaac <at> systemreboot.net>, 49086 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [bug#49086] [PATCH] gnu: linkchecker: Update to 10.0.1.
Date: Fri, 18 Jun 2021 16:18:15 +0200

[Message part 1 (text/plain, inline)]

On Fri, Jun 18 2021, Arun Isaac wrote:

> * gnu/packages/web.scm (linkchecker): Update to 10.0.1.
> [source]: Remove patches.
> [inputs]: Add python-beautifulsoup4. Replace python2-dnspython-1.16,
> python2-pyxdg and python2-requests with python-dnspython, python-pyxdg and
> python-requests respectively.
> [native-inputs]: Replace python2-pytest, python2-miniboa and
> python2-parameterized with python-pytest, python-miniboa and
> python-parameterized respectively.
> [arguments]: Use python 3. Replace check phase instead of deleting the
> standard phase and adding a custom one. Use add-installed-pythonpath instead
> of setting PYTHONPATH directly.
> [home-page]: Update URI.
> * gnu/packages/patches/linkchecker-tests-require-network.patch: Delete file.
> * gnu/local.mk (dist_patch_DATA): Unregister it.
> ---
>  gnu/local.mk                                  |   1 -
>  .../linkchecker-tests-require-network.patch   | 182 ------------------
>  gnu/packages/web.scm                          |  47 ++---
>  3 files changed, 16 insertions(+), 214 deletions(-)
>  delete mode 100644 gnu/packages/patches/linkchecker-tests-require-network.patch
>
>  (define-public linkchecker
>    (package
>      (name "linkchecker")
> -    (version "9.4.0")
> +    (version "10.0.1")
>      (source
>       (origin
>         (method git-fetch)
>         (uri (git-reference
>               (url "https://github.com/linkchecker/linkchecker")
>               (commit (string-append "v" version))))
> -       (patches
> -        (search-patches "linkchecker-tests-require-network.patch"))
>         (file-name (git-file-name name version))
>         (sha256
>          (base32
> -         "03ihjmc4bqxxqv71bb43r2f23sx0xnbq1k2fsg9fw05qa5s9x187"))))
> +         "1j97dc9a4yhpscwadhv5dxp7036pnrxiaky18l8ddr3pvxdjvkxs"))))
>      (build-system python-build-system)
>      (inputs
> -     `(("python2-dnspython" ,python2-dnspython-1.16)
> -       ("python2-pyxdg" ,python2-pyxdg)
> -       ("python2-requests" ,python2-requests)))
> +     `(("python-beautifulsoup4" ,python-beautifulsoup4)
> +       ("python-dnspython" ,python-dnspython)
> +       ("python-pyxdg" ,python-pyxdg)
> +       ("python-requests" ,python-requests)))
>      (native-inputs
>       `(("gettext" ,gettext-minimal)
> -       ("python2-pytest" ,python2-pytest)
> -       ("python2-miniboa" ,python2-miniboa)
> -       ("python2-parameterized" ,python2-parameterized)))
> +       ("python-pytest" ,python-pytest)
> +       ("python-miniboa" ,python-miniboa)
> +       ("python-parameterized" ,python-parameterized)))
>      (arguments
> -     `(#:python ,python-2
> -       #:phases
> +     `(#:phases
>         (modify-phases %standard-phases
> -         ;; Move the 'check phase to after 'install, so that the installed
> -         ;; library can be used
> -         (delete 'check)
> -         (add-after 'install 'check
> -           (lambda* (#:key outputs #:allow-other-keys)
> -             (let ((out (assoc-ref outputs "out")))
> -               ;; Set PYTHONPATH so that the installed linkchecker is used
> -               (setenv "PYTHONPATH"
> -                       (string-append out "/lib/python2.7/site-packages"
> -                                      ":"
> -                                      (getenv "PYTHONPATH")))
> -               ;; Remove this directory to avoid it being used when running
> -               ;; the tests
> -               (delete-file-recursively "linkcheck")
> -
> -               (invoke "py.test" "tests"))
> -             #t)))))
> -    (home-page "https://linkcheck.github.io/linkchecker")
> +         (replace 'check
> +           (lambda* (#:key inputs outputs #:allow-other-keys)
> +             (add-installed-pythonpath inputs outputs)
> +             (invoke "py.test" "tests"))))))

This 'check phase doesn’t respect the ‘--with-tests’ transformation.
Something like the following would be better:

  (replace 'check
    (lambda* (#:key inputs outputs tests? #:allow-other-keys)
      (when tests?
        (with-directory-excursion "tests"
          (add-installed-pythonpath inputs outputs)
          (invoke "py.test" "tests")))))

Otherwise, LGTM.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 49086 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Xinglu Chen <public <at> yoctocell.xyz>, 49086 <at> debbugs.gnu.org
Subject: Re: [bug#49086] [PATCH] gnu: linkchecker: Update to 10.0.1.
Date: Sat, 19 Jun 2021 13:55:13 +0530

[Message part 1 (text/plain, inline)]

> This 'check phase doesn’t respect the ‘--with-tests’ transformation.
> Something like the following would be better:
>
>   (replace 'check
>     (lambda* (#:key inputs outputs tests? #:allow-other-keys)
>       (when tests?
>         (with-directory-excursion "tests"
>           (add-installed-pythonpath inputs outputs)
>           (invoke "py.test" "tests")))))

Makes sense. I'll add the `when tests?', but what's the
with-directory-excursion for?

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 49086 <at> debbugs.gnu.org (full text, mbox):

From: Xinglu Chen <public <at> yoctocell.xyz>
To: Arun Isaac <arunisaac <at> systemreboot.net>, 49086 <at> debbugs.gnu.org
Subject: Re: [bug#49086] [PATCH] gnu: linkchecker: Update to 10.0.1.
Date: Sat, 19 Jun 2021 11:06:34 +0200

[Message part 1 (text/plain, inline)]

On Sat, Jun 19 2021, Arun Isaac wrote:

>> This 'check phase doesn’t respect the ‘--with-tests’ transformation.
>> Something like the following would be better:
>>
>>   (replace 'check
>>     (lambda* (#:key inputs outputs tests? #:allow-other-keys)
>>       (when tests?
>>         (with-directory-excursion "tests"
>>           (add-installed-pythonpath inputs outputs)
>>           (invoke "py.test" "tests")))))
>
> Makes sense. I'll add the `when tests?', but what's the
> with-directory-excursion for?

Oops, ignore the ‘with-directory-excursion’, sorry about that.

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 49086-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Xinglu Chen <public <at> yoctocell.xyz>, 49086-done <at> debbugs.gnu.org
Subject: Re: [bug#49086] [PATCH] gnu: linkchecker: Update to 10.0.1.
Date: Sun, 20 Jun 2021 00:37:58 +0530

[Message part 1 (text/plain, inline)]

Pushed to master. Thanks for the review!

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.