Subject: bug#25177: Test failures don't cause some Python packages to fail [was Re: [PATCH 05/11] gnu: Add python-pygit2.]

From: Marius Bakke <[email-masked]>
Date: Mon, 12 Dec 2016 00:04:38 +0100

Leo Famulari <[email-masked]> writes:

> I noticed while building the package added by ng0's patch (below) that
> the test suite fails, but the check phase succeeds:
>
> [...]
> ======================================================================
> ERROR: test_clone_with_credentials (test.test_repository.CloneRepositoryTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/tmp/guix-build-python-pygit2-0.24.2.drv-0/pygit2-0.24.2/test/test_repository.py", line 544, in test_clone_with_credentials
>     self._temp_dir, callbacks=pygit2.RemoteCallbacks(credentials=pygit2.UserPass("libgit2", "libgit2")))
>   File "/tmp/guix-build-python-pygit2-0.24.2.drv-0/pygit2-0.24.2/pygit2/__init__.py", line 255, in clone_repository
>     check_error(err)
>   File "/tmp/guix-build-python-pygit2-0.24.2.drv-0/pygit2-0.24.2/pygit2/errors.py", line 64, in check_error
>     raise GitError(message)
> _pygit2.GitError: curl error: Couldn't resolve host 'bitbucket.org'
>
>
> ----------------------------------------------------------------------
> Ran 262 tests in 5.771s
>
> FAILED (errors=3)
> phase `check' succeeded after 8.9 seconds
> [...]
>
> Any ideas?

It looks like the 'check' phase ends with 'delete-file-recursively',
which has an unspecified return value, and that eventual failures from
'call-setuppy' are lost.

--8<---------------cut here---------------start------------->8---
(define* (check #:key tests? test-target use-setuptools? #:allow-other-keys)
  "Run the test suite of a given Python package."
  (if tests?
      ;; Running `setup.py test` creates an additional .egg-info directory in
      ;; build/lib in some cases, e.g. if the source is in a sub-directory
      ;; (given with `package_dir`). This will by copied to the output, too,
      ;; so we need to remove.
      (let ((before (find-files "build" "\\.egg-info$" #:directories? #t)))
        (call-setuppy test-target '() use-setuptools?)
        (let* ((after (find-files "build" "\\.egg-info$" #:directories? #t))
               (inter (lset-difference eqv? after before)))
          (for-each delete-file-recursively inter)))
    #t))
--8<---------------cut here---------------end--------------->8---

Perhaps something like this would work (untested)?