GNU bug report logs - #14781
coding system changes cause make check to hang in icalendar tests

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 14781 in the body.
You can then email your comments to 14781 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #3 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: submit <at> debbugs.gnu.org
Subject: coding system changes cause make check to hang in icalendar tests
Date: Wed, 03 Jul 2013 15:02:11 -0400

Package: emacs
Severity: important
Version: 24.3.50

Current trunk, GNU/Linux, make check is broken by recent changes related
to coding systems (I suspect):

make check
...
Converting iCalendar...done
   passed  122/331  icalendar-import-with-uid
Preparing iCalendar...
Preparing iCalendar...done
Reading iCalendar...
Reading iCalendar...done
Converting iCalendar...
Converting iCalendar...done
Saving file /tmp/icalendar-test-diary13327GoO...
Wrote /tmp/icalendar-test-diary13327GoO
Converting iCalendar...done
Preparing iCalendar...
Preparing iCalendar...done
Reading iCalendar...
Reading iCalendar...done
Converting iCalendar...
Converting iCalendar...done
Saving file /tmp/icalendar-test-diary13327TyU...
Wrote /tmp/icalendar-test-diary13327TyU
Converting iCalendar...done
Preparing iCalendar...
Preparing iCalendar...done
Reading iCalendar...
Reading iCalendar...done
Converting iCalendar...
Converting iCalendar...done
Saving file /tmp/icalendar-test-diary13327g8a...
Wrote /tmp/icalendar-test-diary13327g8a
Converting iCalendar...done
Preparing iCalendar...
Preparing iCalendar...done
Reading iCalendar...
Reading iCalendar...done
Converting iCalendar...
Converting iCalendar...done
Saving file /tmp/icalendar-test-diary13327tGh...
Wrote /tmp/icalendar-test-diary13327tGh
Converting iCalendar...done
Preparing iCalendar...
Preparing iCalendar...done
Reading iCalendar...
Reading iCalendar...done
Converting iCalendar...
Converting iCalendar...done
Saving file /tmp/icalendar-test-diary133276Qn...
Select coding system (default chinese-iso-8bit): 

...hangs waiting for input.

Message #6 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Kenichi Handa <handa <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Mon, 08 Jul 2013 21:08:02 +0900

In article <6zju35ux8.fsf <at> fencepost.gnu.org>, Glenn Morris <rgm <at> gnu.org> writes:

> Current trunk, GNU/Linux, make check is broken by recent changes related
> to coding systems (I suspect):

> make check
> ...
> Converting iCalendar...done
>    passed  122/331  icalendar-import-with-uid
[...]
> Saving file /tmp/icalendar-test-diary133276Qn...
> Select coding system (default chinese-iso-8bit): 

> ...hangs waiting for input.

I've just committed a fix to the trunk.  Actually, the
culprit was test/automated/decoder-tests.el which changes
priorities of coding systems.  I didn't know that all tests
under test/automated are executed in a single Emacs session,
and thus they must be side-effect free.  Isn't it better to
execute each test file in its own Emacs session?

---
Kenichi Handa
handa <at> gnu.org

Message #9 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Kenichi Handa <handa <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Mon, 08 Jul 2013 12:52:29 -0400

Kenichi Handa wrote:

> I've just committed a fix to the trunk.

Thanks.

>  Actually, the culprit was test/automated/decoder-tests.el which
> changes priorities of coding systems. I didn't know that all tests
> under test/automated are executed in a single Emacs session, and thus
> they must be side-effect free.

I think they should all be side-effect free (as much as possible)
anyway. Someone might well want to run an individual test in an existing
Emacs session, without that session getting messed up.

> Isn't it better to execute each test file in its own Emacs session?

I don't know. Running them all in one is the only way to get the final
number of tests run / passed / failed to be accurate. Running them in
separate Emacs's would allow them to run in parallel, which would be
faster, but might be confusing?

Message #14 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Kenichi Handa <handa <at> gnu.org>, 14781 <at> debbugs.gnu.org
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Mon, 08 Jul 2013 16:22:52 -0400

> number of tests run / passed / failed to be accurate. Running them in
> separate Emacs's would allow them to run in parallel, which would be
> faster, but might be confusing?

The time to run each test tends to be small, so the overhead of starting
a separate Emacs session for each one may dwarf the potential speed up
from parallel execution, unless you have hundreds of cores.


        Stefan

Message #17 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: Kenichi Handa <handa <at> gnu.org>, 14781 <at> debbugs.gnu.org
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Mon, 08 Jul 2013 21:46:35 -0400

Stefan Monnier wrote:

>> number of tests run / passed / failed to be accurate. Running them in
>> separate Emacs's would allow them to run in parallel, which would be
>> faster, but might be confusing?
>
> The time to run each test tends to be small, so the overhead of starting
> a separate Emacs session for each one may dwarf the potential speed up
> from parallel execution, unless you have hundreds of cores.

I was meaning one Emacs per file, not per test.
I get about a factor of 2 speed-up on 4 cores this way.
The main issue holding it back is the wildly varying length of time that
each test file takes (the file-notify ones are slow for me).

Message #20 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Handa Kenichi <handa <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org, monnier <at> IRO.UMontreal.CA
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Tue, 09 Jul 2013 09:47:51 -0400

In article <dmr4f96lki.fsf <at> fencepost.gnu.org>, Glenn Morris
<rgm <at> gnu.org> writes:

> >  Actually, the culprit was test/automated/decoder-tests.el which
> > changes priorities of coding systems. I didn't know that all tests
> > under test/automated are executed in a single Emacs session, and
> > thus
> > they must be side-effect free.

> I think they should all be side-effect free (as much as possible)
> anyway. Someone might well want to run an individual test in an
> existing
> Emacs session, without that session getting messed up.

I generally agree that keeping a test code site-effect free
is a good thing, and it makes testing a test code easier.
But, sometimes achieving side-effect free is difficult, and
in such a case, I think we don't have to spend much effort
on it.  The main purpose of test is to find bugs.

> > Isn't it better to execute each test file in its own Emacs session?

> I don't know. Running them all in one is the only way to get the final
> number of tests run / passed / failed to be accurate.

Here's a rough idea to do that in test/automated/Makefile.in.

TEST_RESULTS = $(patsubst %.el, %.dat, $(wildcard *.el))

.SUFFIXES: .dat .el 

.el.dat:
        @$(emacs) -f ert-run-tests-batch-and-exit-with-report $< $@

check: $(TEST_RESULTS)
        @$(emacs) -f ert-summarize-test-report $(TEST_RESULTS)

Note that ert-run-tests-batch-and-exit-with-report and
ert-report-test-results are not yet implemented.

> Running them in
> separate Emacs's would allow them to run in parallel, which would be
> faster, but might be confusing?

The faster the better, but I think the speed is not that
important here.  The merit of the above method is that we
can run a single test file just by "make XXXX.dat".

---
Kenichi Handa
handa <at> gnu.org

Message #23 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Handa Kenichi <handa <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org, monnier <at> IRO.UMontreal.CA
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Tue, 09 Jul 2013 12:08:31 -0400

Handa Kenichi wrote:

> The merit of the above method is that we can run a single test file
> just by "make XXXX.dat".

You can do that now with not too much effort.

emacs -batch -l ./man-tests.el -f ert-run-tests-batch-and-exit

Message #26 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Handa Kenichi <handa <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org, monnier <at> IRO.UMontreal.CA
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Tue, 09 Jul 2013 12:15:29 -0400

Handa Kenichi wrote:

> TEST_RESULTS = $(patsubst %.el, %.dat, $(wildcard *.el))

Not portable?
(Though TBH I don't know why we don't just require GNU make and make our
lives easier. But that's a separate issue! :) )

Message #29 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Handa Kenichi <handa <at> gnu.org>, 14781 <at> debbugs.gnu.org
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Wed, 10 Jul 2013 04:49:25 -0400

>> TEST_RESULTS = $(patsubst %.el, %.dat, $(wildcard *.el))
> Not portable?
> (Though TBH I don't know why we don't just require GNU make and make our
> lives easier. But that's a separate issue! :) )

Indeed I think we should use GNUmakefile in `test'.  I already made
that change in the `elpa' branch.

I'm not sure about requiring GNU Make to build Emacs out of a tarball,
but for "side operations" it should not be a problem.

        Stefan

Message #32 received at 14781 <at> debbugs.gnu.org (full text, mbox):

From: Handa Kenichi <handa <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 14781 <at> debbugs.gnu.org, monnier <at> IRO.UMontreal.CA
Subject: Re: bug#14781: coding system changes cause make check to hang in
 icalendar tests
Date: Thu, 11 Jul 2013 09:38:00 -0400

In article <3ea9lvemla.fsf <at> fencepost.gnu.org>, Glenn Morris
<rgm <at> gnu.org> writes:
> Handa Kenichi wrote:
> > TEST_RESULTS = $(patsubst %.el, %.dat, $(wildcard *.el))

> Not portable?
> (Though TBH I don't know why we don't just require GNU make and make
> our
> lives easier. But that's a separate issue! :) )

If portability matters, we can ask the configure script to
generate a proper values:

TEST_RESULTS = %TEST_RESULTS%

> > The merit of the above method is that we can run a single test file
> > just by "make XXXX.dat".

> You can do that now with not too much effort.

> emacs -batch -l ./man-tests.el -f ert-run-tests-batch-and-exit

Not that simple. We must do:

EMACSLOADPATH=../../lisp:. LC_ALL=C ../../src/emacs -batch --no-site-file --no-site-lisp -l ./man-tests.el -f ert-run-tests-batch-and-exit

In article <jwvbo6akdj2.fsf-monnier+emacs <at> gnu.org>, Stefan Monnier
<monnier <at> IRO.UMontreal.CA> writes:

> Indeed I think we should use GNUmakefile in `test'.  I already made
> that change in the `elpa' branch.

> I'm not sure about requiring GNU Make to build Emacs out of a tarball,
> but for "side operations" it should not be a problem.

I fully agree.

---
Kenichi Handa
handa <at> gnu.org

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.