GNU bug report logs - #10437
parallel-tests: `recheck' recipe can cause sed to be invoked with too long input lines (was: Re: bug#10427: coreutils-8.14.116-1e18d: testsuite failures on NetBSD 5.1)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10437 in the body.
You can then email your comments to 10437 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: bug-automake <at> gnu.org, jim <at> meyering.net, 10427 <at> debbugs.gnu.org
Subject: parallel-tests: `recheck' recipe can cause sed to be invoked with
	too long input lines (was: Re: bug#10427: coreutils-8.14.116-1e18d:
	testsuite failures on NetBSD 5.1)
Date: Thu, 05 Jan 2012 15:07:06 +0100

[adding bug-automake in CC:]

Reference: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10427#8>

Hi Paul, thanks for the report and diagnosis.

On 01/05/2012 10:00 AM, Paul Eggert wrote:
> The latest coreutils snapshot fail to build
> 
>> On 01/03/2012 06:10 PM, Jim Meyering wrote:
>>> FYI, here's a snapshot of what will soon be coreutils-8.15,
>>> expected on Thursday or Friday.
>>>
>>> coreutils snapshot:
>>>   http://meyering.net/cu/coreutils-ss.tar.xz      5.2 MB
>>>   http://meyering.net/cu/coreutils-ss.tar.xz.sig
>>>   http://meyering.net/cu/coreutils-8.14.116-1e18d.tar.xz
> 
> This snapshot doesn't build on Solaris 8 (sparc) with native tools,
> for a couple of reasons.  I don't expect Solaris 8 is an active
> porting target any more, but these problems could well happen on
> active targets.
>
I agree.

> Second, there's code like this in tests/Makefile.in:
> 
> 	@list='$(TEST_LOGS)'; \
> 	list=`for i in $$list; do \
> 	  test .log = $$i || echo $$i; \
> 	done | tr '\012\015' '  '`; \
> 	list=`echo "$$list" | sed 's/ *$$//'`; \
> 
> This generates a long line and sends it to 'sed',
> which complains "Output line too long." and outputs nothing.
>
And if I'm not mistaken, sed is allowed such a behaviour by POSIX, so this
is a portability problem in automake.

> This code is also generated by Automake.  How about changing Automake
> to generate something like this instead?
> 
> 	@test_logs='$(TEST_LOGS)'; \
> 	list=; \
> 	for i in $$test_logs; do \
> 	  test .log = "$$i" || list="$$list $$i"; \
> 	done; \
> 
> This avoids the business with echo and tr and ` sed and
> avoids the sed limitation with long lines.
>
Good idea.  I will followed your idea (with some tweaks).
Patch coming up soon.

> Automake does this latter sort of thing in about 4 places,
>
Which "sort of thing" exactly?  I could find only one place which suffers
of the problem you've pointed out, i.e., the `recheck recheck-html' rules
in lib/am/check.am.  Am I missing something?

> and I figure it's done that way for a reason, but I don't
> know what the reason is.
> 
The comments in lib/am/check.am should be explicative enough.  if not,
that's a (minor) bug, so feel free to report it!

Thanks,
  Stefano

Message #8 received at 10437 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 10437 <at> debbugs.gnu.org, automake-patches <at> gnu.org, 10427 <at> debbugs.gnu.org
Subject: Re: bug#10437: parallel-tests: `recheck' recipe can cause sed to
	be invoked with too long input lines
Date: Thu, 05 Jan 2012 15:43:54 +0100

[Message part 1 (text/plain, inline)]

Reference:
  <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10437>

On 01/05/2012 03:07 PM, Stefano Lattarini wrote:
>
> Patch coming up soon.
> 
And here it is.  I will push by this evening if there is no objection.

Regards,
  Stefano

[0001-parallel-tests-avoid-issue-with-overly-long-lines-in.patch (text/x-diff, attachment)]

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: bug-automake <at> gnu.org, jim <at> meyering.net, 10427 <at> debbugs.gnu.org
Subject: Re: parallel-tests: `recheck' recipe can cause sed to be invoked
	with too long input lines
Date: Thu, 05 Jan 2012 10:06:57 -0800

On 01/05/12 06:07, Stefano Lattarini wrote:
> Which "sort of thing" exactly?  I could find only one place which suffers
> of the problem you've pointed out, i.e., the `recheck recheck-html' rules
> in lib/am/check.am.  Am I missing something?

Sorry, that appears to have been a miscount on my part:
I was counting some files that Automake generates for itself
while building.  In Automake source there are only two instances,
which your patch caught: the 'recheck recheck-html' rule and
the 'check-TESTS' rule (the latter is what actually triggered
the problem with coreutils).  So this should be OK.

Thanks for the quick fixes, by the way!

Message #14 received at 10437 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 10437 <at> debbugs.gnu.org, 10427 <at> debbugs.gnu.org
Subject: Re: bug#10437: parallel-tests: `recheck' recipe can cause sed to
	be invoked with too long input lines
Date: Thu, 05 Jan 2012 19:24:48 +0100

On 01/05/2012 07:06 PM, Paul Eggert wrote:
> On 01/05/12 06:07, Stefano Lattarini wrote:
>> Which "sort of thing" exactly?  I could find only one place which suffers
>> of the problem you've pointed out, i.e., the `recheck recheck-html' rules
>> in lib/am/check.am.  Am I missing something?
> 
> Sorry, that appears to have been a miscount on my part:
> I was counting some files that Automake generates for itself
> while building.  In Automake source there are only two instances,
> which your patch caught: the 'recheck recheck-html' rule and
> the 'check-TESTS' rule (the latter is what actually triggered
> the problem with coreutils).
>
Wait, the `check-TESTS' rules didn't use any sed invocation, so it wouldn't make
sense for it to trip for a sed failure ...  What am I missing?

Thanks,
  Stefano

Message #17 received at 10437 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 10437 <at> debbugs.gnu.org, 10427 <at> debbugs.gnu.org
Subject: Re: bug#10427: bug#10437: parallel-tests: `recheck' recipe can cause
	sed to be invoked with too long input lines
Date: Thu, 05 Jan 2012 19:38:39 +0100

On 01/05/2012 07:24 PM, Stefano Lattarini wrote:
> On 01/05/2012 07:06 PM, Paul Eggert wrote:
>> On 01/05/12 06:07, Stefano Lattarini wrote:
>>> Which "sort of thing" exactly?  I could find only one place which suffers
>>> of the problem you've pointed out, i.e., the `recheck recheck-html' rules
>>> in lib/am/check.am.  Am I missing something?
>>
>> Sorry, that appears to have been a miscount on my part:
>> I was counting some files that Automake generates for itself
>> while building.  In Automake source there are only two instances,
>> which your patch caught: the 'recheck recheck-html' rule and
>> the 'check-TESTS' rule (the latter is what actually triggered
>> the problem with coreutils).
>>
> Wait, the `check-TESTS' rules didn't use any sed invocation, so it wouldn't make
> sense for it to trip for a sed failure ...  What am I missing?
>
I will answer myself: I was missing the fact that such a sed invocation had
been added to check-TESTS, *but in master only*.  Anyway, the maint -> master
merge will take care of everything.

Thanks, and sorry for the noise,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.