GNU bug report logs - #10436
New testsuite driver and extra trailing backslash in recipes (was: Re: bug#10427: coreutils-8.14.116-1e18d: testsuite failures on NetBSD 5.1)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10436 in the body.
You can then email your comments to 10436 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: bug-automake <at> gnu.org, jim <at> meyering.net, 10427 <at> debbugs.gnu.org
Subject: New testsuite driver and extra trailing backslash in recipes (was:
	Re: bug#10427: coreutils-8.14.116-1e18d: testsuite failures on NetBSD
	5.1)
Date: Thu, 05 Jan 2012 13:47:36 +0100

[Message part 1 (text/plain, inline)]

[adding bug-automake in CC:]

Reference: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10427#8>

Hi Paul, thanks for the report and diagnosis.

On 01/05/2012 10:00 AM, Paul Eggert wrote:
> I'm sending this to bug-automake because I think it's an automake
> issue.  However, the problem causes the latest coreutils snapshot
> to fail to build, so I'm CC'ing to bug-coreutils.
> 
>> On 01/03/2012 06:10 PM, Jim Meyering wrote:
>>> FYI, here's a snapshot of what will soon be coreutils-8.15,
>>> expected on Thursday or Friday.
>>>
>>> coreutils snapshot:
>>>   http://meyering.net/cu/coreutils-ss.tar.xz      5.2 MB
>>>   http://meyering.net/cu/coreutils-ss.tar.xz.sig
>>>   http://meyering.net/cu/coreutils-8.14.116-1e18d.tar.xz
> 
> This snapshot doesn't build on Solaris 8 (sparc) with native tools,
> for a couple of reasons.  I don't expect Solaris 8 is an active
> porting target any more, but these problems could well happen on
> active targets.
>
I agree that this issues might prove a liability on some modern systems
too, and that we should fix them.  (BTW, if you feel like running the
whole automake testsuite on Solaris 8 to find more similar issues, I
wouldn't object ;-)

> First, there's code like this in tests/Makefile.in:
> 
> 	$(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) "$$tst" \
> 	$(AM_TESTS_FD_REDIRECT)
> 
> This code is generated by Automake.  Here, AM_TESTS_FD_REDIRECT
> is empty.  Solaris 8 'make' executes the above as follows:
> 
>    bash -c '[expansion of previous line] \'
> 
> and Bash complains about a syntax error with the trailing backslash.
>
I can reproduce this with bash 2.05b on Debian.

Attached is a fix for this bug, with a testcase.  The test is quite
elaborate and somewhat hacky, but I'd rather keep it anyway, since
I'm planning to refactor it into an external helper script that will
be used to guard against other unwanted trailing `\' in *all* the
make recipes run in the automake testsuite (that's for a follow-up
patch obviously).

> How about changing Automake to generate something like this instead,
> with no backslash-newline?
> 
> 	$(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) "$$tst" $(AM_TESTS_FD_REDIRECT)
> 
> This should avoid the problem.
>
I ended up breaking the line in a "safer" place instead.  It is enough
to fix the bug.

Regards,
  Stefano

[0001-parallel-tests-avoid-trailing-backslashes-in-make-re.patch (text/x-diff, attachment)]

Message #8 received at 10436 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 10436 <at> debbugs.gnu.org, automake-patches <at> gnu.org, 10427 <at> debbugs.gnu.org
Subject: Re: bug#10436: New testsuite driver and extra trailing backslash
	in recipes
Date: Thu, 05 Jan 2012 14:08:41 +0100

[Message part 1 (text/plain, inline)]

On 01/05/2012 01:47 PM, Stefano Lattarini wrote:
> [adding bug-automake in CC:]
> 
> Reference: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=104367#8>
> 
And here is the definitive version of the patch that I'll push by this
evening (to master) if there is no objection.

Thanks,
  Stefano

[0001-parallel-tests-avoid-trailing-backslashes-in-make-re.patch (text/x-diff, attachment)]

Message #11 received at 10436 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: autoconf-patches <at> gnu.org, 10436 <at> debbugs.gnu.org, automake-patches <at> gnu.org,
	10427 <at> debbugs.gnu.org
Subject: Re: bug#10427: bug#10436: New testsuite driver and extra trailing
	backslash in recipes
Date: Thu, 05 Jan 2012 12:39:39 -0800

I pushed the following doc fix into Autoconf, so that these two portability
issues are documented there.  It turns out that the second issue
is actually due to an old Bash bug -- it's not Solaris-specific.

From b1f0e147aa7aa259dea2c34c5a0ac7965d6efd7e Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert <at> cs.ucla.edu>
Date: Thu, 5 Jan 2012 11:00:45 -0800
Subject: [PATCH 1/2] doc: clarify sed buffer limit

* doc/autoconf.texi (Limitations of Usual Tools):
That 4000-byte limit applies to output and internal buffers, too.
---
 ChangeLog         |    6 ++++++
 doc/autoconf.texi |    5 +++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 30312be..238c09f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2012-01-05  Paul Eggert  <eggert <at> cs.ucla.edu>
+
+	doc: clarify sed buffer limit
+	* doc/autoconf.texi (Limitations of Usual Tools):
+	That 4000-byte limit applies to output and internal buffers, too.
+
 2012-01-03  Paul Eggert  <eggert <at> cs.ucla.edu>
 
 	maint: update copyright year
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 72ff7f8..ea2419b 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -19256,8 +19256,9 @@ $ @kbd{echo a | sed 's/x/x/;;s/x/x/'}
 sed: 1: "s/x/x/;;s/x/x/": invalid command code ;
 @end example
 
-Input should not have unreasonably long lines, since some @command{sed}
-implementations have an input buffer limited to 4000 bytes.  Likewise,
+Some @command{sed} implementations have a buffer limited to 4000 bytes,
+and this limits the size of input lines, output lines, and internal
+buffers that can be processed portably.  Likewise,
 not all @command{sed} implementations can handle embedded @code{NUL} or
 a missing trailing newline.
 
-- 
1.7.6.5


From 0445b4ad121e6356f409833dec3678ae613e1d08 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert <at> cs.ucla.edu>
Date: Thu, 5 Jan 2012 12:32:12 -0800
Subject: [PATCH 2/2] doc: mention Bash 2.03 bug with backslash-newline

* doc/autoconf.texi (Invoking the Shell): New section.
(Backslash-Newline-Empty): Rename from Backslash-Newline-Newline.
Mention problem with Bash 2.03.
---
 ChangeLog         |    5 ++++
 doc/autoconf.texi |   56 +++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 57 insertions(+), 4 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 238c09f..69df405 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
 2012-01-05  Paul Eggert  <eggert <at> cs.ucla.edu>
 
+	doc: mention Bash 2.03 bug with backslash-newline
+	* doc/autoconf.texi (Invoking the Shell): New section.
+	(Backslash-Newline-Empty): Rename from Backslash-Newline-Newline.
+	Mention problem with Bash 2.03.
+
 	doc: clarify sed buffer limit
 	* doc/autoconf.texi (Limitations of Usual Tools):
 	That 4000-byte limit applies to output and internal buffers, too.
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index ea2419b..5c3feef 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -501,6 +501,7 @@ Dependencies Between Macros
 Portable Shell Programming
 
 * Shellology::                  A zoology of shells
+* Invoking the Shell::          Invoking the shell as a command
 * Here-Documents::              Quirks and tricks
 * File Descriptors::            FDs and redirections
 * Signal Handling::             Shells, signals, and headaches
@@ -520,7 +521,7 @@ Portable Make Programming
 * $< in Ordinary Make Rules::   $< in ordinary rules
 * Failure in Make Rules::       Failing portably in rules
 * Special Chars in Names::      Special Characters in Macro Names
-* Backslash-Newline-Newline::   Empty last lines in macro definitions
+* Backslash-Newline-Empty::     Empty lines after backslash-newline
 * Backslash-Newline Comments::  Spanning comments across line boundaries
 * Long Lines in Makefiles::     Line length limitations
 * Macros and Submakes::         @code{make macro=value} and submakes
@@ -15067,6 +15068,7 @@ subset described above, is fairly portable nowadays.  Also please see
 
 @menu
 * Shellology::                  A zoology of shells
+* Invoking the Shell::          Invoking the shell as a command
 * Here-Documents::              Quirks and tricks
 * File Descriptors::            FDs and redirections
 * Signal Handling::             Shells, signals, and headaches
@@ -15205,6 +15207,28 @@ The default Mac OS X @command{sh} was originally Zsh; it was changed to
 Bash in Mac OS X 10.2.
 @end table
 
+@node Invoking the Shell
+@section Invoking the Shell
+@cindex invoking the shell
+@cindex shell invocation
+
+Bash 2.03 has a bug when invoked with the @option{-c} option: if the
+option-argument ends in backslash-newline, Bash incorrectly reports a
+syntax error.  The problem does not occur if a character follows the
+backslash:
+
+@example
+$ @kbd{$ bash -c 'echo foo \}
+> @kbd{'}
+bash: -c: line 2: syntax error: unexpected end of file
+$ @kbd{bash -c 'echo foo \}
+> @kbd{ '}
+foo
+@end example
+
+@noindent
+@xref{Backslash-Newline-Empty}, for how this can cause problems in makefiles.
+
 @node Here-Documents
 @section Here-Documents
 @cindex Here-documents
@@ -19663,7 +19687,7 @@ itself.
 * $< in Ordinary Make Rules::   $< in ordinary rules
 * Failure in Make Rules::       Failing portably in rules
 * Special Chars in Names::      Special Characters in Macro Names
-* Backslash-Newline-Newline::   Empty last lines in macro definitions
+* Backslash-Newline-Empty::     Empty lines after backslash-newline
 * Backslash-Newline Comments::  Spanning comments across line boundaries
 * Long Lines in Makefiles::     Line length limitations
 * Macros and Submakes::         @code{make macro=value} and submakes
@@ -19741,8 +19765,32 @@ this is test
 @noindent
 However, this problem is no longer of practical concern.
 
-@node Backslash-Newline-Newline
-@section Backslash-Newline-Newline in Make Macro Values
+@node Backslash-Newline-Empty
+@section Backslash-Newline Before Empty Lines
+
+A bug in Bash 2.03 can cause problems if a Make rule contains a
+backslash-newline followed by line that expands to nothing.
+For example, on Solaris 8:
+
+@example
+SHELL = /bin/bash
+EMPTY =
+foo:
+	touch foo \
+	$(EMPTY)
+@end example
+
+@noindent
+executes
+
+@example
+/bin/bash -c 'touch foo \
+'
+@end example
+
+@noindent
+which fails with a syntax error, due to the Bash bug.  To avoid this
+problem, avoid nullable macros in the last line of a multiline command.
 
 @c  This has been seen on ia64 hpux 11.20, and on one hppa hpux 10.20,
 @c  but another hppa hpux 10.20 didn't have it.  Bob Proulx
-- 
1.7.6.5

Message #14 received at 10436 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 10436 <at> debbugs.gnu.org, automake-patches <at> gnu.org, 10427 <at> debbugs.gnu.org
Subject: Re: bug#10436: New testsuite driver and extra trailing backslash
	in recipes
Date: Fri, 06 Jan 2012 10:57:57 +0100

On 01/05/2012 02:08 PM, Stefano Lattarini wrote:
> On 01/05/2012 01:47 PM, Stefano Lattarini wrote:
>> [adding bug-automake in CC:]
>>
>> Reference: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=104367#8>
>>
> And here is the definitive version of the patch that I'll push by this
> evening (to master) if there is no objection.
> 
Pushed now.

Regards,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.