GNU bug report logs - #23222
test: incorrect operator-precedence

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23222 in the body.
You can then email your comments to 23222 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mattias Andrée <maandree <at> member.fsf.org>
To: bug-coreutils <at> gnu.org
Subject: test: incorrect operator-precedence
Date: Tue, 5 Apr 2016 13:57:16 +0200

[Message part 1 (text/plain, inline)]

Failing test-case #1:

./test -n -a -n

fails and outputs

  ./test: extra argument ‘-n’

Expected behaviour is silent success, as seen in
Bash's implementation.

Explanation: -a has higher precedence than -n.
Therefore the test is equivalent to
./test -n && ./test -n. ./test -n shall succeed
because it is not an operator, but rather a
non-empty string.


Failing test-case #2:

./test ! '' -a ''

shall fail (exit value 1), but in this
implementation it succeeds.

Explanation: ! has higher precedence than -a,
therefore the test shall be equivalent to
./test ! '' && ./test '', not ! ./test '' -a ''.

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 23222 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Mattias Andrée <maandree <at> member.fsf.org>,
 23222 <at> debbugs.gnu.org
Subject: Re: bug#23222: test: incorrect operator-precedence
Date: Tue, 5 Apr 2016 09:09:40 -0600

[Message part 1 (text/plain, inline)]

tag 23222 confirmed
thanks

On 04/05/2016 05:57 AM, Mattias Andrée wrote:
> Failing test-case #1:
> 
> ./test -n -a -n
> 
> fails and outputs
> 
>   ./test: extra argument ‘-n’
> 
> Expected behaviour is silent success, as seen in
> Bash's implementation.

Thanks for the report.  POSIX indeed says that -a is a binary primary,
and also says that
"3 arguments:
        If $2 is a binary primary, perform the binary test of $1 and $3."
(http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html)

So by that reading, coreutils MUST parse your expression as:

test \( -n \) -a \( -n \)

whereas it appears that coreutils has instead accidentally parsed it as:

test \( -n -a \) -n

Not everyday someone finds a real bug in 'test'!

That said, POSIX _also_ says that portable scripts MUST NOT use -a or
-o; it is inherently ambiguous.

You are better off fixing your script to use:

test -n && test -n

whether or not someone is able to quickly patch coreutils' bug.


> 
> Explanation: -a has higher precedence than -n.

Not quite true.  The POSIX rules state that -a is optional (non-XSI
systems don't even have to have it), and that precedence has no bearing
(rather, it is whether you have exactly three arguments that determines
whether -a must be used in its role as a binary operator).


> 
> Failing test-case #2:
> 
> ./test ! '' -a ''
> 
> shall fail (exit value 1), but in this
> implementation it succeeds.

Sorry, but here, you're wrong.  POSIX states:

"4 arguments:
        If $1 is '!', negate the three-argument test of $2, $3, and $4."

which means this parses as:

test ! \( '' -a '' \)

or as:

test ! \( \( '' \) -a \( '' \) \)

The negation of 'false and false' is true, so the exit status is
correctly 0.


> 
> Explanation: ! has higher precedence than -a,

Wrong.  As mentioned above, the rules are based not on precedence, but
on how many operands are present.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #13 received at 23222 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Andrée <maandree <at> kth.se>
To: Eric Blake <eblake <at> redhat.com>
Cc: 23222 <at> debbugs.gnu.org,
 Mattias Andrée <maandree <at> member.fsf.org>
Subject: Re: bug#23222: test: incorrect operator-precedence
Date: Tue, 5 Apr 2016 20:36:07 +0200

[Message part 1 (text/plain, inline)]

On Tue, 5 Apr 2016 09:09:40 -0600
Eric Blake <eblake <at> redhat.com> wrote:

> tag 23222 confirmed
> thanks
> 
> On 04/05/2016 05:57 AM, Mattias Andrée wrote:
> > Failing test-case #1:
> > 
> > ./test -n -a -n
> > 
> > fails and outputs
> > 
> >   ./test: extra argument ‘-n’
> > 
> > Expected behaviour is silent success, as seen in
> > Bash's implementation.  
> 
> Thanks for the report.  POSIX indeed says that -a is a
> binary primary, and also says that
> "3 arguments:
>         If $2 is a binary primary, perform the binary
> test of $1 and
> $3." (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html)
> 
> So by that reading, coreutils MUST parse your expression
> as:
> 
> test \( -n \) -a \( -n \)
> 
> whereas it appears that coreutils has instead
> accidentally parsed it as:
> 
> test \( -n -a \) -n
> 
> Not everyday someone finds a real bug in 'test'!
> 
> That said, POSIX _also_ says that portable scripts MUST
> NOT use -a or -o; it is inherently ambiguous.
> 
> You are better off fixing your script to use:
> 
> test -n && test -n
> 
> whether or not someone is able to quickly patch
> coreutils' bug.

I'm not actually using -a, -o, (, ), and !. I just
noticed the bug by running some test-cases.

> 
> 
> > 
> > Explanation: -a has higher precedence than -n.  
> 
> Not quite true.  The POSIX rules state that -a is
> optional (non-XSI systems don't even have to have it),
> and that precedence has no bearing (rather, it is whether
> you have exactly three arguments that determines whether
> -a must be used in its role as a binary operator).
> 
> 
> > 
> > Failing test-case #2:
> > 
> > ./test ! '' -a ''
> > 
> > shall fail (exit value 1), but in this
> > implementation it succeeds.  
> 
> Sorry, but here, you're wrong.  POSIX states:
> 
> "4 arguments:
>         If $1 is '!', negate the three-argument test of
> $2, $3, and $4."
> 
> which means this parses as:
> 
> test ! \( '' -a '' \)
> 
> or as:
> 
> test ! \( \( '' \) -a \( '' \) \)
> 
> The negation of 'false and false' is true, so the exit
> status is correctly 0.
> 
> 
> > 
> > Explanation: ! has higher precedence than -a,  
> 
> Wrong.  As mentioned above, the rules are based not on
> precedence, but on how many operands are present.
> 

XSI-conformant systems shall use the following precedence
rules (highest to lowest), as documented (not quite as clearly)
in the POSIX specifications:

* parentheses

(1)

* string binary primaries

* unary non-string primaries

* algebraic binary primaries

(2)

* unary string primary

with

* (3)

* !

* -a

* -o

* (4)

in that order but not necessarily directly
followed by each other somewhere between
(1) and (2), and with other binary primitives
somewhere between (3) and (5).

The POSIX specification does indeed that that
the number of arguments shall determine the
precedence for 1 to 4 arguments. And the given
rules do conflict with XSI. But if we run

test \( \( ... \) \) instead of test it seems
only reasonable to use the XSI rules. Is it
too much to ask that test ... gives the same
result? Your implementation appear to be unaffected
if it is given extra parentheses in this way.
But it does evaluate test ! \( '' \) -a \( '' \)
as expected. It is very weird that
test ! \( '' \) -a \( '' \) and
test ! '' -a '' do not give the same result.

I believe that POSIX rule for 4 arguments shall
be disregarded. It is clearly only intended for
other binary operators than logical operators.

[Message part 2 (application/pgp-signature, inline)]

Message #16 received at 23222 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Mattias Andrée <maandree <at> kth.se>
Cc: 23222 <at> debbugs.gnu.org,
 Mattias Andrée <maandree <at> member.fsf.org>
Subject: Re: bug#23222: test: incorrect operator-precedence
Date: Tue, 5 Apr 2016 13:15:39 -0600

[Message part 1 (text/plain, inline)]

On 04/05/2016 12:36 PM, Mattias Andrée wrote:

>> Sorry, but here, you're wrong.  POSIX states:
>>
>> "4 arguments:
>>         If $1 is '!', negate the three-argument test of
>> $2, $3, and $4."
>>
>> which means this parses as:
>>
>> test ! \( '' -a '' \)
>>

> 
> XSI-conformant systems shall use the following precedence
> rules (highest to lowest), as documented (not quite as clearly)
> in the POSIX specifications:
> 

Only for more than four arguments:


>4 arguments:
    The results are unspecified.

    [OB XSI] [Option Start] On XSI-conformant systems, combinations of
primaries and operators shall be evaluated using the precedence and
associativity rules described previously. In addition, the string
comparison binary primaries '=' and "!=" shall have a higher precedence
than any unary primary. [Option End]


> The POSIX specification does indeed that that
> the number of arguments shall determine the
> precedence for 1 to 4 arguments. And the given
> rules do conflict with XSI. But if we run
> 
> test \( \( ... \) \) instead of test it seems
> only reasonable to use the XSI rules.

Yeah, other than the fact that -o and -a are marked obsolescent by POSIX
because they are inherently ambiguous, and therefore "The results are
unspecified" is a better phrase to rely on than any particular precedence.


> I believe that POSIX rule for 4 arguments shall
> be disregarded. It is clearly only intended for
> other binary operators than logical operators.

Sorry, but that's not how POSIX is worded.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #19 received at 23222 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: 23222 <at> debbugs.gnu.org
Subject: Re: bug#23222: test: incorrect operator-precedence
Date: Mon, 29 Oct 2018 22:14:42 -0600

tags 23222 fixed
close 23222
stop

On 2016-04-05 9:09 a.m., Eric Blake wrote:
> tag 23222 confirmed
> thanks
> 
> On 04/05/2016 05:57 AM, Mattias Andrée wrote:
>> Failing test-case #1:
>>
>> ./test -n -a -n
>>
>> fails and outputs
>>
>>    ./test: extra argument ‘-n’
>>
>> Expected behaviour is silent success, as seen in
>> Bash's implementation.
> 
> Thanks for the report.  POSIX indeed says that -a is a binary primary,
> and also says that



With this recent commit:
  test: remove support for the ambigous -a unary operator

https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=88c32fa68ee7057744bfb6d41f6e8eb68801306f

test(1) no longer accepts unary "-a".

Closing this item as "fixed".

-assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.