GNU bug report logs - #74427
tests/printf/printf-indexed failure on macOS

Previous Next

Package: coreutils;

Reported by: Grisha Levit <grishalevit <at> gmail.com>

Date: Tue, 19 Nov 2024 04:39:02 UTC

Severity: normal

Done: Pádraig Brady <P <at> draigBrady.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 74427 in the body.
You can then email your comments to 74427 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 04:39:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Grisha Levit <grishalevit <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-coreutils <at> gnu.org. (Tue, 19 Nov 2024 04:39:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Grisha Levit <grishalevit <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: tests/printf/printf-indexed failure on macOS
Date: Mon, 18 Nov 2024 23:38:02 -0500
See the following two chunks from the test suite log on macOS 15:

+ printf_check '   0 1  ' '%100$*d %s %s %s\n' 4 1
+ cat
+ shift
+ env printf '%100$*d %s %s %s\n' 4 1
printf: '': Invalid argument
+ fail=1

+ printf_check_err 'printf: '\''A'\'': expected a numeric value' '%0x2$s %2$s\n' A B
+ cat
+ shift
+ returns_ 1 env printf '%0x2$s %2$s\n' A B
02$s B
+ compare exp out
+ compare_dev_null_ exp out
+ test 2 = 2
+ test xexp = x/dev/null
+ test xout = x/dev/null
+ return 2
+ case $? in
+ compare_ exp out
+ LC_ALL=C
+ diff -u exp out
--- exp	2024-11-18 23:32:01.486843241 -0500
+++ out	2024-11-18 23:32:01.492438694 -0500
@@ -1 +1 @@
-printf: 'A': expected a numeric value
+printf: 'A': Invalid argument
+ fail=1





Reply sent to Pádraig Brady <P <at> draigBrady.com>:
You have taken responsibility. (Tue, 19 Nov 2024 15:12:02 GMT) Full text and rfc822 format available.

Notification sent to Grisha Levit <grishalevit <at> gmail.com>:
bug acknowledged by developer. (Tue, 19 Nov 2024 15:12:02 GMT) Full text and rfc822 format available.

Message #10 received at 74427-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Grisha Levit <grishalevit <at> gmail.com>, 74427-done <at> debbugs.gnu.org
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 15:10:37 +0000
[Message part 1 (text/plain, inline)]
On 19/11/2024 04:38, Grisha Levit wrote:
> See the following two chunks from the test suite log on macOS 15:
> 
> + printf_check '   0 1  ' '%100$*d %s %s %s\n' 4 1
> + cat
> + shift
> + env printf '%100$*d %s %s %s\n' 4 1
> printf: '': Invalid argument
> + fail=1
> 
> + printf_check_err 'printf: '\''A'\'': expected a numeric value' '%0x2$s %2$s\n' A B
> + cat
> + shift
> + returns_ 1 env printf '%0x2$s %2$s\n' A B
> 02$s B
> + compare exp out
> + compare_dev_null_ exp out
> + test 2 = 2
> + test xexp = x/dev/null
> + test xout = x/dev/null
> + return 2
> + case $? in
> + compare_ exp out
> + LC_ALL=C
> + diff -u exp out
> --- exp	2024-11-18 23:32:01.486843241 -0500
> +++ out	2024-11-18 23:32:01.492438694 -0500
> @@ -1 +1 @@
> -printf: 'A': expected a numeric value
> +printf: 'A': Invalid argument
> + fail=1

This is due to inconsistency in how strtol and strtoimax operate on macos.
The man page for those on macos states:

  "If no conversion could be performed, 0 is returned and the global variable errno   is set to EINVAL (the last feature is not portable across all platforms)."

So in particular, we'll have to handle the NUL string in code,
to ensure it returns 0, and adjust the test to treat EINVAL
like "expected a numeric value".

I've not got access to a macos system to test currently,
but the attached should address this.

Marking this as done.

thanks,
Pádraig
[printf-indexed-macos.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 16:48:02 GMT) Full text and rfc822 format available.

Message #13 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 74427 <at> debbugs.gnu.org, P <at> draigBrady.com, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 08:47:03 -0800
[Message part 1 (text/plain, inline)]
On 2024-11-19 07:10, Pádraig Brady wrote:
> I've not got access to a macos system to test currently,
> but the attached should address this.

That patch addresses only the empty string, not other strings that 
cannot be converted (e.g., a string consisting of a single space). This 
turns out to be a longstanding bug in printf. I installed the attached 
fancier patch which should address the more-general problem, and which 
adds some test cases. I tested this on cfarm104.cfarm.net which is 
running macOS 12.6.
[0001-printf-diagnose-empty-args-correctly.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 17:55:02 GMT) Full text and rfc822 format available.

Message #16 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, 74427 <at> debbugs.gnu.org,
 grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 17:53:01 +0000
On 19/11/2024 16:47, Paul Eggert wrote:
> On 2024-11-19 07:10, Pádraig Brady wrote:
>> I've not got access to a macos system to test currently,
>> but the attached should address this.
> 
> That patch addresses only the empty string, not other strings that
> cannot be converted (e.g., a string consisting of a single space). This
> turns out to be a longstanding bug in printf. I installed the attached
> fancier patch which should address the more-general problem, and which
> adds some test cases. I tested this on cfarm104.cfarm.net which is
> running macOS 12.6.

Do we really want to treat NUL and NULL strings differently?

  $ src/printf '%d\n'
  0
  $ src/printf '%d\n' ''
  printf: ‘’: expected a numeric value
  0

That's different to bash and dash.
More problematically I think is it's also inconsistent with %s

  $ src/printf '%s'
  $ src/printf '%s' ''

BTW after we decide the best operation, we should update NEWS.
I had done this locally like:

  printf consistently outputs 0 for numeric conversions of empty strings.
  Previously on systems like macos, it would have failed with an error.

cheers,
Pádraig




Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 20:03:02 GMT) Full text and rfc822 format available.

Message #19 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 74427 <at> debbugs.gnu.org, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 12:02:41 -0800
On 2024-11-19 09:53, Pádraig Brady wrote:
> Do we really want to treat NUL and NULL strings differently?
> 
>    $ src/printf '%d\n'
>    0
>    $ src/printf '%d\n' ''
>    printf: ‘’: expected a numeric value
>    0

Both behaviors are required by POSIX, so yes, we want to treat them 
differently. Also, it makes sense to treat '' like ' '; we report an 
error for the latter so we should report it for the former too.


> More problematically I think is it's also inconsistent with %s
> 
>   $ src/printf '%s'
>   $ src/printf '%s' '' 

I don't see a problem there. As per POSIX, missing string arguments are 
treated as empty strings, whereas missing numeric arguments are treated 
as zero.


> That's different to bash and dash.

AIX and Solaris 10 printf do things the right way (i.e., conform to 
POSIX) - not surprising as AIX at least has gone through POSIX 
conformance tests. I expect the GNU divergence from tradition was 
accidental, as it doesn't make sense for printf to treat '' differently 
from other non-numbers.

I filed bug reports for both Bash and Dash; see:

https://lists.gnu.org/r/bug-bash/2024-11/msg00149.html
https://lore.kernel.org/dash/20241119193211.1585716-1-eggert <at> cs.ucla.edu/T/#u


> BTW after we decide the best operation, we should update NEWS. 

Thanks, good point. How about something like the following:

printf now diagnoses attempts to treat empty strings as numbers, as per 
POSIX. For example, "printf '%d' ''" now issues a diagnostic and fails 
instead of silently succeeding.




Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 20:30:02 GMT) Full text and rfc822 format available.

Message #22 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 74427 <at> debbugs.gnu.org, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 20:28:22 +0000
On 19/11/2024 20:02, Paul Eggert wrote:
> On 2024-11-19 09:53, Pádraig Brady wrote:
>> Do we really want to treat NUL and NULL strings differently?
>>
>>     $ src/printf '%d\n'
>>     0
>>     $ src/printf '%d\n' ''
>>     printf: ‘’: expected a numeric value
>>     0
> 
> Both behaviors are required by POSIX, so yes, we want to treat them
> differently. Also, it makes sense to treat '' like ' '; we report an
> error for the latter so we should report it for the former too.

Well POSIX isn't explicit about the empty case.
The closest it specifies I think is:

  "If an argument operand cannot be completely converted into an
   internal value appropriate to the corresponding conversion specification,
   a diagnostic message shall be written to standard error"

But one could consider an empty argument as completely converted?

I'm not very strongly against the change,
it just seems a little inconsistent to me to error for the empty argument
to numeric conversion case, when we don't do that for strings.

Also I'm a bit worried about quoting consequences for shell scripts.
For example scripts with `printf '%d\n' "$int"` will start erroring if int is not defined,
whereas arguably less protected scripts with `printf '%d\n' $int` will behave (correctly) as before.

>> That's different to bash and dash.
> 
> AIX and Solaris 10 printf do things the right way (i.e., conform to
> POSIX) - not surprising as AIX at least has gone through POSIX
> conformance tests. I expect the GNU divergence from tradition was
> accidental, as it doesn't make sense for printf to treat '' differently
> from other non-numbers.
> 
> I filed bug reports for both Bash and Dash; see:
> 
> https://lists.gnu.org/r/bug-bash/2024-11/msg00149.html
> https://lore.kernel.org/dash/20241119193211.1585716-1-eggert <at> cs.ucla.edu/T/#u

For completeness ksh on Solaris 11 treats empty as 0

  solaris11$ printf --version
  version         printf (AT&T Research) 2009-02-02
  solaris11:~$ printf '%d\n' ''
  0

As does newer ksh:
  fedora40$ printf --version
  version         printf (ksh 93u+m) 2023-03-23
  fedora40$ printf '%d\n' ''
  0

Though I also see that ksh treats any non number as 0.

cheers,
Pádraig




Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 21:49:02 GMT) Full text and rfc822 format available.

Message #25 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 74427 <at> debbugs.gnu.org, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 13:48:03 -0800
On 2024-11-19 12:28, Pádraig Brady wrote:
> But one could consider an empty argument as completely converted?

No, because POSIX says the argument must be a C integer constant, and 
empty strings are not C integer constants. Similar reasoning applies for 
floating point specs like %g, where POSIX says the argument is converted 
as if by strtod, and for an empty string POSIX says no conversion is 
performed by strtod.


> For example scripts with `printf '%d\n' "$int"` will start erroring if int is not defined

It shouldn't be that big a deal, as the change doesn't affect stdout. 
The only issues are stderr and exit status, and that sort of script 
typically ignores both.





Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 21:57:02 GMT) Full text and rfc822 format available.

Message #28 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 74427 <at> debbugs.gnu.org, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 21:54:59 +0000
On 19/11/2024 21:48, Paul Eggert wrote:
> On 2024-11-19 12:28, Pádraig Brady wrote:
>> But one could consider an empty argument as completely converted?
> 
> No, because POSIX says the argument must be a C integer constant, and
> empty strings are not C integer constants. Similar reasoning applies for
> floating point specs like %g, where POSIX says the argument is converted
> as if by strtod, and for an empty string POSIX says no conversion is
> performed by strtod.
> 
> 
>> For example scripts with `printf '%d\n' "$int"` will start erroring if int is not defined
> 
> It shouldn't be that big a deal, as the change doesn't affect stdout.
> The only issues are stderr and exit status, and that sort of script
> typically ignores both.
> 

OK assuming all the above, the NEWS change looks good.

thanks!
Pádraig




Information forwarded to bug-coreutils <at> gnu.org:
bug#74427; Package coreutils. (Tue, 19 Nov 2024 22:18:01 GMT) Full text and rfc822 format available.

Message #31 received at 74427 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 74427 <at> debbugs.gnu.org, grishalevit <at> gmail.com
Subject: Re: bug#74427: tests/printf/printf-indexed failure on macOS
Date: Tue, 19 Nov 2024 22:16:37 +0000
On 19/11/2024 20:02, Paul Eggert wrote:
> On 2024-11-19 09:53, Pádraig Brady wrote:
>> Do we really want to treat NUL and NULL strings differently?
>>
>>     $ src/printf '%d\n'
>>     0
>>     $ src/printf '%d\n' ''
>>     printf: ‘’: expected a numeric value
>>     0
> 
> Both behaviors are required by POSIX, so yes, we want to treat them
> differently. Also, it makes sense to treat '' like ' '; we report an
> error for the latter so we should report it for the former too.
> 
> 
>> More problematically I think is it's also inconsistent with %s
>>
>>    $ src/printf '%s'
>>    $ src/printf '%s' ''
> 
> I don't see a problem there. As per POSIX, missing string arguments are
> treated as empty strings, whereas missing numeric arguments are treated
> as zero.
> 
> 
>> That's different to bash and dash.
> 
> AIX and Solaris 10 printf do things the right way (i.e., conform to
> POSIX) - not surprising as AIX at least has gone through POSIX
> conformance tests. I expect the GNU divergence from tradition was
> accidental, as it doesn't make sense for printf to treat '' differently
> from other non-numbers.
> 
> I filed bug reports for both Bash and Dash; see:
> 
> https://lists.gnu.org/r/bug-bash/2024-11/msg00149.html
> https://lore.kernel.org/dash/20241119193211.1585716-1-eggert <at> cs.ucla.edu/T/#u

For completeness I noticed a previous related dash discussion about this:
https://patchwork.kernel.org/project/dash/patch/62c16598-2d44-f4c9-991d-9880a8839786 <at> inlv.org/#21564895

At the _shell level_, unset and empty variables are to be treated similarly,
and as per POSIX, for shells, uninitialized variables should be treated as 0


  dash -c 'unset x; echo $((x))'
  0
  $ dash -c 'x=; echo $((x))'
  0

I know shell arithmetic is separate to printf processing,
so just posting this for completeness and to perhaps aid
any discussions on the shell bug reports above.

cheers,
Pádraig




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Wed, 18 Dec 2024 12:24:09 GMT) Full text and rfc822 format available.

This bug report was last modified 238 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.