GNU bug report logs - #8103
tac RFE: accept -z, --zero-terminated option

Previous Next

To reply to this bug, email your comments to 8103 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bjartur Thorlacius <svartman95 <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: NUL terminated lines
Date: Wed, 23 Feb 2011 23:25:14 +0000

> Maybe we should modify tac to add the -z option.  Would you care to write a patch?
It would be redundant, as tac -s $'\0' is equivalent.

If you want to switch between newline characters dynamically (a bad
idea, IMO) you could edit libc/stdio-common/getline.c to pass read in
the newline character from an environment variable.
More preferably, you could rewrite getline to read in self-delimeted lines.

Message #8 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bjartur Thorlacius <svartman95 <at> gmail.com>
Cc: 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Thu, 24 Feb 2011 08:43:59 +0100

Bjartur Thorlacius wrote:
>> Maybe we should modify tac to add the -z option.  Would you care to
> write a patch?
> It would be redundant, as tac -s $'\0' is equivalent.

Are you using a non-GNU version of tac?
If so, please tell us which one -- that may influence
the decision of whether to make "-s ''" work or to add -z.

With GNU tac, that has never worked:

    $ tac -s ''
    tac: separator cannot be empty

Making -s accommodate an empty string argument is a possibility,
but that change looks like it'd be relatively disruptive.

Unless there is already an implementation for which -s '' works,
I like the idea of adding -z.

Message #11 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Bjartur Thorlacius <svartman95 <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Thu, 24 Feb 2011 14:19:40 +0000

On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
> Bjartur Thorlacius wrote:
>>> Maybe we should modify tac to add the -z option.  Would you care to
>> write a patch?
>> It would be redundant, as tac -s $'\0' is equivalent.
>
> Are you using a non-GNU version of tac?
I don't remember whether I was using FreeBSD or GNU tac.

> If so, please tell us which one -- that may influence
> the decision of whether to make "-s ''" work or to add -z.
>
> With GNU tac, that has never worked:
>
>     $ tac -s ''
>     tac: separator cannot be empty
>
NUL!=the empty string.

> Making -s accommodate an empty string argument is a possibility,
> but that change looks like it'd be relatively disruptive.
>
I don't understand what that would do. Self-delimited strings would be
quite disruptive, indeed, but I gather that's not what you're talking
about.

Message #14 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bjartur Thorlacius <svartman95 <at> gmail.com>
Cc: 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Thu, 24 Feb 2011 20:11:21 +0100

Bjartur Thorlacius wrote:
> On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
>> Bjartur Thorlacius wrote:
>>>> Maybe we should modify tac to add the -z option.  Would you care to
>>> write a patch?
>>> It would be redundant, as tac -s $'\0' is equivalent.
>>
>> Are you using a non-GNU version of tac?
> I don't remember whether I was using FreeBSD or GNU tac.

Please confirm.  I may make a difference.

>> If so, please tell us which one -- that may influence
>> the decision of whether to make "-s ''" work or to add -z.
>>
>> With GNU tac, that has never worked:
>>
>>     $ tac -s ''
>>     tac: separator cannot be empty
>>
> NUL!=the empty string.

tac treats them the same way.

>> Making -s accommodate an empty string argument is a possibility,
>> but that change looks like it'd be relatively disruptive.
>>
> I don't understand what that would do. Self-delimited strings would be
> quite disruptive, indeed, but I gather that's not what you're talking
> about.

Changing tac.c to make -s accommodate an empty string
looks like it would require changes that are too invasive.

Message #17 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: "Alan Curry" <pacman-cu <at> kosh.dhis.org>
To: svartman95 <at> gmail.com (Bjartur Thorlacius)
Cc: 8103 <at> debbugs.gnu.org, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#8103: NUL terminated lines
Date: Thu, 24 Feb 2011 17:13:19 -0500 (GMT+5)

Bjartur Thorlacius writes:
> 
> On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
> > Bjartur Thorlacius wrote:
> >>> Maybe we should modify tac to add the -z option.  Would you care to
> >> write a patch?
> >> It would be redundant, as tac -s $'\0' is equivalent.

Note that a $'\0' argument in a shell command line is exactly equivalent to
an empty string, since it must be passed from the shell to the program using
execve() which takes NUL-terminated strings.

There is no way to run a program with an actual NUL byte contained in one of
its arguments. execve will stop copying at the NUL, and even if it didn't,
the new program receives its arguments in int argc, char **argv form so how
is it supposed to know that there's a NUL in there that's not a terminator?

This limitation can't be avoided. It's not just a C language thing. The
execve interface is based on NUL-terminated strings at the asm level too.

If tac -s $'\0' did something different from tac -s '', it could only have
been a shell builtin. (Assuming the shell supported the $'...' notation at
all)

-- 
Alan Curry

Message #20 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Bjartur Thorlacius <svartman95 <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Fri, 25 Feb 2011 17:35:08 +0000

On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
> Bjartur Thorlacius wrote:
>> On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
>>> Bjartur Thorlacius wrote:
>>>>> Maybe we should modify tac to add the -z option.  Would you care to
>>>> write a patch?
>>>> It would be redundant, as tac -s $'\0' is equivalent.
>>>
>>> Are you using a non-GNU version of tac?
>> I don't remember whether I was using FreeBSD or GNU tac.
>
> Please confirm.  I may make a difference.
>
I was using GNU tac, but didn't actually test tac -s $'\0'.
It parses it as a C string. Makes sense, once you've written it.

>>> If so, please tell us which one -- that may influence
>>> the decision of whether to make "-s ''" work or to add -z.
>>>
>>> With GNU tac, that has never worked:
>>>
>>>     $ tac -s ''
>>>     tac: separator cannot be empty
>>>
>> NUL!=the empty string.
>
> tac treats them the same way.
>
>>> Making -s accommodate an empty string argument is a possibility,
>>> but that change looks like it'd be relatively disruptive.
>>>
>> I don't understand what that would do. Self-delimited strings would be
>> quite disruptive, indeed, but I gather that's not what you're talking
>> about.
>
> Changing tac.c to make -s accommodate an empty string
> looks like it would require changes that are too invasive.
>
Maybe, but there's nothing stopping us from using non-standard string
functions .

Message #23 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: Bjartur Thorlacius <svartman95 <at> gmail.com>, 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Fri, 25 Feb 2011 19:14:31 +0100

Jim Meyering <jim <at> meyering.net> writes:

> Bjartur Thorlacius wrote:
>> On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
>>> With GNU tac, that has never worked:
>>>
>>>     $ tac -s ''
>>>     tac: separator cannot be empty
>>>
>> NUL!=the empty string.
>
> tac treats them the same way.

Actually the shell does.  $'\0' and '' are indistinguishable once passed
as argument.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #26 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: Bjartur Thorlacius <svartman95 <at> gmail.com>, 8103 <at> debbugs.gnu.org
Subject: Re: bug#8103: NUL terminated lines
Date: Fri, 25 Feb 2011 21:09:17 +0100

Andreas Schwab wrote:
> Jim Meyering <jim <at> meyering.net> writes:
>
>> Bjartur Thorlacius wrote:
>>> On 2/24/11, Jim Meyering <jim <at> meyering.net> wrote:
>>>> With GNU tac, that has never worked:
>>>>
>>>>     $ tac -s ''
>>>>     tac: separator cannot be empty
>>>>
>>> NUL!=the empty string.
>>
>> tac treats them the same way.
>
> Actually the shell does.  $'\0' and '' are indistinguishable once passed
> as argument.

I thought so, too.  Then I tried this:

    $ zsh  -c "printf \$'\0'|wc -c"
    1
    $ bash -c "printf \$'\0'|wc -c"
    0

The former prints a NUL byte.
The latter prints nothing.

So it depends on your shell.

Message #29 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 8103 <at> debbugs.gnu.org, Bjartur Thorlacius <svartman95 <at> gmail.com>,
	Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#8103: NUL terminated lines
Date: Fri, 25 Feb 2011 13:45:28 -0700

[Message part 1 (text/plain, inline)]

On 02/25/2011 01:09 PM, Jim Meyering wrote:
>> Actually the shell does.  $'\0' and '' are indistinguishable once passed
>> as argument.
> 
> I thought so, too.  Then I tried this:
> 
>     $ zsh  -c "printf \$'\0'|wc -c"
>     1
>     $ bash -c "printf \$'\0'|wc -c"
>     0
> 
> The former prints a NUL byte.
> The latter prints nothing.
> 
> So it depends on your shell.

zsh does some magic to preserve NUL bytes _when passed as arguments to
shell builtins_.  In this case, printf is a zsh shell builtin,
therefore, zsh's $'\0' is able to preserve the fact that you passed in a
NUL byte, then printf outputs that NUL byte again, such that wc sees it
on stdin.

But zsh is special in that regards, and it violates the upcoming POSIX
wording that will be standardized regarding $'' behavior (which means
zsh will have to add yet another shell option soon, if it hasn't
already, to choose between posix and zsh behavior regarding NUL):

http://austingroupbugs.net/view.php?id=249

        If a \xXX or \XXX escape sequence yields a byte whose value
        is 0, that byte and any following regular characters and
        escape sequences up to the terminating unescaped single-quote
        shall be evaluated and discarded.

Bash, on the other hand, strips NUL bytes everywhere.  And zsh _can't_
preserve NUL bytes to non-builtins.

[As a side note, printf '\0' portably prints a NUL byte, but that's
because you're passing two bytes on input which get interpreted as an
escape sequence, and not passing a literal NUL byte as in zsh printf $'\0']

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #32 received at 8103 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bjartur Thorlacius <svartman95 <at> gmail.com>
Cc: 8103 <at> debbugs.gnu.org
Subject: tac RFE: accept -z,
	--zero-terminated option (was: bug#8103: NUL terminated lines
Date: Sat, 23 Jul 2011 00:05:57 +0200

retitle 8103 tac RFE: accept -z, --zero-terminated option
tags 8103 + notabug
thanks

Jim Meyering wrote:
> Bjartur Thorlacius wrote:
>>> Maybe we should modify tac to add the -z option.  Would you care to
>> write a patch?
>> It would be redundant, as tac -s $'\0' is equivalent.
>
> Are you using a non-GNU version of tac?
> If so, please tell us which one -- that may influence
> the decision of whether to make "-s ''" work or to add -z.
>
> With GNU tac, that has never worked:
>
>     $ tac -s ''
>     tac: separator cannot be empty
>
> Making -s accommodate an empty string argument is a possibility,
> but that change looks like it'd be relatively disruptive.
>
> Unless there is already an implementation for which -s '' works,
> I like the idea of adding -z.

I've changed the title and marking this as "not a bug."

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.