Subject: bug#50701: cannot append or insert to empty file or stream

From: Assaf Gordon <[email-masked]>
Date: Mon, 20 Sep 2021 13:28:42 -0600

tag 50701 notabug
close 50701
stop

Hello,

Thank you for providing clear reproducible example of the situation -
it makes troubleshooting much easier.

This is, however, not a bug - but the intended behavior:

On 2021-09-20 2:53 a.m., lexi hale wrote:
> there is an inconsistency in the handling of the a and i commands in
> sed. if the input stream or file immediately yields EOL, sed ignores
> the commands and produces an empty stream.
> 

First, let's clarify what the files are:

> 	$ echo -n >file line-1
 > 	$ touch empty

The file 'file' is not empty, it has one line.
This line just happens to be empty (i.e. no characters in the line 
before the newline character).

The file 'empty' is empty, it contains NO lines.

The above might seem obvious,
but the distinction is important for the next step:

> 	$ sed -i aline-2 file

The sed command "aline-2", means append ("a") the text "line-2",
to *every* line.
The "every line" parts is due to the command "a" having no address
part (contrast with sed command "3aline-2" which would add "line-2" only 
to the third line).

If the input file has (any) lines, the command will be executed.
If the input does not have any lines, the command will not be executed.
And that is the behavior you are seeing.

> however, >
>   $ sed -i aline-1 empty >   $ cat empty>
> prints nothing,  instead of the expected result "line-1". >

Compare the situation to this (slightly more obvious) case:

     $ printf "%s\n" hello world > in1
     $ cat in1
     hello
     world

Add 'FOO' after every line:

     $ sed 'aFOO' in1
     hello
     FOO
     world
     FOO

Add 'FOO' after the second line:

     $ sed '2aFOO' in1
     hello
     world
     FOO

Add 'FOO' after the third line:

     $ sed '3aFOO' in1
     hello
     world

Since there is NO third line in the input, the command wasn't executed.

And similarly, since the 'empty' file does not have ANY lines,
the command (which is programmed to run on every line) is not executed.

> this is an extremely surprising behavior that limits the utility of sed
> when one cannot predict the contents of the file in question.  

I hope the explanation above makes this behavior less surprising.

> [...] it would be helpful to at least have a flag that would turn
> on consistent behavior for these commands.
> 
> however, making the consistent behavior the new default might also fix
> a few extremely rare bugs in existing shell scripts :)

I humbly think that this behavior is consistent and does not require any
modification or flags - but other opinions are welcomed.

When integrating with shell scripts, edge-cases (e.g. empty input)
should probably be checked explicitly.

Few suggestion to check for empty files (contrived and not tested):

     if test -s INPUTFILE ; then
        # process the file
     else
        echo "FILE IS EMPTY"
     fi

     ---

     lines=$(wc -l < INPUTFILE)
     if test "$lines" -gt 0 ; then
        # process the file
     else
        echo "FILE IS EMPTY"
     fi

And lastly, since you are already using GNU sed extensions
(and not worrying about portability), you can use
the 'qNUM' command extension to exit with a specific code
if there was any input:

     sed -e 'aline-1'  -e '$q42' INPUTFILE > OUTPUTFILE
     exit_code=$?
     if test $exit_code -eq 42 ; then
        # file processed OK, wasn't empty
     elif test $exit_code -eq 0 ; then
        # file was empty
     else
        # another sed error (e.g. bad program, I/O error, etc)
     fi

As such, I'm marking this as "not a bug", but discussion can continue
by replying to this thread.

regards,
  - assaf