GNU bug report logs - #27354
strange behavior of sed

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27354 in the body.
You can then email your comments to 27354 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Han Lu <luhanwa <at> gmail.com>
To: bug-sed <at> gnu.org
Subject: strange behavior of sed
Date: Wed, 14 Jun 2017 11:48:41 +0800

[Message part 1 (text/plain, inline)]

Hi Team,

Please check this below:

~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.5 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.5 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

~$ sed --version
sed (GNU sed) 4.2.2
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
and Paolo Bonzini.
GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed <at> gnu.org>.
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.


Problem reproduce process:

~$ cat testfile
127.0.0.1 localhost hostname
::1     localhost ip6-localhost ip6-loopback

The below command works as expected.
~$ sed -e 's/127.0.0.1/127/' -e "1i 127.0.0.1 localhost\n127.0.0.1
$HOSTNAME.domainname $HOSTNAME" testfile
127.0.0.1 localhost
127.0.0.1 node1.domainname node1
127 localhost hostname
::1     localhost ip6-localhost ip6-loopback

But command below is not working as expeced:
~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.1 localhost\n127.0.0.1
$HOSTNAME.domainname $HOSTNAME" testfile
::1     localhost ip6-localhost ip6-loopback

expected output:
127.0.0.1 localhost
127.0.0.1 node1.domainname node1
::1     localhost ip6-localhost ip6-loopback

If I add a comment at first line. then it's working.
~$ cat testfile
# comment
127.0.0.1 localhost hostname
::1     localhost ip6-localhost ip6-loopback

~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.2 localhost\n127.0.0.2
$HOSTNAME.domainname $HOSTNAME" testfile
127.0.0.2 localhost
127.0.0.2 node1.domainname node1
# comment
::1     localhost ip6-localhost ip6-loopback

[Message part 2 (text/html, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Han Lu <luhanwa <at> gmail.com>
To: bug-sed <at> gnu.org
Subject: Re: strange behavior of sed
Date: Wed, 14 Jun 2017 11:50:10 +0800

[Message part 1 (text/plain, inline)]

sorry for the typo, the last command should be the below:

~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.1 localhost\n127.0.0.1
$HOSTNAME.domainname $HOSTNAME" testfile
127.0.0.1 localhost
127.0.0.1 node1.domainname node1
# comment
::1     localhost ip6-localhost ip6-loopback

2017-06-14 11:48 GMT+08:00 Han Lu <luhanwa <at> gmail.com>:

> Hi Team,
>
> Please check this below:
>
> ~$ cat /etc/os-release
> NAME="Ubuntu"
> VERSION="14.04.5 LTS, Trusty Tahr"
> ID=ubuntu
> ID_LIKE=debian
> PRETTY_NAME="Ubuntu 14.04.5 LTS"
> VERSION_ID="14.04"
> HOME_URL="http://www.ubuntu.com/"
> SUPPORT_URL="http://help.ubuntu.com/"
> BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
>
> ~$ sed --version
> sed (GNU sed) 4.2.2
> Copyright (C) 2012 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.
> html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by Jay Fenlason, Tom Lord, Ken Pizzini,
> and Paolo Bonzini.
> GNU sed home page: <http://www.gnu.org/software/sed/>.
> General help using GNU software: <http://www.gnu.org/gethelp/>.
> E-mail bug reports to: <bug-sed <at> gnu.org>.
> Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
>
>
> Problem reproduce process:
>
> ~$ cat testfile
> 127.0.0.1 localhost hostname
> ::1     localhost ip6-localhost ip6-loopback
>
> The below command works as expected.
> ~$ sed -e 's/127.0.0.1/127/' -e "1i 127.0.0.1 localhost\n127.0.0.1
> $HOSTNAME.domainname $HOSTNAME" testfile
> 127.0.0.1 localhost
> 127.0.0.1 node1.domainname node1
> 127 localhost hostname
> ::1     localhost ip6-localhost ip6-loopback
>
> But command below is not working as expeced:
> ~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.1 localhost\n127.0.0.1
> $HOSTNAME.domainname $HOSTNAME" testfile
> ::1     localhost ip6-localhost ip6-loopback
>
> expected output:
> 127.0.0.1 localhost
> 127.0.0.1 node1.domainname node1
> ::1     localhost ip6-localhost ip6-loopback
>
> If I add a comment at first line. then it's working.
> ~$ cat testfile
> # comment
> 127.0.0.1 localhost hostname
> ::1     localhost ip6-localhost ip6-loopback
>
> ~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.2 localhost\n127.0.0.2
> $HOSTNAME.domainname $HOSTNAME" testfile
> 127.0.0.2 localhost
> 127.0.0.2 node1.domainname node1
> # comment
> ::1     localhost ip6-localhost ip6-loopback
>
>

[Message part 2 (text/html, inline)]

Message #15 received at 27354-done <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Han Lu <luhanwa <at> gmail.com>
Cc: 27354-done <at> debbugs.gnu.org
Subject: Re: bug#27354: strange behavior of sed
Date: Wed, 14 Jun 2017 14:58:35 +0000

tag 27354 notabug
thanks

Hello, 

On Wed, Jun 14, 2017 at 11:48:41AM +0800, Han Lu wrote:
[...]
>The below command works as expected.
>~$ sed -e 's/127.0.0.1/127/' -e "1i 127.0.0.1 localhost\n127.0.0.1
[...]
>But command below is not working as expeced:
>~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.1 localhost\n127.0.0.1

First,
thank you for providing such clear information on how to
reproduce your output - it makes troubleshooting much easier.

This is not a bug in sed, but a side-effect of the 'd' (=delete)
command: when sed executes the 'd' command, it immediately starts
a new cycle (i.e. skips all other commands, reads the next line of the 
input, and starts from the first sed command).

In your example, the sequence is:
1. sed reads the first input line (it contains '127.0.0.1')
2. the 'd' command is executed - line is deleted and not printed,
  and sed restarts a new cycle:
  it skips all other commands and reads the next input line.
4. The first time the '1i' command is executed, sed is already
  on the second input line, and the address '1' is never matched -
  the command is never executed.

This also explains why it does work when you add a dummy comment line:
The '/127.0.0.1/d' does not match the first line,
and then the '1i' command is matched and executed.

A simple work-around would be to put the 'i' command
before the 'd' command, like so:

   sed -e '1i 127.0.0.1 ....' -e '/127/d'

As such, I'm marking this as 'not a bug',
but discussion can continue by replying to this thread.

regards,
- assaf

Message #18 received at 27354-done <at> debbugs.gnu.org (full text, mbox):

From: Han Lu <luhanwa <at> gmail.com>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 27354-done <at> debbugs.gnu.org
Subject: Re: bug#27354: strange behavior of sed
Date: Tue, 20 Jun 2017 14:24:51 +0800

[Message part 1 (text/plain, inline)]

Hi Assaf,

Thanks for the explanation.

But if I put 'i' command before 'd' command, shouldn't the input new lines
be deleted ? Since new inputed lines contain the pattern 'd' command want.

Br,
Xingming

2017-06-14 22:58 GMT+08:00 Assaf Gordon <assafgordon <at> gmail.com>:

> tag 27354 notabug
> thanks
>
> Hello,
> On Wed, Jun 14, 2017 at 11:48:41AM +0800, Han Lu wrote:
> [...]
>
>> The below command works as expected.
>> ~$ sed -e 's/127.0.0.1/127/' -e "1i 127.0.0.1 localhost\n127.0.0.1
>>
> [...]
>
>> But command below is not working as expeced:
>> ~$ sed -e '/127.0.0.1/d' -e "1i 127.0.0.1 localhost\n127.0.0.1
>>
>
> First,
> thank you for providing such clear information on how to
> reproduce your output - it makes troubleshooting much easier.
>
> This is not a bug in sed, but a side-effect of the 'd' (=delete)
> command: when sed executes the 'd' command, it immediately starts
> a new cycle (i.e. skips all other commands, reads the next line of the
> input, and starts from the first sed command).
>
> In your example, the sequence is:
> 1. sed reads the first input line (it contains '127.0.0.1')
> 2. the 'd' command is executed - line is deleted and not printed,
>   and sed restarts a new cycle:
>   it skips all other commands and reads the next input line.
> 4. The first time the '1i' command is executed, sed is already
>   on the second input line, and the address '1' is never matched -
>   the command is never executed.
>
> This also explains why it does work when you add a dummy comment line:
> The '/127.0.0.1/d' does not match the first line,
> and then the '1i' command is matched and executed.
>
> A simple work-around would be to put the 'i' command
> before the 'd' command, like so:
>
>    sed -e '1i 127.0.0.1 ....' -e '/127/d'
>
> As such, I'm marking this as 'not a bug',
> but discussion can continue by replying to this thread.
>
> regards,
> - assaf
>
>
>
>

[Message part 2 (text/html, inline)]

Message #21 received at 27354-done <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Han Lu <luhanwa <at> gmail.com>
Cc: 27354-done <at> debbugs.gnu.org
Subject: Re: bug#27354: strange behavior of sed
Date: Wed, 21 Jun 2017 20:18:59 +0000

Hello,

On Tue, Jun 20, 2017 at 02:24:51PM +0800, Han Lu wrote:
>But if I put 'i' command before 'd' command, shouldn't the input new lines
>be deleted ? Since new inputed lines contain the pattern 'd' command want.

There is a subtlely in the 'i' (insert) command:
The added content is not stored in the pattern space and is
not subject to later manipulations (such as pattern matching, 's///'
and 'd').

The GNU sed manual says:

   i = Immediately output the lines of text which follow this command.

Observe the following:

   $ seq 3 | sed -e 1iX -e 's/X/Y/'
   X
   1
   2
   3

And similarly with match+delete command:

   $ seq 3 | sed -e 1iX -e '/X/d'
   X
   1
   2
   3


Similar subtlety also applies to 'a' (append) and 'c' (change) commands.

The manual should probably be improved to explicitly mention
these points (added to my mental 'todo' list).


regards,
- assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.