Subject: Re: bug#30336: sed

From: Assaf Gordon <[email-masked]>
Date: Sat, 3 Feb 2018 12:24:35 -0700

Hello,

On 2018-02-03 09:11 AM, Roger Adair wrote:
> when I execute the following command:
> 
> $ sed 's/c$/7/g' test.data.asc
> 
> where the contents of test.data.asc is:
> 
> abc  abc
> 123456
> 
> I get the following output:
> 
> abc  abc
> 123456
> 
> Whereas I would expect the following:
> 
> abc  ab7
> 123456
> 
> If I leave the $ out of the command "ab7 ab7" appears.
> 
> What am I doing wrong?  Many thanks for reading my email.

My immediate guess would be that the file has dos/mac line endings
(e.g. CRLF "\r\n" instead of just line-feed "\n").

You can test it by using 'file', like so:

   $ file test.data.sac
   test.data.asc: ASCII text, with CRLF line terminators

Or by printing it's content like so:

   $ od -tcz test.data.asc
   0000000   a   b   c       a   b   c  \r  \n      >abc  abc..<

If you see only "\r" in the output - the file has MAC line-endings.
If you see both "\r\n" - the file has dos/windows line endings.

In unix, GNU sed "$" only matches "\n", and "\r" is just another character.

Some ways to work-around it:
1. Use "dos2unix" or "mac2unix" on the file (simplest and recommend).

2. Use the following to strip "\r" from the file
(assuming it also has "\n"):

     cat test.data.asc \
         | tr -d '\r' \
         | sed 's/c$/7/g'

If the file has only "\r", convert them to "\n":

     cat test.data.asc \
         | tr '\r' '\n' \
         | sed 's/c$/7/g'

3. Add sed command to remove "\r" before doing the
other substitution:

    sed -e 's/\r$//g' -e 's/c$/7/g' test.data.asc

4. Or modify the regex itself to optionally match "\r":

    sed 's/c\r*$/7/g' test.data.asc

---

If none of the above work, and you suspect another problem,
please provide a small fragment of your input file so we can reproduce 
the issue.

regards,
  - assaf