Subject: bug#22237: closed (Re: bug#22237: sed no longer removes high-ascii characters as it did formerly.)

From: [email-masked] (GNU bug Tracking System)
Date: Sat, 26 Dec 2015 21:20:01 +0000

Your bug report

#22237: sed no longer removes high-ascii characters as it did formerly.

which was filed against the sed package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
22237: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22237
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#22237: sed no longer removes high-ascii characters as it did formerly.
From: Jim Meyering <[email-masked]>
To: Brian Tew <[email-masked]>
Cc: [email-masked]
Date: Sat, 26 Dec 2015 13:19:07 -0800
List-Id: <debbugs-submit.debbugs.gnu.org>

On Fri, Dec 25, 2015 at 4:21 AM, Brian Tew <[email-masked]> wrote:
> Well, sometimes it do and sometimes it don't.
>
> Script started on Fri 25 Dec 2015 05:53:04 AM CS
> ~$ed sample
> 50
> l
> subject now that thanksgiving has come and gone\342\246$
> q
> ~$
> ~$sed -i 's/[^a-z 0-9]//g' sample

To remove all but the matched bytes, you probably want something like
this instead:

  LC_ALL=C sed -i 's/[^[:alnum:] ]//'

Note I've done two things: used LC_ALL=C to override your default
locale (probably a UTF8 one), and to use [:alnum:] in place of that
nonportable a-z range and 0-9.

In general, with UTF8-based locales, a byte sequence like your
\342\246 will match no regular expression, since it is not a valid
UTF8 character.

What probably changed is that older versions of sed did not properly
handle multi-byte locales, or your other experience was using a
single-byte locale.

If you still think there is a problem with sed-4.22, please provide
more detail and I'll reopen this issue.

Subject: sed no longer removes high-ascii characters as it did formerly.
From: Brian Tew <[email-masked]>
To: [email-masked]
Date: Fri, 25 Dec 2015 06:21:41 -0600
List-Id: <debbugs-submit.debbugs.gnu.org>

Well, sometimes it do and sometimes it don't.

Script started on Fri 25 Dec 2015 05:53:04 AM CS
~$ed sample
50
l
subject now that thanksgiving has come and gone\342\246$
q
~$
~$sed -i 's/[^a-z 0-9]//g' sample
~$ed sample
50
l
subject now that thanksgiving has come and gone\342\246$
q
~$
~$un[Ksed --version
sed (GNU sed) 4.2.2
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
and Paolo Bonzini.
GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
E-mail bug reports to: <[email-masked]>.
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
~$exit

Script done on Fri 25 Dec 2015 05:59:12 AM CS