Subject: bug#25455: uniq considers all the full-width punctuation and Japanese kana as the same under zh_CN.UTF-8 locale

From: "Mingye Wang (Arthur2e5)" <[email-masked]>
Date: Tue, 17 Jan 2017 18:22:48 +0000

15.01.2017, 20:01, "Icenowy Zheng" <[email-masked]>:
> Problem:
> When dealing lines with only a Chinese full-width punctuation or Japanese kana
> and locale is zh_CN.UTF-8, uniq command will consider all the lines are the
> same, and wrongly removed different punctuations.

To narrow the scope down a bit, I should mention that LC_COLLATE is enough to trigger the bug:

printf '%s\n' 。 ， ？ ！ a b c | LC_COLLATE=zh_CN.UTF-8 uniq

-- 
Regards,

Arthur2e5