From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 08 03:08:14 2014 Received: (at submit) by debbugs.gnu.org; 8 Nov 2014 08:08:14 +0000 Received: from localhost ([127.0.0.1]:53338 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xn14A-0007pn-91 for submit@debbugs.gnu.org; Sat, 08 Nov 2014 03:08:14 -0500 Received: from eggs.gnu.org ([208.118.235.92]:56916) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xn148-0007pg-UV for submit@debbugs.gnu.org; Sat, 08 Nov 2014 03:08:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xn13z-0004FN-TJ for submit@debbugs.gnu.org; Sat, 08 Nov 2014 03:08:12 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:39979) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xn13z-0004FI-QG for submit@debbugs.gnu.org; Sat, 08 Nov 2014 03:08:03 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58615) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xn13s-0006WB-33 for bug-grep@gnu.org; Sat, 08 Nov 2014 03:08:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xn13k-0004EU-Al for bug-grep@gnu.org; Sat, 08 Nov 2014 03:07:56 -0500 Received: from mailgw06.kcn.ne.jp ([61.86.7.213]:36553) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xn13j-0004DQ-R2 for bug-grep@gnu.org; Sat, 08 Nov 2014 03:07:48 -0500 Received: from imp02 (mailgw6.kcn.ne.jp [61.86.15.232]) by mailgw06.kcn.ne.jp (Postfix) with ESMTP id 695E1E8001D for ; Sat, 8 Nov 2014 17:07:42 +0900 (JST) Received: from mail03.kcn.ne.jp ([61.86.6.182]) by imp02 with bizsmtp id Cw7i1p00M3veGq501w7i8C; Sat, 08 Nov 2014 17:07:42 +0900 X-OrgRCPT: bug-grep@gnu.org Received: from [10.120.1.54] (i118-21-128-66.s30.a048.ap.plala.or.jp [118.21.128.66]) by mail03.kcn.ne.jp (Postfix) with ESMTPA id 2B35914100BC for ; Sat, 8 Nov 2014 17:07:42 +0900 (JST) Date: Sat, 08 Nov 2014 17:07:40 +0900 From: Norihiro Tanaka To: bug-grep@gnu.org Subject: [PATCH] tests: fix encoding with `tr' to support multibyte in test Message-Id: <20141108170723.F3E7.27F6AC2D@kcn.ne.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------_545DCCC900000000F3F9_MULTIPART_MIXED_" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.65.07 [ja] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --------_545DCCC900000000F3F9_MULTIPART_MIXED_ Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit It seems that `tr' in GNU coreutils does not recoginize multibyte character, but other imprementation, e.g. HP-UX, Solaris, recoginizes it. As a result, [ echo AB | LC_ALL=ja_JP.eucJP tr AB '\244\263' ] is transformed as [ echo AB | LC_ALL=ja_JP.eucJP tr A '\244\263' ], so that '\244\263' is recognized as a single multibyte character. We do not expect that. --------_545DCCC900000000F3F9_MULTIPART_MIXED_ Content-Type: text/plain; charset="US-ASCII"; name="0001-grep-fix-encoding-with-tr-to-support-multibyte-in-te.patch" Content-Disposition: attachment; filename="0001-grep-fix-encoding-with-tr-to-support-multibyte-in-te.patch" Content-Transfer-Encoding: base64 RnJvbSA4NTJiMTI5MDQ1MDk5NTY0NGM2MDQyNzA3MjBlYzViNDMyZTMyM2I1IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBOb3JpaGlybyBUYW5ha2EgPG5vcml0bmtAa2NuLm5lLmpwPgpE YXRlOiBTYXQsIDggTm92IDIwMTQgMTQ6MjY6NTYgKzA5MDAKU3ViamVjdDogW1BBVENIXSB0ZXN0 czogZml4IGVuY29kaW5nIHdpdGggYHRyJyB0byBzdXBwb3J0IG11bHRpYnl0ZSBpbiB0ZXN0CiBz dWl0ZQoKdGVzdHMvZXVjLW1iLCB0ZXN0cy9mZ3JlcC1pbmZsb29wLCB0ZXN0cy9pbnZhbGlkLW11 bHRpYnl0ZS1pbmZsb29wLAp0ZXN0cy9wcmVmaXgtb2YtbXVsdGlieXRlLCB0ZXN0cy9zamlzLW1i OiBGaXggdGhlbS4KLS0tCiB0ZXN0cy9ldWMtbWIgICAgICAgICAgICAgICAgICAgIHwgMiArLQog dGVzdHMvZmdyZXAtaW5mbG9vcCAgICAgICAgICAgICB8IDIgKy0KIHRlc3RzL2ludmFsaWQtbXVs dGlieXRlLWluZmxvb3AgfCAyICstCiB0ZXN0cy9wcmVmaXgtb2YtbXVsdGlieXRlICAgICAgIHwg MiArLQogdGVzdHMvc2ppcy1tYiAgICAgICAgICAgICAgICAgICB8IDIgKy0KIDUgZmlsZXMgY2hh bmdlZCwgNSBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3Rlc3Rz L2V1Yy1tYiBiL3Rlc3RzL2V1Yy1tYgppbmRleCBiNjI1MDQ2Li5jNjJmOTlmIDEwMDc1NQotLS0g YS90ZXN0cy9ldWMtbWIKKysrIGIvdGVzdHMvZXVjLW1iCkBAIC0xMiw3ICsxMiw3IEBAIHJlcXVp cmVfY29tcGlsZWRfaW5fTUJfc3VwcG9ydAogbG9jYWxlPWphX0pQLkVVQy1KUAogCiBtYWtlX2lu cHV0ICgpIHsKLSAgZWNobyAiJDEiIHwgdHIgQUIgJ1wyNDRcMjYzJworICBlY2hvICIkMSIgfCBM Q19BTEw9QyB0ciBBQiAnXDI0NFwyNjMnCiB9CiAKIGV1Y19ncmVwICgpIHsKZGlmZiAtLWdpdCBh L3Rlc3RzL2ZncmVwLWluZmxvb3AgYi90ZXN0cy9mZ3JlcC1pbmZsb29wCmluZGV4IDAxNWVjNzQu LjBkYTViZTEgMTAwNzU1Ci0tLSBhL3Rlc3RzL2ZncmVwLWluZmxvb3AKKysrIGIvdGVzdHMvZmdy ZXAtaW5mbG9vcApAQCAtNiw3ICs2LDcgQEAgcmVxdWlyZV9lbl91dGY4X2xvY2FsZV8KIHJlcXVp cmVfdGltZW91dF8KIHJlcXVpcmVfY29tcGlsZWRfaW5fTUJfc3VwcG9ydAogCi1lbmNvZGUoKSB7 IGVjaG8gIiQxIiB8IHRyIEFCQyAnXDM1N1wyNzRcMjQxJzsgfQorZW5jb2RlKCkgeyBlY2hvICIk MSIgfCBMQ19BTEw9QyB0ciBBQkMgJ1wzNTdcMjc0XDI0MSc7IH0KIAogZW5jb2RlIEFCQyA+IGlu IHx8IGZyYW1ld29ya19mYWlsdXJlXwogZmFpbD0wCmRpZmYgLS1naXQgYS90ZXN0cy9pbnZhbGlk LW11bHRpYnl0ZS1pbmZsb29wIGIvdGVzdHMvaW52YWxpZC1tdWx0aWJ5dGUtaW5mbG9vcAppbmRl eCBkN2M2MTY1Li44YjM0NThmIDEwMDc1NQotLS0gYS90ZXN0cy9pbnZhbGlkLW11bHRpYnl0ZS1p bmZsb29wCisrKyBiL3Rlc3RzL2ludmFsaWQtbXVsdGlieXRlLWluZmxvb3AKQEAgLTcsNyArNyw3 IEBAIHJlcXVpcmVfZW5fdXRmOF9sb2NhbGVfCiByZXF1aXJlX2NvbXBpbGVkX2luX01CX3N1cHBv cnQKIHJlcXVpcmVfdGltZW91dF8KIAotZW5jb2RlKCkgeyBlY2hvICIkMSIgfCB0ciBBICdcMjAy JzsgfQorZW5jb2RlKCkgeyBlY2hvICIkMSIgfCBMQ19BTEw9QyB0ciBBICdcMjAyJzsgfQogCiBl bmNvZGUgQUEgPiBpbnB1dAogCmRpZmYgLS1naXQgYS90ZXN0cy9wcmVmaXgtb2YtbXVsdGlieXRl IGIvdGVzdHMvcHJlZml4LW9mLW11bHRpYnl0ZQppbmRleCAyMjI4YTIyLi5iMDBjNjRlIDEwMDc1 NQotLS0gYS90ZXN0cy9wcmVmaXgtb2YtbXVsdGlieXRlCisrKyBiL3Rlc3RzL3ByZWZpeC1vZi1t dWx0aWJ5dGUKQEAgLTUsNyArNSw3IEBACiByZXF1aXJlX2VuX3V0ZjhfbG9jYWxlXwogcmVxdWly ZV9jb21waWxlZF9pbl9NQl9zdXBwb3J0CiAKLWVuY29kZSgpIHsgZWNobyAiJDEiIHwgdHIgQUJD ICdcMzU3XDI3NFwyNDEnOyB9CitlbmNvZGUoKSB7IGVjaG8gIiQxIiB8IExDX0FMTD1DIHRyIEFC QyAnXDM1N1wyNzRcMjQxJzsgfQogCiBlbmNvZGUgQUJDID5leHAxCiBlbmNvZGUgYUFCQyA+ZXhw MgpkaWZmIC0tZ2l0IGEvdGVzdHMvc2ppcy1tYiBiL3Rlc3RzL3NqaXMtbWIKaW5kZXggOTkwYzc3 MC4uODczNTc5MiAxMDA3NTUKLS0tIGEvdGVzdHMvc2ppcy1tYgorKysgYi90ZXN0cy9zamlzLW1i CkBAIC0yMSw3ICsyMSw3IEBAIGxvY2FsZT1qYV9KUC5TSElGVF9KSVMKICMgICBtYXRjaGluZyAi QSIgYWdhaW5zdCAiQEEiLCBvciBtaXN0YWtpbmcgYSB2YWxpZCAiQSIgbWF0Y2ggZm9yIHRoZSBz ZWNvbmQKICMgICBieXRlIG9mIGEgbXVsdGktYnl0ZSBjaGFyYWN0ZXIuCiAKLWVuY29kZSgpIHsg ZWNobyAiJDEiIHwgdHIgQCUgJ1wyMDNcMzAxJzsgfQorZW5jb2RlKCkgeyBlY2hvICIkMSIgfCBM Q19BTEw9QyB0ciBAJSAnXDIwM1wzMDEnOyB9CiAKIGs9MAogdGVzdF9ncmVwX3JlamVjdCgpIHsK LS0gCjIuMS4zCgo= --------_545DCCC900000000F3F9_MULTIPART_MIXED_-- From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 08 22:01:18 2014 Received: (at 18991-done) by debbugs.gnu.org; 9 Nov 2014 03:01:18 +0000 Received: from localhost ([127.0.0.1]:54119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XnIkg-0004bA-BQ for submit@debbugs.gnu.org; Sat, 08 Nov 2014 22:01:18 -0500 Received: from mail-yk0-f179.google.com ([209.85.160.179]:46683) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XnIke-0004az-25 for 18991-done@debbugs.gnu.org; Sat, 08 Nov 2014 22:01:16 -0500 Received: by mail-yk0-f179.google.com with SMTP id 131so3296062ykp.38 for <18991-done@debbugs.gnu.org>; Sat, 08 Nov 2014 19:01:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=kzXX1B7Pu4vxhwOHt4qVARVP2YX0/RQQmht1doSkqLk=; b=fkb6mtAnclx/mJ8yCwqH1chHGmWz+Fr19jMkHmREc4gZW78bFpQPRvFcOXgYzE2d8l hKnaLV///R4TsShtGIycJV6Ur1X8Ue9S94JZU3wh9xBu1RLuGFgsC7XAeMuQZPLQcYAC Tl8ISWw+OX162+7x3Zkiz0XRIfFMYbEw9I8I8U89zx6+AcKH+xNHKDz0rHh/pAW6U8Y2 MeLJBkeFaMBhwGoBm41BjaMIo+Ji+sDdEkKjdqzO7ugRpwrFlHLO5goIec/pN5EUpMP/ k/kr7sJxKPrhmR6Nqd3k9dBmw4uvIfW4OF7T4snWPJ23Wq+k6tZfv/wNJBtLzhkn7s7s niuw== X-Received: by 10.236.110.35 with SMTP id t23mr21660784yhg.126.1415502075482; Sat, 08 Nov 2014 19:01:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.170.157.9 with HTTP; Sat, 8 Nov 2014 19:00:55 -0800 (PST) In-Reply-To: <20141108170723.F3E7.27F6AC2D@kcn.ne.jp> References: <20141108170723.F3E7.27F6AC2D@kcn.ne.jp> From: Jim Meyering Date: Sat, 8 Nov 2014 19:00:55 -0800 X-Google-Sender-Auth: EUjoJbgMWtCVq6vLj6E9PdyETIQ Message-ID: Subject: Re: bug#18991: [PATCH] tests: fix encoding with `tr' to support multibyte in test To: Norihiro Tanaka Content-Type: multipart/mixed; boundary=001a1133358a63748205076442dd X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 18991-done Cc: 18991-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a1133358a63748205076442dd Content-Type: text/plain; charset=ISO-8859-1 On Sat, Nov 8, 2014 at 12:07 AM, Norihiro Tanaka wrote: > It seems that `tr' in GNU coreutils does not recoginize multibyte > character, but other imprementation, e.g. HP-UX, Solaris, recoginizes it. > > As a result, [ echo AB | LC_ALL=ja_JP.eucJP tr AB '\244\263' ] is > transformed as [ echo AB | LC_ALL=ja_JP.eucJP tr A '\244\263' ], so that > '\244\263' is recognized as a single multibyte character. We do not > expect that. Thank you for the report and patch. However, it is not maintainable to modify every use of "tr" in the tests. Instead, I've addressed this by making all of the tests use tr through a wrapper that always sets LC_ALL=C: --001a1133358a63748205076442dd Content-Type: application/octet-stream; name="0001-tests-avoid-a-multibyte-tr-portability-problem.patch" Content-Disposition: attachment; filename="0001-tests-avoid-a-multibyte-tr-portability-problem.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i29tcjyw1 RnJvbSAyMWI1ZGM4YmI1YTMzMGI4NGY2MTgwZWRkNDhkMDgxNzdlYjM3MWY3IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKaW0gTWV5ZXJpbmcgPG1leWVyaW5nQGZiLmNvbT4KRGF0ZTog U2F0LCA4IE5vdiAyMDE0IDE4OjU5OjI0IC0wODAwClN1YmplY3Q6IFtQQVRDSF0gdGVzdHM6IGF2 b2lkIGEgbXVsdGlieXRlIHRyIHBvcnRhYmlsaXR5IHByb2JsZW0KCiogdGVzdHMvaW5pdC5jZmcg KHRyKTogTmV3IHdyYXBwZXIgZnVuY3Rpb24uClJlcG9ydGVkIGJ5IE5vcmloaXJvIFRhbmFrYSBp biBodHRwOi8vZGViYnVncy5nbnUub3JnLzE4OTkxCi0tLQogdGVzdHMvaW5pdC5jZmcgfCA5ICsr KysrKysrKwogMSBmaWxlIGNoYW5nZWQsIDkgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdpdCBhL3Rl c3RzL2luaXQuY2ZnIGIvdGVzdHMvaW5pdC5jZmcKaW5kZXggNzVhOTQ5NC4uZTdhMjM4OSAxMDA2 NDQKLS0tIGEvdGVzdHMvaW5pdC5jZmcKKysrIGIvdGVzdHMvaW5pdC5jZmcKQEAgLTE2MiwzICsx NjIsMTIgQEAgaGV4X3ByaW50Zl8oKQogICBwcmludGYgJChwcmludGYgJ1xcJW8nIFwKICAgICAk KHByaW50ZiAlcyAiJDEifHNlZCAncyxcXHhcKFswLTlhLWZBLUZdWzAtOWEtZkEtRl1cKSwgMHhc MSxnJykpCiB9CisKKyMgV3JhcCB0ciBzbyB0aGF0IGl0IGFsd2F5cyBydW5zIGluIHRoZSBDIGxv Y2FsZS4KKyMgT3RoZXJ3aXNlLCBpbiBhIG11bHRpYnl0ZSBsb2NhbGUsIEdOVSB0ciAod2hpY2gg aXMgbm90IG11bHRpYnl0ZS1hd2FyZQorIyBhcyBvZiAyMDE0LTExLTA4KSwgd291bGQgd29yayBk aWZmZXJlbnRseSB0aGFuIG90aGVycy4gIEZvciBleGFtcGxlLAorIyB0aGlzIGNvbW1hbmQsIHdo aWNoIHdhcyB3cml0dGVuIHdpdGggdW5pYnl0ZSBHTlUgdHIgaW4gbWluZCwKKyMgICBMQ19BTEw9 amFfSlAuZXVjSlAgdHIgQUIgJ1wyNDRcMjYzJworIyB3b3VsZCBhY3QgbGlrZSB0aGlzIHdpdGgg dGhlIG11bHRpYnl0ZSB0ciBmcm9tIEhQLVVYIGFuZCBTb2xhcmlzOgorIyAgIExDX0FMTD1qYV9K UC5ldWNKUCB0ciBBICAnXDI0NFwyNjMnCit0cigpIHsgTENfQUxMPUMgZW52IC0tIHRyICIkQCI7 IH0KLS0gCjIuMS4yCgo= --001a1133358a63748205076442dd-- From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 09 00:02:27 2014 Received: (at 18991-done) by debbugs.gnu.org; 9 Nov 2014 05:02:27 +0000 Received: from localhost ([127.0.0.1]:54143 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XnKdv-0007jv-F6 for submit@debbugs.gnu.org; Sun, 09 Nov 2014 00:02:27 -0500 Received: from mailgw06.kcn.ne.jp ([61.86.7.213]:37429) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XnKdt-0007jm-0t for 18991-done@debbugs.gnu.org; Sun, 09 Nov 2014 00:02:25 -0500 Received: from imp01 (mailgw5.kcn.ne.jp [61.86.15.231]) by mailgw06.kcn.ne.jp (Postfix) with ESMTP id 3ADE4E8001C for <18991-done@debbugs.gnu.org>; Sun, 9 Nov 2014 14:02:23 +0900 (JST) Received: from mail07.kcn.ne.jp ([61.86.6.186]) by imp01 with bizsmtp id DH2P1p00540oyB901H2PL4; Sun, 09 Nov 2014 14:02:23 +0900 X-OrgRCPT: 18991-done@debbugs.gnu.org Received: from [10.120.1.12] (i118-21-128-66.s30.a048.ap.plala.or.jp [118.21.128.66]) by mail07.kcn.ne.jp (Postfix) with ESMTPA id EFF40D5009B; Sun, 9 Nov 2014 14:02:22 +0900 (JST) Date: Sun, 09 Nov 2014 14:02:21 +0900 From: Norihiro Tanaka To: Jim Meyering Subject: Re: bug#18991: [PATCH] tests: fix encoding with `tr' to support multibyte in test In-Reply-To: References: <20141108170723.F3E7.27F6AC2D@kcn.ne.jp> Message-Id: <20141109140221.2832.27F6AC2D@kcn.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.65.07 [ja] X-Spam-Score: -0.6 (/) X-Debbugs-Envelope-To: 18991-done Cc: 18991-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) On Sat, 8 Nov 2014 19:00:55 -0800 Jim Meyering wrote: > Thank you for the report and patch. > However, it is not maintainable to modify every use of "tr" in > the tests. Instead, I've addressed this by making all of the > tests use tr through a wrapper that always sets LC_ALL=C: Thanks, I also like it. From unknown Sat Aug 09 13:17:09 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 07 Dec 2014 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator