GNU bug report logs - #66760
29.1; [BUG] GB18030 Incorrect Encoding

Previous Next

Full log

Message #11 received at 66760 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: "Ruijie Yu" <yuruijie <at> sics.ac.cn>
Cc: 66760 <at> debbugs.gnu.org
Subject: Re: bug#66760: 29.1; [BUG] GB18030 Incorrect Encoding
Date: Thu, 26 Oct 2023 16:20:59 +0200

On Okt 26 2023, Ruijie Yu wrote:

> I have noticed that in GB18030 encoding, certain ranges of characters
> have incorrect encodings.
>
> One example is U+217A (SMALL ROMAN NUMERAL ELEVEN).  The expected
> encoding is 81 36 C5 30 (as can be seen from the GB18030 standard [1]
> and verified from other programs such as iconv and MySQL), whereas the
> observed encoding within Emacs is 81 36 C4 39, with a 1-codepoint
> offset.

This is a bug in the generation of GB180304.map.  The gb180303.awk
script assumes that the 4-byte encodings of GB18030 are filling the
holes in sequence of characters with a 2-byte encoding by Unicode
codepoint order, but there are some places where codepoints from the PUA
area are inserted into the sequence.  For example, U+1E3E maps to 81 35
F4 36, the next codepoint not mapped to a 2-byte code is U+1E40, but
that maps to 81 35 F4 38, whereas 81 35 F4 37 is the encoding of U+E7C7.
So the output gets out of sync.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.