From politza@fh-trier.de Fri Sep 19 12:37:24 2008 X-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02 (2007-08-08) on rzlab.ucr.edu X-Spam-Level: X-Spam-Status: No, score=-4.9 required=4.0 tests=BAYES_00,FOURLA, RCVD_IN_DNSWL_LOW autolearn=ham version=3.2.3-bugs.debian.org_2005_01_02 Received: (at submit) by emacsbugs.donarmstrong.com; 19 Sep 2008 19:37:24 +0000 Received: from lists.gnu.org (lists.gnu.org [199.232.76.165]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m8JJbJmk006662 for ; Fri, 19 Sep 2008 12:37:21 -0700 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Kgln5-0007ju-3N for bug-gnu-emacs@gnu.org; Fri, 19 Sep 2008 15:37:19 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Kgln3-0007hy-2U for bug-gnu-emacs@gnu.org; Fri, 19 Sep 2008 15:37:17 -0400 Received: from [199.232.76.173] (port=34464 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Kgln2-0007hr-Jx for bug-gnu-emacs@gnu.org; Fri, 19 Sep 2008 15:37:16 -0400 Received: from gateway-a.fh-trier.de ([143.93.54.181]:51008) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1Kgln2-0005ho-2t for bug-gnu-emacs@gnu.org; Fri, 19 Sep 2008 15:37:16 -0400 Received: from [84.59.201.36] (dslb-084-059-201-036.pools.arcor-ip.net [84.59.201.36]) (Authenticated sender: politza) by gateway-a.fh-trier.de (Postfix) with ESMTP id 75C7912C42B7 for ; Fri, 19 Sep 2008 21:37:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=fh-trier.de; s=default; t=1221853028; bh=Ejq2g2yzLZX9rFj1fgO4gwlDw4E=; h=Message-ID:Date: From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=AsxxzPRqjERiKZjSxTnRYe8nPqerVgRPqr/Y+ 8R/xGz7n0Lqv9ctofDE4FnC/eXd1kOsNmvdjpeD7cN6CUV9xUvIn+cs+1coTBv/zuXm OtZHMnOqwx79UhLYq0NNV6oyPBKRnDkMbe9No2/B37lIjJ0e6bNgY8nk12dy955bM9k = Message-ID: <48D3FEDB.30009@fh-trier.de> Date: Fri, 19 Sep 2008 21:34:51 +0200 From: Andreas Politz User-Agent: Mozilla-Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: bug-gnu-emacs@gnu.org Subject: garbled unicode characters in M-x term Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by Amavisd-new + McAfee uvscan + ClamAV [Rechenzentrum FH-Trier (r/ft)] X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 3) Please write in English if possible, because the Emacs maintainers usually do not have translators to read other languages for them. Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list, and to the gnu.emacs.bug news group. Please describe exactly what actions triggered the bug and the precise symptoms of the bug: Problem : Under certain circumstances multibyte characters in M-x term become garbled and display as single byte escape sequences. Example : debians aptitude (character U+2592) From a post I made to gnu.emacs.help: Ok, I think I found the problem. term uses `binary' as input coding. After it has examined the input, it inserts the relevant/visible parts of it into the buffer. Only at this point it decodes the bytes with the apropriate coding (variable:locale-coding-system). At some point it splits the input string, to make it suitable for the with of the `terminal'. The problem is, that it measures bytes not characters. So the 3-byte character in question in aptitude, which is mostly on the last column, gets split in 2 strings a 1 and 2 byte. This 2 strings, when encoded and inserted independently, will result in what was described as the problem. Solution would be to encode the string before checking the length of it. -ap If Emacs crashed, and you have the Emacs process in the gdb debugger, please include the output from the following gdb commands: `bt full' and `xbacktrace'. If you would like to further debug the crash, please read the file /usr/share/emacs/22.2/etc/DEBUG for instructions. In GNU Emacs 22.2.1 (i486-pc-linux-gnu, GTK+ Version 2.12.11) of 2008-07-25 on raven, modified by Debian Windowing system distributor `The X.Org Foundation', version 11.0.10402000 configured using `configure '--build=i486-linux-gnu' '--host=i486-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var/lib' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs22:/etc/emacs:/usr/local/share/emacs/22.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/22.2/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/22.2/leim' '--with-x=yes' '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars' 'build_alias=i486-linux-gnu' 'host_alias=i486-linux-gnu' 'CFLAGS=-DDEBIAN -g -O2' 'LDFLAGS=-g' 'CPPFLAGS='' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: en_US.UTF-8 locale-coding-system: utf-8 default-enable-multibyte-characters: t Major mode: Fundamental Minor modes in effect: shell-dirtrack-mode: t auto-fill-function: do-auto-fill show-paren-mode: t savehist-mode: t icomplete-mode: t global-hi-lock-mode: t hi-lock-mode: t display-time-mode: t tooltip-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t unify-8859-on-encoding-mode: t utf-translate-cjk-mode: t auto-compression-mode: t column-number-mode: t line-number-mode: t Recent input: C-x C-s M-x d i f f SPC u DEL C-g C-x o M-? m C-M-v C-x k RET C-x C-g M-x d i f f RET RET t e r m . RET C-x o C-v C-v C-v C-v C-v M-< M-x w o m a n RET d i f f RET C-v C-v C-v M-v C-r i g n o r e C-r C-g C-x b t e r C-s C-s C-g C-x o M-x C-g C-u M-x d i f f RET RET t e r C-s RET w C-x o C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-x o C-x o M-< C-x k RET C-x o C-u M-x d i f f RET RET t e r C-s RET DEL w C-x C-g C-u C-g M-x d i f f RET RET t e r m . RET C-x o C-v C-v C-v C-v C-v M-v M-v M-v M-v M-v C-x o C-x C-w ~ / . e m / t e r m . e l C-x b f o RET C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n M-x r e p o SPC r t RET g r a a r b e l e d DEL DEL DEL DEL l e d C-g Recent messages: Repeating command 1 other-window Quit Repeating command 1 other-window [2 times] Saving file /home/andy/.emacs.d/term.el... Wrote /home/andy/.emacs.d/term.el Making completion list... Loading emacsbug...done Quit From cyd@stupidchicken.com Wed Sep 24 13:30:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02 (2007-08-08) on rzlab.ucr.edu X-Spam-Level: X-Spam-Status: No, score=-3.8 required=4.0 tests=AWL,BAYES_00, MURPHY_DRUGS_REL8 autolearn=ham version=3.2.3-bugs.debian.org_2005_01_02 Received: (at 1006) by emacsbugs.donarmstrong.com; 24 Sep 2008 20:30:00 +0000 Received: from cyd.mit.edu (CYD.MIT.EDU [18.115.2.24]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m8OKTw8c025921 for <1006@emacsbugs.donarmstrong.com>; Wed, 24 Sep 2008 13:29:59 -0700 Received: by cyd.mit.edu (Postfix, from userid 1000) id B177257E189; Wed, 24 Sep 2008 16:30:25 -0400 (EDT) From: Chong Yidong To: Andreas Politz Cc: 1006@debbugs.gnu.org Subject: Re: garbled unicode characters in M-x term Date: Wed, 24 Sep 2008 16:30:25 -0400 Message-ID: <87iqslthge.fsf@cyd.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii > Ok, I think I found the problem. term uses `binary' as input coding. > After it has examined the input, it inserts the relevant/visible parts > of it into the buffer. Only at this point it decodes the bytes with > the apropriate coding (variable:locale-coding-system). At some point > it splits the input string, to make it suitable for the with of the > `terminal'. The problem is, that it measures bytes not characters. So > the 3-byte character in question in aptitude, which is mostly on the > last column, gets split in 2 strings a 1 and 2 byte. This 2 strings, > when encoded and inserted independently, will result in what was > described as the problem. Thanks for the analysis. Could you try to write a patch to fix this? From cyd@stupidchicken.com Wed Sep 24 17:07:20 2008 X-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02 (2007-08-08) on rzlab.ucr.edu X-Spam-Level: X-Spam-Status: No, score=-3.8 required=4.0 tests=AWL,BAYES_00, MURPHY_DRUGS_REL8 autolearn=ham version=3.2.3-bugs.debian.org_2005_01_02 Received: (at 1006-done) by emacsbugs.donarmstrong.com; 25 Sep 2008 00:07:20 +0000 Received: from cyd.mit.edu (CYD.MIT.EDU [18.115.2.24]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m8P07HCV016879 for <1006-done@emacsbugs.donarmstrong.com>; Wed, 24 Sep 2008 17:07:18 -0700 Received: by cyd.mit.edu (Postfix, from userid 1000) id 0EA9757E189; Wed, 24 Sep 2008 20:07:46 -0400 (EDT) From: Chong Yidong To: Andreas Politz Cc: 1006-done@debbugs.gnu.org Subject: Re: garbled unicode characters in M-x term References: <87iqslthge.fsf@cyd.mit.edu> <48DAA59B.9040906@fh-trier.de> <87abdx5jbc.fsf@cyd.mit.edu> <48DAB985.9030600@fh-trier.de> Date: Wed, 24 Sep 2008 20:07:46 -0400 In-Reply-To: <48DAB985.9030600@fh-trier.de> (Andreas Politz's message of "Thu, 25 Sep 2008 00:04:53 +0200") Message-ID: <87myhxhyul.fsf@cyd.mit.edu> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii >>>> Thanks for the analysis. Could you try to write a patch to fix >>>> this? >>>> >>> I did. It's a followup in the thread on emacs.bug . >> >> Hmm, I don't see your message. Could you please mail it directly to >> me? > > Shure, here it comes : The patch looks good. I've installed it into the Emacs CVS trunk, with a few minor cosmetic changes. Thanks very much for debugging and fixing this. From unknown Fri Aug 15 18:53:54 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: $requester Subject: Internal Control Message-Id: bug archived. Date: Thu, 23 Oct 2008 14:24:03 +0000 User-Agent: Fakemail v42.6.9 # A New Hope # A log time ago, in a galaxy far, far away # something happened. # # Magically this resulted in the following # action being taken, but this fake control # message doesn't tell you why it happened # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator