From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 14 03:09:54 2012 Received: (at submit) by debbugs.gnu.org; 14 Sep 2012 07:09:54 +0000 Received: from localhost ([127.0.0.1]:32886 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCQ2D-00083U-3f for submit@debbugs.gnu.org; Fri, 14 Sep 2012 03:09:54 -0400 Received: from eggs.gnu.org ([208.118.235.92]:53707) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCNQN-00049m-BJ for submit@debbugs.gnu.org; Fri, 14 Sep 2012 00:22:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TCNPS-0000fn-2o for submit@debbugs.gnu.org; Fri, 14 Sep 2012 00:21:43 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00,BODY_8BITS, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:45796) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TCNPR-0000fj-RN for submit@debbugs.gnu.org; Fri, 14 Sep 2012 00:21:41 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37078) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TCNPQ-0000fo-Bv for bug-gnu-emacs@gnu.org; Fri, 14 Sep 2012 00:21:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TCNPO-0000by-Hb for bug-gnu-emacs@gnu.org; Fri, 14 Sep 2012 00:21:40 -0400 Received: from [207.195.216.3] (port=3729 helo=luna.soncom.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TCNPO-0000XA-8c for bug-gnu-emacs@gnu.org; Fri, 14 Sep 2012 00:21:38 -0400 Received: from peano.jtcl.org ([204.246.91.63]) by luna.soncom.com (8.11.5/8.11.0) with ESMTP id q8E3mxK38993 for ; Thu, 13 Sep 2012 22:49:00 -0500 (CDT) Received: by peano.jtcl.org (Postfix, from userid 201) id ABB42150C9; Thu, 13 Sep 2012 22:48:56 -0500 (CDT) From: jtowler@soncom.com To: bug-gnu-emacs@gnu.org Subject: 24.2; emacs-24.2 bidi Hebrew switch introduces editing error Date: Thu, 13 Sep 2012 21:37:27 -0500 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: FreeBSD 2.0-4.2 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 208.118.235.17 X-Spam-Score: -6.9 (------) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Fri, 14 Sep 2012 03:09:51 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) bash-4.2$ emacs & using M-x report-emacs-bug, bounced, and later S-D-b in gnus C-x i xmodmap-tuple-bug-bidi-emacs-Q file inserted at point with bug data was previously run with=20 bash-4.2$ emacs -Q & to show the editing error at the end. Thu Sep 13 11:21:29 2012 John R. Towler, Yohanan Towler, jtowler@soncom.com On preparing a sketch for the logical problem of all the various keypress keys, codes, glyphs, and encoding-systems and their codes because the solution I needed for Hebrew basic Israeli typewriter unpointed (simplest first solution) X11 input with utf-8 wasn't obviously done already via Xkb or Xmodmap, and the prev. solutions as per the stds of the time might well work but are for iso-8859-8 or its non-Unix etc., man pages for xmodmap, xsrc, x.org docs for xmodmap, xkb, etc. all get at the problem but make logical encoding and ordering assumptions about what the xsrcs etc. assume in their frames for solution paths in the config files etc.: I ran into the following problem with emacs-24.2 and bidi (Language Environment: Hebrew, input method: hebrew) and Language Environment switching to L2R, Language Environment: English, input method either unspecified as non l10n specified emacs default or specified as ipa-praat which has some utility for my interests in (formal and computational) linguistics, other academ. areas etc., Hebrew language, other L2s, or Ln | n >=3D2 studi= ed or just as linguistic data, etc.: file: xmodmap-tuple on C-x f xmodmap-tuple but here duplicated contents of xmodmap-tuple copied to xmodmap-tuple-bug-bidi-baseline up to last typed line as described to show what emacs does. from bof, all is English, emacs default up until around the following point: (the last bit is the result of several typing and editing attempts which led to this report) for point in buffer indicated below: C-h L RET English language environment Sample text: Hello!, Hi!, How are you? Input methods: english-dvorak ("DV@" in mode line) Character sets: ascii: ASCII (ISO646 IRV) Coding systems: nothing specific to English [back] point is here: at C-j bol before 'collating sequence' collating sequence,=20 for hebrew unpointed, k =3D [=D7=90,=D7=91,=D7=92,=D7=93,=D7=94,=D7=95,=D7= =97,=D7=98,=D7=99,=D7=9B,=D7=9C,=D7=9E,=D7=A0,=D7=A1,=D7=A2,=D7=A6,=D7=A5,= =D7=A7,=D7=A8,=D7=A9,=D7=AA] std emacs-24.x, here x=3D2 setting: Hebrew language environment Bidirectional editing is supported. Sample text: Hebrew =D7=A9=D7=9C=D7=95=D7=9D Input methods (default hebrew) hebrew ("=D7=A2" in mode line) yiddish-keyman ("=D7=B2=D7=A7" in mode line) yiddish-royal ("=D7=B2=D7=A8" in mode line) hebrew-biblical-sil ("=D7=A1=D6=B4=D7=9C" in mode line) hebrew-biblical-tiro ("=D7=AA=D6=B4=D7=A8=D6=B9" in mode line) hebrew-full ("=D7=A2=D6=B4=D6=AB" in mode line) hebrew-lyx ("=D7=9C=D6=B4" in mode line) hebrew-new ("=D7=A2" in mode line) Character sets: iso-8859-8: ISO/IEC 8859/8 Coding systems: hebrew-iso-8bit (`8' in mode line): ISO 2022 based 8-bit encoding for Hebrew (MIME:ISO-8859-8). (alias: hebrew-iso-8bit iso-8859-8 iso-8859-8-e iso-8859-8-i) windows-1255 (`h' in mode line): windows-1255 (Hebrew) encoding (MIME: WINDOWS-1255) (alias: windows-1255 cp1255) cp862 (`D' in mode line): DOS codepage 862 (Hebrew) (alias: cp862 ibm862) [back] C-u C-x =3D on the aleph of the sequence/set typed in the orig. file position: 3037 of 5940 (51%), column: 27 character: =D7=90=E2=80=8E (displayed as =D7=90=E2=80=8E) (code= point 1488, #o2720, #x5d0) preferred charset: iso-8859-8 (ISO/IEC 8859/8) code point in charset: 0xE0 syntax: w which means: word category: .:Base, R:Right-to-left (strong) to input: type "t" with hebrew buffer code: #xD7 #x90 file code: #xD7 #x90 (encoded by coding system utf-8-unix) display: by this font (glyph code) x:-misc-fixed-medium-r-normal--15-140-75-75-c-90-iso8859-8 (#xE0) Character code properties: customize what to show name: HEBREW LETTER ALEF general-category: Lo (Letter, Other) decomposition: (1488) ('=D7=90') switching via C-x RET L Hebrew, C-x RET C-\ hebrew works correctly to type the Heb. alph. with R2L correctly handled, all of this is unshifted keyboard keys, knowing the punct keys etc. were not typed. C-x RET L English, C-x RET C-\ ipa-praat %% prev. experiments have shown that switching back to Language %% Environment: English, input-method unchanged results in L2R %% Hebrew chars for qwerty keys, and I don't know the Dvorak %% keyboard, and Praat is a very useful libre portable acoustic %% phonetics laboratory base, and some version of IPA is most %% useful, so why not ipa-praat lc-keys mapped to punct other than qwerty ascii =3D [\,, \. , \q, \' ] The escaped glyphs in the sequence/set are the result of switching between Language Environment:Hebrew, input-method: hebrew to type the remaining unshifted qwerty keys from z to /, a to ', q to p and back to Language Environment:English, input-method:ipa-praat on a pc10[145] laptop keyboard set to swap ctrl and caps-lock so the broken pc keyboard keys are in the right place. C-x RET L English, C-x RET C-\ ipa-praat punct =3D [\?, \,, \., \[, \], \\, \|, \{, \}, \'\', \', \`, \~, \!, \@, \#= , \$, \%, \^, \&, \*, \(, \), \-, \=3D, \+, \_, \maqqeph, \;, \:, \"] num=3D[1, 2, 3, 4, 5, 6, 7, 8, 9, 0] access-feature-val-pr(keybd-array(key), print-name(glyph)) qwerty unshifted etc. bottom row of keys to m with switch to Language Envir= onment:Hebrew, input-mode:hebrew for the corresponding hebrew glyph [(z, =D7=96), (x, =D7=A1), (c, =D7=91), (v, =D7=94), (b, =D7=A0), (n, =D7= =9E), (m, =D7=A6),=20 next few keys across the bottom of the keyboard switching to Hebrew:hebrew for the key press of each key until the error appears, using C-k on the row above to save typing and then typing after the mem rparen: , (, , C-x RET L Hebrew C-x RET C-\ hebrew keypress qwerty , for the expected Hebrew glyph [(z, =D7=96), (x, =D7=A1), (c, =D7=91), (v, =D7=94), (b, =D7=A0), (n, =D7= =9E), (m, =D7=A6), (, , =D7=AA EOF ------ just in case: bash-4.2$ uname -a NetBSD peano.jtcl.org 6.0_BETA2 NetBSD 6.0_BETA2 (Peano3 -r netbsd-6 -D 8/1= 0/2012 -current auich* bind* ) #0: Thu Aug 23 11:56:58 UTC 2012 root@peano= .jtcl.org:/usr/obj/sys/arch/i386/compile/Peano3 i386 --- In GNU Emacs 24.2.1 (i386-unknown-netbsdelf6.0., X toolkit, Xaw3d scroll ba= rs) of 2012-08-28 on peano.jtcl.org Windowing system distributor `The Xorg Foundation', version 11.0.11003000 Configured using: `configure '--x-includes=3D/usr/X11R7/include' '--x-libraries=3D/usr/X11R7/lib' '--enable-asserts' '--enable-checking' '--enable-profiling' '--enable-dependency-tracking' '--without-mail-unlink' '--with-sound' '--with-x-toolkit=3Dlucid' '--with-wide-int' '--without-xpm' '--without-jpeg' '--without-tiff' '--without-gif' '--without-png' '--without-rsvg' '--without-imagemagick' '--without-gpm' '--without-dbus' '--without-gconf' '--without-gsettings' '--without-selinux' '--with-x' '--without-xft' 'LDFLAGS=3D-L/usr/X11R7/lib -L/usr/local/lib' 'CPPFLAGS=3D-I/usr/X11R7/include -I/usr/local/include'' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: nil value of $XMODIFIERS: nil locale-coding-system: nil default enable-multibyte-characters: t Major mode: Info Minor modes in effect: display-time-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent input: q C-h i m e m a c s m e m a c s C-g=20 m B u g s SPC SPC SPC SPC SPC SPC SPC SPC=20 ` x C-g =20 x r e p o r t - e m a c s - b u g Recent messages: Loading /usr/local/home/jtowler/elisp/haskell-mode-2.8.0/haskell-site-file.= el (source)...done Loading ispell...done Loading timeclock...done Preparing diary...done For information about GNU Emacs and the GNU system, type C-h C-a. Composing main Info directory...done Making completion list... Quit Quit Load-path shadows: /usr/local/home/jtowler/elisp/ispell hides /usr/local/share/emacs/24.2/lisp= /textmodes/ispell /usr/local/share/emacs/site-lisp/esh-mode hides /usr/local/share/emacs/24.2= /lisp/eshell/esh-mode Features: (shadow sort gnus-util mail-extr warnings emacsbug message format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils help-mode jka-compr info view cal-china lunar cal-bahai holidays hol-loaddefs cal-julian solar cal-dst diary-lib cal-hebrew diary-loaddefs mule-util cal-move cal-menu calendar cal-loaddefs time timeclock ispell preview-latex tex-site auto-loads bbdb-autoloads bbdb timezone printing ps-print ps-def lpr slime byte-opt bytecomp byte-compile cconv macroexp derived help-fns edmacro kmacro easy-mmode easymenu pp comint regexp-opt ansi-color ring hyperspec thingatpt browse-url cl time-stamp time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces cus-face files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote make-network-process dynamic-setting x-toolkit x multi-tty emacs) From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 14 04:59:45 2012 Received: (at 12438) by debbugs.gnu.org; 14 Sep 2012 08:59:45 +0000 Received: from localhost ([127.0.0.1]:33004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCRkW-00029T-8E for submit@debbugs.gnu.org; Fri, 14 Sep 2012 04:59:44 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:50255) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCRkT-00029K-HB for 12438@debbugs.gnu.org; Fri, 14 Sep 2012 04:59:43 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0MAC009000KG4E00@a-mtaout23.012.net.il> for 12438@debbugs.gnu.org; Fri, 14 Sep 2012 11:58:43 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MAC008WX0XVZO40@a-mtaout23.012.net.il>; Fri, 14 Sep 2012 11:58:43 +0300 (IDT) Date: Fri, 14 Sep 2012 11:58:58 +0300 From: Eli Zaretskii Subject: Re: bug#12438: 24.2; emacs-24.2 bidi Hebrew switch introduces editing error In-reply-to: X-012-Sender: halo1@inter.net.il To: jtowler@soncom.com Message-id: <83ipbhugj1.fsf@gnu.org> References: X-Spam-Score: -1.2 (-) X-Debbugs-Envelope-To: 12438 Cc: 12438@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.2 (-) > From: jtowler@soncom.com > Date: Thu, 13 Sep 2012 21:37:27 -0500 > > bash-4.2$ emacs & > using M-x report-emacs-bug, bounced, and later S-D-b in gnus > > C-x i xmodmap-tuple-bug-bidi-emacs-Q > > file inserted at point with bug data was previously run with > bash-4.2$ emacs -Q & > > to show the editing error at the end. Thank you for your report. However, I cannot figure out what exactly is the problem you are complaining about. What "editing error" did you find, and how to reproduce it starting with "emacs -Q"? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 14 05:41:49 2012 Received: (at 12438) by debbugs.gnu.org; 14 Sep 2012 09:41:49 +0000 Received: from localhost ([127.0.0.1]:33074 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCSPE-0003A4-Ta for submit@debbugs.gnu.org; Fri, 14 Sep 2012 05:41:49 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:46005) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TCSPC-00039v-CK for 12438@debbugs.gnu.org; Fri, 14 Sep 2012 05:41:47 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MAC00A002SD2I00@a-mtaout22.012.net.il> for 12438@debbugs.gnu.org; Fri, 14 Sep 2012 12:40:26 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MAC009FE2VDBWF0@a-mtaout22.012.net.il>; Fri, 14 Sep 2012 12:40:26 +0300 (IDT) Date: Fri, 14 Sep 2012 12:40:41 +0300 From: Eli Zaretskii Subject: Re: bug#12438: 24.2; emacs-24.2 bidi Hebrew switch introduces editing error In-reply-to: <83ipbhugj1.fsf@gnu.org> To: jtowler@soncom.com Message-id: <83ehm5ueli.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: QUOTED-PRINTABLE X-012-Sender: halo1@inter.net.il References: <83ipbhugj1.fsf@gnu.org> X-Spam-Score: -1.2 (-) X-Debbugs-Envelope-To: 12438 Cc: 12438@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.2 (-) > Date: Fri, 14 Sep 2012 11:58:58 +0300 > From: Eli Zaretskii > Cc: 12438@debbugs.gnu.org >=20 > However, I cannot figure out what exactly is the problem you are > complaining about. What "editing error" did you find, and how to > reproduce it starting with "emacs -Q"? If you mean the seemingly strange arrangement of characters (marked with "^^^^") at the end of this line: [(z, =D7=96), (x, =D7=A1), (c, =D7=91), (v, =D7=94), (b, =D7=A0), (= n, =D7=9E), (m, =D7=A6), (, , =D7=AA ^^^^^^^^^^ then this is not a bug. This is how the Unicode standard prescribes to display the sequence of characters at the end of that line. Because the characters between =D7=A6 and =D7=AA are all "neutral", i= .e. have no strong directionality attribute, these two Hebrew characters, which are strong R2L characters, cause this entire run of characters betwee= n them be rendered as R2L text, which includes mirroring of '(' as ')'. In the other tuples on the above line, Latin characters, which are strong L2R characters, after the '(' that begins the next tuple prevent the text to be rendered as R2L. But in the last tuple, there's a space instead of a Latin character, and space has no strong directionality. If you want the end of that line be rendered as you expect, use the LRM character, like this: [(z, =D7=96), (x, =D7=A1), (c, =D7=91), (v, =D7=94), (b, =D7=A0), (= n, =D7=9E), (m, =D7=A6), (=E2=80=8E,, =D7=AA) ^ Here I inserted the LRM character (u+200E) before the comma, as marke= d by "^". LRM is a strong L2R character whose purpose is precisely to "fix" display of mixed L2R/R2L text in such cases. This character is displayed by Emacs as a thin space, which makes it almost invisible o= n display. (See "C-h v glyphless-char-display" for other options of displaying such characters.) From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 04 04:52:39 2013 Received: (at control) by debbugs.gnu.org; 4 Jan 2013 09:52:39 +0000 Received: from localhost ([127.0.0.1]:42647 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tr3x8-0000IX-A3 for submit@debbugs.gnu.org; Fri, 04 Jan 2013 04:52:38 -0500 Received: from fencepost.gnu.org ([208.118.235.10]:50119) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tr3x5-0000IP-LN for control@debbugs.gnu.org; Fri, 04 Jan 2013 04:52:36 -0500 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1Tr3wz-0006eg-Du for control@debbugs.gnu.org; Fri, 04 Jan 2013 04:52:29 -0500 Date: Fri, 04 Jan 2013 04:52:29 -0500 Message-Id: Subject: control message for bug 12438 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -4.2 (----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.2 (----) tag 12438 notabug close 12438 From unknown Mon Aug 18 11:27:49 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 01 Feb 2013 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator