From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 15:21:20 2018 Received: (at submit) by debbugs.gnu.org; 13 Dec 2018 20:21:20 +0000 Received: from localhost ([127.0.0.1]:46787 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXU0-0000Rj-0j for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:21:20 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44958) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXTw-0000RU-Fs for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:21:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXXTp-0002a2-S1 for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:21:11 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:42732) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXXTn-0002XI-4h for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:21:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXXTl-000535-M7 for bug-gnu-emacs@gnu.org; Thu, 13 Dec 2018 15:21:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXXTk-0002Tu-G9 for bug-gnu-emacs@gnu.org; Thu, 13 Dec 2018 15:21:05 -0500 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]:39835) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gXXTk-0002Rs-7Y for bug-gnu-emacs@gnu.org; Thu, 13 Dec 2018 15:21:04 -0500 Received: by mail-lf1-x12e.google.com with SMTP id n18so2555582lfh.6 for ; Thu, 13 Dec 2018 12:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=oDp1LZwmbyT6V1HUfXWhOarj9kkCDbckMwjzncSKgMc=; b=jO9iEzvoxo2blLCspWK+OEfFm2uUrEogq+07fNNvvp77xFliwGjp/NHRpgIE5iHLfR ZRczRwY9iWEJjdwkZoFVdjCuKwcjI+v6uSPxg3NXOX2mA+C4sRoPun3qta2f8/ABOZhR 0jISp9DVCA7ohX0xPJ9Cq1we5dII/vPTDlcdvRcc0iVs3DMLXsrktdpzDmPSQLKPdG3m /lBLxPJoF5kP7ESQiMyjFQtJkSububf9M6iNFfr7sdZCjmXXgYxUC9dIVpsNM24n5WML jlPHyBkViYNCrtT1izkyOMDqqLTpKen3c5AZH5ozATCtNXYyp3/YKONApNG9DYSIpS2G 7EQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=oDp1LZwmbyT6V1HUfXWhOarj9kkCDbckMwjzncSKgMc=; b=hXuAIilt3y3bI1gbcu9pdOv9DXIp5E2sdj22dE76R9QrT7R/wJw5uDfiIHO3U2i3Gx z70fhuhTfbjQz+sJ7lTzsYqo+9xmrLwbDr20HEQ05TkIXE6qM5SmfUo5GvbaTR1FeTRi CfGyyc1r107ityYzwpMrxpRUWFU9ICNOtJQNdv++EA8qXkqBNeqAkB1wb2qMx3Z4NISK lbUqEW/ngvSTnNNYYweCdlynJMICruRG+JAeNCqTPl3WWOUlRr0w+0FpSpccNlUnaZXM 9sZV7aT0xEagNL5nAUtctXq2CKwbve6pb72UA3wOgKWzbJZ5N9Hbltgji9Far/6G+oDK jD9A== X-Gm-Message-State: AA+aEWblc8Y0jvrKB927W07oT4vJBrEM3w9t33SYDbmxmt10h9O4Wep3 4rC243WGUYirUZHxvUKMp+VhhmQHrmLqR+W0rOg4VBGoXrE= X-Google-Smtp-Source: AFSGD/X2yJE+xDKIsECUnqwj2YpO6LAqSPMSPDLk/w0GR5QSsCrfMYrs4Wo9+bLzpu10XE+DybzUWN1DgdqJhEnuxAU= X-Received: by 2002:a19:a86:: with SMTP id 128mr97784lfk.139.1544732460792; Thu, 13 Dec 2018 12:21:00 -0800 (PST) MIME-Version: 1.0 From: Kaushal Modi Date: Thu, 13 Dec 2018 15:20:23 -0500 Message-ID: Subject: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: bug-gnu-emacs@gnu.org Content-Type: multipart/alternative; boundary="000000000000994640057ced0d2e" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) --000000000000994640057ced0d2e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello, I built emacs from harfbuzz branch with harfbuzz 1.0.3 installed (RHEL 6.8)= . I quickly compared Hindi and Gujarati rendering difference between emacs built with m17n vs the new harfbuzz branch build. With harfbuzz, it does not render the partial glyphs for Gujarati, but does it fine for Hindi. But on the build with m17n, both Hindi and Gujarati show that partial glyph rendered fine. Screenshot to explain this issue: https://i.imgtc.com/md9Yz7X.png In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 2.24.23) of 2018-12-13 Repository revision: 981b3d292aff49452c2b5f0217b57ec1a2829a8b Repository branch: harfbuzz Windowing system distributor 'The X.Org Foundation', version 11.0.60900000 System Description: Red Hat Enterprise Linux Workstation release 6.8 (Santiago) Recent messages: Emacs version: GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 2.24.23) of 2018-12-13, built using commit 981b3d292aff49452c2b5f0217b57ec1a2829a8b= . ./configure options: --with-modules --prefix=3D/home/kmodi/usr_local/apps/6/emacs/harfbuzz '--program-transform-name=3Ds/^ctags$/ctags_emacs/' --with-harfbuzz 'CPPFLAGS=3D-I/home/kmodi/stowed/include -I/home/kmodi/usr_local/6/include -I/usr/include/freetype2 -I/usr/include' 'CFLAGS=3D-O2 -march=3Dnative' 'LDFLAGS=3D-L/home/kmodi/stowed/lib -L/home/kmodi/stowed/lib64 -L/home/kmodi/usr_local/6/lib -L/home/kmodi/usr_local/6/lib64' PKG_CONFIG_PATH=3D/home/kmodi/usr_local/6/lib/pkgconfig:/home/kmodi/usr_loc= al/6/lib64/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib/pkgconfig:/cad/ad= i/apps/gnu/linux/x86_64/6/lib64/pkgconfig:/home/kmodi/stowed/lib/pkgconfig:= /usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/lib/pkgconfig= :/lib64/pkgconfig Features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK2 X11 XDBE XIM MODULES THREADS GMP Important settings: value of $LANG: en_US.UTF-8 value of $XMODIFIERS: @im=3Dnone locale-coding-system: utf-8-unix -- Kaushal Modi --000000000000994640057ced0d2e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello,

I built emacs from harfbuzz branch wit= h harfbuzz 1.0.3 installed (RHEL 6.8).

I quickly c= ompared Hindi and Gujarati rendering difference between emacs built with m1= 7n vs the new harfbuzz branch build.

With harfbuzz, it does not render the partial glyphs for Gujarati, but does=20 it fine for Hindi. But on the build with m17n, both Hindi and Gujarati=20 show that partial glyph rendered fine.

Screenshot = to explain this issue: https://= i.imgtc.com/md9Yz7X.png



In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Versio= n 2.24.23)
=C2=A0of 2018-12-13
Repository revision: 981b3d292aff49452= c2b5f0217b57ec1a2829a8b
Repository branch: harfbuzz
Windowing system = distributor 'The X.Org Foundation', version 11.0.60900000
System= Description: Red Hat Enterprise Linux Workstation release 6.8 (Santiago)
Recent messages:
Emacs version: GNU Emacs 27.0.50 (build 2, x86_64= -pc-linux-gnu, GTK+ Version 2.24.23)
=C2=A0of 2018-12-13, built using co= mmit 981b3d292aff49452c2b5f0217b57ec1a2829a8b.

./configure options:<= br>=C2=A0 --with-modules --prefix=3D/home/kmodi/usr_local/apps/6/emacs/harf= buzz '--program-transform-name=3Ds/^ctags$/ctags_emacs/' --with-har= fbuzz 'CPPFLAGS=3D-I/home/kmodi/stowed/include -I/home/kmodi/usr_local/= 6/include -I/usr/include/freetype2 -I/usr/include' 'CFLAGS=3D-O2 -m= arch=3Dnative' 'LDFLAGS=3D-L/home/kmodi/stowed/lib -L/home/kmodi/st= owed/lib64 -L/home/kmodi/usr_local/6/lib -L/home/kmodi/usr_local/6/lib64= 9; PKG_CONFIG_PATH=3D/home/kmodi/usr_local/6/lib/pkgconfig:/home/kmodi/usr_= local/6/lib64/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib/pkgconfig:/cad= /adi/apps/gnu/linux/x86_64/6/lib64/pkgconfig:/home/kmodi/stowed/lib/pkgconf= ig:/usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/lib/pkgcon= fig:/lib64/pkgconfig

Features:
=C2=A0 XPM JPEG TIFF GIF PNG RSVG = IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNU= TLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS = GTK2 X11 XDBE XIM MODULES THREADS GMP

Important settings:
=C2=A0 = value of $LANG: en_US.UTF-8
=C2=A0 value of $XMODIFIERS: @im=3Dnone
= =C2=A0 locale-coding-system: utf-8-unix


--
Kaushal Modi<= /div>
--000000000000994640057ced0d2e-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 15:26:03 2018 Received: (at 33729) by debbugs.gnu.org; 13 Dec 2018 20:26:03 +0000 Received: from localhost ([127.0.0.1]:46816 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXYY-0000bV-Ok for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:26:03 -0500 Received: from mail-lf1-f43.google.com ([209.85.167.43]:40242) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXYW-0000au-BX for 33729@debbugs.gnu.org; Thu, 13 Dec 2018 15:26:01 -0500 Received: by mail-lf1-f43.google.com with SMTP id v5so2569234lfe.7 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 12:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=YFDrQBZFIsJn6DKuyqNnw/+ye/KcvCgvo7oZLX6JQYs=; b=gbvH05I+22OlGPq36tWVzMsVF4nd8wnuGMM8ipizBawqZoz8kE4mhAAutLYfGfgpIU 5UlJGK6HaSF1bcoq10wibxF4VF5VtLn7RIfdUNps80oGkP6nEhYuyVIu7oNPR0s/BnFE gRoU0qbGUG83rejsu1Pe89F+v6KjUgXL674O9rf4I2LeGEYDtZfwVqgrrgcoRZZRrPto VbtO2Mrc9cjAhhS/lVvaczWY7DxaQij/wQ1l9iT3U9L+76rT14ADwiT7Ebt5BIYg4PCB 3KsAkpKayV2CLgPMwTB2CLU4kvox0LzUuVW0CW34bd+2k6IQleOQglnCLjWQ5s6lmVau N7IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YFDrQBZFIsJn6DKuyqNnw/+ye/KcvCgvo7oZLX6JQYs=; b=NOhF6gJxXG4yINBZ+UFuQQ2yBGMqIuYibJJ5ifDTbYiahtDdytAfw3ARtL97K/Bcwf gIhhqGzuZdnsidikg7pgx039P1Ca0/DPSsBankhktteXJuWwcwByaj5v63ENgZUgwTKh 4UyE47D5GfdAq6MS4NF8uBJcoVzZjU9pXtmMXS2xqunaZpqCfi8UZyPjXY6oCnf2bDwM 7M4boxTawvmznLQ+0lZbQ9ThM/DTZiJ5HG1OTMl40mGZJ7jlnFrOvPvUAnLlYNYgnAHO jvJGojj4bGGjPcknLlGk3ntll+0m71DuzsZGJLsPymAhRLN7VrPMsWByR99gFeBYdu24 khPw== X-Gm-Message-State: AA+aEWa4llvU8o5vyNo3uDwBZHI9omX/doy5McUtLgOooPhNg2Mbovjv i3aDC66y/KJ51U+jjNNFYS70wRJfq4ntYMxEPixdQThH X-Google-Smtp-Source: AFSGD/VIFDHk7GYZbtv63RFHJ4GdnFAks3xz7yraBr7PldM6c186Bm4Jd6LL48gbwRiOszdtIONfc2xOZQlw1NTRMss= X-Received: by 2002:a19:2395:: with SMTP id j143mr109508lfj.107.1544732753596; Thu, 13 Dec 2018 12:25:53 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Kaushal Modi Date: Thu, 13 Dec 2018 15:25:16 -0500 Message-ID: Subject: Re: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: 33729@debbugs.gnu.org Content-Type: multipart/alternative; boundary="0000000000000d170f057ced1f94" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --0000000000000d170f057ced1f94 Content-Type: text/plain; charset="UTF-8" > > Screenshot to explain this issue: https://i.imgtc.com/md9Yz7X.png > I don't know Arabic. But from that same screenshot, it's evident that the rendering of that same text is quite different between m17n and harfbuzz. --0000000000000d170f057ced1f94 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Screenshot to exp= lain this issue: https://i.imgtc.com/md9Yz7X.png
=
I don't know Arabic. But from that same screenshot, it&#= 39;s evident that the rendering of that same text is quite different betwee= n m17n and harfbuzz.

--0000000000000d170f057ced1f94-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 15:34:01 2018 Received: (at 33729) by debbugs.gnu.org; 13 Dec 2018 20:34:01 +0000 Received: from localhost ([127.0.0.1]:46867 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXgG-0000vZ-QA for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:34:01 -0500 Received: from mail-wm1-f50.google.com ([209.85.128.50]:35118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXdY-0000oo-5g for 33729@debbugs.gnu.org; Thu, 13 Dec 2018 15:31:12 -0500 Received: by mail-wm1-f50.google.com with SMTP id c126so3734219wmh.0 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 12:31:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gzqAOn5n3m6E+bfjb5C6g5rwvxzYnUzUUFhhiST//10=; b=kkFlKQ9TyxoV+iuvizMdkPWWeQZNyQOCaIyvsA/cl94zl0yQg1fZFW85Mmw1DEj0Ra RRRNJjrL3vYZyBcDAvdywxeh/ZL0uwkmar37EN2rX4qPXgZytgJt641VERxFuketZo2w 0+KhQKdf4U6N8JxzyifgfXbxsJrZKsjueOLDtk150hDecfQOY/nmKraKDrxXwe738G0C ogiLxZ62ZAMNpW/d1xhnQW6OX7/bysnFRgDRbl7sX30Kh9By/Tqhqt1fCtQPh99CDCdO llnjxw7OeiKuOrJE+8bbh0FekD1vyzKELS4lzj1Co7ClC5FddvoH8PZ/0MRbSGVFvuAn tdYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=gzqAOn5n3m6E+bfjb5C6g5rwvxzYnUzUUFhhiST//10=; b=kC1IUGl1btQ1wWczUx//N6WrqegexE1ray5VtEFUkragW6nzMWDCUz3vfRBwA5dJmf XoDOMHsAcpHh7GWRlbYteEyG+XodBMrOQ9LrTOkw88nT1uiYRsyfbTHxylAc/mM0ZfKd G7V86srWlExwlFAS69n2qez7vNYOxU+kXo58RkQeY0zk9r2rR1fwbpj2fIcRCh3oY0dU r516oC5vzkwCtlEPYyKfwAnla5+jmTLyvAPlqcEIJ4b5BFpsGJkckQ1TO2lMr9WdIkr4 fga10Iuw3uda2WOy5wbXZZvlUnI5LkCxY968tHlDHnMhJGd/hvdvL6JP8gTYAYegATAU RjSA== X-Gm-Message-State: AA+aEWb4Jf2D3rTHPWJT42fIt9pnK8WbiDGBK8CZrxPWSaNoTjHro+2E a0UOcp1PlXq0wrGFAhvWwwk= X-Google-Smtp-Source: AFSGD/Wf33wZ3dc1SNUo4/rRI0mdEQGKROQJ+p8Nx0P9iHyQWzd5Q0Wa7JjEJEoDwpy1nDLWRA2siA== X-Received: by 2002:a1c:aad2:: with SMTP id t201mr798795wme.148.1544733066299; Thu, 13 Dec 2018 12:31:06 -0800 (PST) Received: from macbook.localdomain ([197.58.120.254]) by smtp.gmail.com with ESMTPSA id u10sm2441415wrr.33.2018.12.13.12.31.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 13 Dec 2018 12:31:05 -0800 (PST) Date: Thu, 13 Dec 2018 22:31:02 +0200 From: Khaled Hosny To: Kaushal Modi Subject: Re: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181213203102.GF2244@macbook.localdomain> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 X-Mailman-Approved-At: Thu, 13 Dec 2018 15:33:59 -0500 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Thu, Dec 13, 2018 at 03:25:16PM -0500, Kaushal Modi wrote: > > > > Screenshot to explain this issue: https://i.imgtc.com/md9Yz7X.png > > > > I don't know Arabic. But from that same screenshot, it's evident that the > rendering of that same text is quite different between m17n and harfbuzz. The HarfBuzz rendering of Arabic is the correct one in this screenshot. For debugging the such rendering differences, the actual font used by Emacs for a given part of the text need to be known, then the text and the font can be checked against vanilla HarfBuzz (e.g. using the hb-view command line tool); if it gives the same rendering then it is either a HarfBuzz or font issue, if not then it is a bug in the HarfBuzz integration code in Emacs. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 15:44:37 2018 Received: (at 33729) by debbugs.gnu.org; 13 Dec 2018 20:44:37 +0000 Received: from localhost ([127.0.0.1]:46910 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXqW-0001Jw-Fn for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:44:37 -0500 Received: from mail-lj1-f179.google.com ([209.85.208.179]:44653) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXqU-0001Jj-99 for 33729@debbugs.gnu.org; Thu, 13 Dec 2018 15:44:35 -0500 Received: by mail-lj1-f179.google.com with SMTP id k19-v6so2960112lji.11 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 12:44:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=N6hXOEaMk68qB7NM/pjswjlDhIvzZ4KiUpxCPmJLBBU=; b=QrUfAppicL0sTimwaXwSl4l6CTogDONpMYe96EXB/Ll/lAM6VXNCOXmBmWowkOo+Lx PmA3XBf71RTACH0VoRdeUr5s1NgbjFVyXU5MmKBwvs/VgzoSQObedEWvQ3z7yCxrKCgf tEd34AOPebZiWT1PGcrTKHmgGlmbESF67dHLIfDtDPeqf7dia9WgzKXfNQi+cbUtXWEG LXHx2fYHU0ftQ11Zv1WsJL7bZZCgm5F+seN0IjX+qLP5GRpb4FtexINpjiXR0P8bKhnc 1LKm/a1r4MkQ9GMrEIG7OvDmCpvuR2BCBA83hcAuCALgazJtsaEAMdF1JO1eSYkum3+N D/4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=N6hXOEaMk68qB7NM/pjswjlDhIvzZ4KiUpxCPmJLBBU=; b=HmQgMNemn1AOmicOwxUI8nZAiCT1RBtIeygzcMpbu2j0InfHai9C02lQa6hMhnBP/p qdsiIv2fuejCwqSxbkFd1nE72SGN03eHb9mP1UNsbofXj63fCKZmb9wLGLfFUZfGXJTe ZansnxDjuOOZ3L+NEmVZEd3jZuWul5nBE8YG52T8bJhcRc4cJrj/Zy9dpVUpzkDI9DcY 0YUtVnlOggN85pDw1Ty8GVQl+L/fFtz9kxenkj0Grubg8yJzJxnFOrDjdtgqDtIyEWl/ z6HFr6txlBKTPZux8/Fto7tDyKAKuNkEM87CFRjxv309+XFWdhXxxk/2c9iBHl7ZH6b3 T9ZQ== X-Gm-Message-State: AA+aEWaZxvAv9x3gR5f2X2x7P5ErDcAHysEPgwrYchg6XHlmA2ZR4SWc uaAJjvwd/YqJ0VczX0XrugORDq1QXUxiU65r8ro= X-Google-Smtp-Source: AFSGD/Wo3PXAIoSHuyH8oYaPquyZW91d/WkypGdoCfS91z0DKH0Q3NFt2qlb0585sEMji1kvwy030J11EAVnjDTmrYE= X-Received: by 2002:a2e:96c6:: with SMTP id d6-v6mr166285ljj.35.1544733867730; Thu, 13 Dec 2018 12:44:27 -0800 (PST) MIME-Version: 1.0 References: <20181213203102.GF2244@macbook.localdomain> In-Reply-To: <20181213203102.GF2244@macbook.localdomain> From: Kaushal Modi Date: Thu, 13 Dec 2018 15:43:50 -0500 Message-ID: Subject: Re: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: dr.khaled.hosny@gmail.com Content-Type: multipart/alternative; boundary="000000000000756f12057ced615b" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000756f12057ced615b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 13, 2018 at 3:31 PM Khaled Hosny wrote: > > The HarfBuzz rendering of Arabic is the correct one in this screenshot. > Thanks. So here's the status so far: Rendering of Namaste as seen in C-h h (M-x view-hello-file): | | harfbuzz | m17b | |----------+----------+---------| | Hindi | correct | correct | | Gujarati | wrong | correct | | Arabic | correct | wrong | > For debugging the such rendering differences, the actual font used by > Emacs for a given part of the text need to be known, I am using Mukta Vaani font for Gujarati. It is a free font and be downloaded from https://ektype.in/mukta-vaani.html. The string being rendered is "=E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4= =E0=AB=87". By placing the cursor on each of those characters and doing C-u x =3D (on t= he m17n build), I get: (1) =E0=AA=A8 position: 1610 of 3509 (46%), column: 32 character: =E0=AA=A8 (displayed as =E0=AA=A8) (codepoint 2728, = #o5250, #xaa8) charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.) code point in charset: 0x3968 script: gujarati syntax: w which means: word category: .:Base, L:Left-to-right (strong) to input: type "C-x 8 RET aa8" or "C-x 8 RET GUJARATI LETTER NA" buffer code: #xE0 #xAA #xA8 file code: #xE0 #xAA #xA8 (encoded by coding system utf-8-unix) display: by this font (glyph code) xft:-unknown-Mukta Vaani-normal-normal-normal-*-18-*-*-*-*-0-iso10646-1 (#x234) Character code properties: customize what to show name: GUJARATI LETTER NA general-category: Lo (Letter, Other) decomposition: (2728) ('=E0=AA=A8') There are text properties here: charset mule-unicode-0100-24ff (2) =E0=AA=AE position: 1611 of 3509 (46%), column: 33 character: =E0=AA=AE (displayed as =E0=AA=AE) (codepoint 2734, = #o5256, #xaae) charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.) code point in charset: 0x396E script: gujarati syntax: w which means: word category: .:Base, L:Left-to-right (strong) to input: type "C-x 8 RET aae" or "C-x 8 RET GUJARATI LETTER MA" buffer code: #xE0 #xAA #xAE file code: #xE0 #xAA #xAE (encoded by coding system utf-8-unix) display: by this font (glyph code) xft:-unknown-Mukta Vaani-normal-normal-normal-*-18-*-*-*-*-0-iso10646-1 (#x239) Character code properties: customize what to show name: GUJARATI LETTER MA general-category: Lo (Letter, Other) decomposition: (2734) ('=E0=AA=AE') There are text properties here: charset mule-unicode-0100-24ff (3) =E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87 position: 1612 of 3509 (46%), column: 34 character: =E0=AA=B8 (displayed as =E0=AA=B8) (codepoint 2744, = #o5270, #xab8) charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.) code point in charset: 0x3978 script: gujarati syntax: w which means: word category: .:Base, L:Left-to-right (strong) to input: type "C-x 8 RET ab8" or "C-x 8 RET GUJARATI LETTER SA" buffer code: #xE0 #xAA #xB8 file code: #xE0 #xAA #xB8 (encoded by coding system utf-8-unix) display: composed to form "=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB= =87" (see below) Composed with the following character(s) "=E0=AB=8D=E0=AA=A4=E0=AB=87" usin= g this font: xft:-unknown-Mukta Vaani-normal-normal-normal-*-18-*-*-*-*-0-iso10646-1 by these glyphs: [0 3 0 645 8 0 11 11 0 [0 0 8]] [0 3 2724 560 11 1 11 11 1 nil] [0 3 2759 589 0 -9 -2 16 -11 [-1 0 0]] Character code properties: customize what to show name: GUJARATI LETTER SA general-category: Lo (Letter, Other) decomposition: (2744) ('=E0=AA=B8') There are text properties here: charset mule-unicode-0100-24ff =3D=3D=3D=3D=3D On harfbuzz build, the "=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87" part is diffe= rent.. I can place the cursor separately on =E0=AA=B8=E0=AB=8D and =E0=AA=A4=E0=AB=87, do C-u x =3D and I= get: (3.1) =E0=AA=B8=E0=AB=8D position: 1612 of 3509 (46%), column: 34 character: =E0=AA=B8 (displayed as =E0=AA=B8) (codepoint 2744, = #o5270, #xab8) charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.) code point in charset: 0x3978 script: gujarati syntax: w which means: word category: .:Base, L:Left-to-right (strong) to input: type "C-x 8 RET ab8" or "C-x 8 RET GUJARATI LETTER SA" buffer code: #xE0 #xAA #xB8 file code: #xE0 #xAA #xB8 (encoded by coding system utf-8-unix) display: by this font (glyph code) xft:-unknown-Mukta Vaani-normal-normal-normal-*-18-*-*-*-*-0-iso10646-1 (#x241) Character code properties: customize what to show name: GUJARATI LETTER SA general-category: Lo (Letter, Other) decomposition: (2744) ('=E0=AA=B8') There are text properties here: charset mule-unicode-0100-24ff (3.2) =E0=AA=A4=E0=AB=87 position: 1614 of 3509 (46%), column: 35 character: =E0=AA=A4 (displayed as =E0=AA=A4) (codepoint 2724, = #o5244, #xaa4) charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.) code point in charset: 0x3964 script: gujarati syntax: w which means: word category: .:Base, L:Left-to-right (strong) to input: type "C-x 8 RET aa4" or "C-x 8 RET GUJARATI LETTER TA" buffer code: #xE0 #xAA #xA4 file code: #xE0 #xAA #xA4 (encoded by coding system utf-8-unix) display: by this font (glyph code) xft:-unknown-Mukta Vaani-normal-normal-normal-*-18-*-*-*-*-0-iso10646-1 (#x230) Character code properties: customize what to show name: GUJARATI LETTER TA general-category: Lo (Letter, Other) decomposition: (2724) ('=E0=AA=A4') There are text properties here: charset mule-unicode-0100-24ff then the text and > the font can be checked against vanilla HarfBuzz (e.g. using the hb-view > command line tool); if it gives the same rendering then it is either a > HarfBuzz or font issue, if not then it is a bug in the HarfBuzz > integration code in Emacs. > --000000000000756f12057ced615b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0 ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9 Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBk aXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiPk9uIFRodSwg RGVjIDEzLCAyMDE4IGF0IDM6MzEgUE0gS2hhbGVkIEhvc255ICZsdDs8YSBocmVmPSJtYWlsdG86 ZHIua2hhbGVkLmhvc255QGdtYWlsLmNvbSI+ZHIua2hhbGVkLmhvc255QGdtYWlsLmNvbTwvYT4m Z3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxl PSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4IHNvbGlkIHJnYigyMDQs MjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCI+PGJyPg0KVGhlIEhhcmZCdXp6IHJlbmRlcmluZyBv ZiBBcmFiaWMgaXMgdGhlIGNvcnJlY3Qgb25lIGluIHRoaXMgc2NyZWVuc2hvdC48YnI+PC9ibG9j a3F1b3RlPjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhhbmtzLiBTbyBoZXJlJiMzOTtzIHRoZSBzdGF0 dXMgc28gZmFyOjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+UmVuZGVyaW5nIG9mIE5hbWFzdGUg YXMgc2VlbiBpbiBDLWggaCAoTS14IHZpZXctaGVsbG8tZmlsZSk6PGJyPjxicj58wqDCoMKgwqDC oMKgwqDCoMKgIHwgaGFyZmJ1enogfCBtMTdiwqDCoMKgIHw8YnI+fC0tLS0tLS0tLS0rLS0tLS0t LS0tLSstLS0tLS0tLS18PGJyPnwgSGluZGnCoMKgwqAgfCBjb3JyZWN0wqAgfCBjb3JyZWN0IHw8 YnI+fCBHdWphcmF0aSB8IHdyb25nwqDCoMKgIHwgY29ycmVjdCB8PGJyPnwgQXJhYmljwqDCoCB8 IGNvcnJlY3TCoCB8IHdyb25nwqDCoCB8PGJyPjxicj48L2Rpdj48ZGl2PsKgPC9kaXY+PGJsb2Nr cXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjBweCAwcHggMHB4IDAuOGV4 O2JvcmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoMjA0LDIwNCwyMDQpO3BhZGRpbmctbGVmdDoxZXgi Pg0KRm9yIGRlYnVnZ2luZyB0aGUgc3VjaCByZW5kZXJpbmcgZGlmZmVyZW5jZXMsIHRoZSBhY3R1 YWwgZm9udCB1c2VkIGJ5PGJyPg0KRW1hY3MgZm9yIGEgZ2l2ZW4gcGFydCBvZiB0aGUgdGV4dCBu ZWVkIHRvIGJlIGtub3duLDwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PkkgYW0gdXNp bmcgTXVrdGEgVmFhbmkgZm9udCBmb3IgR3VqYXJhdGkuIEl0IGlzIGEgZnJlZSBmb250IGFuZCBi ZSBkb3dubG9hZGVkIGZyb20gPGEgaHJlZj0iaHR0cHM6Ly9la3R5cGUuaW4vbXVrdGEtdmFhbmku aHRtbCI+aHR0cHM6Ly9la3R5cGUuaW4vbXVrdGEtdmFhbmkuaHRtbDwvYT4uPGJyPjwvZGl2Pjxk aXY+wqA8L2Rpdj48ZGl2PlRoZSBzdHJpbmcgYmVpbmcgcmVuZGVyZWQgaXMgJnF1b3Q74Kqo4Kqu 4Kq44KuN4Kqk4KuHJnF1b3Q7LiANCjxkaXY+QnkgcGxhY2luZyB0aGUgY3Vyc29yIG9uIGVhY2gg b2YgdGhvc2UgY2hhcmFjdGVycyBhbmQgZG9pbmcgQy11IHggPSAob24gdGhlIG0xN24gYnVpbGQp LCBJIGdldDo8L2Rpdj48ZGl2PjwvZGl2Pg0KDQo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pigx KSANCuCqqA0KDQo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PsKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCBwb3NpdGlvbjogMTYxMCBvZiAzNTA5ICg0NiUpLCBjb2x1bW46IDMyPGJyPsKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgY2hhcmFjdGVyOiDgqqggKGRpc3BsYXllZCBhcyDgqqgpIChjb2RlcG9p bnQgMjcyOCwgI281MjUwLCAjeGFhOCk8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgY2hh cnNldDogbXVsZS11bmljb2RlLTAxMDAtMjRmZiAoVW5pY29kZSBjaGFyYWN0ZXJzIG9mIHRoZSBy YW5nZSBVKzAxMDAuLlUrMjRGRi4pPGJyPmNvZGUgcG9pbnQgaW4gY2hhcnNldDogMHgzOTY4PGJy PsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2NyaXB0OiBndWphcmF0aTxicj7CoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN5bnRheDogdyDCoMKgwqAgd2hpY2ggbWVhbnM6IHdvcmQ8 YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNhdGVnb3J5OiAuOkJhc2UsIEw6TGVmdC10by1y aWdodCAoc3Ryb25nKTxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdG8gaW5wdXQ6IHR5cGUg JnF1b3Q7Qy14IDggUkVUIGFhOCZxdW90OyBvciAmcXVvdDtDLXggOCBSRVQgR1VKQVJBVEkgTEVU VEVSIE5BJnF1b3Q7PGJyPsKgwqDCoMKgwqDCoMKgwqDCoCBidWZmZXIgY29kZTogI3hFMCAjeEFB ICN4QTg8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBmaWxlIGNvZGU6ICN4RTAgI3hBQSAjeEE4 IChlbmNvZGVkIGJ5IGNvZGluZyBzeXN0ZW0gdXRmLTgtdW5peCk8YnI+wqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgZGlzcGxheTogYnkgdGhpcyBmb250IChnbHlwaCBjb2RlKTxicj7CoMKgwqAg eGZ0Oi11bmtub3duLU11a3RhIFZhYW5pLW5vcm1hbC1ub3JtYWwtbm9ybWFsLSotMTgtKi0qLSot Ki0wLWlzbzEwNjQ2LTEgKCN4MjM0KTxicj48YnI+Q2hhcmFjdGVyIGNvZGUgcHJvcGVydGllczog Y3VzdG9taXplIHdoYXQgdG8gc2hvdzxicj7CoCBuYW1lOiBHVUpBUkFUSSBMRVRURVIgTkE8YnI+ wqAgZ2VuZXJhbC1jYXRlZ29yeTogTG8gKExldHRlciwgT3RoZXIpPGJyPsKgIGRlY29tcG9zaXRp b246ICgyNzI4KSAoJiMzOTvgqqgmIzM5Oyk8YnI+PGJyPlRoZXJlIGFyZSB0ZXh0IHByb3BlcnRp ZXMgaGVyZTo8YnI+wqAgY2hhcnNldMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIG11bGUtdW5p Y29kZS0wMTAwLTI0ZmY8YnI+PC9kaXY+PGJyPjxkaXY+KDIpIOCqrjwvZGl2PjxkaXY+PGJyPjwv ZGl2PjxkaXY+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHBvc2l0aW9uOiAxNjExIG9mIDM1MDkg KDQ2JSksIGNvbHVtbjogMzM8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjaGFyYWN0ZXI6IOCq riAoZGlzcGxheWVkIGFzIOCqrikgKGNvZGVwb2ludCAyNzM0LCAjbzUyNTYsICN4YWFlKTxicj7C oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjaGFyc2V0OiBtdWxlLXVuaWNvZGUtMDEwMC0yNGZm IChVbmljb2RlIGNoYXJhY3RlcnMgb2YgdGhlIHJhbmdlIFUrMDEwMC4uVSsyNEZGLik8YnI+Y29k ZSBwb2ludCBpbiBjaGFyc2V0OiAweDM5NkU8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCBzY3JpcHQ6IGd1amFyYXRpPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3ludGF4 OiB3IMKgwqDCoCB3aGljaCBtZWFuczogd29yZDxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg Y2F0ZWdvcnk6IC46QmFzZSwgTDpMZWZ0LXRvLXJpZ2h0IChzdHJvbmcpPGJyPsKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCB0byBpbnB1dDogdHlwZSAmcXVvdDtDLXggOCBSRVQgYWFlJnF1b3Q7IG9y ICZxdW90O0MteCA4IFJFVCBHVUpBUkFUSSBMRVRURVIgTUEmcXVvdDs8YnI+wqDCoMKgwqDCoMKg wqDCoMKgIGJ1ZmZlciBjb2RlOiAjeEUwICN4QUEgI3hBRTxicj7CoMKgwqDCoMKgwqDCoMKgwqDC oMKgIGZpbGUgY29kZTogI3hFMCAjeEFBICN4QUUgKGVuY29kZWQgYnkgY29kaW5nIHN5c3RlbSB1 dGYtOC11bml4KTxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBkaXNwbGF5OiBieSB0aGlz IGZvbnQgKGdseXBoIGNvZGUpPGJyPsKgwqDCoCB4ZnQ6LXVua25vd24tTXVrdGEgVmFhbmktbm9y bWFsLW5vcm1hbC1ub3JtYWwtKi0xOC0qLSotKi0qLTAtaXNvMTA2NDYtMSAoI3gyMzkpPGJyPjxi cj5DaGFyYWN0ZXIgY29kZSBwcm9wZXJ0aWVzOiBjdXN0b21pemUgd2hhdCB0byBzaG93PGJyPsKg IG5hbWU6IEdVSkFSQVRJIExFVFRFUiBNQTxicj7CoCBnZW5lcmFsLWNhdGVnb3J5OiBMbyAoTGV0 dGVyLCBPdGhlcik8YnI+wqAgZGVjb21wb3NpdGlvbjogKDI3MzQpICgmIzM5O+CqriYjMzk7KTxi cj48YnI+VGhlcmUgYXJlIHRleHQgcHJvcGVydGllcyBoZXJlOjxicj7CoCBjaGFyc2V0wqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgbXVsZS11bmljb2RlLTAxMDAtMjRmZjxicj48YnI+PC9kaXY+ PGRpdj4oMykgDQoNCuCquOCrjeCqpOCrhyA8YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj7C oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcG9zaXRpb246IDE2MTIgb2YgMzUwOSAoNDYlKSwgY29s dW1uOiAzNDxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNoYXJhY3Rlcjog4Kq4IChkaXNwbGF5 ZWQgYXMg4Kq4KSAoY29kZXBvaW50IDI3NDQsICNvNTI3MCwgI3hhYjgpPGJyPsKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIGNoYXJzZXQ6IG11bGUtdW5pY29kZS0wMTAwLTI0ZmYgKFVuaWNvZGUg Y2hhcmFjdGVycyBvZiB0aGUgcmFuZ2UgVSswMTAwLi5VKzI0RkYuKTxicj5jb2RlIHBvaW50IGlu IGNoYXJzZXQ6IDB4Mzk3ODxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHNjcmlwdDog Z3VqYXJhdGk8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzeW50YXg6IHcgwqDCoMKg IHdoaWNoIG1lYW5zOiB3b3JkPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjYXRlZ29yeTog LjpCYXNlLCBMOkxlZnQtdG8tcmlnaHQgKHN0cm9uZyk8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIHRvIGlucHV0OiB0eXBlICZxdW90O0MteCA4IFJFVCBhYjgmcXVvdDsgb3IgJnF1b3Q7Qy14 IDggUkVUIEdVSkFSQVRJIExFVFRFUiBTQSZxdW90Ozxicj7CoMKgwqDCoMKgwqDCoMKgwqAgYnVm ZmVyIGNvZGU6ICN4RTAgI3hBQSAjeEI4PGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZmlsZSBj b2RlOiAjeEUwICN4QUEgI3hCOCAoZW5jb2RlZCBieSBjb2Rpbmcgc3lzdGVtIHV0Zi04LXVuaXgp PGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGRpc3BsYXk6IGNvbXBvc2VkIHRvIGZvcm0g JnF1b3Q74Kq44KuN4Kqk4KuHJnF1b3Q7IChzZWUgYmVsb3cpPGJyPjxicj5Db21wb3NlZCB3aXRo IHRoZSBmb2xsb3dpbmcgY2hhcmFjdGVyKHMpICZxdW90O+CrjeCqpOCrhyZxdW90OyB1c2luZyB0 aGlzIGZvbnQ6PGJyPsKgIHhmdDotdW5rbm93bi1NdWt0YSBWYWFuaS1ub3JtYWwtbm9ybWFsLW5v cm1hbC0qLTE4LSotKi0qLSotMC1pc28xMDY0Ni0xPGJyPmJ5IHRoZXNlIGdseXBoczo8YnI+wqAg WzAgMyAwIDY0NSA4IDAgMTEgMTEgMCBbMCAwIDhdXTxicj7CoCBbMCAzIDI3MjQgNTYwIDExIDEg MTEgMTEgMSBuaWxdPGJyPsKgIFswIDMgMjc1OSA1ODkgMCAtOSAtMiAxNiAtMTEgWy0xIDAgMF1d PGJyPjxicj5DaGFyYWN0ZXIgY29kZSBwcm9wZXJ0aWVzOiBjdXN0b21pemUgd2hhdCB0byBzaG93 PGJyPsKgIG5hbWU6IEdVSkFSQVRJIExFVFRFUiBTQTxicj7CoCBnZW5lcmFsLWNhdGVnb3J5OiBM byAoTGV0dGVyLCBPdGhlcik8YnI+wqAgZGVjb21wb3NpdGlvbjogKDI3NDQpICgmIzM5O+CquCYj Mzk7KTxicj48YnI+VGhlcmUgYXJlIHRleHQgcHJvcGVydGllcyBoZXJlOjxicj7CoCBjaGFyc2V0 wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgbXVsZS11bmljb2RlLTAxMDAtMjRmZjxicj48YnI+ PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj49PT09PTwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+ PGJyPjwvZGl2PjxkaXY+T24gaGFyZmJ1enogYnVpbGQsIHRoZSAmcXVvdDvgqrjgq43gqqTgq4cm cXVvdDsgcGFydCBpcyBkaWZmZXJlbnQuLiBJIGNhbiBwbGFjZSB0aGUgY3Vyc29yIHNlcGFyYXRl bHkgb24g4Kq44KuNIGFuZCDgqqTgq4csIGRvIEMtdSB4ID0gYW5kIEkgZ2V0OjwvZGl2PjxkaXY+ PGJyPjwvZGl2PjxkaXY+KDMuMSkg4Kq44KuNwqANCg0KPC9kaXY+PGRpdj7CoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgcG9zaXRpb246IDE2MTIgb2YgMzUwOSAoNDYlKSwgY29sdW1uOiAzNDxicj7C oMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNoYXJhY3Rlcjog4Kq4IChkaXNwbGF5ZWQgYXMg4Kq4KSAo Y29kZXBvaW50IDI3NDQsICNvNTI3MCwgI3hhYjgpPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIGNoYXJzZXQ6IG11bGUtdW5pY29kZS0wMTAwLTI0ZmYgKFVuaWNvZGUgY2hhcmFjdGVycyBv ZiB0aGUgcmFuZ2UgVSswMTAwLi5VKzI0RkYuKTxicj5jb2RlIHBvaW50IGluIGNoYXJzZXQ6IDB4 Mzk3ODxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHNjcmlwdDogZ3VqYXJhdGk8YnI+ wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzeW50YXg6IHcgwqDCoMKgIHdoaWNoIG1lYW5z OiB3b3JkPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjYXRlZ29yeTogLjpCYXNlLCBMOkxl ZnQtdG8tcmlnaHQgKHN0cm9uZyk8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRvIGlucHV0 OiB0eXBlICZxdW90O0MteCA4IFJFVCBhYjgmcXVvdDsgb3IgJnF1b3Q7Qy14IDggUkVUIEdVSkFS QVRJIExFVFRFUiBTQSZxdW90Ozxicj7CoMKgwqDCoMKgwqDCoMKgwqAgYnVmZmVyIGNvZGU6ICN4 RTAgI3hBQSAjeEI4PGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZmlsZSBjb2RlOiAjeEUwICN4 QUEgI3hCOCAoZW5jb2RlZCBieSBjb2Rpbmcgc3lzdGVtIHV0Zi04LXVuaXgpPGJyPsKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIGRpc3BsYXk6IGJ5IHRoaXMgZm9udCAoZ2x5cGggY29kZSk8YnI+ wqDCoMKgIHhmdDotdW5rbm93bi1NdWt0YSBWYWFuaS1ub3JtYWwtbm9ybWFsLW5vcm1hbC0qLTE4 LSotKi0qLSotMC1pc28xMDY0Ni0xICgjeDI0MSk8YnI+PGJyPkNoYXJhY3RlciBjb2RlIHByb3Bl cnRpZXM6IGN1c3RvbWl6ZSB3aGF0IHRvIHNob3c8YnI+wqAgbmFtZTogR1VKQVJBVEkgTEVUVEVS IFNBPGJyPsKgIGdlbmVyYWwtY2F0ZWdvcnk6IExvIChMZXR0ZXIsIE90aGVyKTxicj7CoCBkZWNv bXBvc2l0aW9uOiAoMjc0NCkgKCYjMzk74Kq4JiMzOTspPGJyPjxicj5UaGVyZSBhcmUgdGV4dCBw cm9wZXJ0aWVzIGhlcmU6PGJyPsKgIGNoYXJzZXTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBt dWxlLXVuaWNvZGUtMDEwMC0yNGZmPGJyPjxicj48L2Rpdj4NCigzLjIpIOCqpOCrhw0KDQoNCg0K PGRpdj48YnI+PC9kaXY+PGRpdj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcG9zaXRpb246IDE2 MTQgb2YgMzUwOSAoNDYlKSwgY29sdW1uOiAzNTxicj7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNo YXJhY3Rlcjog4KqkIChkaXNwbGF5ZWQgYXMg4KqkKSAoY29kZXBvaW50IDI3MjQsICNvNTI0NCwg I3hhYTQpPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNoYXJzZXQ6IG11bGUtdW5pY29k ZS0wMTAwLTI0ZmYgKFVuaWNvZGUgY2hhcmFjdGVycyBvZiB0aGUgcmFuZ2UgVSswMTAwLi5VKzI0 RkYuKTxicj5jb2RlIHBvaW50IGluIGNoYXJzZXQ6IDB4Mzk2NDxicj7CoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIHNjcmlwdDogZ3VqYXJhdGk8YnI+wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCBzeW50YXg6IHcgwqDCoMKgIHdoaWNoIG1lYW5zOiB3b3JkPGJyPsKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCBjYXRlZ29yeTogLjpCYXNlLCBMOkxlZnQtdG8tcmlnaHQgKHN0cm9uZyk8YnI+ wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRvIGlucHV0OiB0eXBlICZxdW90O0MteCA4IFJFVCBh YTQmcXVvdDsgb3IgJnF1b3Q7Qy14IDggUkVUIEdVSkFSQVRJIExFVFRFUiBUQSZxdW90Ozxicj7C oMKgwqDCoMKgwqDCoMKgwqAgYnVmZmVyIGNvZGU6ICN4RTAgI3hBQSAjeEE0PGJyPsKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgZmlsZSBjb2RlOiAjeEUwICN4QUEgI3hBNCAoZW5jb2RlZCBieSBjb2Rp bmcgc3lzdGVtIHV0Zi04LXVuaXgpPGJyPsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGRpc3Bs YXk6IGJ5IHRoaXMgZm9udCAoZ2x5cGggY29kZSk8YnI+wqDCoMKgIHhmdDotdW5rbm93bi1NdWt0 YSBWYWFuaS1ub3JtYWwtbm9ybWFsLW5vcm1hbC0qLTE4LSotKi0qLSotMC1pc28xMDY0Ni0xICgj eDIzMCk8YnI+PGJyPkNoYXJhY3RlciBjb2RlIHByb3BlcnRpZXM6IGN1c3RvbWl6ZSB3aGF0IHRv IHNob3c8YnI+wqAgbmFtZTogR1VKQVJBVEkgTEVUVEVSIFRBPGJyPsKgIGdlbmVyYWwtY2F0ZWdv cnk6IExvIChMZXR0ZXIsIE90aGVyKTxicj7CoCBkZWNvbXBvc2l0aW9uOiAoMjcyNCkgKCYjMzk7 4KqkJiMzOTspPGJyPjxicj5UaGVyZSBhcmUgdGV4dCBwcm9wZXJ0aWVzIGhlcmU6PGJyPsKgIGNo YXJzZXTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBtdWxlLXVuaWNvZGUtMDEwMC0yNGZmPGJy Pjxicj48YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1 b3RlIiBzdHlsZT0ibWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0OjFweCBzb2xp ZCByZ2IoMjA0LDIwNCwyMDQpO3BhZGRpbmctbGVmdDoxZXgiPiB0aGVuIHRoZSB0ZXh0IGFuZDxi cj4NCnRoZSBmb250IGNhbiBiZSBjaGVja2VkIGFnYWluc3QgdmFuaWxsYSBIYXJmQnV6eiAoZS5n LiB1c2luZyB0aGUgaGItdmlldzxicj4NCmNvbW1hbmQgbGluZSB0b29sKTsgaWYgaXQgZ2l2ZXMg dGhlIHNhbWUgcmVuZGVyaW5nIHRoZW4gaXQgaXMgZWl0aGVyIGE8YnI+DQpIYXJmQnV6eiBvciBm b250IGlzc3VlLCBpZiBub3QgdGhlbiBpdCBpcyBhIGJ1ZyBpbiB0aGUgSGFyZkJ1eno8YnI+DQpp bnRlZ3JhdGlvbiBjb2RlIGluIEVtYWNzLjxicj4NCjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj48 L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rp dj48L2Rpdj4NCg== --000000000000756f12057ced615b-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 15:53:47 2018 Received: (at 33729) by debbugs.gnu.org; 13 Dec 2018 20:53:47 +0000 Received: from localhost ([127.0.0.1]:46936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXzP-0003YM-2O for submit@debbugs.gnu.org; Thu, 13 Dec 2018 15:53:47 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:40367) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXXzO-0003Y9-0p for 33729@debbugs.gnu.org; Thu, 13 Dec 2018 15:53:46 -0500 Received: by mail-wm1-f47.google.com with SMTP id q26so3756001wmf.5 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 12:53:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=4xwe32xkFcAwGc7g/aZIKI5v5ZroCBBo+J8S7AX1XHY=; b=V8F7N2sI0IZJv1GYTT/g8h/R4QrFOn+dzsUZ7x8xv1EmeyJuVtzXm4uBhDtI8ORDXM ti1cqC4iWReboEPyHRa7b2i7rzCWCbHVt3SeRMf2FSCtoLv5Hm1Q26juQZHEHvU8SZJf dVpzPkUCAwRq8OtEFlNHSEcC93ZGr8sZ6/5a5Hv9FxAWmbGWEC9TmcnuVrFx39ewLXzv j1xQlPOrzli8v43rMWNsM1ldRFzKO1GQ0ddnz8mtLg9JIQ5UfO6fG4nUin34jH/TtJQw aPe79iCQvxqenUqKpr9DYsdkYTS7AtaHUSOw/UxLh0ylFPtJXJRed6LDb6Dg3QRRks4H mydQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=4xwe32xkFcAwGc7g/aZIKI5v5ZroCBBo+J8S7AX1XHY=; b=QAtgHunXdvUMnCsKRv+n93uEQ365aub+e/GhmnsN5MbzMyWsNlMzN8GXMp8TgonqiT d9Wtsa3gz7+PaVZS64L30I8j2dBr9e/vhMYEYHbjc4W8ISoV+EC0/zwcQZ519wvQHgFo kUAB4LMOwbQpQ+PZuabZy3nZ0yq404PQ4y9UxcQ5+fGWpGeonKgENZxA9ayJ2osXlH/R UypdU38eajZyBPPU+yPQokFnCA6+0BPAqqwQ7VOPBF9qK9iDiGuFmxC5/oOwHVVz5hXV YGLm5kKTQemXvKbYMY2RNT5YaXxZflcr5f3A6MMBYKDsIrpLzVz1WGSeVhldHaCSzpud 5i9w== X-Gm-Message-State: AA+aEWb1ClfULgd8Y6GE9nUhpg3pEVDz/WNkRUi51gvA9EzFr+MfN9Mv Plhk3VvFP0+ENk977G1PvFY= X-Google-Smtp-Source: AFSGD/VFCJGkZQjV7BQ46oJLGVHwATFC+ybTsIuw53RO20nDy4zvfAJ912/CEZ29tPucikFDtTTByw== X-Received: by 2002:a1c:864f:: with SMTP id i76mr835269wmd.83.1544734420065; Thu, 13 Dec 2018 12:53:40 -0800 (PST) Received: from macbook.localdomain ([197.58.120.254]) by smtp.gmail.com with ESMTPSA id r69sm4773376wmd.4.2018.12.13.12.53.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 13 Dec 2018 12:53:39 -0800 (PST) Date: Thu, 13 Dec 2018 22:53:36 +0200 From: Khaled Hosny To: Kaushal Modi Subject: Re: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181213205336.GG2244@macbook.localdomain> References: <20181213203102.GF2244@macbook.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Thu, Dec 13, 2018 at 03:43:50PM -0500, Kaushal Modi wrote: > On Thu, Dec 13, 2018 at 3:31 PM Khaled Hosny > wrote: > > > > > The HarfBuzz rendering of Arabic is the correct one in this screenshot. > > > > Thanks. So here's the status so far: > > Rendering of Namaste as seen in C-h h (M-x view-hello-file): > > | | harfbuzz | m17b | > |----------+----------+---------| > | Hindi | correct | correct | > | Gujarati | wrong | correct | > | Arabic | correct | wrong | > > > > > For debugging the such rendering differences, the actual font used by > > Emacs for a given part of the text need to be known, > > > I am using Mukta Vaani font for Gujarati. It is a free font and be > downloaded from https://ektype.in/mukta-vaani.html. > > The string being rendered is "નમસ્તે". I tried that font and text with hb-view and the output I get is identical to m17b. If I pass a wrong script to HarfBuzz (e.g. --script=latn), I get the same broken output you see in Emacs. So I’m guessing something is not correctly working in script itemization. Most likely the FIXME in uni_script(), or the FIXME above the call to hb_buffer_guess_segment_properties(). Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 16:05:11 2018 Received: (at 33729) by debbugs.gnu.org; 13 Dec 2018 21:05:11 +0000 Received: from localhost ([127.0.0.1]:46945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXYAR-0005nf-G3 for submit@debbugs.gnu.org; Thu, 13 Dec 2018 16:05:11 -0500 Received: from mail-lj1-f170.google.com ([209.85.208.170]:41965) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXYAP-0005nN-Hg for 33729@debbugs.gnu.org; Thu, 13 Dec 2018 16:05:10 -0500 Received: by mail-lj1-f170.google.com with SMTP id k15-v6so3027254ljc.8 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 13:05:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=30xQwH9KIBHuF/FxGaY4SOpxucDrnQC2CXHg13t1td0=; b=jJju5qJ4sVp1UwaCEcGcJldumPQTC7sGgbsrKhfJkdC9xRzDPRO2AnSuxSKZegOzSL aUTmO6QqGG1fO4YyXgqnFXW52v4Rvf0/NY1d+vheR1I6qiD6LJgsyQCdC0Xcy7NaxIMA 5e+PoXfFqp/NoF1YvfUAtQd5bMZjnNyuFgjWFz9cloHHLq8aTYXYEPPYOrY6p34i16f7 nT21p9KvuIwf9o6OA/tTHa6iQrln6PFZMNp691hSbf4sLRHdz4zTGxXAHSzQ8ClSMRzf jGUO1h97111OtQTaUV3zAc2mRi7faDmPCz9qZncoOQkqt8FRk9zcdUV7RBOv6V0OEJZB obYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=30xQwH9KIBHuF/FxGaY4SOpxucDrnQC2CXHg13t1td0=; b=RKnvcZlgtqY8s5UJBGrbmpHGv6WaTxXJxR9LujiQQ8QlQ4gHRZmHjj2cX3GZcunHlV +rIwQtI0FafbkTsfpeiDR/AjP01hcCRWVtASN4AYphRT7/H3nmN2VUuwwEG1c9tuI+76 qJ9o+pb0+mu3xmh+hyE2bHjOITtR5OWoZbxeGZYCU5b1orWeZEcVmlLAww0po2EhL3s2 nvfFp+wPxjNhWpGIReM52NZus7fHhex48soDtPIpw/lSTytLdQt8r21lpwa8VDTMptx9 8pVb7GuC4arlcq6p8sdcyTqICoXHVHl2t6cJq5HC3MX//+44i/9vhP86Fnp9L5yqd7MU mS9Q== X-Gm-Message-State: AA+aEWYxpvgH/tNoUU9a1NZVn3eB9kqDtsGo8Ng+81XpTGNmUWGX98/f C7w4n6UshgjvhVjbMkri4F2sh+6C4zQzYqnq22Q= X-Google-Smtp-Source: AFSGD/X5hD0ZU9dhhJFLyYaCr02P3wrA0CZnJGXJZOm841+h78z/lG2HNSI9hhDHG6B8EBhbov8GgyY9zkclImHOYsg= X-Received: by 2002:a2e:5109:: with SMTP id f9-v6mr216621ljb.52.1544735103303; Thu, 13 Dec 2018 13:05:03 -0800 (PST) MIME-Version: 1.0 References: <20181213203102.GF2244@macbook.localdomain> <20181213205336.GG2244@macbook.localdomain> In-Reply-To: <20181213205336.GG2244@macbook.localdomain> From: Kaushal Modi Date: Thu, 13 Dec 2018 16:04:25 -0500 Message-ID: Subject: Re: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: dr.khaled.hosny@gmail.com Content-Type: multipart/alternative; boundary="0000000000001ac933057cedab9c" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --0000000000001ac933057cedab9c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 13, 2018 at 3:53 PM Khaled Hosny wrote: > > I tried that font and text with hb-view and the output I get is > identical to m17b. hb-view is nifty! I wasn't sure if it would work for me (because I haven't set my terminal to show unicode, etc.). But even with the older Harfbuzz 1.0.3 that I have, hb-view gave this: https://i.imgtc.com/d1N177Z.png I am impressed. That shows the correct rendering of =E0=AA=A8=E0=AA=AE=E0= =AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87. (I just blindly pasted =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87 as the secon= d argument and hit enter, my terminal doesn't even show the pasted text. But the hb-view rendering is correct.) > If I pass a wrong script to HarfBuzz (e.g. > --script=3Dlatn), I get the same broken output you see in Emacs. So I=E2= =80=99m > guessing something is not correctly working in script itemization. Most > likely the FIXME in uni_script(), or the FIXME above the call to > hb_buffer_guess_segment_properties(). > I am not a C developer. But hopefully this information would help you to fix the Harfbuzz integration with Emacs. I am surprised that the rendering of Hindi =E0=A4=A8=E0=A4=AE=E0=A4=B8=E0= =A5=8D=E0=A4=A4=E0=A5=87 using Harfbuzz in Emacs is correct, while the rendering of Gujarati =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0= =AB=8D=E0=AA=A4=E0=AB=87 is not, when in fact the two scripts are so similar to each other. [Fun fact: Most of Gujarati script if superimposed with a line at the top will look like valid Hindi. You can see that in the case of =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA= =A4=E0=AB=87 vs =E0=A4=A8=E0=A4=AE=E0=A4=B8=E0=A5=8D=E0=A4=A4=E0=A5=87 :) = ] --0000000000001ac933057cedab9c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Dec 13, 2018 at 3:53 PM Khaled Hosny <dr.khaled.hosny@gmail.com> wr= ote:

I tried that font and text with hb-view and the output I get is
identical to m17b.

hb-view is nifty! I wasn= 't sure if it would work for me (because I haven't set my terminal = to show unicode, etc.). But even with the older Harfbuzz 1.0.3 that I have,= hb-view gave this: https://i.i= mgtc.com/d1N177Z.png

I am impressed. That show= s the correct rendering of=20 =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87. (I just blindly pas= ted=C2=A0 =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87 as the second argume= nt and hit enter, my terminal doesn't even show the pasted text. But th= e hb-view rendering is correct.)
=C2=A0
I= f I pass a wrong script to HarfBuzz (e.g.
--script=3Dlatn), I get the same broken output you see in Emacs. So I=E2=80= =99m
guessing something is not correctly working in script itemization. Most
likely the FIXME in uni_script(), or the FIXME above the call to
hb_buffer_guess_segment_properties().

I= am not a C developer. But hopefully this information would help you to fix= the Harfbuzz integration with Emacs.=C2=A0

I am s= urprised that the rendering of Hindi =E0=A4=A8=E0=A4=AE=E0=A4=B8=E0=A5=8D= =E0=A4=A4=E0=A5=87 using Harfbuzz in Emacs is correct, while the=C2=A0 rend= ering of Gujarati=20 =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87 is not, when in fact= the two scripts are so similar to each other. [Fun fact: Most of Gujarati = script if superimposed with a line at the top will look like valid Hindi. Y= ou can see that in the case of=C2=A0 =E0=AA=A8=E0=AA=AE=E0=AA=B8=E0=AB=8D=E0=AA=A4=E0=AB=87 vs=C2=A0 =E0=A4=A8=E0=A4=AE=E0=A4=B8=E0=A5=8D=E0=A4=A4=E0=A5=87 :) ]
--0000000000001ac933057cedab9c-- From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 00:58:11 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 05:58:11 +0000 Received: from localhost ([127.0.0.1]:47206 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXgUF-0001OE-BR for submit@debbugs.gnu.org; Fri, 14 Dec 2018 00:58:11 -0500 Received: from eggs.gnu.org ([208.118.235.92]:53812) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXgUD-0001O0-RC for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 00:58:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXgU7-0004rx-N7 for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 00:58:04 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52459) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXgU2-0004pB-PA; Fri, 14 Dec 2018 00:57:58 -0500 Received: from [176.228.60.248] (port=4536 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gXgU2-0003kN-Af; Fri, 14 Dec 2018 00:57:58 -0500 Date: Fri, 14 Dec 2018 07:57:55 +0200 Message-Id: <83h8fghcpo.fsf@gnu.org> From: Eli Zaretskii To: Kaushal Modi In-reply-to: (message from Kaushal Modi on Thu, 13 Dec 2018 15:43:50 -0500) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Kaushal Modi > Date: Thu, 13 Dec 2018 15:43:50 -0500 > Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com > > For debugging the such rendering differences, the actual font used by > Emacs for a given part of the text need to be known, > > I am using Mukta Vaani font for Gujarati. It is a free font and be downloaded from > https://ektype.in/mukta-vaani.html. Your data indicates that the m17n build performs character composition at buffer position 34, whereas the harfbuzz build does not. The question is why. From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 01:46:03 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 06:46:03 +0000 Received: from localhost ([127.0.0.1]:47215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXhEZ-0002yP-6l for submit@debbugs.gnu.org; Fri, 14 Dec 2018 01:46:03 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:60386) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXhEV-0002xd-AT for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 01:46:02 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3AB5516094D; Thu, 13 Dec 2018 22:45:53 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MSTROIeOh4LC; Thu, 13 Dec 2018 22:45:52 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3B05E16092C; Thu, 13 Dec 2018 22:45:52 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id dEfXmBtFqzvu; Thu, 13 Dec 2018 22:45:52 -0800 (PST) Received: from [192.168.1.9] (cpe-23-242-74-103.socal.res.rr.com [23.242.74.103]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 03DA31608FF; Thu, 13 Dec 2018 22:45:51 -0800 (PST) To: 33729@debbugs.gnu.org From: Paul Eggert Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Organization: UCLA Computer Science Department Message-ID: <7cf3b43b-ab7c-1bac-48b3-b27e61570aeb@cs.ucla.edu> Date: Thu, 13 Dec 2018 22:45:48 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, Florian Beck , far.nasiri.m@gmail.com, Kaushal Modi X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Florian Beck pointed out some examples of possible related problems when=20 rendering Emacs's etc/HELLO file; see: https://lists.gnu.org/r/emacs-devel/2018-12/msg00271.html For the names of the languages in the languages, Harfbuzz seems to be bet= ter for=20 Burmese (=E1=80=99=E1=80=BC=E1=80=94=E1=80=BA=E1=80=99=E1=80=AC) (where m= aster is wrong); conversely Harfbuzz seems to be wrong=20 for Maldivian (=DE=8B=DE=A8=DE=88=DE=AC=DE=80=DE=A8) (where master is bet= ter). Please see the following for=20 what these should look like: https://en.wikipedia.org/wiki/File:Dhivehiscript.svg https://en.wikipedia.org/wiki/File:Burmese_script_sample.svg From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 02:49:09 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 07:49:10 +0000 Received: from localhost ([127.0.0.1]:47245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXiDd-0004Ue-MD for submit@debbugs.gnu.org; Fri, 14 Dec 2018 02:49:09 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50258) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXiDb-0004UQ-IV for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 02:49:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXiDV-0004J5-Q6 for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 02:49:02 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54040) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXiDR-0004HE-53; Fri, 14 Dec 2018 02:48:57 -0500 Received: from [176.228.60.248] (port=3401 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gXiDQ-0003GH-NE; Fri, 14 Dec 2018 02:48:57 -0500 Date: Fri, 14 Dec 2018 09:48:54 +0200 Message-Id: <838t0sh7kp.fsf@gnu.org> From: Eli Zaretskii To: kaushal.modi@gmail.com In-reply-to: <83h8fghcpo.fsf@gnu.org> (message from Eli Zaretskii on Fri, 14 Dec 2018 07:57:55 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Fri, 14 Dec 2018 07:57:55 +0200 > From: Eli Zaretskii > Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, > far.nasiri.m@gmail.com > > Your data indicates that the m17n build performs character composition > at buffer position 34 Sorry, wrong number: I meant buffer position 1612. From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 02:51:09 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 07:51:09 +0000 Received: from localhost ([127.0.0.1]:47253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXiFZ-0004YA-1i for submit@debbugs.gnu.org; Fri, 14 Dec 2018 02:51:09 -0500 Received: from mail-wr1-f51.google.com ([209.85.221.51]:41019) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXiFW-0004Xd-Jq for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 02:51:07 -0500 Received: by mail-wr1-f51.google.com with SMTP id x10so4433370wrs.8 for <33729@debbugs.gnu.org>; Thu, 13 Dec 2018 23:51:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=2EhnsLXf16PIk49rid2dop6LbhX1stDWTDO7cqfCidU=; b=WJdEX+N6EtOgXlPbG44mBRHgK18y6GfAz6RhM3wd9Er7mfojrZXUmI0lYzrIyV0in7 r3hIHHhOOzdaqG9psqBMJnwGIni1wjDLf2HmkxLjeYdm/JlU/ici1FLh2hpQJJt7OR7p o+Of4VX1mELRqKav/lq1/Le7yuNCyBFRJWSGTLumHbITjI3X4ZYx+dKs1ZGCWNg1AvPS egCDydmAiYxxtGGVKmsPlL4sGSOemAFKHCfl7k4/OjpFuFiTKJhgOuuQQ56zTuJ12lYO 6w2rBqECRknscAMPcsZehAUKAXht2O0rHKs/t+KUdF8ZBkFA9TkwJULpyR2zGIUfI3jz HQCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=2EhnsLXf16PIk49rid2dop6LbhX1stDWTDO7cqfCidU=; b=K5UiLfL50Hoe5tJEpFBArKuejGviLbiYBF4gLCIhL16L+lY3i44LAeqIoCPOOA+KHb ZhNi7Z0upTTD30v1xzlhmP/JJZB7dFpQH+Q6sVXL1hcPOpzO5AZB+Rs0nJya9cEhqsLP dFJaBIO9vPmAnDItdItRl6NsSVXNP8uljD3TCi5THVcguwCgYFmQAgOgW+ZUecHROnkd dchq+kBILc11ZzsKyYBNfWITa2Kb65+LtWLGtpZIZF7ESzzhNTRIocAKurscbbaowVvn YHC+QzVPLXec6SwVB2bbdTF3ea4TCdgG/Nbn6VHb9UQK7PSkzg62HXCYzfCejv9nY4xv 0Oyg== X-Gm-Message-State: AA+aEWa8FbuXjeyl/nNd7GRAiu0HNge5pz+0X4jQyOpFAX0yxs6ZDSIP Lhyyrd/utGcsltepWzCX4p4= X-Google-Smtp-Source: AFSGD/WC8IJ97o2R9Lb0JYk/iUpgwdvbrie8LXocOOIZ4r57oRW4m6iPkcre8YI8DQJhS/ITI7T/nw== X-Received: by 2002:adf:c711:: with SMTP id k17mr1553771wrg.197.1544773860672; Thu, 13 Dec 2018 23:51:00 -0800 (PST) Received: from macbook.localdomain ([197.58.120.254]) by smtp.gmail.com with ESMTPSA id f66sm4312371wmd.28.2018.12.13.23.50.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 13 Dec 2018 23:50:59 -0800 (PST) Date: Fri, 14 Dec 2018 09:50:56 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181214075056.GI2244@macbook.localdomain> References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <83h8fghcpo.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, Kaushal Modi X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Dec 14, 2018 at 07:57:55AM +0200, Eli Zaretskii wrote: > > From: Kaushal Modi > > Date: Thu, 13 Dec 2018 15:43:50 -0500 > > Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com > > > > For debugging the such rendering differences, the actual font used by > > Emacs for a given part of the text need to be known, > > > > I am using Mukta Vaani font for Gujarati. It is a free font and be downloaded from > > https://ektype.in/mukta-vaani.html. > > Your data indicates that the m17n build performs character composition > at buffer position 34, whereas the harfbuzz build does not. The > question is why. See my earlier email, most likely the culprit is the broken Emacs to HarfBuzz script code mapping that we discussed earlier. HarfBuzz needs to know the correct script of the text to perform shaping, and it looks like we are passing nonsense values for certain scripts (or rather for certain scripts we are lucky that the mapping is not broken). From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 05:03:49 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 10:03:49 +0000 Received: from localhost ([127.0.0.1]:47337 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXkJx-0007si-2S for submit@debbugs.gnu.org; Fri, 14 Dec 2018 05:03:49 -0500 Received: from eggs.gnu.org ([208.118.235.92]:40589) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXkJv-0007sX-5v for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 05:03:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXkJo-0005zO-Sx for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 05:03:41 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57034) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXkJj-0005vJ-QU; Fri, 14 Dec 2018 05:03:35 -0500 Received: from [176.228.60.248] (port=3980 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gXkJj-0007bb-Cu; Fri, 14 Dec 2018 05:03:35 -0500 Date: Fri, 14 Dec 2018 12:03:32 +0200 Message-Id: <8336r0h1cb.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181214075056.GI2244@macbook.localdomain> (message from Khaled Hosny on Fri, 14 Dec 2018 09:50:56 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Fri, 14 Dec 2018 09:50:56 +0200 > From: Khaled Hosny > Cc: Kaushal Modi , behdad@behdad.org, > 33729@debbugs.gnu.org, far.nasiri.m@gmail.com > > > Your data indicates that the m17n build performs character composition > > at buffer position 34, whereas the harfbuzz build does not. The > > question is why. > > See my earlier email, most likely the culprit is the broken Emacs to > HarfBuzz script code mapping that we discussed earlier. HarfBuzz needs > to know the correct script of the text to perform shaping, and it looks > like we are passing nonsense values for certain scripts (or rather for > certain scripts we are lucky that the mapping is not broken). Thanks. I don't yet have access to a GNU/Linux system with HarfBuzz installed, so I cannot myself debug it. I hope Mohammad will be able to look into this and either fix it or provide more focused and detailed analysis of what is wrong, so we could fix it. Or maybe you could point to the problematic code and tell more details. FWIW, I looked at ftfont.c:uni_script, and I cannot find a problem with it; in particular looking up in char-script-table each character of the Gujarati welcome in HELLO yields 'gujarati', so I couldn't see any evident Emacs issue. Or are you saying that hb_script_from_string doesn't DTRT? Or maybe Kaushal should upgrade to a newer version of HarfBuzz? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 06:03:28 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 11:03:28 +0000 Received: from localhost ([127.0.0.1]:47380 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXlFg-0000v8-2u for submit@debbugs.gnu.org; Fri, 14 Dec 2018 06:03:28 -0500 Received: from mail-wm1-f49.google.com ([209.85.128.49]:33831) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXlFe-0000ut-Iz for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 06:03:27 -0500 Received: by mail-wm1-f49.google.com with SMTP id y185so15969436wmd.1 for <33729@debbugs.gnu.org>; Fri, 14 Dec 2018 03:03:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=8k+AVQH5MB7DJfbVezJtjbDqtv3OFog7S/ao1HL+SRE=; b=WlEeO9fZk4mmoWuJcLfnyyAiNwAu7wolckVijYRy5xW7RDFLfLrNqthteyhKMIdihl +sMcOwMZ7fhREEMlEwJvGK+h0Eb5qBxJ7T9RdLTjwE9wTOvhrYOuAl7AGjj3U3exwgrI LmnPQRCukoUqbY1KsGAyHTUp6hY0578Cn7eEDk8Z96BqbL+galpVkGGalyQndlgCe8MG dnBYc3wjPQIx6L7GfvE8MVaw910k0ua2dtxNArcQub+O/KIYpOxf1mVtrM+4afKEUKZ4 CjPcUNKnc7etaorsv312Ly+o2qixQmePtDqTxWPIKmaKWGdbwUhhP+OAC3UgEPZAd7t3 4kuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=8k+AVQH5MB7DJfbVezJtjbDqtv3OFog7S/ao1HL+SRE=; b=OzA8ckWfjILi5UnMXKS1x+tOBGkUyLrU36YdlSQw/ZjROjB6zBYB8lhfTRP1qlgEIO 93RBHPjQu2nGr85cYlZ6YgEl2eJXsncDErpvl03K1Rlpll+73TU4/noSbgdM9MGDLSXt kGjjH0JvmRmemmJq4JRqRj+a0DZWE0rnji4oEnFXiH2VgZFgheIaPzesF91fejrZK0C6 8y6CCjqU18aLzh2IPJqJgdIMiIncNqB4KVhQnbCINADBM1TBepgx7lpVYJszU09Ftbmn XWeCjAphqUh+TVu4KgA6NZCl2YDlOLezr4g7noPLJ1V0j3xe06je7kmuBl2WRTt1CFi5 XTgQ== X-Gm-Message-State: AA+aEWYJUt+BJ6qn6Q93Ca1G8fWujRl23rA7Hvc+mmj/jwhtWGdj0N6N QUnORzDBBJSg1nENmCv2M8o= X-Google-Smtp-Source: AFSGD/W9CHYec424oiAJccQx38NkJQ36ksH+v1QEMpm3NDKPDeMSrOskQ9pAHgmEQRQFEOSMfcfMFw== X-Received: by 2002:a7b:c191:: with SMTP id y17mr2885711wmi.10.1544785400716; Fri, 14 Dec 2018 03:03:20 -0800 (PST) Received: from macbook.localdomain ([197.58.120.254]) by smtp.gmail.com with ESMTPSA id 199sm4913398wmh.21.2018.12.14.03.03.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Dec 2018 03:03:20 -0800 (PST) Date: Fri, 14 Dec 2018 13:03:16 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181214110316.GK2244@macbook.localdomain> References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8336r0h1cb.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Dec 14, 2018 at 12:03:32PM +0200, Eli Zaretskii wrote: > > Date: Fri, 14 Dec 2018 09:50:56 +0200 > > From: Khaled Hosny > > Cc: Kaushal Modi , behdad@behdad.org, > > 33729@debbugs.gnu.org, far.nasiri.m@gmail.com > > > > > Your data indicates that the m17n build performs character composition > > > at buffer position 34, whereas the harfbuzz build does not. The > > > question is why. > > > > See my earlier email, most likely the culprit is the broken Emacs to > > HarfBuzz script code mapping that we discussed earlier. HarfBuzz needs > > to know the correct script of the text to perform shaping, and it looks > > like we are passing nonsense values for certain scripts (or rather for > > certain scripts we are lucky that the mapping is not broken). > > Thanks. > > I don't yet have access to a GNU/Linux system with HarfBuzz installed, > so I cannot myself debug it. > > I hope Mohammad will be able to look into this and either fix it or > provide more focused and detailed analysis of what is wrong, so we > could fix it. Or maybe you could point to the problematic code and > tell more details. > > FWIW, I looked at ftfont.c:uni_script, and I cannot find a problem > with it; in particular looking up in char-script-table each character > of the Gujarati welcome in HELLO yields 'gujarati', so I couldn't see > any evident Emacs issue. Or are you saying that hb_script_from_string > doesn't DTRT? Or maybe Kaushal should upgrade to a newer version of > HarfBuzz? There is this FIXME: /* FIXME: from_string wants an ISO 15924 script tag here. */ As we discussed earlier, hb_script_from_string() expects ISO 15924 script tags, but char_script_table does not provide such tags (I don’t recall what it does provide exactly). We need a way to get ISO 15924 script tags from Emacs. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 08:43:06 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 13:43:06 +0000 Received: from localhost ([127.0.0.1]:47431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXnkA-00077i-5X for submit@debbugs.gnu.org; Fri, 14 Dec 2018 08:43:06 -0500 Received: from eggs.gnu.org ([208.118.235.92]:35934) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXnk9-00077B-2o for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 08:43:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXnk2-0005nx-TB for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 08:42:59 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:44702) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXnjz-0005m6-5s; Fri, 14 Dec 2018 08:42:55 -0500 Received: from [176.228.60.248] (port=1640 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gXnjy-0001de-JB; Fri, 14 Dec 2018 08:42:55 -0500 Date: Fri, 14 Dec 2018 15:42:49 +0200 Message-Id: <83y38sfcme.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181214110316.GK2244@macbook.localdomain> (message from Khaled Hosny on Fri, 14 Dec 2018 13:03:16 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Fri, 14 Dec 2018 13:03:16 +0200 > From: Khaled Hosny > Cc: kaushal.modi@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, > far.nasiri.m@gmail.com > > > FWIW, I looked at ftfont.c:uni_script, and I cannot find a problem > > with it; in particular looking up in char-script-table each character > > of the Gujarati welcome in HELLO yields 'gujarati', so I couldn't see > > any evident Emacs issue. Or are you saying that hb_script_from_string > > doesn't DTRT? Or maybe Kaushal should upgrade to a newer version of > > HarfBuzz? > > There is this FIXME: > > /* FIXME: from_string wants an ISO 15924 script tag here. */ > > As we discussed earlier, hb_script_from_string() expects ISO 15924 > script tags, but char_script_table does not provide such tags (I don’t > recall what it does provide exactly). We need a way to get ISO 15924 > script tags from Emacs. Right, I forgot about that. So you are saying that we need to generate Gujr instead of gujarati, is that right? Mohammad, do you need help in comping up with a solution? There's otf-script-alist (see fontest.el), but it goes in the opposite direction. We could use rassq (Frassq in C) to find the OTF script tag by its Emacs symbol (which is returned by indexing into Vchar_script_table), by looking in otf-script-alist. Or maybe you prefer a seperat data structure, not limited to the OTF tags? Let me know if you need more help. From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 10:25:59 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 15:25:59 +0000 Received: from localhost ([127.0.0.1]:48435 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXpLj-0001dL-Ft for submit@debbugs.gnu.org; Fri, 14 Dec 2018 10:25:59 -0500 Received: from eggs.gnu.org ([208.118.235.92]:43639) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXpLi-0001d9-1H for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 10:25:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXpLW-0008VN-PY for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 10:25:52 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47041) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXpLW-0008UH-Hf; Fri, 14 Dec 2018 10:25:46 -0500 Received: from [176.228.60.248] (port=4231 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gXpLW-0000sY-2c; Fri, 14 Dec 2018 10:25:46 -0500 Date: Fri, 14 Dec 2018 17:25:42 +0200 Message-Id: <83tvjgf7ux.fsf@gnu.org> From: Eli Zaretskii To: far.nasiri.m@gmail.com In-reply-to: <83y38sfcme.fsf@gnu.org> (message from Eli Zaretskii on Fri, 14 Dec 2018 15:42:49 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Fri, 14 Dec 2018 15:42:49 +0200 > From: Eli Zaretskii > Cc: behdad@behdad.org, kaushal.modi@gmail.com, 33729@debbugs.gnu.org, > far.nasiri.m@gmail.com > > Mohammad, do you need help in comping up with a solution? There's > otf-script-alist (see fontest.el), but it goes in the opposite > direction. We could use rassq (Frassq in C) to find the OTF script > tag by its Emacs symbol (which is returned by indexing into > Vchar_script_table), by looking in otf-script-alist. > > Or maybe you prefer a separate data structure, not limited to the OTF > tags? After some thinking, my conclusion is that we should import the ISO 15924 database from https://unicode.org/iso15924/, use a script similar to admin/unidata/blocks.awk to generate an alist from it that maps Emacs script names to ISO 15924 tags, and then access that alist from uni_script to get the correct script information to Harfbuzz. Patches implementing that are welcome. From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 14 17:47:58 2018 Received: (at 33729) by debbugs.gnu.org; 14 Dec 2018 22:47:58 +0000 Received: from localhost ([127.0.0.1]:48643 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXwFR-0004Ht-MT for submit@debbugs.gnu.org; Fri, 14 Dec 2018 17:47:57 -0500 Received: from mail-wm1-f41.google.com ([209.85.128.41]:39020) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXwFP-0004Hg-Ix for 33729@debbugs.gnu.org; Fri, 14 Dec 2018 17:47:55 -0500 Received: by mail-wm1-f41.google.com with SMTP id f81so7221267wmd.4 for <33729@debbugs.gnu.org>; Fri, 14 Dec 2018 14:47:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=mDRiAB2kztbHc7Sm7QlEeC7GrjyvMYRKWiz5jLIFjJs=; b=Zn+doqxF6R/Awrsw02Y9qvFHgXVXOp2SqTbmtVWvTxrSzj+PwWcIPp9DFkQdo8axil eHCkJVkuSe/vsagrzF81HhIOUgMvjuzRQ2FRQi17x0Wj7KiNxIwpbtvqJ4VKQdN/CGYI QJjQRKbhkFUqFQbaMd4YXktYntyOG6iukicFiW2fHFMYdORFuNy1tnVfMqQicLcMIWC5 oSpZnK0h6KtFqNL30G/XZQc3e0JAlq/ZT6juKgAG+9/scsP+zh77qfJCd8UE1K39swSw QRfYzusSf/tQqlnuy2CfnYCp+lqM9oFyvwBv0RKz+HuvQ7sQnzpM8FqIFMv+EQ/XrUn4 iyqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=mDRiAB2kztbHc7Sm7QlEeC7GrjyvMYRKWiz5jLIFjJs=; b=ZWl5vU/NFqoSHC+V0HJalfAbpEfz3P/Mi0yZ719fhohXUrBxy/0sH0s16elVCGwqyl 2y38BsuIBY7zmdTWpWJxTcuoJHJc/vIdjGnslg3ptmd3EhI4Qgzs3IPCDe/1fstrQVtN 2F+qQYBvgFmtBT2LsVSDryZqG2V03QHX5EWbsFOZWkoqHyGxA1m4ZTvR725kuVncuIO9 Dnu5rml2flf/irceOebcklOrVuNrz66GdM52IjGODWPZMFLe1uycY1CVjzYH6FYq6Y9Y Roe1S7QjXzqmzdqdvTywQPYnInFclQZOsamhf5ugV6bSSzYa7jh1F2GHOfyZNHTM0NBs cH+Q== X-Gm-Message-State: AA+aEWYAu4pF+XKucM5TfTGV7jNJ8zpC+VL0UCz4SOZhRl9pYIMm2mi3 kw6SQ/36+tEe1Qsa5GYGUjo= X-Google-Smtp-Source: AFSGD/Vzj+x+GNDov4vJDBgdedfXMuNr9PmsWe9lwRML+d531Wt9gLPrh8xeine/7rVzqB3ux1STsw== X-Received: by 2002:a1c:a401:: with SMTP id n1mr4952069wme.114.1544827669746; Fri, 14 Dec 2018 14:47:49 -0800 (PST) Received: from macbook.localdomain ([197.58.120.254]) by smtp.gmail.com with ESMTPSA id c12sm6847240wrs.82.2018.12.14.14.47.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Dec 2018 14:47:48 -0800 (PST) Date: Sat, 15 Dec 2018 00:47:43 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181214224743.GL2244@macbook.localdomain> References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83y38sfcme.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Dec 14, 2018 at 03:42:49PM +0200, Eli Zaretskii wrote: > > Date: Fri, 14 Dec 2018 13:03:16 +0200 > > From: Khaled Hosny > > Cc: kaushal.modi@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, > > far.nasiri.m@gmail.com > > > > > FWIW, I looked at ftfont.c:uni_script, and I cannot find a problem > > > with it; in particular looking up in char-script-table each character > > > of the Gujarati welcome in HELLO yields 'gujarati', so I couldn't see > > > any evident Emacs issue. Or are you saying that hb_script_from_string > > > doesn't DTRT? Or maybe Kaushal should upgrade to a newer version of > > > HarfBuzz? > > > > There is this FIXME: > > > > /* FIXME: from_string wants an ISO 15924 script tag here. */ > > > > As we discussed earlier, hb_script_from_string() expects ISO 15924 > > script tags, but char_script_table does not provide such tags (I don’t > > recall what it does provide exactly). We need a way to get ISO 15924 > > script tags from Emacs. > > Right, I forgot about that. > > So you are saying that we need to generate Gujr instead of gujarati, > is that right? Yes (and the equivalent for all other scripts, of course). Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 16 09:48:14 2018 Received: (at 33729) by debbugs.gnu.org; 16 Dec 2018 14:48:14 +0000 Received: from localhost ([127.0.0.1]:49993 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYXiI-0004wO-Ew for submit@debbugs.gnu.org; Sun, 16 Dec 2018 09:48:14 -0500 Received: from odoacer.turtle-trading.net ([93.241.193.16]:37145) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYXiG-0004w9-GN for 33729@debbugs.gnu.org; Sun, 16 Dec 2018 09:48:13 -0500 Received: from justinian.turtle-trading.net ([192.168.2.118]) by odoacer.turtle-trading.net with esmtp (Exim 4.80) (envelope-from ) id 1gYXi1-0007Vm-B0; Sun, 16 Dec 2018 15:47:57 +0100 Received: from benny by justinian.turtle-trading.net with local (Exim 4.84_2) (envelope-from ) id 1gYXi1-0005NA-6V; Sun, 16 Dec 2018 15:47:57 +0100 From: Benjamin Riefenstahl To: Khaled Hosny Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> Date: Sun, 16 Dec 2018 15:47:56 +0100 In-Reply-To: <20181214110316.GK2244@macbook.localdomain> (Khaled Hosny's message of "Fri, 14 Dec 2018 13:03:16 +0200") Message-ID: <87y38pcyub.fsf@turtle-trading.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, Eli Zaretskii , 33729@debbugs.gnu.org, far.nasiri.m@gmail.com, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Khaled Hosny writes: > /* FIXME: from_string wants an ISO 15924 script tag here. */ > > As we discussed earlier, hb_script_from_string() expects ISO 15924 > script tags, but char_script_table does not provide such tags (I don=E2= =80=99t > recall what it does provide exactly). We need a way to get ISO 15924 > script tags from Emacs. The same mismatch also prevents Syriac text from actually shaping. Syriac shaping works in m17n with the required setup in composition-function-table and using the Meltho fonts. With Harfbuzz it doesn't work, unless I change "syriac" to "syrc" in charscript.el, just for testing of course. As a success story OTOH, Mandaic, using the Noto font, works OOTB ;-) benny From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 16 19:30:17 2018 Received: (at 33729) by debbugs.gnu.org; 17 Dec 2018 00:30:17 +0000 Received: from localhost ([127.0.0.1]:50698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYgnY-00073t-QY for submit@debbugs.gnu.org; Sun, 16 Dec 2018 19:30:17 -0500 Received: from eggs.gnu.org ([208.118.235.92]:35860) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYgnW-00073e-Dv for 33729@debbugs.gnu.org; Sun, 16 Dec 2018 19:30:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gYgnQ-00038r-B9 for 33729@debbugs.gnu.org; Sun, 16 Dec 2018 19:30:09 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52049) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYgnJ-0002vx-G7; Sun, 16 Dec 2018 19:30:01 -0500 Received: from rgm by fencepost.gnu.org with local (Exim 4.82) (envelope-from ) id 1gYgnI-0000qx-7A; Sun, 16 Dec 2018 19:30:00 -0500 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> X-Spook: H5N1 SBI Abu Sayyaf Cyber Command Emergency Relief X-Ran: Ca%JpP]j (Eli Zaretskii's message of "Fri, 14 Dec 2018 17:25:42 +0200") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Eli Zaretskii wrote: > After some thinking, my conclusion is that we should import the > ISO 15924 database from https://unicode.org/iso15924/, use a script > similar to admin/unidata/blocks.awk to generate an alist from it that > maps Emacs script names to ISO 15924 tags, and then access that alist > from uni_script to get the correct script information to Harfbuzz. > > Patches implementing that are welcome. I live to write awk scripts. I'm not 100% sure what you want, but as a first example, the following takes http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt as input and outputs lines of the form "(gujr . gujarati)". The aliases are so that the RHS matches charscript.el. If this is not right, please clarify exactly what the inputs and output should be. #!/usr/bin/awk -f function name2alias(name) { name = tolower(name) if (name ~ /arabic/) return "arabic" else if (name ~ /aramaic/) return "aramaic" else if (name ~ /cypriot/) return "cypriot-syllabary" else if (name ~ /katakana|hiragana/) return "kana" else if (name ~ /myanmar/) return "burmese" else if (name ~ /duployan|shorthand/) return "duployan-shorthand" else if (name ~ /signwriting/) return "sutton-sign-writing" sub(/^new_/, "", name) sub(/_(hieroglyphs|cursive)$/, "", name) gsub(/_/,"-",name) return name } $1 == "sc" { abbrev = tolower($3) alias = name2alias($5) if (alias ~ /^inherited|common|unknown/) next print "(" abbrev, ".", alias ")" } From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 17 10:56:04 2018 Received: (at 33729) by debbugs.gnu.org; 17 Dec 2018 15:56:04 +0000 Received: from localhost ([127.0.0.1]:52004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYvFT-00011K-Mr for submit@debbugs.gnu.org; Mon, 17 Dec 2018 10:56:04 -0500 Received: from eggs.gnu.org ([208.118.235.92]:60549) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gYvFR-00010o-OE for 33729@debbugs.gnu.org; Mon, 17 Dec 2018 10:56:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gYvFK-0005tV-NZ for 33729@debbugs.gnu.org; Mon, 17 Dec 2018 10:55:56 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38770) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYvFD-0005nl-GS; Mon, 17 Dec 2018 10:55:47 -0500 Received: from [176.228.60.248] (port=3863 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gYvFB-0005hs-UW; Mon, 17 Dec 2018 10:55:46 -0500 Date: Mon, 17 Dec 2018 17:55:52 +0200 Message-Id: <83mup4du5z.fsf@gnu.org> From: Eli Zaretskii To: Glenn Morris In-reply-to: (message from Glenn Morris on Sun, 16 Dec 2018 19:30:00 -0500) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Glenn Morris > Cc: far.nasiri.m@gmail.com, dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com > Date: Sun, 16 Dec 2018 19:30:00 -0500 > > > After some thinking, my conclusion is that we should import the > > ISO 15924 database from https://unicode.org/iso15924/, use a script > > similar to admin/unidata/blocks.awk to generate an alist from it that > > maps Emacs script names to ISO 15924 tags, and then access that alist > > from uni_script to get the correct script information to Harfbuzz. > > > > Patches implementing that are welcome. > > I live to write awk scripts. I'm not 100% sure what you want, but as a > first example, the following takes > http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt > as input and outputs lines of the form "(gujr . gujarati)". > > The aliases are so that the RHS matches charscript.el. > > If this is not right, please clarify exactly what the inputs and output > should be. Thanks. It turns out I didn't have this figured out completely, and your proposal forced me to dig some more into the relevant parts of Unicode and Emacs. I found a few additional issues and considerations; for at least some of them I'd like to hear the opinions of the Harfbuzz developers. Here are the issues: . Contrary to my original thoughts, I now tend to think that a separate char-table, say char-iso159240tag-table, that maps character codepoints directly to the script tags, is a better solution: - it will allow a faster look up, obviously - the subdivision of characters into scripts, as shown in Unicode's Scripts.txt, is slightly different from what char-script-table does, so a simple mapping from Emacs scripts to ISO 15924 script tag will not do. For example, many characters Emacs puts into 'latin' or 'symbol' scripts are in the Common script according to Scripts.txt, and similarly for the Inherited script. I imagine this is important for Harfbuzz. . Whether to produce the character-to-script-tag mapping using the UCD files, such as Scripts.txt and PropertyValueAliases.txt, or the canonical ISO 15924 tags from https://unicode.org/iso15924/, depends on whether the slight differences mentioned in https://www.unicode.org/reports/tr24/#Relation_To_ISO15924 matter for Harfbuzz. For example, ISO 15924 has separate tags for the Fraktur and Gaelic varieties of the Latin script: does this distinction matter for Harfbuzz? . Does Harfbuzz handle the issues mentioned in https://www.unicode.org/reports/tr24/#Script_Anomalies, and in particular the use case of decomposed characters which yield a different script than their precomposed variants? This use case is quite common in handling of character compositions, so it's important to understand its implications before we decide on the implementation. To summarize, unless the Harfbuzz guys advise differently, I'd prefer processing Scripts.txt and PropertyValueAliases.txt into a list similar to the one we produce in charscript.el, then generate a char-table from that list. Thanks again for working on this. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 20 13:58:21 2018 Received: (at 33729) by debbugs.gnu.org; 20 Dec 2018 18:58:21 +0000 Received: from localhost ([127.0.0.1]:56737 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ga3WX-0003nc-64 for submit@debbugs.gnu.org; Thu, 20 Dec 2018 13:58:21 -0500 Received: from eggs.gnu.org ([208.118.235.92]:57740) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ga3WU-0003nO-Tr for 33729@debbugs.gnu.org; Thu, 20 Dec 2018 13:58:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ga3WO-00013D-Jm for 33729@debbugs.gnu.org; Thu, 20 Dec 2018 13:58:13 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42745) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ga3WJ-0000yC-Mq; Thu, 20 Dec 2018 13:58:07 -0500 Received: from [176.228.60.248] (port=3934 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ga3WH-0000tp-Ox; Thu, 20 Dec 2018 13:58:07 -0500 Date: Thu, 20 Dec 2018 20:58:06 +0200 Message-Id: <8336qsc9fl.fsf@gnu.org> From: Eli Zaretskii To: dr.khaled.hosny@gmail.com, behdad@behdad.org,, far.nasiri.m@gmail.com In-reply-to: <83mup4du5z.fsf@gnu.org> (message from Eli Zaretskii on Mon, 17 Dec 2018 17:55:52 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: rgm@gnu.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Ping! Could someone on the Harfbuzz team please comment on the thoughts below? Khaled, Mohammad, Behdad? > Date: Mon, 17 Dec 2018 17:55:52 +0200 > From: Eli Zaretskii > Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, > far.nasiri.m@gmail.com, kaushal.modi@gmail.com > > > From: Glenn Morris > > Cc: far.nasiri.m@gmail.com, dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > Date: Sun, 16 Dec 2018 19:30:00 -0500 > > > > > After some thinking, my conclusion is that we should import the > > > ISO 15924 database from https://unicode.org/iso15924/, use a script > > > similar to admin/unidata/blocks.awk to generate an alist from it that > > > maps Emacs script names to ISO 15924 tags, and then access that alist > > > from uni_script to get the correct script information to Harfbuzz. > > > > > > Patches implementing that are welcome. > > > > I live to write awk scripts. I'm not 100% sure what you want, but as a > > first example, the following takes > > http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt > > as input and outputs lines of the form "(gujr . gujarati)". > > > > The aliases are so that the RHS matches charscript.el. > > > > If this is not right, please clarify exactly what the inputs and output > > should be. > > Thanks. > > It turns out I didn't have this figured out completely, and your > proposal forced me to dig some more into the relevant parts of Unicode > and Emacs. I found a few additional issues and considerations; for at > least some of them I'd like to hear the opinions of the Harfbuzz > developers. > > Here are the issues: > > . Contrary to my original thoughts, I now tend to think that a > separate char-table, say char-iso159240tag-table, that maps > character codepoints directly to the script tags, is a better > solution: > - it will allow a faster look up, obviously > - the subdivision of characters into scripts, as shown in > Unicode's Scripts.txt, is slightly different from what > char-script-table does, so a simple mapping from Emacs scripts > to ISO 15924 script tag will not do. For example, many > characters Emacs puts into 'latin' or 'symbol' scripts are in > the Common script according to Scripts.txt, and similarly for > the Inherited script. I imagine this is important for > Harfbuzz. > > . Whether to produce the character-to-script-tag mapping using the > UCD files, such as Scripts.txt and PropertyValueAliases.txt, or the > canonical ISO 15924 tags from https://unicode.org/iso15924/, > depends on whether the slight differences mentioned in > https://www.unicode.org/reports/tr24/#Relation_To_ISO15924 matter > for Harfbuzz. For example, ISO 15924 has separate tags for the > Fraktur and Gaelic varieties of the Latin script: does this > distinction matter for Harfbuzz? > > . Does Harfbuzz handle the issues mentioned in > https://www.unicode.org/reports/tr24/#Script_Anomalies, and in > particular the use case of decomposed characters which yield a > different script than their precomposed variants? This use case is > quite common in handling of character compositions, so it's > important to understand its implications before we decide on the > implementation. > > To summarize, unless the Harfbuzz guys advise differently, I'd prefer > processing Scripts.txt and PropertyValueAliases.txt into a list > similar to the one we produce in charscript.el, then generate a > char-table from that list. > > Thanks again for working on this. > > > > From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 20 15:46:12 2018 Received: (at 33729) by debbugs.gnu.org; 20 Dec 2018 20:46:12 +0000 Received: from localhost ([127.0.0.1]:56771 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ga5Ct-0001yE-Ub for submit@debbugs.gnu.org; Thu, 20 Dec 2018 15:46:12 -0500 Received: from mail-lj1-f171.google.com ([209.85.208.171]:36175) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ga5Cr-0001y0-29 for 33729@debbugs.gnu.org; Thu, 20 Dec 2018 15:46:09 -0500 Received: by mail-lj1-f171.google.com with SMTP id g11-v6so2804269ljk.3 for <33729@debbugs.gnu.org>; Thu, 20 Dec 2018 12:46:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qiztYbbWTjmkNotaTEfjLPxecDdh6Y8MJ72zjmNgeRE=; b=nEH9heIpZvf8cfrdXOZBRHEYORYRLMSRc7eACZ/DLs8N540Lzbbj5nZ4txqr3ueihp c28pYsGIccDLCYWxZiGy73pE2WrKbfRloWOKhSFWqPNiBmWIYRHC0PWzUfbXv6wHdmBx KPqJ4UwmAEmNLl6kcKHmsw2h47ZrkXLmdWXeW9q8L/5YT2PRoetv6dcOoQoP01pVa3j1 XQEDXSk/FrM2R/htoBuO9U4W0E4sJ9yflDwQgaSSH1CnRKiF8/2y+7D7JqY1w8p5ZBSz XkTd/s9jM6UEErcySpEtmGxH8c+m43K+rBEQavAKwxH7hHdIrUsYG1UTNO32H0vEPtZg DEDQ== X-Gm-Message-State: AA+aEWZoQ7HzKL6+J2ue/j+uZTQvQonSnVHff9HyIVLwnGosvz0Ua8lo KcqjciybTchixVTAk+80kqHsGZpvlv8Ldw4Lp+c= X-Google-Smtp-Source: AFSGD/WO7fqX0kO5W6YJY5W9DHMJBNd6iAnhQbvfdyNZuixlZh1fULf9t2jcDNaTogaRaokY/PVBn4JkDaTjfRjWQNI= X-Received: by 2002:a2e:1b47:: with SMTP id b68-v6mr13847877ljb.104.1545338762795; Thu, 20 Dec 2018 12:46:02 -0800 (PST) MIME-Version: 1.0 References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <8336qsc9fl.fsf@gnu.org> In-Reply-To: <8336qsc9fl.fsf@gnu.org> From: Behdad Esfahbod Date: Thu, 20 Dec 2018 15:45:50 -0500 Message-ID: Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: Eli Zaretskii Content-Type: multipart/alternative; boundary="000000000000039de7057d7a3814" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: Khaled Hosny , rgm@gnu.org, kaushal.modi@gmail.com, Mohammad Nasirifar , 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000039de7057d7a3814 Content-Type: text/plain; charset="UTF-8" Sounds good to me. On Thu, Dec 20, 2018 at 1:58 PM Eli Zaretskii wrote: > Ping! Could someone on the Harfbuzz team please comment on the > thoughts below? Khaled, Mohammad, Behdad? > > > Date: Mon, 17 Dec 2018 17:55:52 +0200 > > From: Eli Zaretskii > > Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, > > far.nasiri.m@gmail.com, kaushal.modi@gmail.com > > > > > From: Glenn Morris > > > Cc: far.nasiri.m@gmail.com, dr.khaled.hosny@gmail.com, > behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > Date: Sun, 16 Dec 2018 19:30:00 -0500 > > > > > > > After some thinking, my conclusion is that we should import the > > > > ISO 15924 database from https://unicode.org/iso15924/, use a script > > > > similar to admin/unidata/blocks.awk to generate an alist from it that > > > > maps Emacs script names to ISO 15924 tags, and then access that alist > > > > from uni_script to get the correct script information to Harfbuzz. > > > > > > > > Patches implementing that are welcome. > > > > > > I live to write awk scripts. I'm not 100% sure what you want, but as a > > > first example, the following takes > > > http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt > > > as input and outputs lines of the form "(gujr . gujarati)". > > > > > > The aliases are so that the RHS matches charscript.el. > > > > > > If this is not right, please clarify exactly what the inputs and output > > > should be. > > > > Thanks. > > > > It turns out I didn't have this figured out completely, and your > > proposal forced me to dig some more into the relevant parts of Unicode > > and Emacs. I found a few additional issues and considerations; for at > > least some of them I'd like to hear the opinions of the Harfbuzz > > developers. > > > > Here are the issues: > > > > . Contrary to my original thoughts, I now tend to think that a > > separate char-table, say char-iso159240tag-table, that maps > > character codepoints directly to the script tags, is a better > > solution: > > - it will allow a faster look up, obviously > > - the subdivision of characters into scripts, as shown in > > Unicode's Scripts.txt, is slightly different from what > > char-script-table does, so a simple mapping from Emacs scripts > > to ISO 15924 script tag will not do. For example, many > > characters Emacs puts into 'latin' or 'symbol' scripts are in > > the Common script according to Scripts.txt, and similarly for > > the Inherited script. I imagine this is important for > > Harfbuzz. > > > > . Whether to produce the character-to-script-tag mapping using the > > UCD files, such as Scripts.txt and PropertyValueAliases.txt, or the > > canonical ISO 15924 tags from https://unicode.org/iso15924/, > > depends on whether the slight differences mentioned in > > https://www.unicode.org/reports/tr24/#Relation_To_ISO15924 matter > > for Harfbuzz. For example, ISO 15924 has separate tags for the > > Fraktur and Gaelic varieties of the Latin script: does this > > distinction matter for Harfbuzz? > > > > . Does Harfbuzz handle the issues mentioned in > > https://www.unicode.org/reports/tr24/#Script_Anomalies, and in > > particular the use case of decomposed characters which yield a > > different script than their precomposed variants? This use case is > > quite common in handling of character compositions, so it's > > important to understand its implications before we decide on the > > implementation. > > > > To summarize, unless the Harfbuzz guys advise differently, I'd prefer > > processing Scripts.txt and PropertyValueAliases.txt into a list > > similar to the one we produce in charscript.el, then generate a > > char-table from that list. > > > > Thanks again for working on this. > > > > > > > > > -- behdad http://behdad.org/ --000000000000039de7057d7a3814 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Sounds good to me.

On Thu, Dec 20, 2018 at 1:58 PM Eli Zaretskii <eliz@gnu.org> wrote:
Ping!=C2=A0 Could someone on the Harfbuzz = team please comment on the
thoughts below?=C2=A0 Khaled, Mohammad, Behdad?

> Date: Mon, 17 Dec 2018 17:55:52 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: dr.= khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0far.nasiri.m@gmail.com, kaushal.modi@gmail.com
>
> > From: Glenn Morris <rgm@gnu.org>
> > Cc: f= ar.nasiri.m@gmail.com,=C2=A0 dr.khaled.hosny@gmail.com,=C2=A0 behdad@behdad.org,=C2=A0 33729@debbugs.gnu.org,=C2=A0 kaush= al.modi@gmail.com
> > Date: Sun, 16 Dec 2018 19:30:00 -0500
> >
> > > After some thinking, my conclusion is that we should import = the
> > > ISO 15924 database from https://unicode.org/iso15924/, use a script
> > > similar to admin/unidata/blocks.awk to generate an alist fro= m it that
> > > maps Emacs script names to ISO 15924 tags, and then access t= hat alist
> > > from uni_script to get the correct script information to Har= fbuzz.
> > >
> > > Patches implementing that are welcome.
> >
> > I live to write awk scripts. I'm not 100% sure what you want,= but as a
> > first example, the following takes
> > http://www.unicode.or= g/Public/UCD/latest/ucd/PropertyValueAliases.txt
> > as input and outputs lines of the form "(gujr . gujarati)&qu= ot;.
> >
> > The aliases are so that the RHS matches charscript.el.
> >
> > If this is not right, please clarify exactly what the inputs and = output
> > should be.
>
> Thanks.
>
> It turns out I didn't have this figured out completely, and your > proposal forced me to dig some more into the relevant parts of Unicode=
> and Emacs.=C2=A0 I found a few additional issues and considerations; f= or at
> least some of them I'd like to hear the opinions of the Harfbuzz > developers.
>
> Here are the issues:
>
>=C2=A0 . Contrary to my original thoughts, I now tend to think that a >=C2=A0 =C2=A0 separate char-table, say char-iso159240tag-table, that ma= ps
>=C2=A0 =C2=A0 character codepoints directly to the script tags, is a be= tter
>=C2=A0 =C2=A0 solution:
>=C2=A0 =C2=A0 =C2=A0- it will allow a faster look up, obviously
>=C2=A0 =C2=A0 =C2=A0- the subdivision of characters into scripts, as sh= own in
>=C2=A0 =C2=A0 =C2=A0 =C2=A0Unicode's Scripts.txt, is slightly diffe= rent from what
>=C2=A0 =C2=A0 =C2=A0 =C2=A0char-script-table does, so a simple mapping = from Emacs scripts
>=C2=A0 =C2=A0 =C2=A0 =C2=A0to ISO 15924 script tag will not do.=C2=A0 F= or example, many
>=C2=A0 =C2=A0 =C2=A0 =C2=A0characters Emacs puts into 'latin' o= r 'symbol' scripts are in
>=C2=A0 =C2=A0 =C2=A0 =C2=A0the Common script according to Scripts.txt, = and similarly for
>=C2=A0 =C2=A0 =C2=A0 =C2=A0the Inherited script.=C2=A0 I imagine this i= s important for
>=C2=A0 =C2=A0 =C2=A0 =C2=A0Harfbuzz.
>
>=C2=A0 . Whether to produce the character-to-script-tag mapping using t= he
>=C2=A0 =C2=A0 UCD files, such as Scripts.txt and PropertyValueAliases.t= xt, or the
>=C2=A0 =C2=A0 canonical ISO 15924 tags from https://unicode.org/iso= 15924/,
>=C2=A0 =C2=A0 depends on whether the slight differences mentioned in >=C2=A0 =C2=A0 https://www.unicode.org/= reports/tr24/#Relation_To_ISO15924 matter
>=C2=A0 =C2=A0 for Harfbuzz.=C2=A0 For example, ISO 15924 has separate t= ags for the
>=C2=A0 =C2=A0 Fraktur and Gaelic varieties of the Latin script: does th= is
>=C2=A0 =C2=A0 distinction matter for Harfbuzz?
>
>=C2=A0 . Does Harfbuzz handle the issues mentioned in
>=C2=A0 =C2=A0 https://www.unicode.org/repo= rts/tr24/#Script_Anomalies, and in
>=C2=A0 =C2=A0 particular the use case of decomposed characters which yi= eld a
>=C2=A0 =C2=A0 different script than their precomposed variants?=C2=A0 T= his use case is
>=C2=A0 =C2=A0 quite common in handling of character compositions, so it= 's
>=C2=A0 =C2=A0 important to understand its implications before we decide= on the
>=C2=A0 =C2=A0 implementation.
>
> To summarize, unless the Harfbuzz guys advise differently, I'd pre= fer
> processing Scripts.txt and PropertyValueAliases.txt into a list
> similar to the one we produce in charscript.el, then generate a
> char-table from that list.
>
> Thanks again for working on this.
>
>
>
>


--
behdad
http://behdad.org/
--000000000000039de7057d7a3814-- From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 03:55:01 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 08:55:01 +0000 Received: from localhost ([127.0.0.1]:59115 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gad3l-00031p-BA for submit@debbugs.gnu.org; Sat, 22 Dec 2018 03:55:01 -0500 Received: from mail-wr1-f43.google.com ([209.85.221.43]:43291) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gad3j-00031b-VR for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 03:55:00 -0500 Received: by mail-wr1-f43.google.com with SMTP id r10so7419539wrs.10 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 00:54:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=rliGVy5bZ70psuVdS/D/r1p74GeUq82RaemItH03TYI=; b=YuEzu+sIfleHp5GNHEqHqh642SjR+XmFLP1PHOHmtt8qhVZf1Fl7jZoo85KYbaPuks ZKsXKSXtLqX56dWyFH7/ycmiAZDNqN5ostkOjpMQKY0XZTApfj5QkFKy65arp8SFZ0jw QgInCNBWGSSVOVnI1FeEFu4vvQRnYqSGkIBjM6G6dN8P7ODAiG6lr2Y6CUTqgWZJp/0l odw/N31zfWasGj2v13bcAueI1u4dbNq5Myi7deY2TBddA2dpO5fWpb0OZbSJhUdBDxLy 8SmvO+PKykfNxFwKAH+WMPOsTmiWR5n607C5n30wjHdPgN8GfWQQ7Xk0HBTqH3E4LHiL oPWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=rliGVy5bZ70psuVdS/D/r1p74GeUq82RaemItH03TYI=; b=CMJnk0wMlrjd+ntx3B1fLHwXXstWMDitJ7N3RG3qRBtDcv99PXQeRYNItiXUSxMZkx dVEYAEjTelC6LCwSvOFbb0YROyKTZAxzuWmj0LCuuW0Ahv4SFZNLXWqK9UiCxnBuhMRi 7bd81eloqfpkGBE7KjLt3kIpnFgi2LFRVTYbpN7lE7hF16dIcbSpt3hoYgY/TcNvqysL ubqE9opRDCQfA0vOTVp9TR0xthWiIvtXW6EDSPTeqm5WpNxHPrKfIrFjEs8VSpALQ75j aZl2Kxy8uk61qUp+NT5XrLGPokQiI7YN7C5qDFrp1IHqIbQqTarUWRuj49Y+lFSFiCNP KJuQ== X-Gm-Message-State: AJcUukc6XPwtRuPW55qAhC74oH4ZN9cw3lywIJprOY4vYR5HMkc8Zsgz n5gUIIf8gsfa+f/esSPNn1o= X-Google-Smtp-Source: ALg8bN5R62WqAdCQ89223V0P/hfCbZBZSsyZ5DSOFOOmb9oHtu8seaovPZF+aEaLPEfFTPB+cvU+pg== X-Received: by 2002:adf:c452:: with SMTP id a18mr5644838wrg.145.1545468894122; Sat, 22 Dec 2018 00:54:54 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id c77sm16246250wmh.12.2018.12.22.00.54.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 00:54:52 -0800 (PST) Date: Sat, 22 Dec 2018 10:54:48 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222085448.GA2244@macbook.localdomain> References: <20181213203102.GF2244@macbook.localdomain> <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83mup4du5z.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, Glenn Morris , kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, Dec 17, 2018 at 05:55:52PM +0200, Eli Zaretskii wrote: > > From: Glenn Morris > > Cc: far.nasiri.m@gmail.com, dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > Date: Sun, 16 Dec 2018 19:30:00 -0500 > > > > > After some thinking, my conclusion is that we should import the > > > ISO 15924 database from https://unicode.org/iso15924/, use a script > > > similar to admin/unidata/blocks.awk to generate an alist from it that > > > maps Emacs script names to ISO 15924 tags, and then access that alist > > > from uni_script to get the correct script information to Harfbuzz. > > > > > > Patches implementing that are welcome. > > > > I live to write awk scripts. I'm not 100% sure what you want, but as a > > first example, the following takes > > http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt > > as input and outputs lines of the form "(gujr . gujarati)". > > > > The aliases are so that the RHS matches charscript.el. > > > > If this is not right, please clarify exactly what the inputs and output > > should be. > > Thanks. > > It turns out I didn't have this figured out completely, and your > proposal forced me to dig some more into the relevant parts of Unicode > and Emacs. I found a few additional issues and considerations; for at > least some of them I'd like to hear the opinions of the Harfbuzz > developers. > > Here are the issues: > > . Contrary to my original thoughts, I now tend to think that a > separate char-table, say char-iso159240tag-table, that maps > character codepoints directly to the script tags, is a better > solution: > - it will allow a faster look up, obviously > - the subdivision of characters into scripts, as shown in > Unicode's Scripts.txt, is slightly different from what > char-script-table does, so a simple mapping from Emacs scripts > to ISO 15924 script tag will not do. For example, many > characters Emacs puts into 'latin' or 'symbol' scripts are in > the Common script according to Scripts.txt, and similarly for > the Inherited script. I imagine this is important for > Harfbuzz. Alternatively, we could just use HarfBuzz’s own built in ucdn-based Unicode function for this. The only reason for overriding this in Emacs was to keep HarfBuzz and Emacs Unicode support in sync, but if we are going to duplicate the Unicode script data then better use what HarfBuzz has. I’m going to try this now. > . Whether to produce the character-to-script-tag mapping using the > UCD files, such as Scripts.txt and PropertyValueAliases.txt, or the > canonical ISO 15924 tags from https://unicode.org/iso15924/, > depends on whether the slight differences mentioned in > https://www.unicode.org/reports/tr24/#Relation_To_ISO15924 matter > for Harfbuzz. For example, ISO 15924 has separate tags for the > Fraktur and Gaelic varieties of the Latin script: does this > distinction matter for Harfbuzz? We want the UCD data. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 04:06:58 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 09:06:58 +0000 Received: from localhost ([127.0.0.1]:59123 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gadFJ-0003Kt-O2 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 04:06:58 -0500 Received: from mail-wm1-f54.google.com ([209.85.128.54]:32970) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gadFH-0003Kg-4V for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 04:06:55 -0500 Received: by mail-wm1-f54.google.com with SMTP id r24so15772214wmh.0 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 01:06:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=qcVpSqdJ7le6hJGh1xq2XYove73A1hkwlqmQilswBlg=; b=IqdSXy7NYYfTJf6JVpXXQfMzClq47agW0S6D4RGKHraKF2hJwLXX8cBqUyhrmVRJXx eQKerx6JYZ6yZHk8HSBAYxS5XS5mEwmsmZsdAtJNa/URJE2NiqatQeB90AdBLxlOWPIy S6ScewS6zuF3VNOpnlEvxeviqQZqTSzoML//x+iap0YSX2hewAJLjeK8g3cKFEpT1t/V YtnQ9BR+pPIAAVuWj9mtr0CWdzz15nHb0Y9+Iz3i1cV1bi3z4yb3mmpNhwZp9KOQdRge pUtcRMyZ/Is6dJUI326kPklBzqb+GR4CRGE8Ox/nxsBhZIwrbCZCuYwp+sdO2j9koyf1 BMAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=qcVpSqdJ7le6hJGh1xq2XYove73A1hkwlqmQilswBlg=; b=t8VEV0Obds3tVm6xW2CMUIm0Fc76rozaMVkGnpw6SgcoHdW2VhtD+Wm/Y/7KtNBeub aFVBfi37aAm7WJBp3Kf+gGoQ5RqBT99BUKHFg1BvnaalAm0GVyaOScpodSJFuNbcI7Kp 3EY+38ptRevGGKJ4cruLaPhHZSpkYt2WstD+3quXGi8p+wpOn8TlDw5yFxn03W2Fbjdn 972nrEdFHJBEUk0sKt19zs7yO5lkPF36Q1/FbjNfi7xRKSh+hZxOq5Q8u7gCU6syRbKX BCdeGCyP1jzfvA+MyauHUbc9e1xgKNEYfyG8OlkXp7k31DB/gdPPy0lLZ1yZSNH5uPOC iVww== X-Gm-Message-State: AA+aEWYPAy/HGtTC+YtA8y87aBgIs8cUGwVJct7wpfN+DVnr1pa1HPUx tEGV+CZi7+Po9ti4C0kq/EY= X-Google-Smtp-Source: ALg8bN7NCM1gVRooZKeSJbI1hoZxNuxkcFb/pev7neADnmX+XmJ+bFAEZeAaoqZH/gDK8Pjyv7XFnA== X-Received: by 2002:a1c:2787:: with SMTP id n129mr5997486wmn.128.1545469609316; Sat, 22 Dec 2018 01:06:49 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id f66sm16252507wmd.28.2018.12.22.01.06.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 01:06:48 -0800 (PST) Date: Sat, 22 Dec 2018 11:06:44 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222090644.GB2244@macbook.localdomain> References: <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20181222085448.GA2244@macbook.localdomain> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, Glenn Morris , kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Dec 22, 2018 at 10:54:48AM +0200, Khaled Hosny wrote: > On Mon, Dec 17, 2018 at 05:55:52PM +0200, Eli Zaretskii wrote: > > > From: Glenn Morris > > > Cc: far.nasiri.m@gmail.com, dr.khaled.hosny@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > Date: Sun, 16 Dec 2018 19:30:00 -0500 > > > > > > > After some thinking, my conclusion is that we should import the > > > > ISO 15924 database from https://unicode.org/iso15924/, use a script > > > > similar to admin/unidata/blocks.awk to generate an alist from it that > > > > maps Emacs script names to ISO 15924 tags, and then access that alist > > > > from uni_script to get the correct script information to Harfbuzz. > > > > > > > > Patches implementing that are welcome. > > > > > > I live to write awk scripts. I'm not 100% sure what you want, but as a > > > first example, the following takes > > > http://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt > > > as input and outputs lines of the form "(gujr . gujarati)". > > > > > > The aliases are so that the RHS matches charscript.el. > > > > > > If this is not right, please clarify exactly what the inputs and output > > > should be. > > > > Thanks. > > > > It turns out I didn't have this figured out completely, and your > > proposal forced me to dig some more into the relevant parts of Unicode > > and Emacs. I found a few additional issues and considerations; for at > > least some of them I'd like to hear the opinions of the Harfbuzz > > developers. > > > > Here are the issues: > > > > . Contrary to my original thoughts, I now tend to think that a > > separate char-table, say char-iso159240tag-table, that maps > > character codepoints directly to the script tags, is a better > > solution: > > - it will allow a faster look up, obviously > > - the subdivision of characters into scripts, as shown in > > Unicode's Scripts.txt, is slightly different from what > > char-script-table does, so a simple mapping from Emacs scripts > > to ISO 15924 script tag will not do. For example, many > > characters Emacs puts into 'latin' or 'symbol' scripts are in > > the Common script according to Scripts.txt, and similarly for > > the Inherited script. I imagine this is important for > > Harfbuzz. > > Alternatively, we could just use HarfBuzz’s own built in ucdn-based > Unicode function for this. The only reason for overriding this in Emacs > was to keep HarfBuzz and Emacs Unicode support in sync, but if we are > going to duplicate the Unicode script data then better use what HarfBuzz > has. > > I’m going to try this now. I pushed a commit to harfbuzz branch that I think fixes this issue now. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 05:11:44 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 10:11:44 +0000 Received: from localhost ([127.0.0.1]:59169 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaeFz-0006vp-Rp for submit@debbugs.gnu.org; Sat, 22 Dec 2018 05:11:44 -0500 Received: from eggs.gnu.org ([208.118.235.92]:35315) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaeFy-0006vd-JY for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 05:11:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gaeFs-0000O9-P7 for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 05:11:37 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36070) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gaeFn-0000IT-Gh; Sat, 22 Dec 2018 05:11:31 -0500 Received: from [176.228.60.248] (port=2954 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gaeFn-0003NC-2Y; Sat, 22 Dec 2018 05:11:31 -0500 Date: Sat, 22 Dec 2018 12:11:15 +0200 Message-Id: <83va3lan24.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181222090644.GB2244@macbook.localdomain> (message from Khaled Hosny on Sat, 22 Dec 2018 11:06:44 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 22 Dec 2018 11:06:44 +0200 > From: Khaled Hosny > Cc: Glenn Morris , far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > Alternatively, we could just use HarfBuzz’s own built in ucdn-based > > Unicode function for this. The only reason for overriding this in Emacs > > was to keep HarfBuzz and Emacs Unicode support in sync, but if we are > > going to duplicate the Unicode script data then better use what HarfBuzz > > has. > > > > I’m going to try this now. > > I pushed a commit to harfbuzz branch that I think fixes this issue now. Thanks. There's a FIXME in the change you pushed (which I believe just repeats what was already in the previous version). Ca you tell more about the problem we need to fix there? From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 10:15:22 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 15:15:22 +0000 Received: from localhost ([127.0.0.1]:60129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaizq-0007y4-39 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 10:15:22 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:55774) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaizo-0007xn-LN for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:15:21 -0500 Received: by mail-wm1-f46.google.com with SMTP id y139so7788077wmc.5 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 07:15:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=FzSop95AfBQMCxG+TGKBaRMY54mqX2F9//fu2gbW0OE=; b=jSnaPYC6GXjyBkU1t2nOvCO68GY5ToFmKnk6ZZ1Hew1uFjCB/0Z/uLgjM3hW6mKszM /QZogQ40lRTAMOEDk/6uwqZm/NQQAcFFI7TekYnB82lsTFi2YWHqvq3QNZEpQTRTuXWl JTaV4y+H4PpX7WU9lcOXd7QAWRpMZd0Q90AtRMGwPtt74+eLI9Xh2jn+h9iq3VQCJjWW cmqQ9eQhxGhkYy5QmL15OjRNiPNFqijnNVy0+4cjAElOxNLr4+dn811DcSvTjcVBBpAG atFRP6ioklZTs7XzAgxZEggZebgr7ICxWtEiw0prHE7vCV1go6nojz93TgUSWj6+Ayec C53A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=FzSop95AfBQMCxG+TGKBaRMY54mqX2F9//fu2gbW0OE=; b=LTlkO2+dp+8H21jHO9RKSvV4Fucoalfaac3svW4XTAinFQQzTcdx4XdntbmmssOx5Z 0ye5bE1URyq6NOEr7M8iDkPtFgPtCeL4eqS0nzP/6u58D+X7Y0yPPMK5kEX6jmtf4NQJ xU4CVDyO660ctesAbR29I/uFyCbDDcA869Vl9Y+fpMlaqglOQB3BumCULtdngs489JlF QeoaUXKjy4W+g9eeFilZDpiqgR1ERC5Ukv4ktiN2U9+OO8TGd6k5hxwUrKhX4q1Qrz9R As+ACneEPLBnjgDjgQSUnfFls2I3Xji6dSZA1jTFQnbEZze6dgXdrKSdOANPeg+2MB8M XHLw== X-Gm-Message-State: AA+aEWa8Ac8tdxgSGmtCaKXOG1VviRcYPwqMxlFBS8kyahbaiZTdZR9c EViq/TYFbS7un5ZouIO6uUw= X-Google-Smtp-Source: AFSGD/XCqMPRmxTkUgGXu9TJu+t9b7UCcB/QkNUnVx4T4t8QkZw6EqrsEAlh2GMcNUX90vxUZNhHGg== X-Received: by 2002:a1c:6508:: with SMTP id z8mr6800796wmb.28.1545491714902; Sat, 22 Dec 2018 07:15:14 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id z12sm12993079wrh.35.2018.12.22.07.15.12 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 07:15:13 -0800 (PST) Date: Sat, 22 Dec 2018 17:15:09 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222151509.GC2244@macbook.localdomain> References: <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83va3lan24.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Dec 22, 2018 at 12:11:15PM +0200, Eli Zaretskii wrote: > > Date: Sat, 22 Dec 2018 11:06:44 +0200 > > From: Khaled Hosny > > Cc: Glenn Morris , far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > Alternatively, we could just use HarfBuzz’s own built in ucdn-based > > > Unicode function for this. The only reason for overriding this in Emacs > > > was to keep HarfBuzz and Emacs Unicode support in sync, but if we are > > > going to duplicate the Unicode script data then better use what HarfBuzz > > > has. > > > > > > I’m going to try this now. > > > > I pushed a commit to harfbuzz branch that I think fixes this issue now. > > Thanks. > > There's a FIXME in the change you pushed (which I believe just > repeats what was already in the previous version). Ca you tell more > about the problem we need to fix there? We need a way to get Unicode composition and decomposition for the a given character (implementing the uni_compose and uni_decompose functions I deleted). I recall you suggested something earlier that I tried but couldn’t get to work, the exact detail escapes me. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 10:27:28 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 15:27:28 +0000 Received: from localhost ([127.0.0.1]:60137 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajBX-0008Hg-UC for submit@debbugs.gnu.org; Sat, 22 Dec 2018 10:27:28 -0500 Received: from mail-lf1-f54.google.com ([209.85.167.54]:41951) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajBW-0008HN-Ie for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:27:27 -0500 Received: by mail-lf1-f54.google.com with SMTP id c16so5887986lfj.8 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 07:27:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gvhgE1f2i0MXn9vr1uuf2mM1ML0wOw3R/mhcd4CaT/I=; b=j0Hw+t1l4MJ2yGGcBy5phKB7UEAS+4eh29x0xZJS/8idDrXpeGN3mGY6dY7CQk48Jp 8vz+Z3wjrjMH6Vwxw52UaEK4oqaCzukWB8sMEzltviZhPvAap2bemWJ7wvv3+PcP3XHN ggF35v2dt0rUzZXgFrFX78K/Xor03x6RAwo8+zmEq/E6T9rXKiF+Z14IHVVF83u2f5VO /govw4+Mr5AKZclqfUe8Qw9xbZVFxinAQHUST/h2iO6iRkiSMlJvJok1WZkQXakxZFEa qGnDel4fK7rmYEuWIguZZAHu4AjtlL2AQQ/N36ddl6jBKlAmh/2082j585VQjO23XR+q oerg== X-Gm-Message-State: AA+aEWYGd5cHIC0Vb5Cgqz2xF8wBpQNW5oKPKgPCCyfckTchptFsebqS dHKQX4kT3xcaC63nQB/lEWye1CBRi0FP2kLRiiw= X-Google-Smtp-Source: AFSGD/XzRZsTFCF6zIIpYsSysfcPgBHBH5iL/rprmIND9DMYTnQYRmziRxUoitr4bHoN6lGxnzWd1r6WESjhGJAFAAQ= X-Received: by 2002:a19:9fcd:: with SMTP id i196mr3661715lfe.82.1545492438962; Sat, 22 Dec 2018 07:27:18 -0800 (PST) MIME-Version: 1.0 References: <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> In-Reply-To: <20181222151509.GC2244@macbook.localdomain> From: Behdad Esfahbod Date: Sat, 22 Dec 2018 10:27:06 -0500 Message-ID: Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: Khaled Hosny Content-Type: multipart/alternative; boundary="000000000000d3d646057d9dffc4" X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 33729 Cc: rgm@gnu.org, Eli Zaretskii , Kaushal Modi , Mohammad Nasirifar , 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --000000000000d3d646057d9dffc4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I suggest you enabled UCDN. On Sat, Dec 22, 2018 at 10:15 AM Khaled Hosny wrote: > On Sat, Dec 22, 2018 at 12:11:15PM +0200, Eli Zaretskii wrote: > > > Date: Sat, 22 Dec 2018 11:06:44 +0200 > > > From: Khaled Hosny > > > Cc: Glenn Morris , far.nasiri.m@gmail.com, > behdad@behdad.org, > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > > Alternatively, we could just use HarfBuzz=E2=80=99s own built in uc= dn-based > > > > Unicode function for this. The only reason for overriding this in > Emacs > > > > was to keep HarfBuzz and Emacs Unicode support in sync, but if we a= re > > > > going to duplicate the Unicode script data then better use what > HarfBuzz > > > > has. > > > > > > > > I=E2=80=99m going to try this now. > > > > > > I pushed a commit to harfbuzz branch that I think fixes this issue no= w. > > > > Thanks. > > > > There's a FIXME in the change you pushed (which I believe just > > repeats what was already in the previous version). Ca you tell more > > about the problem we need to fix there? > > We need a way to get Unicode composition and decomposition for the > a given character (implementing the uni_compose and uni_decompose > functions I deleted). I recall you suggested something earlier that I > tried but couldn=E2=80=99t get to work, the exact detail escapes me. > --=20 behdad http://behdad.org/ --000000000000d3d646057d9dffc4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I suggest you enabled UCDN.

On Sat, Dec 22, 2018 at 10:15 AM Khaled Hosny <dr.khaled.hosny@gmail.com>= ; wrote:
On Sat,= Dec 22, 2018 at 12:11:15PM +0200, Eli Zaretskii wrote:
> > Date: Sat, 22 Dec 2018 11:06:44 +0200
> > From: Khaled Hosny <dr.khaled.hosny@gmail.com>
> > Cc: Glenn Morris <rgm@gnu.org>, far.nasiri.m@gmail.com, behdad@behdad.org,
> >=C2=A0 =C2=A0 =C2=A033729@debbugs.gnu.org, kaushal.modi@gmail.com
> >
> > > Alternatively, we could just use HarfBuzz=E2=80=99s own buil= t in ucdn-based
> > > Unicode function for this. The only reason for overriding th= is in Emacs
> > > was to keep HarfBuzz and Emacs Unicode support in sync, but = if we are
> > > going to duplicate the Unicode script data then better use w= hat HarfBuzz
> > > has.
> > >
> > > I=E2=80=99m going to try this now.
> >
> > I pushed a commit to harfbuzz branch that I think fixes this issu= e now.
>
> Thanks.
>
> There's a FIXME in the change you pushed (which I believe just
> repeats what was already in the previous version).=C2=A0 Ca you tell m= ore
> about the problem we need to fix there?

We need a way to get Unicode composition and decomposition for the
a given character (implementing the uni_compose and uni_decompose
functions I deleted). I recall you suggested something earlier that I
tried but couldn=E2=80=99t get to work, the exact detail escapes me.


--
behdad
http://behdad.org/
--000000000000d3d646057d9dffc4-- From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 10:43:02 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 15:43:02 +0000 Received: from localhost ([127.0.0.1]:60142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajQa-0000Eg-E6 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 10:43:02 -0500 Received: from mail-wm1-f54.google.com ([209.85.128.54]:33402) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajQY-0000ER-1n for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:42:58 -0500 Received: by mail-wm1-f54.google.com with SMTP id r24so16261713wmh.0 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 07:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=nJnHhLFuY382kjGKA/qEXxBl/HR+fJyjYgQk1K1NYDM=; b=AAGZnlh6xYpwoTyIYZxSQvqC0RFMRzYRAWccz4HwCeQVchCoA19GNyoeUjRGTqgfhF TYV3fNxnE1T4g0EMs1e7Dt/82EA3fPW6TBQGCXZPouvQ97QthBJBrt5xcMdBfu15DeSk go3Vno8v3xAAIHolH5yXnbjgTjKrNP/hCEFSE5RQuANuEqAKJJN02Jd9hwdZp1l6CLWB nbykLnLcTjaxFpRMOQ6ahA3H7vIYTMLkpvx+AR6CR+w6Wt/uhJurvqJmKNIZeLB8dp6o cwnn4VufxDbiaTk3jkGdT78bwZHS7L4qsM4AdkrjXUoe6W2jhmerJcKxFg/gxMA1k4dL ZusA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=nJnHhLFuY382kjGKA/qEXxBl/HR+fJyjYgQk1K1NYDM=; b=BCWDoDYLUe9vDme/N0PR8E0ZCYnbMovlod3yMPnN4Z4WNzdXP3w+VI+jK//MeiApGZ Sklaqlf5Gtgi0V+GDjbeFwJ+kuOYBeSV0jUK7J8dFnrEtB/NQg1FVjDZ8Rz4BYqDIhTT ka1yT/ZG2gTHvjG8cWDKrzuDWxXgoxYeYhDdw1CMAGO1F/t3h406oPlJOGAAOwZicnro rNpmSKU1eA0WLQGi9RBfnXCdzwnxsyQQ5qwIVdaH9LqqiD/LyPeltWDnDAzPwPoAVhqU MiBwTNFpatARTmbOp8zCUsR73DEh6CyEnmrqPl2cCsRU8efFfRH5TXyZZARMKUDnvEnl qWQQ== X-Gm-Message-State: AJcUukf4dWlIWPK93BogbQMPIox1NKnZxMLnAJrxZ0Ti1ierSk/Txtd1 zNeBmrZ5Fya9/cIc+5F2zSY= X-Google-Smtp-Source: ALg8bN5N2+RsLJH7rj/jK/z6DM/PBYiFt/FInkHcaIxFW7tiYmF/QKPHJcbMdpCANpCEbGCA4J9nvA== X-Received: by 2002:a1c:5dce:: with SMTP id r197mr6363901wmb.130.1545493372134; Sat, 22 Dec 2018 07:42:52 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id t5sm13026704wmg.43.2018.12.22.07.42.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 07:42:51 -0800 (PST) Date: Sat, 22 Dec 2018 17:42:47 +0200 From: Khaled Hosny To: Behdad Esfahbod Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222154247.GD2244@macbook.localdomain> References: <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: rgm@gnu.org, Eli Zaretskii , Kaushal Modi , Mohammad Nasirifar , 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I’m sub-classing the default Unicode functions, so for the callback we don’t implement the default implementation will be used already. On Sat, Dec 22, 2018 at 10:27:06AM -0500, Behdad Esfahbod wrote: > I suggest you enabled UCDN. > > On Sat, Dec 22, 2018 at 10:15 AM Khaled Hosny > wrote: > > > On Sat, Dec 22, 2018 at 12:11:15PM +0200, Eli Zaretskii wrote: > > > > Date: Sat, 22 Dec 2018 11:06:44 +0200 > > > > From: Khaled Hosny > > > > Cc: Glenn Morris , far.nasiri.m@gmail.com, > > behdad@behdad.org, > > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > > > > Alternatively, we could just use HarfBuzz’s own built in ucdn-based > > > > > Unicode function for this. The only reason for overriding this in > > Emacs > > > > > was to keep HarfBuzz and Emacs Unicode support in sync, but if we are > > > > > going to duplicate the Unicode script data then better use what > > HarfBuzz > > > > > has. > > > > > > > > > > I’m going to try this now. > > > > > > > > I pushed a commit to harfbuzz branch that I think fixes this issue now. > > > > > > Thanks. > > > > > > There's a FIXME in the change you pushed (which I believe just > > > repeats what was already in the previous version). Ca you tell more > > > about the problem we need to fix there? > > > > We need a way to get Unicode composition and decomposition for the > > a given character (implementing the uni_compose and uni_decompose > > functions I deleted). I recall you suggested something earlier that I > > tried but couldn’t get to work, the exact detail escapes me. > > > > > -- > behdad > http://behdad.org/ From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 10:43:15 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 15:43:15 +0000 Received: from localhost ([127.0.0.1]:60145 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajQo-0000FQ-T4 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 10:43:15 -0500 Received: from eggs.gnu.org ([208.118.235.92]:32841) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajQm-0000FB-P5 for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:43:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gajQg-0006q5-Qu for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:43:07 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42125) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gajQd-0006eN-Lk; Sat, 22 Dec 2018 10:43:03 -0500 Received: from [176.228.60.248] (port=4079 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gajQc-0007EU-GN; Sat, 22 Dec 2018 10:43:03 -0500 Date: Sat, 22 Dec 2018 17:42:43 +0200 Message-Id: <83h8f5a7po.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181222151509.GC2244@macbook.localdomain> (message from Khaled Hosny on Sat, 22 Dec 2018 17:15:09 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 22 Dec 2018 17:15:09 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > There's a FIXME in the change you pushed (which I believe just > > repeats what was already in the previous version). Ca you tell more > > about the problem we need to fix there? > > We need a way to get Unicode composition and decomposition for the > a given character (implementing the uni_compose and uni_decompose > functions I deleted). Yes, but what does that entail? Are these compositions and decompositions defined by the Unicode UCD? And how does Harfbuzz use the results for a given character? > I recall you suggested something earlier that I tried but couldn’t > get to work, the exact detail escapes me. I probably suggested using the 'decomposition' property of a character, and perhaps als the facilities in ucs-normalize.el. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 10:49:57 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 15:49:57 +0000 Received: from localhost ([127.0.0.1]:60154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajXJ-0000Qg-1h for submit@debbugs.gnu.org; Sat, 22 Dec 2018 10:49:57 -0500 Received: from mail-wr1-f48.google.com ([209.85.221.48]:46746) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gajXH-0000QR-RT for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 10:49:56 -0500 Received: by mail-wr1-f48.google.com with SMTP id l9so8013838wrt.13 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 07:49:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=1K5JLOIUUfWCkBSSly4IU8p75v2teXxSFhcXrPVjtLE=; b=t/PSPscNvcxKVjepv6zx98h+xvIOGImei/68MZ23eeJVJjy5S3OrkL6GKDt03PuYNy rNtEqPnxIA4Fq41JYa4V3R1+kNEpWO/NYLQZXQXh8E+z7bVafqMpEeRrOTjOiacvuWIt Kls2WN992c5iuluH0q/g34/ddhC0i36/CnnRAEGgrUQqHPvxsidThU6pMSeBKWstz1Vs KIZjHK4ZohpxxRjHz87kDv4CQe8NyB4Ve8Z4NJX9d5AyAE8izr6r0f6VIqzWPtVoNeQ5 JNdHDE5pk8HIXyp1s7TSQrHQBvsSJY3idj+4Wh4C4dDMp7KH9H6ODLnESpCi4MqkACyM C9Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=1K5JLOIUUfWCkBSSly4IU8p75v2teXxSFhcXrPVjtLE=; b=APr/MbDPRLlNHBQ36AL0RLx5TkiuGGBPQok6VvNPHtOhaxjNFHaS27agTTrBLeW1No UMzp3Zk3l3tyE0Vd3YciyhWPJZN2w/ZZ5DTZMrs/f7am6x1PKDeedOLLOJhBzp9wcLMs ep4i/vf2TTP1D6CD8CTKJoRFM3Vy+zMwNTrHHAdXVYRkr+uTh/S/fPce6f6CVo/Li5B1 hD61rSCW0MbtbCscSv6DZdwJcUVWtF7bAefDd/1SZ+//XqzOJxnhs7mjVBeW7yfXcSRm mdKiTld4mijMRBv+0T+pT2mBYE0NUUCYCS0OOsOvtNkhP8zktAmnjXjPD2uyrUzPtF8O g5AA== X-Gm-Message-State: AJcUukclDMKjXbtEU+WFrJwyUheehyvU5QxoLxVF9X2/+MFWz8C6Ciwl Ks8KFpflmajsW870QwqqTy4= X-Google-Smtp-Source: ALg8bN5TL6HupD5aAxchA5RkaWr3x/N/Zo2GGAz1xi29gdUBQ145AIKNSwLliIGOzoPYy0b8WAujwQ== X-Received: by 2002:adf:e08c:: with SMTP id c12mr6159100wri.199.1545493790051; Sat, 22 Dec 2018 07:49:50 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id e16sm17561378wrn.72.2018.12.22.07.49.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 07:49:49 -0800 (PST) Date: Sat, 22 Dec 2018 17:49:45 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222154945.GE2244@macbook.localdomain> References: <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83h8f5a7po.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Dec 22, 2018 at 05:42:43PM +0200, Eli Zaretskii wrote: > > Date: Sat, 22 Dec 2018 17:15:09 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > There's a FIXME in the change you pushed (which I believe just > > > repeats what was already in the previous version). Ca you tell more > > > about the problem we need to fix there? > > > > We need a way to get Unicode composition and decomposition for the > > a given character (implementing the uni_compose and uni_decompose > > functions I deleted). > > Yes, but what does that entail? Are these compositions and > decompositions defined by the Unicode UCD? And how does Harfbuzz use > the results for a given character? Yes, the standard Unicode composition and decomposition. HarfBuzz uses these during shaping (it prefers composed form for a given sequence if supported by the font, and falls back to decomposed form otherwise). > > I recall you suggested something earlier that I tried but couldn’t > > get to work, the exact detail escapes me. > > I probably suggested using the 'decomposition' property of a > character, and perhaps als the facilities in ucs-normalize.el. How can this be done from C. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 11:34:35 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 16:34:35 +0000 Received: from localhost ([127.0.0.1]:60179 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gakEV-0003Wv-LM for submit@debbugs.gnu.org; Sat, 22 Dec 2018 11:34:35 -0500 Received: from eggs.gnu.org ([208.118.235.92]:43932) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gakEU-0003Wj-CE for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 11:34:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gakEO-0003KL-DS for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 11:34:29 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:43615) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gakEL-0003GF-3v; Sat, 22 Dec 2018 11:34:25 -0500 Received: from [176.228.60.248] (port=3240 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gakEF-0007DF-0r; Sat, 22 Dec 2018 11:34:23 -0500 Date: Sat, 22 Dec 2018 18:33:52 +0200 Message-Id: <83efa9a5cf.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181222154945.GE2244@macbook.localdomain> (message from Khaled Hosny on Sat, 22 Dec 2018 17:49:45 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 22 Dec 2018 17:49:45 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > Yes, the standard Unicode composition and decomposition. HarfBuzz uses > these during shaping (it prefers composed form for a given sequence if > supported by the font, and falls back to decomposed form otherwise). > > > > I recall you suggested something earlier that I tried but couldn’t > > > get to work, the exact detail escapes me. > > > > I probably suggested using the 'decomposition' property of a > > character, and perhaps als the facilities in ucs-normalize.el. > > How can this be done from C. There are several examples in the sources of calling Lisp from C. As just a random example: if (STRINGP (curdir)) val = call1 (intern ("file-remote-p"), curdir); This calls the Lisp function file-remote-p with one argument, curdir. If you tell me more about the arguments and the expected effects of calling uni_compose and uni_decompose, maybe I could propose a specific implementation. The Harfbuzz documentation doesn't seem to tell enough, or maybe I didn't find the right text. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 14:39:30 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 19:39:30 +0000 Received: from localhost ([127.0.0.1]:60230 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gan7S-0001eO-D8 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 14:39:30 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34170) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gan7Q-0001eB-Vk for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 14:39:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gan7G-0007pR-Qp for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 14:39:23 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48612) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gan75-0007hR-AR; Sat, 22 Dec 2018 14:39:07 -0500 Received: from [176.228.60.248] (port=2851 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gan73-0000wr-MG; Sat, 22 Dec 2018 14:39:07 -0500 Date: Sat, 22 Dec 2018 21:38:43 +0200 Message-Id: <83bm5d9wsc.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181222154945.GE2244@macbook.localdomain> (message from Khaled Hosny on Sat, 22 Dec 2018 17:49:45 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 22 Dec 2018 17:49:45 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > Yes, the standard Unicode composition and decomposition. HarfBuzz uses > these during shaping (it prefers composed form for a given sequence if > supported by the font, and falls back to decomposed form otherwise). Btw, how is this problem solved in the other projects that use Harfuzz? Does every project need to provide this functionality, or does Harfuzz have it built-in, like with the script tags? If there's built-in support for this, perhaps Emacs could just use that? From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 16:00:14 2018 Received: (at 33729) by debbugs.gnu.org; 22 Dec 2018 21:00:14 +0000 Received: from localhost ([127.0.0.1]:60255 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaoNZ-0003e4-W3 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 16:00:14 -0500 Received: from mail-wr1-f50.google.com ([209.85.221.50]:44816) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaoNY-0003d9-EC for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 16:00:12 -0500 Received: by mail-wr1-f50.google.com with SMTP id z5so8458695wrt.11 for <33729@debbugs.gnu.org>; Sat, 22 Dec 2018 13:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=83Jp+mUfRfb0L2jJcFNrs4u30VELLK5Of5HGV3dZK30=; b=Uqw9Qgaz2Lf/rzK+GcbOayZ2CW4M0vKZSa07sDk5gPjmWC3VJ0GS7FMpu06wJPYapg C5tpJytqDMczP+4m9VmZuaM9CT+W7a6M5U549fHPkllTNsPrpIZC8iVuxuvSemSpYhrT 8hMe5vzuWF7YD2ZC/2/1GgGQteIyBHBxWoI0lGjLrLGex2IewOk6S7qeQtr4+O8qQ9NJ FGiV/kaO66MHbtQaj6d7Hbb/Nf/6YjvfMNumfNoLmBOmA1CxWfc5X/TYUdTaMIiEM89j pTIa/iBQe0BoZy7jL3k3gl9IpHqJxDT58wEOED604UUIlQ2fXCBnTTOMEezPyd9M30dD UZKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=83Jp+mUfRfb0L2jJcFNrs4u30VELLK5Of5HGV3dZK30=; b=KubyR0gQsPPWb1x+ciEv9uOAz/YptNUb91bAg7PVV5aKGMjs7b7mjAKmCiyhigChPX Ieh1FllWbzbHTw2PnSFuBwn9REOUkCuMp6Yq5aD8DC6Ka+LDW7I3n1vN+mfNMAYvkV41 EvS/KldvBtvM8ify4LajHuEgWyQqLAHUe77mf2R1KuW5nqhvkTFApMQXeMou63LxdhCw htK2o8daweI72Fs1463F4H6Ol2pepE6XUqJGY8wzlH6oQ4B6WeDOh6D2+zT+jvpfj7qI Vpyc7KfqIbTgWcl0d/nbx2DHE/Nbin1ZDT3bj7wzi0jRLF6YgEcpDyBAyErLMn9L8MhF 1//g== X-Gm-Message-State: AJcUukd22fLxLdYT5iSs375YWSrK03HUbI6DUmPcxDVzUtgs0z2rFUhP 45wl+cVYscqcNck4bvPyWpc= X-Google-Smtp-Source: ALg8bN42HsM13oKnlzrfWpvUk6Y8d8/Ow+WkqaFOiJKILWEWCXLGE1uiWj5hcq8BM3p24V45zHtnEQ== X-Received: by 2002:adf:aa9c:: with SMTP id h28mr6937626wrc.216.1545512406534; Sat, 22 Dec 2018 13:00:06 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id v19sm26904844wrd.46.2018.12.22.13.00.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 13:00:05 -0800 (PST) Date: Sat, 22 Dec 2018 22:59:48 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181222205948.GF2244@macbook.localdomain> References: <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <83bm5d9wsc.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Dec 22, 2018 at 09:38:43PM +0200, Eli Zaretskii wrote: > > Date: Sat, 22 Dec 2018 17:49:45 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > Yes, the standard Unicode composition and decomposition. HarfBuzz uses > > these during shaping (it prefers composed form for a given sequence if > > supported by the font, and falls back to decomposed form otherwise). > > Btw, how is this problem solved in the other projects that use > Harfuzz? Does every project need to provide this functionality, or > does Harfuzz have it built-in, like with the script tags? If there's > built-in support for this, perhaps Emacs could just use that? There is built-in support, and currently we are using that. I can just remove the FIXME. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 22 22:34:32 2018 Received: (at 33729) by debbugs.gnu.org; 23 Dec 2018 03:34:32 +0000 Received: from localhost ([127.0.0.1]:60388 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gauX9-0000Jd-OP for submit@debbugs.gnu.org; Sat, 22 Dec 2018 22:34:31 -0500 Received: from eggs.gnu.org ([208.118.235.92]:47846) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gauX7-0000JQ-QZ for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 22:34:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gauX1-0006pt-MH for 33729@debbugs.gnu.org; Sat, 22 Dec 2018 22:34:24 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33628) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gauWv-0006k9-Uu; Sat, 22 Dec 2018 22:34:17 -0500 Received: from [176.228.60.248] (port=4169 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gauWv-0005Od-Fr; Sat, 22 Dec 2018 22:34:17 -0500 Date: Sun, 23 Dec 2018 05:34:04 +0200 Message-Id: <838t0gapcj.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181222205948.GF2244@macbook.localdomain> (message from Khaled Hosny on Sat, 22 Dec 2018 22:59:48 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 22 Dec 2018 22:59:48 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > On Sat, Dec 22, 2018 at 09:38:43PM +0200, Eli Zaretskii wrote: > > > Date: Sat, 22 Dec 2018 17:49:45 +0200 > > > From: Khaled Hosny > > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > Yes, the standard Unicode composition and decomposition. HarfBuzz uses > > > these during shaping (it prefers composed form for a given sequence if > > > supported by the font, and falls back to decomposed form otherwise). > > > > Btw, how is this problem solved in the other projects that use > > Harfuzz? Does every project need to provide this functionality, or > > does Harfuzz have it built-in, like with the script tags? If there's > > built-in support for this, perhaps Emacs could just use that? > > There is built-in support, and currently we are using that. I can just > remove the FIXME. Are there any disadvantages in using the built-in support? I mean, why did you envision an Emacs-specific implementation in the first place? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 23 08:51:21 2018 Received: (at 33729) by debbugs.gnu.org; 23 Dec 2018 13:51:21 +0000 Received: from localhost ([127.0.0.1]:60611 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb4A5-0000jr-7p for submit@debbugs.gnu.org; Sun, 23 Dec 2018 08:51:21 -0500 Received: from mail-wm1-f52.google.com ([209.85.128.52]:37036) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb4A4-0000ja-0j for 33729@debbugs.gnu.org; Sun, 23 Dec 2018 08:51:20 -0500 Received: by mail-wm1-f52.google.com with SMTP id g67so9876886wmd.2 for <33729@debbugs.gnu.org>; Sun, 23 Dec 2018 05:51:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=oHqB464v2DcOygKq/1ULYmQb8jGmH7avhL9st7Y1wUM=; b=uiBnUO36mRbYXD/UX7P6FG3DuzBj0qdTzbB74W4cYW0Y1Y/5pniOMjuWXn6/UlMFny p4uz8ge6QXDhKc7yqEMBDdr0dldEX/y2dLcz9uua9OdOOQ01LqupNFR4Jx3aqlnIprqs AqdH9ERtcLfvUZlp2EsqV3CNEYptp0sqA46Y6Fk9Pr2LaoQJOtTn4OZ03RW5fjFsk8CI eiCY1vszJAaA9wpVLuWtQfoAwNFxdcsA2hbA3b2kGBc4890O0ltgLHQlqIRlN8692YgQ SxkAesK4Hfs5Fy9pXzdabqfo7zr2wxgxgYwRr81uCVprMJX8QtfurmIg7kFnDPqgOWRI h14g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=oHqB464v2DcOygKq/1ULYmQb8jGmH7avhL9st7Y1wUM=; b=r9/3675aqdOC2FDP7YY3yQ3QgBjvZ0n9/4TbaQdj30eSNm6XURdU0EVLoL6S+iG0MZ AqNGY5TyPfpuRBzF+cqei52Fzwh5+m/XF07hUyJE5NFOOeKbnMxyqMffo+kBUIv+iABA t2lq8y99xE1SlO6k99Dtd3daz2Y/omMClNpmAFq24zZI/HT9l8bdLOruQUEDDL0uxG6B Gi37DrAlJY01MyQMlAUSgv/eko7ogAmWz9oFk8MyTJ8E8A5aGFdiHS9EUw7t8aEajbz7 rMkrWcnFMiGWkkHhMh/UJq4rsEYxwVIZ4xQm9wbufWuV/Bn1zr+k31Tmct6Jhn8BPh8+ ewyg== X-Gm-Message-State: AA+aEWYA2wxRLHzLU788Z+BxiaXa3+Jf7xDvq6rSZ9HIl5FudS52JXV+ wCDzJHu9MwWYVX4TvAl2LxUNzF23fvU= X-Google-Smtp-Source: AFSGD/V3rAMYu7OwSwHrIwLNknMjUVlDnZ/PlmCP44V82bYOwcUgN/klT4c171wqbGwzpagXhHKs/w== X-Received: by 2002:a7b:c156:: with SMTP id z22mr9235362wmi.24.1545573074238; Sun, 23 Dec 2018 05:51:14 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id t5sm16754031wmd.15.2018.12.23.05.51.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 23 Dec 2018 05:51:13 -0800 (PST) Date: Sun, 23 Dec 2018 15:51:09 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181223135109.GA6568@macbook.localdomain> References: <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <838t0gapcj.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, Dec 23, 2018 at 05:34:04AM +0200, Eli Zaretskii wrote: > > Date: Sat, 22 Dec 2018 22:59:48 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > On Sat, Dec 22, 2018 at 09:38:43PM +0200, Eli Zaretskii wrote: > > > > Date: Sat, 22 Dec 2018 17:49:45 +0200 > > > > From: Khaled Hosny > > > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > > > Yes, the standard Unicode composition and decomposition. HarfBuzz uses > > > > these during shaping (it prefers composed form for a given sequence if > > > > supported by the font, and falls back to decomposed form otherwise). > > > > > > Btw, how is this problem solved in the other projects that use > > > Harfuzz? Does every project need to provide this functionality, or > > > does Harfuzz have it built-in, like with the script tags? If there's > > > built-in support for this, perhaps Emacs could just use that? > > > > There is built-in support, and currently we are using that. I can just > > remove the FIXME. > > Are there any disadvantages in using the built-in support? I mean, > why did you envision an Emacs-specific implementation in the first > place? I thought, but I might be mistaken, that Emacs allow changing these character properties at runtime and someone might possibly want to use that to change some character property (e.g. make some PUA character a combining mark) and it would then be nice if HarfBuzz respected that. I admit that is very niche thing if possible at all, and I’m more than happy to let HarfBuzz use it default Unicode functions and simplify the Emacs integration code. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 23 11:01:35 2018 Received: (at 33729) by debbugs.gnu.org; 23 Dec 2018 16:01:36 +0000 Received: from localhost ([127.0.0.1]:33922 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6C7-0004Gu-Le for submit@debbugs.gnu.org; Sun, 23 Dec 2018 11:01:35 -0500 Received: from eggs.gnu.org ([208.118.235.92]:40421) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6C5-0004Gh-VV for 33729@debbugs.gnu.org; Sun, 23 Dec 2018 11:01:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb6C0-0001A1-3E for 33729@debbugs.gnu.org; Sun, 23 Dec 2018 11:01:28 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:55839) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gb6Bp-0000y1-OW; Sun, 23 Dec 2018 11:01:19 -0500 Received: from [176.228.60.248] (port=2445 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gb6Bi-0003t6-BM; Sun, 23 Dec 2018 11:01:11 -0500 Date: Sun, 23 Dec 2018 18:00:58 +0200 Message-Id: <83va3k8c79.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181223135109.GA6568@macbook.localdomain> (message from Khaled Hosny on Sun, 23 Dec 2018 15:51:09 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sun, 23 Dec 2018 15:51:09 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > Are there any disadvantages in using the built-in support? I mean, > > why did you envision an Emacs-specific implementation in the first > > place? > > I thought, but I might be mistaken, that Emacs allow changing these > character properties at runtime and someone might possibly want to use > that to change some character property (e.g. make some PUA character a > combining mark) and it would then be nice if HarfBuzz respected that. I > admit that is very niche thing if possible at all, and I’m more than > happy to let HarfBuzz use it default Unicode functions and simplify the > Emacs integration code. Right, I agree that we should for now leave that to HarfBuzz. It could be added later as an optional feature. (I don't expect many users to want to modify the Unicode character properties.) Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 23 21:08:59 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 02:08:59 +0000 Received: from localhost ([127.0.0.1]:34165 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbFfv-0002xZ-21 for submit@debbugs.gnu.org; Sun, 23 Dec 2018 21:08:59 -0500 Received: from mail-wr1-f47.google.com ([209.85.221.47]:42337) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbFft-0002xK-Bb for 33729@debbugs.gnu.org; Sun, 23 Dec 2018 21:08:57 -0500 Received: by mail-wr1-f47.google.com with SMTP id q18so10410188wrx.9 for <33729@debbugs.gnu.org>; Sun, 23 Dec 2018 18:08:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=nZOV9QV+WhHVBttOHPNwez+nlyT/V5avzsnJt4DlvTg=; b=QnQnf/noCVHWwkrTqhQ54Cby2obBToIBrWsb7CScPrlm0HzvAM0HJARNc14KmOmn6b d0yrh+eO0pFe4ZbaMYe9WjCtJgGoqgWa7DVPgqYLb4pSYxP+Y0unNVw9edWrGwIbAm2A gUyH7JYWakjspsshiVYw9qnvNzadnSgLqJGEYuE9YdIAs1H8+zQohC3ouae5BfCS6hRK RGaUBc82ZjzTIQBrkAvqorJaTGazNlFPaG+87rY1Yv6OI3cCzl7nJOTT1EoDSw1GtcJl EhFPPhs4pzur+6A4j/hcysQFIvb2d1RZIWxjEa+3jF8TfOL8y++b4TmgvDAWdG4kAJ7L kCDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=nZOV9QV+WhHVBttOHPNwez+nlyT/V5avzsnJt4DlvTg=; b=Zri9TIwV1hjZPDdUhMWXpiwZhbvo4KN0kOzYACR0abKTaOPFHde/RQJT9U0d3r/Qy4 QnrUlgKiN8y5Cc40SIXVezfWrPlT63tkFi8co+towqavJSPAE9vpX7xWl6EAnp49spVJ zKB4L/9bq1gOUpGuVJjlB6Ij7qLv7sAa7TbIoOYQr24i+ddcSnjea7AhQmYwkACKA8FB REwqJZ0g//pv4hzOuevuOF3z2oST6vSRpdn87zymqL9ocWcK07HSpdOjm0Kyf9AUxlak SEfmTAUfvQ2f7ekjZ0QZBL4Zjq91htSXX9fHRqy2grDD/IiyWDM3AWQ+qv7QjY3qtRfH 3VKg== X-Gm-Message-State: AJcUukciJt66ZpxtBOI5lvSD5PSJjJ5kLSFVE/gPr1ZsiXwAvvazrkXe 6C2ZyrfhPIEgdXrw/mdVKUM= X-Google-Smtp-Source: ALg8bN7+6WZkVnAkfPppEyeh+YV663s4ryyFPyC0uPHZ0fhCgkASRQ7ay95gXDBDFTy7z/jh2Ddj7g== X-Received: by 2002:adf:a393:: with SMTP id l19mr10315183wrb.110.1545617331697; Sun, 23 Dec 2018 18:08:51 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id 199sm25002083wmh.21.2018.12.23.18.08.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 23 Dec 2018 18:08:50 -0800 (PST) Date: Mon, 24 Dec 2018 04:08:47 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181224020847.GC6568@macbook.localdomain> References: <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83va3k8c79.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, Dec 23, 2018 at 06:00:58PM +0200, Eli Zaretskii wrote: > > Date: Sun, 23 Dec 2018 15:51:09 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > Are there any disadvantages in using the built-in support? I mean, > > > why did you envision an Emacs-specific implementation in the first > > > place? > > > > I thought, but I might be mistaken, that Emacs allow changing these > > character properties at runtime and someone might possibly want to use > > that to change some character property (e.g. make some PUA character a > > combining mark) and it would then be nice if HarfBuzz respected that. I > > admit that is very niche thing if possible at all, and I’m more than > > happy to let HarfBuzz use it default Unicode functions and simplify the > > Emacs integration code. > > Right, I agree that we should for now leave that to HarfBuzz. It > could be added later as an optional feature. (I don't expect many > users to want to modify the Unicode character properties.) I think we are almost good now. There is only one serious FIXME left: /* FIXME: guess_segment_properties is BAD BAD BAD. * we need to get these properties with the LGSTRING. */ #if 1 hb_buffer_guess_segment_properties (hb_buffer); #else hb_buffer_set_direction (hb_buffer, XXX); hb_buffer_set_script (hb_buffer, XXX); hb_buffer_set_language (hb_buffer, XXX); #endif We need to know, for a given lgstring we are shaping: * Its direction (from applying bidi algorithm). Each lgstring we are shaping must be of a single direction. * Its script, possibly after applying something like: http://unicode.org/reports/tr24/#Common * Its language, is Emacs allows setting text language (my understand is that it doesn’t). Some languages really need this for applying language-specfic features (Urdu digits, Serbian alternate glyphs, etc.). From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 23 23:13:45 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 04:13:45 +0000 Received: from localhost ([127.0.0.1]:34192 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbHcb-0006Ec-DM for submit@debbugs.gnu.org; Sun, 23 Dec 2018 23:13:45 -0500 Received: from mail-lj1-f177.google.com ([209.85.208.177]:34350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbHcV-0006EJ-Cv for 33729@debbugs.gnu.org; Sun, 23 Dec 2018 23:13:39 -0500 Received: by mail-lj1-f177.google.com with SMTP id u89-v6so9487643lje.1 for <33729@debbugs.gnu.org>; Sun, 23 Dec 2018 20:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=dhqe8jZgTCOn+XB5CqJhnYG3zn3alCqofZDga6aa8AM=; b=Bm5e3RUpPRnaWocijGpq3y04GMOQCMek6Vb7lR/DqTS3phmYUeRYnLhEV3NvbeHIG8 Kvrlgy/+yi5i75rum8dBDXqUGHWJ1g6KpTUndRU97O0zCEglSzHBPzytJimQ2PsSITR8 bCrUxxacbe46XTI7tYkT1hRw3AICQZPBSOl0aYzYVQHvPwqsQ9EsKsUvuK7B/BHl+r8F I/tIkO6aV4i9IAY2OHpL5nfzWxcdJnSryFDweIWyxT2AY2cdgYDlfeUDEHOAaW2VLCGn mqTCUK/wQiMkgUW3gGHJgGxrFO6LrQdra8jG6JwH22e8oKkOlV3T9lkzLV1lIPN2Bk8o PKTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dhqe8jZgTCOn+XB5CqJhnYG3zn3alCqofZDga6aa8AM=; b=pHzq18VeGdnUm5xRKkNfdBuI8VxL3v6uZGMFMWVUmhQDn0jcGX8z1Hob9vz85BU+Q3 MC9LTyRhmVXa/HLVuI5AdAjyaN2EvpAQEgpIuzKNqq3J+fo6OeWbo3aQlpcRIUuN0irP jR+8s6kpjKOuTPg5+642sXJgRbHH/RV2iuxHDegvHR8PditJ2T9vkBM0IZfeZ1qe3FHd eKeckrG2iyyhHDPzYAmM6ara7Dd5gdiv+aHcP5Xbu8l00LfLLlvmaa63bSTObk0w3hJ+ BQrFkmSqjT1nGxJJWM1gtqGAiGO9nE5cB7Q72Bur+ZA+qqjDc0kWagNt+9izyFndR/EZ PD2w== X-Gm-Message-State: AJcUukcdEfDoYBUMeZEEJ+0ZxBXGIOEbWrZvLEQbFFHxLnlVw8VlPppW uUL0rGsIVr7nQHsgMXFJbCi2QR3tnUMoNdkRLjM= X-Google-Smtp-Source: ALg8bN7EDZdrVmy2SPC9VsoFDr0aFNq5vseXTI//bta8mP64eAnxadO6IFq7NFNbWPc3ku7Ktxl5wyi/ktHDG9pQSHg= X-Received: by 2002:a2e:1f01:: with SMTP id f1-v6mr5870496ljf.129.1545624809332; Sun, 23 Dec 2018 20:13:29 -0800 (PST) MIME-Version: 1.0 References: <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> In-Reply-To: <20181224020847.GC6568@macbook.localdomain> From: Kaushal Modi Date: Sun, 23 Dec 2018 23:12:52 -0500 Message-ID: Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) To: Khaled Hosny Content-Type: multipart/alternative; boundary="000000000000b76e6e057dbcd14d" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, Glenn Morris , Eli Zaretskii , Mohammad Nasirifar , Behdad Esfahbod X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000b76e6e057dbcd14d Content-Type: text/plain; charset="UTF-8" On Sun, Dec 23, 2018 at 9:08 PM Khaled Hosny wrote: > I think we are almost good now. > Thanks for working on this! I confirm that this particular issue related to rendering compound glyphs in Gujarati script is fixed. Proof: https://i.imgtc.com/XYSM5fE.png @Eli I see that discussion related to this fix is on-going. So feel free to mark this bug as DONE when you see fit. Thanks all! --000000000000b76e6e057dbcd14d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Sun, Dec 23, 2018 at 9:08 PM Khaled Hosny <dr.khaled.hosny@gmail.com> wrote:
I think we are almost = good now.

Thanks for working on this!<= /div>

I confirm that this particular issue related to re= ndering compound glyphs in Gujarati script is fixed.

Proof: https://i.imgtc.com/X= YSM5fE.png


@Eli I see that discussion rela= ted to this fix is on-going. So feel free to mark this bug as DONE when you= see fit.

Thanks all!

--000000000000b76e6e057dbcd14d-- From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 24 11:11:12 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 16:11:12 +0000 Received: from localhost ([127.0.0.1]:35576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbSox-0007hw-RK for submit@debbugs.gnu.org; Mon, 24 Dec 2018 11:11:12 -0500 Received: from eggs.gnu.org ([208.118.235.92]:46648) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbSow-0007hi-HF for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 11:11:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gbSoq-0004k6-D1 for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 11:11:05 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33913) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gbSol-0004g0-Pw; Mon, 24 Dec 2018 11:10:59 -0500 Received: from [176.228.60.248] (port=4294 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gbSol-0005Pf-B7; Mon, 24 Dec 2018 11:10:59 -0500 Date: Mon, 24 Dec 2018 18:10:49 +0200 Message-Id: <83lg4e9a7q.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181224020847.GC6568@macbook.localdomain> (message from Khaled Hosny on Mon, 24 Dec 2018 04:08:47 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222090644.GB2244@macbook.localdomain> <83va3lan24.fsf@gnu.org> <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Mon, 24 Dec 2018 04:08:47 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > I think we are almost good now. There is only one serious FIXME left: > > /* FIXME: guess_segment_properties is BAD BAD BAD. > * we need to get these properties with the LGSTRING. */ > #if 1 > hb_buffer_guess_segment_properties (hb_buffer); > #else > hb_buffer_set_direction (hb_buffer, XXX); > hb_buffer_set_script (hb_buffer, XXX); > hb_buffer_set_language (hb_buffer, XXX); > #endif > > We need to know, for a given lgstring we are shaping: > * Its direction (from applying bidi algorithm). Each lgstring we are > shaping must be of a single direction. Communicating this to ftfont_shape_by_hb will need changes in a couple of interfaces (the existing shaping engines didn't need this information). I will work on this soon. > * Its script, possibly after applying something like: > http://unicode.org/reports/tr24/#Common Per previous discussions, we decided to use the Harfbuzz built-in methods for determining the script, since Emacs doesn't have this information, and adding it will just do the same as Harfbuzz does, i.e. find the first character whose script is not Common etc., using the UCD database. I think it was you who suggested to use the Harfbuzz built-ins in this case. > * Its language, is Emacs allows setting text language (my understand is > that it doesn’t). Some languages really need this for applying > language-specfic features (Urdu digits, Serbian alternate glyphs, etc.). We don't currently have a language property for chunks of text, we only have the current global language setting determined from the locale (and there's a command to change that for Emacs, should the user want it). This is not really appropriate for multilingual buffers, but we will have to use that for now, and hope that in the future, infrastructure will be added to allow more flexible determination of the language of each run of text. (I see that Harfbuzz already looks a the locale for its default language, but since Emacs allows user control of this, however unlikely, I think it's best to use the value Emacs uses.) I will work on this as well. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 24 12:37:35 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 17:37:35 +0000 Received: from localhost ([127.0.0.1]:35601 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUAY-0001Nw-Ri for submit@debbugs.gnu.org; Mon, 24 Dec 2018 12:37:35 -0500 Received: from mail-wr1-f50.google.com ([209.85.221.50]:34310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUAX-0001Ni-A7 for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 12:37:33 -0500 Received: by mail-wr1-f50.google.com with SMTP id j2so12126499wrw.1 for <33729@debbugs.gnu.org>; Mon, 24 Dec 2018 09:37:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=Gana4c+e6HGNisJXyoKZtAkhPEAXnBq/NCm0M7y9dF0=; b=DyUZ/k5C5UThWt0BWaHJ+0/mY5j7jqe9wbRtUdKErJZALC9I+ZwhimHzdnoVIRezm8 Us8WjhZDEX957vZ/KhzC23CD5hjAgQAAP/9PydvZizNzkrtQc878bGRXWLWxOKIx2wc6 L0yeJ9CyDJGDeWjuZPubf5MpmIPgxqYN7CtsYkokFcA8Ql1L4mgnP+LB6Uf/Ykj0RB7b BtFQkJe3IcwjlFT67uVOWHvEhgtR6S5aSJ89Z6Oo680kKd+co0l9icdQHbA5W3EPz//6 pi6pZdU9E+N7t7OPNWOakHC5+XzoDs4xn57uDVilQvtOtDqVDJVFgubv+ukY0on+uTL1 PWcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=Gana4c+e6HGNisJXyoKZtAkhPEAXnBq/NCm0M7y9dF0=; b=Okk+e3icq7VEarMkse7U6cGG3+MMRuHaZv8cB9dArP6hjVWks9+15bb9kgUkXRLO7p VYmZf3z+TUwnNOQQ2TstkuuMXNeT8Nyhv92zKw54+ZVgqDydihhpIwb9vUODAKtiQoZZ FInHZsiHSZERr4H5IhQuw48iGHQLvxPkaXDgq+N4Jto8kS7nGVy8qEcz2MYoAyBQsraH u7O8b5sqX5vuXeMcf+KwIjUrvDxAvd9oXUmmwRmTVlztGnFH6STGdS+fwHl4wC6O+A26 q8xlJ8kKQNn5vH3NkuhiSdTsxFBbUKsVgjkOlRfMa0RbSyI1vA0nx5ZTJz7BKuq2Hq55 rJNw== X-Gm-Message-State: AJcUukdXPA3BB1x3l+KiaogL66V3DG4D4ZluVwB6tOD40BYT4CQlvGgV rs+mvyU8vuttJ1R0OUejNfU= X-Google-Smtp-Source: ALg8bN5WEdfwyLaBA5FUy4nvqKraDwg73YonM4xaKazRA5CFuWmPI4xAAKTf6j0F8zH9y/Etc12IkQ== X-Received: by 2002:a5d:694d:: with SMTP id r13mr12059510wrw.323.1545673047517; Mon, 24 Dec 2018 09:37:27 -0800 (PST) Received: from macbook.localdomain ([41.237.113.27]) by smtp.gmail.com with ESMTPSA id t63sm16609962wmt.8.2018.12.24.09.37.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 24 Dec 2018 09:37:26 -0800 (PST) Date: Mon, 24 Dec 2018 19:37:23 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20181224173723.GH6568@macbook.localdomain> References: <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83lg4e9a7q.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, Dec 24, 2018 at 06:10:49PM +0200, Eli Zaretskii wrote: > > Date: Mon, 24 Dec 2018 04:08:47 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > I think we are almost good now. There is only one serious FIXME left: > > > > /* FIXME: guess_segment_properties is BAD BAD BAD. > > * we need to get these properties with the LGSTRING. */ > > #if 1 > > hb_buffer_guess_segment_properties (hb_buffer); > > #else > > hb_buffer_set_direction (hb_buffer, XXX); > > hb_buffer_set_script (hb_buffer, XXX); > > hb_buffer_set_language (hb_buffer, XXX); > > #endif > > > > We need to know, for a given lgstring we are shaping: > > * Its direction (from applying bidi algorithm). Each lgstring we are > > shaping must be of a single direction. > > Communicating this to ftfont_shape_by_hb will need changes in a couple > of interfaces (the existing shaping engines didn't need this > information). I will work on this soon. Great. > > * Its script, possibly after applying something like: > > http://unicode.org/reports/tr24/#Common > > Per previous discussions, we decided to use the Harfbuzz built-in > methods for determining the script, since Emacs doesn't have this > information, and adding it will just do the same as Harfbuzz does, > i.e. find the first character whose script is not Common etc., using > the UCD database. I think it was you who suggested to use the > Harfbuzz built-ins in this case. The built-in HarfBuzz code is for getting the script for a given character, but resolving characters with Common script is left to the client. Suppose you have this string (upper case for RTL) ABC 123 DEF, what HarfBuzz sees during shaping is three separate chunks of text ABC, 123, DEF. The 123 part is all Common script characters and thus hb_buffer_guess_segment_properties won’t be able to guess anything (and based on the font and the script, this can cause rendering differences). Emacs will have to resolve the script of Common characters before applying bidi algorithm and pass that down to HarfBuzz. > > * Its language, is Emacs allows setting text language (my understand is > > that it doesn’t). Some languages really need this for applying > > language-specfic features (Urdu digits, Serbian alternate glyphs, etc.). > > We don't currently have a language property for chunks of text, we > only have the current global language setting determined from the > locale (and there's a command to change that for Emacs, should the > user want it). This is not really appropriate for multilingual > buffers, but we will have to use that for now, and hope that in the > future, infrastructure will be added to allow more flexible > determination of the language of each run of text. (I see that > Harfbuzz already looks a the locale for its default language, but > since Emacs allows user control of this, however unlikely, I think > it's best to use the value Emacs uses.) I will work on this as well. Yes, better pass that from Emacs to HarfBuzz. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 24 12:38:56 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 17:38:56 +0000 Received: from localhost ([127.0.0.1]:35605 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUBs-0001Pw-C9 for submit@debbugs.gnu.org; Mon, 24 Dec 2018 12:38:56 -0500 Received: from odoacer.turtle-trading.net ([93.241.193.16]:46243) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUBq-0001Pi-K3 for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 12:38:55 -0500 Received: from justinian.turtle-trading.net ([192.168.2.118]) by odoacer.turtle-trading.net with esmtp (Exim 4.80) (envelope-from ) id 1gbUBj-0004P5-7q; Mon, 24 Dec 2018 18:38:47 +0100 Received: from benny by justinian.turtle-trading.net with local (Exim 4.89) (envelope-from ) id 1gbUBi-0003FN-TY; Mon, 24 Dec 2018 18:38:46 +0100 From: Benjamin Riefenstahl To: Khaled Hosny Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83h8fghcpo.fsf@gnu.org> <20181214075056.GI2244@macbook.localdomain> <8336r0h1cb.fsf@gnu.org> <20181214110316.GK2244@macbook.localdomain> <83y38sfcme.fsf@gnu.org> <83tvjgf7ux.fsf@gnu.org> <83mup4du5z.fsf@gnu.org> <20181222085448.GA2244@macbook.localdomain> <20181222090644.GB2244@macbook.localdomain> Date: Mon, 24 Dec 2018 18:38:46 +0100 In-Reply-To: <20181222090644.GB2244@macbook.localdomain> (Khaled Hosny's message of "Sat, 22 Dec 2018 11:06:44 +0200") Message-ID: <874lb2j049.fsf@turtle-trading.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: Glenn Morris , far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org, kaushal.modi@gmail.com, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Khaled Hosny writes: > I pushed a commit to harfbuzz branch that I think fixes this issue now. I can confirm that this fixes the issue with Syriac. Thanks, benny From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 24 13:07:26 2018 Received: (at 33729) by debbugs.gnu.org; 24 Dec 2018 18:07:26 +0000 Received: from localhost ([127.0.0.1]:35614 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUdR-0002HP-Vn for submit@debbugs.gnu.org; Mon, 24 Dec 2018 13:07:26 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49911) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUdQ-0002HD-Bf for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 13:07:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gbUdK-00024A-6o for 33729@debbugs.gnu.org; Mon, 24 Dec 2018 13:07:18 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35505) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gbUdG-00021m-6e; Mon, 24 Dec 2018 13:07:14 -0500 Received: from [176.228.60.248] (port=3591 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gbUdF-0006l0-Ml; Mon, 24 Dec 2018 13:07:14 -0500 Date: Mon, 24 Dec 2018 20:07:04 +0200 Message-Id: <83imzi94tz.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181224173723.GH6568@macbook.localdomain> (message from Khaled Hosny on Mon, 24 Dec 2018 19:37:23 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Mon, 24 Dec 2018 19:37:23 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > Per previous discussions, we decided to use the Harfbuzz built-in > > methods for determining the script, since Emacs doesn't have this > > information, and adding it will just do the same as Harfbuzz does, > > i.e. find the first character whose script is not Common etc., using > > the UCD database. I think it was you who suggested to use the > > Harfbuzz built-ins in this case. > > The built-in HarfBuzz code is for getting the script for a given > character, but resolving characters with Common script is left to the > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > what HarfBuzz sees during shaping is three separate chunks of text ABC, > 123, DEF. The 123 part is all Common script characters and thus > hb_buffer_guess_segment_properties won’t be able to guess anything (and > based on the font and the script, this can cause rendering differences). > Emacs will have to resolve the script of Common characters before > applying bidi algorithm and pass that down to HarfBuzz. I'm not sure I understand: why does HarfBuzz care that 123 was in the middle if RTL text. Does it need to shape 123 specially in this case? (In general, AFAIK simple characters like 123 will not even go through HarfBuzz, as Emacs doesn't call the shaper for characters whose entry in composition-function-table is nil. So I guess 123 here should stand for some other characters, not for literal digits? IOW, I don't think I understand the example very well.) From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 29 09:49:52 2018 Received: (at 33729) by debbugs.gnu.org; 29 Dec 2018 14:49:52 +0000 Received: from localhost ([127.0.0.1]:40940 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gdFw0-0000yA-Cq for submit@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:52 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34081) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gdFvz-0000xw-9G for 33729@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gdFvq-00085v-A9 for 33729@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:45 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48587) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdFvq-00085i-55; Sat, 29 Dec 2018 09:49:42 -0500 Received: from [176.228.60.248] (port=3863 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gdFvp-0002Gl-PR; Sat, 29 Dec 2018 09:49:42 -0500 Date: Sat, 29 Dec 2018 16:49:23 +0200 Message-Id: <838t085qx8.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20181224173723.GH6568@macbook.localdomain> (message from Khaled Hosny on Mon, 24 Dec 2018 19:37:23 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Mon, 24 Dec 2018 19:37:23 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > On Mon, Dec 24, 2018 at 06:10:49PM +0200, Eli Zaretskii wrote: > > > Date: Mon, 24 Dec 2018 04:08:47 +0200 > > > From: Khaled Hosny > > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > I think we are almost good now. There is only one serious FIXME left: > > > > > > /* FIXME: guess_segment_properties is BAD BAD BAD. > > > * we need to get these properties with the LGSTRING. */ > > > #if 1 > > > hb_buffer_guess_segment_properties (hb_buffer); > > > #else > > > hb_buffer_set_direction (hb_buffer, XXX); > > > hb_buffer_set_script (hb_buffer, XXX); > > > hb_buffer_set_language (hb_buffer, XXX); > > > #endif > > > > > > We need to know, for a given lgstring we are shaping: > > > * Its direction (from applying bidi algorithm). Each lgstring we are > > > shaping must be of a single direction. > > > > Communicating this to ftfont_shape_by_hb will need changes in a couple > > of interfaces (the existing shaping engines didn't need this > > information). I will work on this soon. > > Great. Done. Please test. I made sure it compiles, but I couldn't actually test the results, as I don't have access to a GNU/Linux system with GUI display. So it could be that I misunderstood the Harfbuzz APIs, as I was essentially flying blind, guided only by the Harfbuzz docs. In particularly, I hope I understood correctly the way we should leave to Harfbuzz guess the properties not explicitly provided by the Emacs context, both for the direction of the text and its script. > > > * Its script, possibly after applying something like: > > > http://unicode.org/reports/tr24/#Common > > > > Per previous discussions, we decided to use the Harfbuzz built-in > > methods for determining the script, since Emacs doesn't have this > > information, and adding it will just do the same as Harfbuzz does, > > i.e. find the first character whose script is not Common etc., using > > the UCD database. I think it was you who suggested to use the > > Harfbuzz built-ins in this case. > > The built-in HarfBuzz code is for getting the script for a given > character, but resolving characters with Common script is left to the > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > what HarfBuzz sees during shaping is three separate chunks of text ABC, > 123, DEF. The 123 part is all Common script characters and thus > hb_buffer_guess_segment_properties won’t be able to guess anything (and > based on the font and the script, this can cause rendering differences). > Emacs will have to resolve the script of Common characters before > applying bidi algorithm and pass that down to HarfBuzz. See my followup questions about this. For now, I left this aspect to HarfBuzz. > > > * Its language, is Emacs allows setting text language (my understand is > > > that it doesn’t). Some languages really need this for applying > > > language-specfic features (Urdu digits, Serbian alternate glyphs, etc.). > > > > We don't currently have a language property for chunks of text, we > > only have the current global language setting determined from the > > locale (and there's a command to change that for Emacs, should the > > user want it). This is not really appropriate for multilingual > > buffers, but we will have to use that for now, and hope that in the > > future, infrastructure will be added to allow more flexible > > determination of the language of each run of text. (I see that > > Harfbuzz already looks a the locale for its default language, but > > since Emacs allows user control of this, however unlikely, I think > > it's best to use the value Emacs uses.) I will work on this as well. > > Yes, better pass that from Emacs to HarfBuzz. Done, but please see the FIXME I left behind. For testing purposes, you can change the current language like this: M-x set-locale-environment RET xx_YY.CODESET RET For example: M-x set-locale-environment RET sr_RS.UTF-8 RET for the Cyrillic Serbian locale. This should change the value of current-iso639-language to the symbol 'sr'. Please tell if you encounter any difficulties with the code I added, or if you need any further help. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 05 15:53:27 2019 Received: (at 33729) by debbugs.gnu.org; 5 Jan 2019 20:53:27 +0000 Received: from localhost ([127.0.0.1]:48085 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gfswh-0001qQ-D8 for submit@debbugs.gnu.org; Sat, 05 Jan 2019 15:53:27 -0500 Received: from mail-ed1-f51.google.com ([209.85.208.51]:34022) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gfswe-0001qB-QB for 33729@debbugs.gnu.org; Sat, 05 Jan 2019 15:53:25 -0500 Received: by mail-ed1-f51.google.com with SMTP id b3so34717762ede.1 for <33729@debbugs.gnu.org>; Sat, 05 Jan 2019 12:53:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=1mAV/ZoaEMJ5L5q3DxakmcuPtg7cHPy33uxHU0PLEqM=; b=HmkiNK8oLQtzsbO/8SaPbikTuLxWNaZZhYPQNShqBm8wHGQ+ruQ1rwHAYVH9+fSsvh koMXWMPGRGEd6a2FeaL9G3TC2rmeO56pYVoubqmPSIFHdRxmZhewEJU3JxDs5onYFhod JBoolcBYaB0+RbOWTssRG5Gh9WNtAue4/SglPnSF0tNhrXHApNNgPAYqE7fM1VimvxuG WlJxCadj5k3FH0sYnQRQbXvxDkJo7X4CPSiQz0m+SgrkSZWSkxYRInY14R6rtys/RZHm 1rLxEVsSl8+NCdNwcstVJUms6H/nxJjggDH89o3Wsow8aX4V9xvvITNTjNA+NvqRW+ax Y3MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=1mAV/ZoaEMJ5L5q3DxakmcuPtg7cHPy33uxHU0PLEqM=; b=FI8zUxt8OimVFsZesPVvPnH7k0tQhZ5ZGub+GM/SrNJ9roZSiIJaYi27rZ21PAYmBS z2c5sK1bnZ60/d6MKh53xnbPcERmHYmrzT7cfNJcJ/XZ7v/DoPbOgUobnY/mgcAAHVjC 3a8NvGF1qteQINY0gHI8oUAygwU1U4ummjxyY8qOUixmfn/fxpsksACrtCQCRpwAhxKk wKD0muATLqGkOgX597T8WzFPjR0jlxSLlzOHMugd+igUGOpFJuozDCBw2QcKg8oZeo0t HOF+jPwxKSwaBotlh/Kiy4NgfOsGWQr7gX/HXw5BmgQWlYnV3i2X2ko3RHvoR6vij+m2 crCw== X-Gm-Message-State: AA+aEWYFfgqnBP/Bj95f50mw8AiZ1yoGm50vgpgzEPYxvXIeAzgawdWg PAsWbSmp9FyuKRgmh2yLbRM= X-Google-Smtp-Source: AFSGD/XS++ozfv2IhzaKjP1fd76uFuf4Yem8uSNN81caLgd8wn64hfgBAALgGCK1fv/qn4QmG6VZ9g== X-Received: by 2002:aa7:d684:: with SMTP id d4mr48434493edr.59.1546721598952; Sat, 05 Jan 2019 12:53:18 -0800 (PST) Received: from macbook.localdomain ([41.237.67.158]) by smtp.gmail.com with ESMTPSA id j31sm27098306eda.46.2019.01.05.12.53.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 05 Jan 2019 12:53:17 -0800 (PST) Date: Sat, 5 Jan 2019 22:53:14 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190105205314.GA28761@macbook.localdomain> References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <838t085qx8.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Dec 29, 2018 at 04:49:23PM +0200, Eli Zaretskii wrote: > > Date: Mon, 24 Dec 2018 19:37:23 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > We need to know, for a given lgstring we are shaping: > > > > * Its direction (from applying bidi algorithm). Each lgstring we are > > > > shaping must be of a single direction. > > > > > > Communicating this to ftfont_shape_by_hb will need changes in a couple > > > of interfaces (the existing shaping engines didn't need this > > > information). I will work on this soon. > > > > Great. > > Done. Please test. I made sure it compiles, but I couldn't actually > test the results, as I don't have access to a GNU/Linux system with > GUI display. So it could be that I misunderstood the Harfbuzz APIs, > as I was essentially flying blind, guided only by the Harfbuzz docs. It seems to work, but still not quite right. You seem to be passing the paragraph direction, but what HarfBuzz needs is resolved direction of the text (i.e. the bidi embedding level of the run). In other words, if Emacs is going to draw this text from right to left, then HarfBuzz must shape it in right to left direction. Both should use the same direction all the time and HarfBuzz direction guessing should never be used (i.e. always pass to it an explicit direction). Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 05 16:04:32 2019 Received: (at 33729) by debbugs.gnu.org; 5 Jan 2019 21:04:32 +0000 Received: from localhost ([127.0.0.1]:48088 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gft7Q-00028a-GU for submit@debbugs.gnu.org; Sat, 05 Jan 2019 16:04:32 -0500 Received: from mail-ed1-f46.google.com ([209.85.208.46]:40023) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gft7O-00028M-PF for 33729@debbugs.gnu.org; Sat, 05 Jan 2019 16:04:31 -0500 Received: by mail-ed1-f46.google.com with SMTP id g22so34593160edr.7 for <33729@debbugs.gnu.org>; Sat, 05 Jan 2019 13:04:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=uQBWwMNoXQXaUx2Yo2mSWIgduhZeZIo/BlHYfo7UUHs=; b=X0JTy63ZEhc+ol9dQNh0N98cZ1PGSc8AnqnaPZ0OQST5lMwqrluzyFokyfswx5t40P CE/zOKfoeYHLjIYwVtjYM3QQR0Lb1qVLX6UVM7wlWAIz/c/0eT/bsnjqdTtLU1n4QcUc gNeUQqsutK2zqv5mkbwwn1spyu7tjIT2EOsbjq9ibDYdlkdiUtRyhix/uEvIc+7BpzWh YMTu+Rlp0dxrljgmJq/RaxM06gfj6MB8igP6p0FwmENlUhyQ/Dmd7DkNQIoPjHd5442w eXihJMs/4tzwRskm1lItY9qX6p+ACAly0EdMUK8Kg1tB40xZ1sXOWy8JPecCcndlfuzX gdEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=uQBWwMNoXQXaUx2Yo2mSWIgduhZeZIo/BlHYfo7UUHs=; b=GEFbCUUHPvGlpdJTYdF7nX9msKYVAs3Wr/SrWX4pxyc1zCkD39iEmZajLJ7jfTypxZ zez4FhF/YFnkc43rFiqOGVZNj5wA+DpU3raMP8FJ5/fGtsVsHztX2mG4St5aJFplQJt9 XsOMc+PsL606y8DyXWPqJOQEVb46K6/pDjiGfNDjgawj4VO8oOj3nwxvGkcq2YfYSWue buf7Pn+wGmbSl1pl/t8TN9PTLkPIEhExYBeX+ux4PfoLHTnsxTROpmK+gIuIoYl/x9z1 30AO96b6ziOizyYyKceAM2sovMP3A2bgZ5joOoerPC8hrb2FNfVPuB9tGGvFJHuhEM/d +Pww== X-Gm-Message-State: AA+aEWZJret9jXoWhmy83ESnl9JMu+3Z5yFu6gBdypxbk24xkiYkrt19 l280pGDTNisysEh67MdxaD8= X-Google-Smtp-Source: AFSGD/WiVnPPvmZz2xj0uw0DCoVmVI88vlAh+L9uG30s2MLNHhjtCbzbtdXj7PulIZQOHC0W3OBVZA== X-Received: by 2002:a50:880d:: with SMTP id b13mr49717222edb.68.1546722264968; Sat, 05 Jan 2019 13:04:24 -0800 (PST) Received: from macbook.localdomain ([41.237.67.158]) by smtp.gmail.com with ESMTPSA id c23-v6sm16886587ejb.62.2019.01.05.13.04.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 05 Jan 2019 13:04:24 -0800 (PST) Date: Sat, 5 Jan 2019 23:04:20 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190105210420.GA31354@macbook.localdomain> References: <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20190105205314.GA28761@macbook.localdomain> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Jan 05, 2019 at 10:53:14PM +0200, Khaled Hosny wrote: > On Sat, Dec 29, 2018 at 04:49:23PM +0200, Eli Zaretskii wrote: > > > Date: Mon, 24 Dec 2018 19:37:23 +0200 > > > From: Khaled Hosny > > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > > > We need to know, for a given lgstring we are shaping: > > > > > * Its direction (from applying bidi algorithm). Each lgstring we are > > > > > shaping must be of a single direction. > > > > > > > > Communicating this to ftfont_shape_by_hb will need changes in a couple > > > > of interfaces (the existing shaping engines didn't need this > > > > information). I will work on this soon. > > > > > > Great. > > > > Done. Please test. I made sure it compiles, but I couldn't actually > > test the results, as I don't have access to a GNU/Linux system with > > GUI display. So it could be that I misunderstood the Harfbuzz APIs, > > as I was essentially flying blind, guided only by the Harfbuzz docs. > > It seems to work, but still not quite right. You seem to be passing the > paragraph direction, but what HarfBuzz needs is resolved direction of > the text (i.e. the bidi embedding level of the run). In other words, if > Emacs is going to draw this text from right to left, then HarfBuzz must > shape it in right to left direction. Both should use the same direction > all the time and HarfBuzz direction guessing should never be used (i.e. > always pass to it an explicit direction). I pushed a couple of commits that does this based on my limited understanding of Emacs code, please check. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 05 16:15:26 2019 Received: (at 33729) by debbugs.gnu.org; 5 Jan 2019 21:15:26 +0000 Received: from localhost ([127.0.0.1]:48094 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gftHx-0002PY-KT for submit@debbugs.gnu.org; Sat, 05 Jan 2019 16:15:25 -0500 Received: from mail-ed1-f45.google.com ([209.85.208.45]:37290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gftHw-0002PK-EP for 33729@debbugs.gnu.org; Sat, 05 Jan 2019 16:15:24 -0500 Received: by mail-ed1-f45.google.com with SMTP id h15so34636573edb.4 for <33729@debbugs.gnu.org>; Sat, 05 Jan 2019 13:15:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=AelQR6wUNPHqIZgJf54xZ43Kxa1MlKefsIXkKACjdI8=; b=l5geqCA/orlispZkOSC0D/LBQRYMr1iw2YRZFsDCrVWnsEgVJnE2QYUvN8I/p7M4a0 0WapYVeYpT/xVm57XyhTzJrEgK8gZBTMdri447LHx1gEyTIKyB5uBxafpsJkvUJuiaiE fV/ZO6kGQ+kQbqe9rkHoU1WyG7CFj4rU/nVyCbJZuFdOatxpa51aPOGJKhUH2zhwPjSY KQFPqFd6TUAf8slCk4G2fsNplA1c23jk+6x5MrAXC2Cs/YJLFTvy8nGRjqDgBlMvEu/x dSftiarKU/9PKwYjRc6k+SdfdD2LCTVgr4T2D4ILxZKScLuSpWg0/riR3ZWuRmjT34Uz VGWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=AelQR6wUNPHqIZgJf54xZ43Kxa1MlKefsIXkKACjdI8=; b=sDkU4xay2hzBSAJuAn93hlif4eeWQe0PJwKavzxO1r8BEiKG4s4YdQsGD6NrbC/1FM 8gfXu+MiWahrohGhkzGHlOZGcHXrchPUsAwYOIevE0RB2cMBler34vWKdivDC+Ua24Tz ej1Yd19bB0cKSQh3ddoOpuD1crvwxRJCPuchM+fVw+fgSszX7GpLFRGaQhGXJqNOXBU2 2xI1oe8KlU1H1E2/P/HFOruTmhRbKFL5dLWtgY5kPD3eIFcLxdrR8mR0nZoVoZm4mvLc 8YnCgCgK9p+jbaaK34X09kDt/3IemaS98W2qD95EbzIIMTEWTwqsxwV+bKv8/0+V7iLA UxfA== X-Gm-Message-State: AA+aEWa0uXyhl05RDdnC9g3bjVm9oxau/Y7SYzp/eO6WwSYU3+n48Pla BdszCB0Csi6j5K2mupgjkoo= X-Google-Smtp-Source: AFSGD/WdWq/ON9zxUF/kCo5pxgF05GviqTI452/OEbv77o2wYz9A1RIsbSIc7tB2lvI/oLuu22Swcw== X-Received: by 2002:a50:9724:: with SMTP id c33mr50788146edb.288.1546722918570; Sat, 05 Jan 2019 13:15:18 -0800 (PST) Received: from macbook.localdomain ([41.237.67.158]) by smtp.gmail.com with ESMTPSA id t9sm27606596edd.25.2019.01.05.13.15.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 05 Jan 2019 13:15:17 -0800 (PST) Date: Sat, 5 Jan 2019 23:15:14 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190105211514.GB28761@macbook.localdomain> References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <83imzi94tz.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83imzi94tz.fsf@gnu.org> User-Agent: Mutt/1.11.1 (2018-12-01) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, rgm@gnu.org, kaushal.modi@gmail.com, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, Dec 24, 2018 at 08:07:04PM +0200, Eli Zaretskii wrote: > > Date: Mon, 24 Dec 2018 19:37:23 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > Per previous discussions, we decided to use the Harfbuzz built-in > > > methods for determining the script, since Emacs doesn't have this > > > information, and adding it will just do the same as Harfbuzz does, > > > i.e. find the first character whose script is not Common etc., using > > > the UCD database. I think it was you who suggested to use the > > > Harfbuzz built-ins in this case. > > > > The built-in HarfBuzz code is for getting the script for a given > > character, but resolving characters with Common script is left to the > > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > > what HarfBuzz sees during shaping is three separate chunks of text ABC, > > 123, DEF. The 123 part is all Common script characters and thus > > hb_buffer_guess_segment_properties won’t be able to guess anything (and > > based on the font and the script, this can cause rendering differences). > > Emacs will have to resolve the script of Common characters before > > applying bidi algorithm and pass that down to HarfBuzz. > > I'm not sure I understand: why does HarfBuzz care that 123 was in the > middle if RTL text. It doesn’t. What it cares about here is the correct script. Because 123 are in the middle of RTL text they will be shaped separately, and thus hb_buffer_guess_segment_properties() will only see 123 and won’t to be able to guess the correct script for them (Arabic, Hebrew, etc., whatever the script for the surrounding RTL text is). The point I’m trying to make is that script detection, even in its simplest form, needs to be done on the text as a whole not just the portion being shaped, which makes hb_buffer_guess_segment_properties() ill equipped for doing this as it only sees a small portion of the text at a time. > Does it need to shape 123 specially in this case? Depending on the font, the digits might be shaped differently if the script is, say Arabic, by e.g. applying script-specific substitutions to forms more suitable for a given script. > (In general, AFAIK simple characters like 123 will not even go through > HarfBuzz, as Emacs doesn't call the shaper for characters whose entry > in composition-function-table is nil. So I guess 123 here should > stand for some other characters, not for literal digits? IOW, I don't > think I understand the example very well.) This is a bug then and needs to be fixed. All text should go through HarfBuzz since even so-called “simple” character often require shaping depending on the text and the font. If this is done for optimization, then it should be revised to see if shaping with HarfBuzz is actually significantly slower and if it is, find more proper ways to optimize it. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 06 10:51:17 2019 Received: (at 33729) by debbugs.gnu.org; 6 Jan 2019 15:51:18 +0000 Received: from localhost ([127.0.0.1]:48959 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggAhp-0007kt-Jg for submit@debbugs.gnu.org; Sun, 06 Jan 2019 10:51:17 -0500 Received: from eggsout.gnu.org ([209.51.188.92]:57199) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggAho-0007km-Nr for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 10:51:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ggAhi-00007v-T8 for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 10:51:16 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38546) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggAhi-00007m-Q1; Sun, 06 Jan 2019 10:51:10 -0500 Received: from [176.228.60.248] (port=4350 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ggAhi-0003Lw-A4; Sun, 06 Jan 2019 10:51:10 -0500 Date: Sun, 06 Jan 2019 17:50:54 +0200 Message-Id: <83y37xzswx.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20190105205314.GA28761@macbook.localdomain> (message from Khaled Hosny on Sat, 5 Jan 2019 22:53:14 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sat, 5 Jan 2019 22:53:14 +0200 > From: Khaled Hosny > Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org > > > Done. Please test. I made sure it compiles, but I couldn't actually > > test the results, as I don't have access to a GNU/Linux system with > > GUI display. So it could be that I misunderstood the Harfbuzz APIs, > > as I was essentially flying blind, guided only by the Harfbuzz docs. > > It seems to work, but still not quite right. You seem to be passing the > paragraph direction, but what HarfBuzz needs is resolved direction of > the text (i.e. the bidi embedding level of the run). It isn't the paragraph direction; at least it wasn't supposed to be that. The code is (or was before your changes): if (charpos < endpos) { if (pdir == L2R) direction = QL2R; else if (pdir == R2L) direction = QR2L; [...] cmp_it->reversed_p = 0; } else { [...] cmp_it->reversed_p = 1; [...] if (pdir == L2R) direction = QR2L; else if (pdir == R2L) direction = QL2R; [...] } So, as you see, when the paragraph direction is L2R, normal text gets L2R direction, while test reversed for display gets R2L, and the other way around when the paragraph direction is R2L. Which AFAIU is what HarfBuzz needs, but maybe I'm missing something. Did you actually see incorrect display with the code I wrote? If so, could you please show the recipes for reproducing that, preferably with screenshots of correct and incorrect display? I'd like to look into that, to understand what I missed. > HarfBuzz direction guessing should never be used (i.e. always pass > to it an explicit direction). This is in general impossible (or at least very hard), since the shaper is sometimes called from Lisp without any display context. See the Lisp callers of the function font-shape-gstring. One use case is when we want to display the composition information for a grapheme cluster to the user, see descr-text.el (used by the "C-u C-x =" command). In these cases, the UBA is not invoked, and so we don't have the direction information. I could provide the direction information in this case by using the directionality of the base character of the grapheme cluster, but I figured out that HarfBuzz already does this as part of its guessing. Doesn't it? > I pushed a couple of commits that does this based on my limited > understanding of Emacs code, please check. Thanks. Do you see any difference in the results? If so, can you please show the text you used and the results of shaping it with both versions. AFAIU, your code should produce exactly the same results, unless I'm missing something. (I didn't want to use the resolved_level attribute because it is ephemeral, and might not provide the correct value where we are using it.) Btw, did you test both paragraph directions (controlled by the bidi-paragraph-direction variable), and also text inside directional override which changes its natural direction? From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 06 11:04:25 2019 Received: (at 33729) by debbugs.gnu.org; 6 Jan 2019 16:04:25 +0000 Received: from localhost ([127.0.0.1]:48962 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggAuW-00087a-TB for submit@debbugs.gnu.org; Sun, 06 Jan 2019 11:04:25 -0500 Received: from eggs.gnu.org ([208.118.235.92]:47948) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggAuU-00087U-IW for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 11:04:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ggAuS-00024d-64 for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 11:04:22 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38821) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggAuK-0001qw-Oi; Sun, 06 Jan 2019 11:04:12 -0500 Received: from [176.228.60.248] (port=1170 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ggAuJ-0002Rt-C9; Sun, 06 Jan 2019 11:04:11 -0500 Date: Sun, 06 Jan 2019 18:03:55 +0200 Message-Id: <83wonhzsb8.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny , Kenichi Handa In-reply-to: <20190105211514.GB28761@macbook.localdomain> (message from Khaled Hosny on Sat, 5 Jan 2019 23:15:14 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <83imzi94tz.fsf@gnu.org> <20190105211514.GB28761@macbook.localdomain> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Sat, 5 Jan 2019 23:15:14 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > The built-in HarfBuzz code is for getting the script for a given > > > character, but resolving characters with Common script is left to the > > > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > > > what HarfBuzz sees during shaping is three separate chunks of text ABC, > > > 123, DEF. The 123 part is all Common script characters and thus > > > hb_buffer_guess_segment_properties won’t be able to guess anything (and > > > based on the font and the script, this can cause rendering differences). > > > Emacs will have to resolve the script of Common characters before > > > applying bidi algorithm and pass that down to HarfBuzz. > > > > I'm not sure I understand: why does HarfBuzz care that 123 was in the > > middle if RTL text. > > It doesn’t. What it cares about here is the correct script. Because 123 > are in the middle of RTL text they will be shaped separately, and thus > hb_buffer_guess_segment_properties() will only see 123 and won’t to be > able to guess the correct script for them (Arabic, Hebrew, etc., > whatever the script for the surrounding RTL text is). That's what I was asking: why it's important for HarfBuzz to know that 123 should be shaped for the Arabic script? > Depending on the font, the digits might be shaped differently if the > script is, say Arabic, by e.g. applying script-specific substitutions to > forms more suitable for a given script. I guess this is what I'm missing, then: these script-specific substitutions. Can you elaborate on that, or point to some place where these substitutions are described in detail? > > (In general, AFAIK simple characters like 123 will not even go through > > HarfBuzz, as Emacs doesn't call the shaper for characters whose entry > > in composition-function-table is nil. So I guess 123 here should > > stand for some other characters, not for literal digits? IOW, I don't > > think I understand the example very well.) > > This is a bug then and needs to be fixed. All text should go through > HarfBuzz since even so-called “simple” character often require shaping > depending on the text and the font. If this is done for optimization, > then it should be revised to see if shaping with HarfBuzz is actually > significantly slower and if it is, find more proper ways to optimize it. (Adding Handa-san to the discussion, in the hope that he could comment on the issue.) I think running all text through a shaper might be prohibitively expensive, because the shaper is called through Lisp code (see composite.el), and we decide which chunk of text to pass to the shaper using regexp search. See the various files under lisp/language/ which set up portions of composition-function-table as appropriate for each language that needs it. So I think we should identify all the cases where "simple" characters surrounded by, or adjacent to, "non-simple" ones need to be passed to a shaper, and add the necessary regular expressions to the data structures in lisp/languages/. Can you describe these cases, or point me to a place where I can find the relevant info? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 06 12:54:51 2019 Received: (at 33729) by debbugs.gnu.org; 6 Jan 2019 17:54:52 +0000 Received: from localhost ([127.0.0.1]:48977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggCdP-0002Wk-ME for submit@debbugs.gnu.org; Sun, 06 Jan 2019 12:54:51 -0500 Received: from eggsout.gnu.org ([209.51.188.92]:41930) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ggCdL-0002Wb-QQ for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 12:54:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ggCdI-0001s1-4F for 33729@debbugs.gnu.org; Sun, 06 Jan 2019 12:54:47 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41702) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggCdH-0001rM-0P; Sun, 06 Jan 2019 12:54:43 -0500 Received: from [176.228.60.248] (port=4362 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ggCdF-0001hm-Tw; Sun, 06 Jan 2019 12:54:42 -0500 Date: Sun, 06 Jan 2019 19:54:24 +0200 Message-Id: <83pnt9zn73.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20190105210420.GA31354@macbook.localdomain> (message from Khaled Hosny on Sat, 5 Jan 2019 23:04:20 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <20190105210420.GA31354@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sat, 5 Jan 2019 23:04:20 +0200 > From: Khaled Hosny > Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org > > I pushed a couple of commits that does this based on my limited > understanding of Emacs code, please check. Can you explain why you moved the call to hb_buffer_guess_segment_properties _after_ the code which sets some of the properties? I cannot find anything about that in the HarfBuzz documentation. Is this because guessing the unset properties can benefit from knowing the properties which _are_ set, such as the direction? I did it the other way around, because my mental model was: first set the defaults, then override them where better info is available. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 27 12:10:18 2019 Received: (at 33729) by debbugs.gnu.org; 27 Jan 2019 17:10:18 +0000 Received: from localhost ([127.0.0.1]:47831 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnwn-0001qQ-Va for submit@debbugs.gnu.org; Sun, 27 Jan 2019 12:10:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnwm-0001qB-HD for 33729@debbugs.gnu.org; Sun, 27 Jan 2019 12:10:16 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36518) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gnnwh-0007oL-7P; Sun, 27 Jan 2019 12:10:11 -0500 Received: from [176.228.60.248] (port=1227 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gnnwg-0000GT-Am; Sun, 27 Jan 2019 12:10:11 -0500 Date: Sun, 27 Jan 2019 19:09:53 +0200 Message-Id: <83lg363u3i.fsf@gnu.org> From: Eli Zaretskii To: Khaled Hosny In-reply-to: <20190105205314.GA28761@macbook.localdomain> (message from Khaled Hosny on Sat, 5 Jan 2019 22:53:14 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sat, 5 Jan 2019 22:53:14 +0200 > From: Khaled Hosny > Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org > > > Done. Please test. I made sure it compiles, but I couldn't actually > > test the results, as I don't have access to a GNU/Linux system with > > GUI display. So it could be that I misunderstood the Harfbuzz APIs, > > as I was essentially flying blind, guided only by the Harfbuzz docs. > > It seems to work, but still not quite right. You seem to be passing the > paragraph direction, but what HarfBuzz needs is resolved direction of > the text (i.e. the bidi embedding level of the run). In other words, if > Emacs is going to draw this text from right to left, then HarfBuzz must > shape it in right to left direction. Both should use the same direction > all the time and HarfBuzz direction guessing should never be used (i.e. > always pass to it an explicit direction). In response to that, I wrote: It isn't the paragraph direction; at least it wasn't supposed to be that. The code is (or was before your changes): if (charpos < endpos) { if (pdir == L2R) direction = QL2R; else if (pdir == R2L) direction = QR2L; [...] cmp_it->reversed_p = 0; } else { [...] cmp_it->reversed_p = 1; [...] if (pdir == L2R) direction = QR2L; else if (pdir == R2L) direction = QL2R; [...] } So, as you see, when the paragraph direction is L2R, normal text gets L2R direction, while test reversed for display gets R2L, and the other way around when the paragraph direction is R2L. Which AFAIU is what HarfBuzz needs, but maybe I'm missing something. Did you actually see incorrect display with the code I wrote? If so, could you please show the recipes for reproducing that, preferably with screenshots of correct and incorrect display? I'd like to look into that, to understand what I missed. > HarfBuzz direction guessing should never be used (i.e. always pass > to it an explicit direction). This is in general impossible (or at least very hard), since the shaper is sometimes called from Lisp without any display context. See the Lisp callers of the function font-shape-gstring. One use case is when we want to display the composition information for a grapheme cluster to the user, see descr-text.el (used by the "C-u C-x =" command). In these cases, the UBA is not invoked, and so we don't have the direction information. I could provide the direction information in this case by using the directionality of the base character of the grapheme cluster, but I figured out that HarfBuzz already does this as part of its guessing. Doesn't it? > I pushed a couple of commits that does this based on my limited > understanding of Emacs code, please check. Thanks. Do you see any difference in the results? If so, can you please show the text you used and the results of shaping it with both versions. AFAIU, your code should produce exactly the same results, unless I'm missing something. (I didn't want to use the resolved_level attribute because it is ephemeral, and might not provide the correct value where we are using it.) Btw, did you test both paragraph directions (controlled by the bidi-paragraph-direction variable), and also text inside directional override which changes its natural direction? Could you please respond and answer the few questions I asked? I'd like us to continue working on the branch. TIA From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 27 12:12:44 2019 Received: (at 33729) by debbugs.gnu.org; 27 Jan 2019 17:12:44 +0000 Received: from localhost ([127.0.0.1]:47836 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnz9-0001tu-Hq for submit@debbugs.gnu.org; Sun, 27 Jan 2019 12:12:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37195) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnz4-0001te-GT for 33729@debbugs.gnu.org; Sun, 27 Jan 2019 12:12:39 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36644) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gnnyp-0001Q9-Gy; Sun, 27 Jan 2019 12:12:27 -0500 Received: from [176.228.60.248] (port=1367 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gnnyn-0000We-QA; Sun, 27 Jan 2019 12:12:23 -0500 Date: Sun, 27 Jan 2019 19:12:04 +0200 Message-Id: <83h8du3tzv.fsf@gnu.org> From: Eli Zaretskii To: dr.khaled.hosny@gmail.com In-reply-to: <83wonhzsb8.fsf@gnu.org> (message from Eli Zaretskii on Sun, 06 Jan 2019 18:03:55 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <83imzi94tz.fsf@gnu.org> <20190105211514.GB28761@macbook.localdomain> <83wonhzsb8.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, handa@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Could you please respond to the below as well? > Date: Sun, 06 Jan 2019 18:03:55 +0200 > From: Eli Zaretskii > Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org > > > Date: Sat, 5 Jan 2019 23:15:14 +0200 > > From: Khaled Hosny > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > The built-in HarfBuzz code is for getting the script for a given > > > > character, but resolving characters with Common script is left to the > > > > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > > > > what HarfBuzz sees during shaping is three separate chunks of text ABC, > > > > 123, DEF. The 123 part is all Common script characters and thus > > > > hb_buffer_guess_segment_properties won’t be able to guess anything (and > > > > based on the font and the script, this can cause rendering differences). > > > > Emacs will have to resolve the script of Common characters before > > > > applying bidi algorithm and pass that down to HarfBuzz. > > > > > > I'm not sure I understand: why does HarfBuzz care that 123 was in the > > > middle if RTL text. > > > > It doesn’t. What it cares about here is the correct script. Because 123 > > are in the middle of RTL text they will be shaped separately, and thus > > hb_buffer_guess_segment_properties() will only see 123 and won’t to be > > able to guess the correct script for them (Arabic, Hebrew, etc., > > whatever the script for the surrounding RTL text is). > > That's what I was asking: why it's important for HarfBuzz to know that > 123 should be shaped for the Arabic script? > > > Depending on the font, the digits might be shaped differently if the > > script is, say Arabic, by e.g. applying script-specific substitutions to > > forms more suitable for a given script. > > I guess this is what I'm missing, then: these script-specific > substitutions. Can you elaborate on that, or point to some place > where these substitutions are described in detail? > > > > (In general, AFAIK simple characters like 123 will not even go through > > > HarfBuzz, as Emacs doesn't call the shaper for characters whose entry > > > in composition-function-table is nil. So I guess 123 here should > > > stand for some other characters, not for literal digits? IOW, I don't > > > think I understand the example very well.) > > > > This is a bug then and needs to be fixed. All text should go through > > HarfBuzz since even so-called “simple” character often require shaping > > depending on the text and the font. If this is done for optimization, > > then it should be revised to see if shaping with HarfBuzz is actually > > significantly slower and if it is, find more proper ways to optimize it. > > (Adding Handa-san to the discussion, in the hope that he could comment > on the issue.) > > I think running all text through a shaper might be prohibitively > expensive, because the shaper is called through Lisp code (see > composite.el), and we decide which chunk of text to pass to the shaper > using regexp search. See the various files under lisp/language/ which > set up portions of composition-function-table as appropriate for each > language that needs it. > > So I think we should identify all the cases where "simple" characters > surrounded by, or adjacent to, "non-simple" ones need to be passed to > a shaper, and add the necessary regular expressions to the data > structures in lisp/languages/. Can you describe these cases, or point > me to a place where I can find the relevant info? > > Thanks. > > > > From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 27 12:13:08 2019 Received: (at 33729) by debbugs.gnu.org; 27 Jan 2019 17:13:08 +0000 Received: from localhost ([127.0.0.1]:47840 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnzY-0001v1-2j for submit@debbugs.gnu.org; Sun, 27 Jan 2019 12:13:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37368) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gnnzW-0001uU-LN for 33729@debbugs.gnu.org; Sun, 27 Jan 2019 12:13:06 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36682) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gnnzR-0001uk-Hq; Sun, 27 Jan 2019 12:13:01 -0500 Received: from [176.228.60.248] (port=1412 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gnnzR-0000az-5G; Sun, 27 Jan 2019 12:13:01 -0500 Date: Sun, 27 Jan 2019 19:12:45 +0200 Message-Id: <83ftte3tyq.fsf@gnu.org> From: Eli Zaretskii To: dr.khaled.hosny@gmail.com In-reply-to: <83pnt9zn73.fsf@gnu.org> (message from Eli Zaretskii on Sun, 06 Jan 2019 19:54:24 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <20190105210420.GA31354@macbook.localdomain> <83pnt9zn73.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sun, 06 Jan 2019 19:54:24 +0200 > From: Eli Zaretskii > Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org > > > Date: Sat, 5 Jan 2019 23:04:20 +0200 > > From: Khaled Hosny > > Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org > > > > I pushed a couple of commits that does this based on my limited > > understanding of Emacs code, please check. > > Can you explain why you moved the call to > hb_buffer_guess_segment_properties _after_ the code which sets some of > the properties? I cannot find anything about that in the HarfBuzz > documentation. Is this because guessing the unset properties can > benefit from knowing the properties which _are_ set, such as the > direction? > > I did it the other way around, because my mental model was: first set > the defaults, then override them where better info is available. > > Thanks. Please respond. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 29 17:26:15 2019 Received: (at 33729) by debbugs.gnu.org; 29 Jan 2019 22:26:15 +0000 Received: from localhost ([127.0.0.1]:50911 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobpf-0000cn-Ii for submit@debbugs.gnu.org; Tue, 29 Jan 2019 17:26:15 -0500 Received: from mail-wm1-f51.google.com ([209.85.128.51]:33016) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobpd-0000ca-6K for 33729@debbugs.gnu.org; Tue, 29 Jan 2019 17:26:13 -0500 Received: by mail-wm1-f51.google.com with SMTP id r24so14030798wmh.0 for <33729@debbugs.gnu.org>; Tue, 29 Jan 2019 14:26:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=nnOqJtG2W1l+4FHp29MoNuVGkq+tymuiswByMiEvwLY=; b=StrGEur11uHFON75mAVhpCYMnqlnEFyOFEsnBHhf/MLLIja5SrwJAsqw6c+V4dhhyo sUz1Pu3E4usEk/XXbKrA3tzs5zv5MOLVYUeMoRUIUKs0Etpi+9ZUt+ROjsGoOCaFFmrK 2Cej6X3YWr324UQlya8kGSI2gLTK7fg7k24+rtVZxUPMN7URXTVdRv66kfVypOc0/U3c +r1x0H4gEugmtrxGD/2Rm4Kn4YESzjZlGjks2sn18lgr0UHu896YPSjH2jl9ZAAqo3aj hvjO0+XgFiCNRYsPHPMWqu0Wk8hqIUQ+tg7MPqVbV7w6wIptMFv0iZBBMHUm+4VXwhbH N4gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=nnOqJtG2W1l+4FHp29MoNuVGkq+tymuiswByMiEvwLY=; b=TSJ5vOQMzPIqCKrRCG9BMJbxhnnFFC31A6IhELUR/oNYBsNFw0Ld3vQOrZMPfd6OxB +WynF/n5H60E53tSiVB7AzZDS6bbdLuynJ1hT9DuWYUUg2KvixEURxGjDsjkPMkdBUTG hbVBqAIugZJXkaDOVvAZ16qQMXsJN2hYcEFSkVLbLxCAtDbCG22V1eNRrTca+6OAScOx yG00yy+PaNUhoGJoDm5v9BnuYBYNggoPbSnBUyOOHYBOhPHBTZrycbAncndBggINCel3 WOnB5+1PIAir186NffPFabR9yUxMxVuEOusSkcugJ1RHP4Lw6NHFuC5UcoR2gm9ySW5m Zueg== X-Gm-Message-State: AJcUukd6/1Tc7hCo9TD5Y1FkVtTUx3Q/N6Hvs4y4Gkp8WytycBe3dUCI CqRHGf8jdKh6QB9h0Bbnevs= X-Google-Smtp-Source: ALg8bN5wDrgLsKF/0Bf6nDkpm81Jp1dLZlzNpM+rnQsGElIKuYyLWJ75Q6jOX95JyJZQExaoZHFmYw== X-Received: by 2002:a1c:a913:: with SMTP id s19mr21811394wme.4.1548800767254; Tue, 29 Jan 2019 14:26:07 -0800 (PST) Received: from macbook.localdomain ([197.58.129.117]) by smtp.gmail.com with ESMTPSA id f130sm72862wme.41.2019.01.29.14.25.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Jan 2019 14:26:05 -0800 (PST) Date: Wed, 30 Jan 2019 00:25:36 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190129222536.GA20784@macbook.localdomain> References: <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <83imzi94tz.fsf@gnu.org> <20190105211514.GB28761@macbook.localdomain> <83wonhzsb8.fsf@gnu.org> <83h8du3tzv.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <83h8du3tzv.fsf@gnu.org> User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: 33729@debbugs.gnu.org, handa@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, Jan 27, 2019 at 07:12:04PM +0200, Eli Zaretskii wrote: > Could you please respond to the below as well? I have no time for angering these questions any more, sorry. Please feel free to do what you find sensible. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 29 17:29:38 2019 Received: (at 33729) by debbugs.gnu.org; 29 Jan 2019 22:29:38 +0000 Received: from localhost ([127.0.0.1]:50915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobsw-0000hQ-2q for submit@debbugs.gnu.org; Tue, 29 Jan 2019 17:29:38 -0500 Received: from mail-wm1-f53.google.com ([209.85.128.53]:35595) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobst-0000h9-Sn for 33729@debbugs.gnu.org; Tue, 29 Jan 2019 17:29:36 -0500 Received: by mail-wm1-f53.google.com with SMTP id t200so19690963wmt.0 for <33729@debbugs.gnu.org>; Tue, 29 Jan 2019 14:29:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=0m8cwCuQ82+6gzIKdbxcnClPxSla5t4Ch5niiQJNV7A=; b=E8vn8VWZ1bIDJS+qhp710BFht/OarRPDSPmMkmPCu6/GxxUnqW5r3WMJ91M66KRZio SpWUcyE9AeD7PjAPOiCGTxk/oZXBY8rKdRYjiIepCGCZThGc0D3Ymq1piVKzl5tU7bHu bKZ7omWCDWd2ZVyjbjnU6bfSTAnn8F2ijjoYcd18olUSPrjOQr96UQfn0YqkPx3bo//l Iu4AesD9N83Fn0SiVwKlCqDpz9R+057kKEoHbufcUXIBPvcx9lQl+tKApEMmrcZmutPb AdvmDj3FQAwJv1sq7DQ3kvCLuYc89ZvZ61GuZcjZN1H0pBuyreXdGYK/9/3DDWP7H2oH jqUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=0m8cwCuQ82+6gzIKdbxcnClPxSla5t4Ch5niiQJNV7A=; b=NvGH7GJpIEb7Z01qYeRvw+KJ3MKG8qqGg1aQidjvr6x8fhZn9ge+gnvTev+Y3xKcuz WFSEiO7ayHqQa76BNFldOlc+xF4I3Y9pH02XYFf4wTOyKLw/N13PRpC2QNo5AnJ6itfG hpvMYwUfaVnB3+iKzjSs20yZ7AfeiI53enq1KCcaNM8GmUch3IupUm8LHsg0LyNnVwhe YOTzluGdm/8VuqMWG5rlzc8v0p6JgsADp8Nj/0b7T2F/dqGxxxj4GBUGdlLyYmWzCCqv VdgP0dbtLf3/yt0yoVE9+GS5pP0/wa4wgZ8FAzgNjAzilAjOPDz2qSKb4+wYQfy2CDLm yMFw== X-Gm-Message-State: AJcUukfaVsN/3aeq9d/Rtb2p9IlWGv99eqH3p7LhyM9Un8V15YSCMD7X 5hcDo1HTWXQzyokthnWXs9o= X-Google-Smtp-Source: ALg8bN6rUSaf1OAGZTpGQSg5VCu113BOaAEd5NlzZAiTB1SlqaS/0mLj2z/Fu2AGw158BDFdJxq8HQ== X-Received: by 2002:a1c:f916:: with SMTP id x22mr23900309wmh.87.1548800970083; Tue, 29 Jan 2019 14:29:30 -0800 (PST) Received: from macbook.localdomain ([197.58.129.117]) by smtp.gmail.com with ESMTPSA id m21sm71776wmi.43.2019.01.29.14.29.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Jan 2019 14:29:29 -0800 (PST) Date: Wed, 30 Jan 2019 00:29:03 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190129222903.GB20784@macbook.localdomain> References: <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <83y37xzswx.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <83y37xzswx.fsf@gnu.org> User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, Jan 06, 2019 at 05:50:54PM +0200, Eli Zaretskii wrote: > > Date: Sat, 5 Jan 2019 22:53:14 +0200 > > From: Khaled Hosny > > I pushed a couple of commits that does this based on my limited > > understanding of Emacs code, please check. > > Thanks. Do you see any difference in the results? Strings with forced direction (e.g. Arabic with LRO) showed difference. Without my change they were shaped RTL by drawn LTR, with my change shaping and drawing used LTR direction. Please feel free to revert that change if you think it is incorrect. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 29 17:34:09 2019 Received: (at 33729) by debbugs.gnu.org; 29 Jan 2019 22:34:09 +0000 Received: from localhost ([127.0.0.1]:50931 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobxI-0000qH-Rw for submit@debbugs.gnu.org; Tue, 29 Jan 2019 17:34:09 -0500 Received: from mail-wm1-f54.google.com ([209.85.128.54]:54701) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gobxG-0000pm-SS for 33729@debbugs.gnu.org; Tue, 29 Jan 2019 17:34:07 -0500 Received: by mail-wm1-f54.google.com with SMTP id a62so19626379wmh.4 for <33729@debbugs.gnu.org>; Tue, 29 Jan 2019 14:34:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=sjpl2j4h3lwodH5ZNN8SBKjW0MscgxcsPGHfxezSapQ=; b=AwtGu5L9LkJqX4Rm6bRKXiUt5rVrAmmd8glIgG/+SBJIhyg799KP4CySc0g8og/bhP 2WhOe+xurnOZIu13WzNPWrVnJweBYGGAkvgHv6U5T9p//+TvmLgiBjsJrNW9U60WHW+W 1c+HLiWn7x00BhkrXa8GOZiJ+7DV8g9YIL6GSFZxYn5kV21bHmxO0rZgFAEebTzeM+5d D6LXo8uwho+L/P5Qn371RwZv1C16K6BAHS5E0VwXfzlst6wzaNGsZDLimdi1UOIrmAfG 9NvMx3IdMC2i+GQgWw+DYVJxrzLNRnO7Iw27i/r2d+ua7J2znxt4IfeDzWG5S4A8SGvv pwEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=sjpl2j4h3lwodH5ZNN8SBKjW0MscgxcsPGHfxezSapQ=; b=TNVbZe7ANHIJflA/KQuKhGj+VJk8TY1qXvInXqS3hkk/7dWws/wrmkq1uyHCOHFrR1 WYVWfywy3EzUL/I+boUkH/xRrkPGQ3FqZMM+l/6e6nbUox3PnCVLbDjJYFboKUE4byjj 7zD+DHTw6J1bdXlv1Fnm/bRbtcoQahLwFwGolBm6HHetd/sN93LHQsqBuGtC6olR0MBQ Qy3S2sYSNC6kUEX7aBsKd6yq46xDzGsQzKqQa5M1i2YhS2lbUEzg6uUylgwctWtwWS/A R3PpyTNel84eoVLZiYO7p5kilYh/cMGMqpVci/pR2aJAJoV2Iw0PH8V0VgdD9PGeRgyS MrAw== X-Gm-Message-State: AJcUukdXtInfkVPy/uJhyJBqh/2LzXaRqNHK7YM6j0wTxnOfDDbHA4N9 5nM4TMcC1XXZ+hCsDxXsz8ywR6W0 X-Google-Smtp-Source: ALg8bN7SXuo5p1oRpVjHHG2k6mQboFN1AKlHfHrqMCl430Tz5Jp2ywCgWz/kn7tHtpoV0Zv5rqccAA== X-Received: by 2002:a1c:d00d:: with SMTP id h13mr22308758wmg.13.1548801240587; Tue, 29 Jan 2019 14:34:00 -0800 (PST) Received: from macbook.localdomain ([197.58.129.117]) by smtp.gmail.com with ESMTPSA id g188sm96124wmf.32.2019.01.29.14.33.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Jan 2019 14:33:59 -0800 (PST) Date: Wed, 30 Jan 2019 00:33:30 +0200 From: Khaled Hosny To: Eli Zaretskii Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Message-ID: <20190129223330.GC20784@macbook.localdomain> References: <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <20190105210420.GA31354@macbook.localdomain> <83pnt9zn73.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83pnt9zn73.fsf@gnu.org> User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, Jan 06, 2019 at 07:54:24PM +0200, Eli Zaretskii wrote: > > Date: Sat, 5 Jan 2019 23:04:20 +0200 > > From: Khaled Hosny > > Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org > > > > I pushed a couple of commits that does this based on my limited > > understanding of Emacs code, please check. > > Can you explain why you moved the call to > hb_buffer_guess_segment_properties _after_ the code which sets some of > the properties? I cannot find anything about that in the HarfBuzz > documentation. Is this because guessing the unset properties can > benefit from knowing the properties which _are_ set, such as the > direction? hb_buffer_guess_segment_properties() won’t guess set properties, so moving it last was to avoid wasting time guessing properties that we will override later anyway. > I did it the other way around, because my mental model was: first set > the defaults, then override them where better info is available. hb_buffer_guess_segment_properties() is not for setting defaults (there is no such thing as default buffer properties in HarfBuzz working model), it is a kind of quick and dirty hack and production code should not use it. Regards, Khaled From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 29 08:48:08 2022 Received: (at 33729) by debbugs.gnu.org; 29 Apr 2022 12:48:08 +0000 Received: from localhost ([127.0.0.1]:51141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkQ2a-0000D1-Id for submit@debbugs.gnu.org; Fri, 29 Apr 2022 08:48:08 -0400 Received: from quimby.gnus.org ([95.216.78.240]:37478) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkQ2Z-0000C3-2Z for 33729@debbugs.gnu.org; Fri, 29 Apr 2022 08:48:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=5mZ04oQw30HXbMHVYdcAtIkpGYNIDCNYFzYOmLQfUXw=; b=tmAV2/JPcRk26kcER66qD7R9n7 FSZNwuC8A4shan4rU261FhNDhvVooLJjFjJqZeqY0GrK8NnBI2GG0dpNYlnZm9MGx4MvQUE0XbEn8 S0ScPTS4myn8B37ZaADNEyQ1Q1CqChI+rzzwwS/0M9gYO1F88pKLkTxjZLH6BCqrXx9c=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nkQ2P-0008JR-4D; Fri, 29 Apr 2022 14:47:59 +0200 From: Lars Ingebrigtsen To: Khaled Hosny Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <83y37xzswx.fsf@gnu.org> <20190129222903.GB20784@macbook.localdomain> X-Now-Playing: Colleen's _A Flame My Love, A Frequency_: "A Flame My Love, A Frequency" Date: Fri, 29 Apr 2022 14:47:55 +0200 In-Reply-To: <20190129222903.GB20784@macbook.localdomain> (Khaled Hosny's message of "Wed, 30 Jan 2019 00:29:03 +0200") Message-ID: <87pml0dpj8.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Khaled Hosny writes: >> Thanks. Do you see any difference in the results? > > Strings with forced direction (e.g. Arabic with LRO) showed difference. > Without my change they were shaped RTL by drawn LTR, with my change > [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 33729 Cc: behdad@behdad.org, Eli Zaretskii , far.nasiri.m@gmail.com, 33729@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Khaled Hosny writes: >> Thanks. Do you see any difference in the results? > > Strings with forced direction (e.g. Arabic with LRO) showed difference. > Without my change they were shaped RTL by drawn LTR, with my change > shaping and drawing used LTR direction. Please feel free to revert that > change if you think it is incorrect. (I'm going through old bug reports that unfortunately weren't resolved at the time.) Skimming this long bug report, it seems like the fixes Khaled pushed fixed the reported issue (but I may well be misreading). Eli, is there anything more to do here? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 29 09:24:13 2022 Received: (at 33729-done) by debbugs.gnu.org; 29 Apr 2022 13:24:13 +0000 Received: from localhost ([127.0.0.1]:51264 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkQbU-0001Dp-T6 for submit@debbugs.gnu.org; Fri, 29 Apr 2022 09:24:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51470) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkQbT-0001Dd-Cj for 33729-done@debbugs.gnu.org; Fri, 29 Apr 2022 09:24:11 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48492) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nkQbN-0003Z5-Nz; Fri, 29 Apr 2022 09:24:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=5DNKYb4xQrSe+Ob3cH0aWJOsPw15I9+0wzha7fKiK7g=; b=F5M4WSpUCp7T W23jd6GwepJzCy2/6HKqUb1jcgvzgGsSHcU9Hjg0wus9WbDCKkGfSx/NvPgsUKtjQ+IQWMZTBAKZn JORMMNyEHUOiYY0hxyFbeqAKQjvcAhl/wDq+nwPbEJlnkWG+fs9o/SPvjB634TcLbWYG6EAUvY5Bl qKaz+O8U3z6hGLV5Ardr8NlYDtcFVQRhKDzlHdR+tOcAi2siqTP27vGpKeUU8yQY81T6R3d8xld/e foEYz1cHlgPcpAhxiCMEP+Ox+5R5/A2uwqYlrzHto+BAoPizDrq5nGhcdQ5eiqTmkuvT4V4LNhR4A H9c/qEWhRYLp9kWwkn9w1w==; Received: from [87.69.77.57] (port=3246 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nkQbN-0007Yl-2m; Fri, 29 Apr 2022 09:24:05 -0400 Date: Fri, 29 Apr 2022 16:24:05 +0300 Message-Id: <83r15g9g5m.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-Reply-To: <87pml0dpj8.fsf@gnus.org> (message from Lars Ingebrigtsen on Fri, 29 Apr 2022 14:47:55 +0200) Subject: Re: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) References: <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> <838t085qx8.fsf@gnu.org> <20190105205314.GA28761@macbook.localdomain> <83y37xzswx.fsf@gnu.org> <20190129222903.GB20784@macbook.localdomain> <87pml0dpj8.fsf@gnus.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 33729-done Cc: dr.khaled.hosny@gmail.com, behdad@behdad.org, far.nasiri.m@gmail.com, 33729-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Lars Ingebrigtsen > Cc: Eli Zaretskii , behdad@behdad.org, > far.nasiri.m@gmail.com, 33729@debbugs.gnu.org > Date: Fri, 29 Apr 2022 14:47:55 +0200 > > Khaled Hosny writes: > > >> Thanks. Do you see any difference in the results? > > > > Strings with forced direction (e.g. Arabic with LRO) showed difference. > > Without my change they were shaped RTL by drawn LTR, with my change > > shaping and drawing used LTR direction. Please feel free to revert that > > change if you think it is incorrect. > > (I'm going through old bug reports that unfortunately weren't resolved > at the time.) > > Skimming this long bug report, it seems like the fixes Khaled pushed > fixed the reported issue (but I may well be misreading). > > Eli, is there anything more to do here? No. The original problem was fixed, and a couple of followup issues were also fixed. So I'm closing this bug. From unknown Thu Jun 19 14:31:00 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 28 May 2022 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator