From unknown Sun Jun 15 08:21:11 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#52459 <52459@debbugs.gnu.org> To: bug#52459 <52459@debbugs.gnu.org> Subject: Status: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t Reply-To: bug#52459 <52459@debbugs.gnu.org> Date: Sun, 15 Jun 2025 15:21:11 +0000 retitle 52459 28.0.90; prin1-to-string does not escape bidi control charact= ers despite print-escape-control-characters=3Dt reassign 52459 emacs submitter 52459 Daniel Mendler severity 52459 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 12 15:13:21 2021 Received: (at submit) by debbugs.gnu.org; 12 Dec 2021 20:13:21 +0000 Received: from localhost ([127.0.0.1]:53226 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwVDl-0002D8-1h for submit@debbugs.gnu.org; Sun, 12 Dec 2021 15:13:21 -0500 Received: from lists.gnu.org ([209.51.188.17]:38928) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwVDk-0002D1-0C for submit@debbugs.gnu.org; Sun, 12 Dec 2021 15:13:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44632) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwVDj-0004tx-CR for bug-gnu-emacs@gnu.org; Sun, 12 Dec 2021 15:13:19 -0500 Received: from [2a01:4f8:121:346::180] (port=59689 helo=mail.qxqx.de) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwVDg-0007wO-Nz for bug-gnu-emacs@gnu.org; Sun, 12 Dec 2021 15:13:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Date: Message-ID:Subject:From:To:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=2SPhhSXO1S0JOZk5SveOCAFDOuiTWXPm/wfjkdaPAjg=; b=ZafLf0mzuE6DgGaeYfNa66CmlC 7Ow2ohIQLF5ahY7MpWTNDnsmMRVpV8yc/mMep+cwDjjFJlGvIt9ujn8XWDaYsz4dYTiFy9QtKBFFY zOwHPw4sYAgwE0dqOE6jsXS8unIJTsUwjJrsI8uE1FdTFzoeIapDs6VIVVCE/IjnWsaE=; To: bug-gnu-emacs@gnu.org From: Daniel Mendler Subject: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t Message-ID: Date: Sun, 12 Dec 2021 21:13:12 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a01:4f8:121:346::180 (failed) Received-SPF: pass client-ip=2a01:4f8:121:346::180; envelope-from=mail@daniel-mendler.de; helo=mail.qxqx.de X-Spam_score_int: -33 X-Spam_score: -3.4 X-Spam_bar: --- X-Spam_report: (-3.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_MED=-2.3, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) 1. Start emacs -Q 2. Enter the following in the scratch buffer: (let ((print-escape-control-characters t)) (prin1-to-string bidi-directional-controls-chars)) 3. Evaluate. The bidi control characters are not escaped despite print-escape-control-characters=t. The bidi characters should probably be treated as control characters since they have the Bidi_Control property according to the Unicode standard. If it is undesirable to treat bidi control characters like other control characters it may make sense to introduce another print configuration variable, print-escape-all-control-characters or print-escape-bidi-control-characters? In GNU Emacs 28.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.5, cairo version 1.16.0) of 2021-12-05 built on projects Repository revision: 34f56561372d83b71dcaff1cdf5d9264ba38fa0e Repository branch: emacs-28 Windowing system distributor 'The X.Org Foundation', version 11.0.12004000 System Description: Debian GNU/Linux 10 (buster) Configured using: 'configure --prefix=/home/user/emacs/install --with-cairo' Configured features: CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json map text-property-search time-date subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils thingatpt help-fns radix-tree help-mode cl-loaddefs cl-lib two-column iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 57250 10751) (symbols 48 6852 1) (strings 32 21147 1607) (string-bytes 1 651816) (vectors 16 13342) (vector-slots 8 183349 13347) (floats 8 22 48) (intervals 56 260 1) (buffers 992 13)) From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 12 15:42:22 2021 Received: (at 52459) by debbugs.gnu.org; 12 Dec 2021 20:42:22 +0000 Received: from localhost ([127.0.0.1]:53278 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwVfp-0002vq-P5 for submit@debbugs.gnu.org; Sun, 12 Dec 2021 15:42:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48152) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwVfn-0002vc-II for 52459@debbugs.gnu.org; Sun, 12 Dec 2021 15:42:19 -0500 Received: from [2001:470:142:3::e] (port=39626 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwVfh-0006I9-7C; Sun, 12 Dec 2021 15:42:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=2uGuzVBShhJdin1cMzh25aFFqqXBSdb342zpqBM7qVE=; b=Xg0YOtyzPU05 Y6u+yfE8wXiyAnbE9D4i1iW8IgxcrDz0hrJY443IhbRVziulvqI3wiTeIZtXSzCN/PMjBy3cpFUTM u46eT7XxBe75MK1wo7+Xzsh558NS66xtbnUppgGC0Hrcwvw93mmL00Mivu89O0hGtWZFrb/l98mRV bMVbW8do4vjDm6rbmo5G4jQO7kIr0oasjsuulZ5r/I+/J2oBfDFLjtYZ6LzgacHO7SOMb403LzFZi 3mW2zyJYwUAO/FEeFmd81bW81lhmNZxmAxxdKaolxJvpjxMXxk1Et9Qzig6e/DWPOaHzwjtk2BHeQ Pke39C8VGnAScVo5dAqfog==; Received: from [87.69.77.57] (port=2781 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwVfh-0002MV-1U; Sun, 12 Dec 2021 15:42:13 -0500 Date: Sun, 12 Dec 2021 22:42:06 +0200 Message-Id: <83v8ztmu75.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: (message from Daniel Mendler on Sun, 12 Dec 2021 21:13:12 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Daniel Mendler > Date: Sun, 12 Dec 2021 21:13:12 +0100 > > 1. Start emacs -Q > 2. Enter the following in the scratch buffer: > (let ((print-escape-control-characters t)) > (prin1-to-string bidi-directional-controls-chars)) > 3. Evaluate. The bidi control characters are not escaped despite > print-escape-control-characters=t. > > The bidi characters should probably be treated as control characters > since they have the Bidi_Control property according to the Unicode > standard. print-escape-control-characters is about ASCII control characters, not about Unicode fomatting controls. > If it is undesirable to treat bidi control characters like other control > characters it may make sense to introduce another print configuration > variable, print-escape-all-control-characters or > print-escape-bidi-control-characters? I don't think it's desirable. Those formatting controls have starkly different roles that ASCII control characters, and we already have features to make them stand out on display. Moreover, escape sequences are not well-defined for codepoints beyond a single byte. So I don't think we should do anything here, and we should close the bug. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 12 16:11:48 2021 Received: (at 52459) by debbugs.gnu.org; 12 Dec 2021 21:11:48 +0000 Received: from localhost ([127.0.0.1]:53294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwW8J-0003dK-P4 for submit@debbugs.gnu.org; Sun, 12 Dec 2021 16:11:48 -0500 Received: from server.qxqx.de ([178.63.65.180]:43175 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwW8H-0003d7-Vp for 52459@debbugs.gnu.org; Sun, 12 Dec 2021 16:11:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=BOX5z7YzQx7oD4XLHd0DPSFAGWFLF0pQZ5hsLwA9Hpk=; b=mrjFsNQxKo19qA84IfjRhlYdDD ZkWQc2zy6VE8GqLzhv/kdjFRGGHLFLrTekFjblJ8/kChzOWg5nkHxfra3i6wDpAlmFXCTa6jeCYqZ FhpCSEvDipIXu48IoPQVJnDNqzT+2453FzD66ceFA02QWbV4RH8wXm5neXrt1frqLfLU=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> From: Daniel Mendler Message-ID: Date: Sun, 12 Dec 2021 22:11:38 +0100 MIME-Version: 1.0 In-Reply-To: <83v8ztmu75.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> 1. Start emacs -Q >> 2. Enter the following in the scratch buffer: >> (let ((print-escape-control-characters t)) >> (prin1-to-string bidi-directional-controls-chars)) >> 3. Evaluate. The bidi control characters are not escaped despite >> print-escape-control-characters=t. >> >> The bidi characters should probably be treated as control characters >> since they have the Bidi_Control property according to the Unicode >> standard. > > print-escape-control-characters is about ASCII control characters, not > about Unicode fomatting controls. I see, but this is not a satisfactory answer. >> If it is undesirable to treat bidi control characters like other control >> characters it may make sense to introduce another print configuration >> variable, print-escape-all-control-characters or >> print-escape-bidi-control-characters? > > I don't think it's desirable. Those formatting controls have starkly > different roles that ASCII control characters, and we already have > features to make them stand out on display. Moreover, escape > sequences are not well-defined for codepoints beyond a single byte. > > So I don't think we should do anything here, and we should close the > bug. I disagree. I would like to turn this report into a feature request then. I've observed that multiple packages which escape strings and print them for debugging or help purposes have difficulties with bidi characters. All I would like to have is a way to print strings safely, such that control characters (any kind of control characters, ascii or unicode) do not affect the output. Example packages which are affected by this issue are for example the Helpful package, which provides an enhanced Help buffer. Another package affected by the issue is the Marginalia package which adds annotations to `describe-variable` in the minibuffer. The annotations show the variable value. I would like to print the variable values in a safe way which does not mess up the display. Instead of "string" these packages show "str"gnandallthatfollowsisgarbage. How is this supposed to be done? I propose the addition of an additional variable which configures prin1-string such that all control characters which affect the display in special ways are escaped. Is there an alternative approach to achieve this goal? From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 12 16:34:01 2021 Received: (at 52459) by debbugs.gnu.org; 12 Dec 2021 21:34:01 +0000 Received: from localhost ([127.0.0.1]:53331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwWTo-0004C6-VZ for submit@debbugs.gnu.org; Sun, 12 Dec 2021 16:34:01 -0500 Received: from server.qxqx.de ([178.63.65.180]:39875 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwWTm-0004Br-U6 for 52459@debbugs.gnu.org; Sun, 12 Dec 2021 16:33:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:References:Cc:To:From:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=0KEpT7tUzCVlRXiYeoWbj1R+BV18jM7OruL5v+YmOdo=; b=FNyElso4FNxxWINerZHdB0L3hu gEPXRtYNCFH3W5T0AEn5Bd0nCbQ6JNPsEhbuh9SHBOXLcQLDKEXgn3q6pBb7DfSvjIFtFsz8G9GKO DBfLwxRIsNScF8JcZkdLcJeBLe8nTSnPaYImfT78jEVbDuCGmuNro4p0kmajWalti/WQ=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t From: Daniel Mendler To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> Message-ID: <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> Date: Sun, 12 Dec 2021 22:33:50 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/12/21 10:11 PM, Daniel Mendler wrote: > Example packages which are affected by this issue are for example the > Helpful package, which provides an enhanced Help buffer. Another package > affected by the issue is the Marginalia package which adds annotations > to `describe-variable` in the minibuffer. The annotations show the > variable value. I would like to print the variable values in a safe way > which does not mess up the display. Instead of "string" these packages > show "str"gnandallthatfollowsisgarbage. How is this supposed to be done? > > I propose the addition of an additional variable which configures > prin1-string such that all control characters which affect the display > in special ways are escaped. Is there an alternative approach to achieve > this goal? There is actually one function which comes close in functionality - `bidi-string-mark-left-to-right`. However this function is not really a pure string manipulation function since it adds display properties. So this function can only be used if the string is directly displayed as is. The function is not a good fit if the resulting string is manipulated further, truncated, etc. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 07:22:53 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 12:22:53 +0000 Received: from localhost ([127.0.0.1]:54299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwkM1-0006Mo-3T for submit@debbugs.gnu.org; Mon, 13 Dec 2021 07:22:53 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwkLz-0006MX-1m for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 07:22:51 -0500 Received: from [2001:470:142:3::e] (port=56882 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwkLq-0001ne-TJ; Mon, 13 Dec 2021 07:22:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=rVk9aNPYuOW5Sqel+ac/8TFVGQLXOb5XsVhBgRHlRto=; b=q0sMaJ4zfr34 Y/bIa7hCDoEWSSKyRiWenmIopNxNy4mHgDCPF0vP19JmjLnZHsZqEVTLstk1S1TxzfJ0EQ1xYybzS doQZhli90aEVAr213mKikRNnAT6MqTEYinAF/aX4pKynAPbJL4sUCXiiHeeI2htr/iqODzdaax+Ib JoSpl2D+iyEFsZ4tb5aL1WMPFRjqFUdOXIQuGdo6M04psIzIRHy7kmZh3Q8gEUQ1iko1uP8xyNrcO Wje5BCR3EYSsUiMC4UCUevzgLN4gfBpX+GvoGkfIqoTT8sS0EV9TFngAeicHtjLFzPXx3Ntlx5k2g lYzbkpeTlQSEBNDv0GPRLA==; Received: from [87.69.77.57] (port=4405 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwkLo-0006zD-NX; Mon, 13 Dec 2021 07:22:41 -0500 Date: Mon, 13 Dec 2021 14:22:35 +0200 Message-Id: <83r1agn184.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> (message from Daniel Mendler on Sun, 12 Dec 2021 22:33:50 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Daniel Mendler > Cc: 52459@debbugs.gnu.org > Date: Sun, 12 Dec 2021 22:33:50 +0100 > > On 12/12/21 10:11 PM, Daniel Mendler wrote: > > Example packages which are affected by this issue are for example the > > Helpful package, which provides an enhanced Help buffer. Another package > > affected by the issue is the Marginalia package which adds annotations > > to `describe-variable` in the minibuffer. The annotations show the > > variable value. I would like to print the variable values in a safe way > > which does not mess up the display. Instead of "string" these packages > > show "str"gnandallthatfollowsisgarbage. How is this supposed to be done? > > > > I propose the addition of an additional variable which configures > > prin1-string such that all control characters which affect the display > > in special ways are escaped. Is there an alternative approach to achieve > > this goal? > > There is actually one function which comes close in functionality - > `bidi-string-mark-left-to-right`. However this function is not really a > pure string manipulation function since it adds display properties. So > this function can only be used if the string is directly displayed as > is. The function is not a good fit if the resulting string is > manipulated further, truncated, etc. Sorry, I don't understand. The examples you provided are of text being displayed. Which is expected, since these controls have no other effect _except_ when the text is displayed. So why isn't the existing function bidi-string-mark-left-to-right (which was introduced precisely for the situations like you describe, and is actually used in Emacs for those purposes) the solution for the class of problems that you described? And can we agree that displaying these characters as escapes would not solve the problems you had in mind, so it is off the table for the rest of this discussion? From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 08:19:11 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 13:19:11 +0000 Received: from localhost ([127.0.0.1]:54404 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwlEV-0001V3-An for submit@debbugs.gnu.org; Mon, 13 Dec 2021 08:19:11 -0500 Received: from server.qxqx.de ([178.63.65.180]:60507 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwlES-0001Un-K7 for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 08:19:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=djXHnPiYnjWMuhfkzwdahJ44CrbeH6pAtwJkqmNJquw=; b=qtuIAjwyTImdyRrcMzE/6aAYzA VkkA41VvKyz4cJSU/X6ehKKkzVj/x7n9Hf+ldquB8rqheB2yYGWVeD01c2PhkPI7Qmuaj3Vw6hy2v IUSOXWnlw8vtRoONLxDUdaSF0OSApQpLJEPyik8YYsSerCVKxU98uS7e2y/aHVXwGsnw=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> From: Daniel Mendler Message-ID: <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> Date: Mon, 13 Dec 2021 14:19:00 +0100 MIME-Version: 1.0 In-Reply-To: <83r1agn184.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Sorry, I don't understand. The examples you provided are of text > being displayed. Which is expected, since these controls have no > other effect _except_ when the text is displayed. So why isn't the > existing function bidi-string-mark-left-to-right (which was introduced > precisely for the situations like you describe, and is actually used > in Emacs for those purposes) the solution for the class of problems > that you described? `bidi-string-mark-left-to-right` is an insufficient solution since it manipulates the string on the level of display properties. It appends an invisible character. If I take the string returned by `bidi-string-mark-left-to-right` I cannot manipulate it freely afterwards. In particular if I truncate the string, the invisible character will be lost again. I need a function which sanitizes a string completely, such that after sanitization I can use and manipulate it without having to worry about display properties or other pecularities. > And can we agree that displaying these characters as escapes would not > solve the problems you had in mind, so it is off the table for the > rest of this discussion? No, I disagree. This should not be off the table. I don't understand why you want to close this issue so quickly. The probem I described is an actual problem, which I've observed in multiple packages. Ideally Emacs would offer a solution on the API level such that package authors and users can sanitize strings in a robust way. Such an API does not exist currently. Escaping all control characters is my preferred solution. What about adding two variables: `print-escape-unicode-control-characters` and `print-escape-ascii-control-characters`, such that it is explicit what is going on? The variable `print-escape-control-characters` could be deprecated or aliased to `print-escape-ascii-control-characters`. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 08:30:24 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 13:30:24 +0000 Received: from localhost ([127.0.0.1]:54411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwlPM-0001mE-Ds for submit@debbugs.gnu.org; Mon, 13 Dec 2021 08:30:24 -0500 Received: from server.qxqx.de ([178.63.65.180]:53057 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwlPJ-0001m0-In for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 08:30:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:References:Cc:To:From:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=96VvO1dhoHBuWBq8HdAu8NoGYHEVpmId8xe8kvoMTzU=; b=E4h99qWseGX4jowOvM2rsQjQAy 8RjCfATEJKP6FkpBX7I2Q9ZKqB6ZizxhnNJl8wCFMoEKLYS6s2Mc6cQBKN/e2JAjUDvu89dJAuBYF fjMdLQste4HLG1GsrxwZrjfbmu0BNjpKgiPFmLHguGAFAndCiBWrJYYLawH9+UornjZQ=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t From: Daniel Mendler To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> Message-ID: <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> Date: Mon, 13 Dec 2021 14:30:13 +0100 MIME-Version: 1.0 In-Reply-To: <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) In other words, there is a need for a functionality which makes it possible to turn a string into a string literal in a form which could be used in source code for example. If you look at the definition of bidi-directional-controls-chars in simple.el, the bidi characters are escaped there. Why is this? Why did you write this definition in this form and not with unescaped bidi characters? (defvar bidi-directional-controls-chars "\x202a-\x202e\x2066-\x2069" "Character set that matches bidirectional formatting control characters.") If I set `print-escape-control-characters` or some hypothetical `print-escape-unicode-control-characters` to t, I expect a string literal of this form. The current behavior of Emacs is counter intuitive since it does not escape all control characters but singles out only the ASCII control characters. This behavior does not seem correct given full Emacs support in Emacs. Therefore my proposal to add two variables `print-escape-ascii-control-characters` and `print-escape-unicode-control-characters`. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 10:24:57 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 15:24:57 +0000 Received: from localhost ([127.0.0.1]:56592 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwnCD-00054f-1s for submit@debbugs.gnu.org; Mon, 13 Dec 2021 10:24:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58772) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwnC8-00054Q-Rj for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 10:24:55 -0500 Received: from [2001:470:142:3::e] (port=34314 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwnC2-0007tV-C3; Mon, 13 Dec 2021 10:24:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=kvdIIFk7sEBLjMHwtZ92Jfc/4GTpsCb3yT06eSazesc=; b=dGQanli5u9yt d50+4dVj7ToPq9UYRxrsniUNrZYbpSklm6Syh5mb0FGjHpYfcL3djPk8hHsjbqspGMoZ55Xl0A+/G ejfKZiwnfmK4H5yaz8F+qcOA5qDquYy4IQA2R5wyVPh27XeoU1BE70bkKfGrFnToI+eogg5Uy7o1K +hp9Q/GgIyVcD9j7Se35lohTY4BJz7+/kOnHHmtG8NPFYZAb74n8ix9Y4LScVbzycrIrhos5cjuAZ FAuhqsfZ9XRXwx8j2ABiQJefXZVVJWEuYbQO2TQ1U6ymPZputuhfa8tfxZdgAePa0UFJRMy+GgZRm qGznSxPr3GdOxTIujAmyxQ==; Received: from [87.69.77.57] (port=3979 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwnC2-0006aD-6I; Mon, 13 Dec 2021 10:24:46 -0500 Date: Mon, 13 Dec 2021 17:24:41 +0200 Message-Id: <8335mwmssm.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> (message from Daniel Mendler on Mon, 13 Dec 2021 14:30:13 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Daniel Mendler > Cc: 52459@debbugs.gnu.org > Date: Mon, 13 Dec 2021 14:30:13 +0100 > > In other words, there is a need for a functionality which makes it > possible to turn a string into a string literal in a form which could be > used in source code for example. > > If you look at the definition of bidi-directional-controls-chars in > simple.el, the bidi characters are escaped there. Why is this? Why did > you write this definition in this form and not with unescaped bidi > characters? > > (defvar bidi-directional-controls-chars "\x202a-\x202e\x2066-\x2069" > "Character set that matches bidirectional formatting control characters.") So you want a feature that would produce strings suitable for using in program source files, like we did in the above example? Is that the meaning of "sanitize" you have been using? Are there other use cases for those "escaped" or "sanitized" strings? If so, please describe them as well. Or if that's not the correct meaning of "sanitized", please define it more accurately. You see, this discussion is hard because I still don't understand what is it that you want Emacs to provide, and for what purposes. Please try to clarify that, to make the discussion more efficient and avoid misunderstandings. For now, I understand that those strings are not necessarily required to be readable on the Emacs display, at least not in all cases, because some of the reordering that these controls produce will be disabled when they are represented by ASCII escapes, and the character order on display will change as result. If the string includes RTL characters, the result might not be easily readable. But AFAIU, this is not a problem for the use cases you have in mind? > Therefore my proposal to add two variables > `print-escape-ascii-control-characters` and > `print-escape-unicode-control-characters`. I'd prefer to wait with concrete proposals until the requirements are clear. It seems like a variable like those you mention, which only affect the Emacs display, but not the string contents, is not what you need. You need to actually produce the ASCII characters \x2023a, so that you could put them in a string like you show above. Or am I misunderstanding again? From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 11:32:58 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 16:32:58 +0000 Received: from localhost ([127.0.0.1]:56660 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwoFx-0006s1-9d for submit@debbugs.gnu.org; Mon, 13 Dec 2021 11:32:58 -0500 Received: from server.qxqx.de ([178.63.65.180]:52715 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwoFt-0006rk-My for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 11:32:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=QCX9EOpbiHWBZIrHgf6OwaI4nrPhNgSKU+tE78PLXXg=; b=VXPjMPYtwCNyxMCEMWjFgIy8Sf 8ZtcH84MTZdKZMt6mWn8pKywGDG0CFqbjvUgQA/o9rdR9v9hFBURklTmq/mTHwksuyIlK01fPKJUD VRtXZkK1morOwNEL7mtxxh00x15ksCetw9hHcLDBi1R/TOEOdCAcaioYTa07Gp4nlEPc=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> From: Daniel Mendler Message-ID: <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> Date: Mon, 13 Dec 2021 17:32:40 +0100 MIME-Version: 1.0 In-Reply-To: <8335mwmssm.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.0 (--) On 12/13/21 4:24 PM, Eli Zaretskii wrote: > So you want a feature that would produce strings suitable for using in > program source files, like we did in the above example? Is that the > meaning of "sanitize" you have been using? > > Are there other use cases for those "escaped" or "sanitized" strings? > If so, please describe them as well. As you said the use case is to produce strings suitable in source files or strings in a form which looks like strings occurring in source files. This use case appears in debuggers and other UIs which inspect variable values at Emacs runtime. Furthermore code editing and refactoring tools produce strings which are supposed to be used directly in source files. For the usage of strings in source files on could simply use `print-escape-multibyte=t` however in particular in debugger UIs this leads to a severe obfuscation of the output, which in particular hurts users wo use Emacs in a setup with a multi-byte bidi language, Hebrew, Arabic, Chinese, etc. Therefore in debugger UIs I only want to escape control characters but not other multi-byte display characters. There are packages which provide such debugging or live inspection of values, I already mentioned the Helpful (MELPA) package and the Marginalia (GNU ELPA) package. Both of these variable inspection utilities are affected by the problem of how to escape string literals or print Emacs values properly, transforming the value to a printable representation as it would appear in source. > For now, I understand that those strings are not necessarily required > to be readable on the Emacs display, at least not in all cases, > because some of the reordering that these controls produce will be > disabled when they are represented by ASCII escapes, and the character > order on display will change as result. If the string includes RTL > characters, the result might not be easily readable. But AFAIU, this > is not a problem for the use cases you have in mind? Yes, if one escapes bidi control characters the readability of the result is affected, but not as severely as with `print-escape-multibyte=t`. The goal is to produce strings which don't contain hidden characters for debugging, so even if readability is affected it is still not as bad as with `print-escape-multibyte=t`. Note that I am not a native speaker and only an occasional user of multi-byte bidi languages, e.g., for educational purpose. Therefore I cannot tell how such bidi strings are usually written in source code. But I suspect that in source code literals ideally all non-visible control characters are written in escaped form. However visible multi-byte characters may be written literally in source code. At least this is the practice I am using in source code regarding other Unicode characters, I may write displayable characters as is but I will always escape control characters (no difference between ASCII or Unicode). From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 12:07:29 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 17:07:29 +0000 Received: from localhost ([127.0.0.1]:56680 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwonR-0007j0-Cu for submit@debbugs.gnu.org; Mon, 13 Dec 2021 12:07:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58358) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwonM-0007ik-MU for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 12:07:28 -0500 Received: from [2001:470:142:3::e] (port=37182 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwonH-0000Fi-4E; Mon, 13 Dec 2021 12:07:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=EQ49dzMRDwzYkrNfVlmlaTWlnqPNwxiUQw3McFDs8Ro=; b=CNP2c3c5ffOR nHCKQmGU17c+h4An5tOTvhG++JLIG4MtJCsgTWVmn4lX07BlKZbXUQF8ns+Ylr0PudgsJ1mgseBQi gPLZfUaEsSe9mZR5mQ86hqu5Bnhbvt0ZqAApOj3G6eg8TigH39SnKeU1yFBHMWIlQHRupelEwyhJk m+SMYRXXumAXyiUiobCrS/gq59UgPUDIjvX6O9uz3s8mfMZenmVatompS1HTjvtKlunhRLu6W1xB/ 5ywX//jg6AhJlvbcnVPu274Zey1c+rGIYeIGvWajX/RqVtXTNll9VaDGsNLBO41U9hATb9Hm1gMOp 1wxyBrWyC4vVuWpNUTGQfg==; Received: from [87.69.77.57] (port=2263 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwonG-0006TS-TY; Mon, 13 Dec 2021 12:07:19 -0500 Date: Mon, 13 Dec 2021 19:07:13 +0200 Message-Id: <83wnk8l9ha.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> (message from Daniel Mendler on Mon, 13 Dec 2021 17:32:40 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Mon, 13 Dec 2021 17:32:40 +0100 > > On 12/13/21 4:24 PM, Eli Zaretskii wrote: > > So you want a feature that would produce strings suitable for using in > > program source files, like we did in the above example? Is that the > > meaning of "sanitize" you have been using? > > > > Are there other use cases for those "escaped" or "sanitized" strings? > > If so, please describe them as well. > > As you said the use case is to produce strings suitable in source files > or strings in a form which looks like strings occurring in source files. > This use case appears in debuggers and other UIs which inspect variable > values at Emacs runtime. Furthermore code editing and refactoring tools > produce strings which are supposed to be used directly in source files. > > For the usage of strings in source files on could simply use > `print-escape-multibyte=t` however in particular in debugger UIs this > leads to a severe obfuscation of the output, which in particular hurts > users wo use Emacs in a setup with a multi-byte bidi language, Hebrew, > Arabic, Chinese, etc. Therefore in debugger UIs I only want to escape > control characters but not other multi-byte display characters. So there are two different use cases: 1) produce strings for using in program source files. 2) produce strings for display in various UIs The solutions should IMO be different, because the first is not about displaying these characters, while the second is about displaying them. For 1), is print-escape-multibyte satisfactory? If not, why not? For 2), we now have in Emacs 29 the glyphless-display-mode, whereby the bidi control characters are shown as small boxes with their acronyms (RLE, FSI, PDI, etc.). Is that satisfactory? If not, why not? From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 13:13:44 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 18:13:44 +0000 Received: from localhost ([127.0.0.1]:56719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwppX-0000qh-TB for submit@debbugs.gnu.org; Mon, 13 Dec 2021 13:13:44 -0500 Received: from server.qxqx.de ([178.63.65.180]:36867 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwppV-0000qI-CL for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 13:13:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ES7ZMoP0sRWRBL5ZhrEuWeRQH2axNFrzU47IhoN0Lxs=; b=Ajtn7Vu+/jCkc16AKMf4STjhet LXaeHMPUXmSi10ud0fhcbmOqZT0tR7TMmG6vXVTswne+2L25/FAvGKfNl0em1c4K7aoi9B8pw0+VO RQhT6eZOW1B5evsOLw7EICArkNECEze6M/lp/HyeWW8aOH/KtsnVgvqGpclWCdz+bc5I=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> From: Daniel Mendler Message-ID: <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> Date: Mon, 13 Dec 2021 19:13:32 +0100 MIME-Version: 1.0 In-Reply-To: <83wnk8l9ha.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > 1) produce strings for using in program source files. > 2) produce strings for display in various UIs > > The solutions should IMO be different, because the first is not about > displaying these characters, while the second is about displaying > them. No, they are not different for my purposes since I want to have the ability to copy strings from the UI to a source file. Working around the problem on the display level (glyphless-display-mode) will preclude this use case. > For 1), is print-escape-multibyte satisfactory? If not, why not? I already explained this. `print-escape-multibyte` obfuscates the string too much, which is undesirable for a debugging UI. Note that I am passing on this experience report from a Russian user who observed that Marginalia (which currently uses `print-escape-multibyte=t`) produces output which is not as helpful as it could be thanks to the escaping of all multi byte characters. The escaping hurts users of multi-byte languages. > For 2), we now have in Emacs 29 the glyphless-display-mode, whereby > the bidi control characters are shown as small boxes with their > acronyms (RLE, FSI, PDI, etc.). Is that satisfactory? If not, why > not? The `glyphless-display-mode` would be a possible workaround if I just pass on the characters unescaped. However I want to produce strings which I can possibly copy to source code buffers. This is not possible if the strings are not escaped and contain the problematic control characters in literal form. Once again - I propose the addition of configuration variables which configure `prin1-string` to produce output where all control characters are escaped. I would even argue that current variable `print-escape-control-characters` is misleading since it only encodes Ascii control characters. Is there anything which prevents the addition of a configuration variable `print-escape-unicode-control-characters`, which ensures full escaping of *all control characters* or we could even further and add `print-escape-glyphless-characters` which would treat the same characters as `glyphless-display-mode`. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 13:28:58 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 18:28:58 +0000 Received: from localhost ([127.0.0.1]:56726 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwq48-0001C3-95 for submit@debbugs.gnu.org; Mon, 13 Dec 2021 13:28:58 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwq43-0001Bl-1E for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 13:28:46 -0500 Received: from [2001:470:142:3::e] (port=39762 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwq3x-0005T9-Az; Mon, 13 Dec 2021 13:28:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=MLrj23F5dcUlymYoBSpGLoUCUbeA1Q8IVNahB/zCz+I=; b=VlJ/te9wbcbc TVPxWOe2PTforl1kjTHddGw078awziKFplHieBMnez/Gu8VzxOhVadKTuu8fj/Hgn6gikrL6mn0Q4 p3pV6bkyX0feUjYzXPwhWHwWRiVn0lzTZxf48whhI9kEzcfwZw4YWMHIJN5THPdKvsOzzaiamQ1VE nCENt/JEejPBNVUats2oTqdgZZrU8uI2l43+kggBt1AMmhQzuF5tDMPex0BSB7VdRrLzex5vEjeND 1WIOggu1tiZABzl82Z0cjyXXqZfOEgLkZ263zWGRxvq66nahmCRmxjMMPdLL6H4nCesB32UQJSGR9 NR7BsKjTxLm7FCzbVfzILA==; Received: from [87.69.77.57] (port=3458 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwq3v-0006sy-3b; Mon, 13 Dec 2021 13:28:37 -0500 Date: Mon, 13 Dec 2021 20:28:10 +0200 Message-Id: <83tufcl5qd.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> (message from Daniel Mendler on Mon, 13 Dec 2021 19:13:32 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Mon, 13 Dec 2021 19:13:32 +0100 > > > 1) produce strings for using in program source files. > > 2) produce strings for display in various UIs > > > > The solutions should IMO be different, because the first is not about > > displaying these characters, while the second is about displaying > > them. > > No, they are not different for my purposes since I want to have the > ability to copy strings from the UI to a source file. Working around the > problem on the display level (glyphless-display-mode) will preclude this > use case. But in the debug UI use-case, you do want to see the text and be able to read it, don't you? Which is why you said you don't want to have escapes there, you want to see characters. Which means that use-case is about _displaying_ these characters, not _replacing_ them with something else. And you _cannot_ copy anything that only exists on display, because that copies the actual codepoints, not their visual representation. > > For 1), is print-escape-multibyte satisfactory? If not, why not? > > I already explained this. `print-escape-multibyte` obfuscates the string > too much, which is undesirable for a debugging UI. Note that I am > passing on this experience report from a Russian user who observed that > Marginalia (which currently uses `print-escape-multibyte=t`) produces > output which is not as helpful as it could be thanks to the escaping of > all multi byte characters. The escaping hurts users of multi-byte languages. The use case you describe with Marginalia is of a different kind -- why do they use print-escape-multibyte in that case? Cyrillic text doesn't use bidi controls, so what does that use case have to do with your request? What I'm suggesting is to use print-escape-multibyte when producing strings for inclusion in the source code, and only for that purpose. You, OTOH, are talking about case 2), where these strings are presented in a UI. Then of course print-escape-multibyte is inappropriate for that. > Once again - I propose the addition of configuration variables which > configure `prin1-string` to produce output where all control characters > are escaped. That could help in case 1), but not in case 2), because there prin1 is not used, or not necessarily used. > `print-escape-control-characters` is misleading since it only encodes > Ascii control characters. Is there anything which prevents the addition > of a configuration variable `print-escape-unicode-control-characters`, > which ensures full escaping of *all control characters* Escaping where? on display? That won't help you to write strings as in bidi-directional-controls-chars. I thought this was one part of your request? But I already said that, and so it sounds like we have some grave misunderstanding that I'm unable to resolve. So maybe it's time for someone else to try. Sorry I couldn't be of more help. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 13:35:49 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 18:35:49 +0000 Received: from localhost ([127.0.0.1]:56755 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqAv-0001Nw-JP for submit@debbugs.gnu.org; Mon, 13 Dec 2021 13:35:49 -0500 Received: from server.qxqx.de ([178.63.65.180]:39345 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqAs-0001Ng-1h for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 13:35:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6EZL9iQdAZyF0QbeA0uEoCMvjBtKPnbbBH0K9ILmMhI=; b=VYvhkwxIo8ZWTVOIg45L/ncodo tQXFrdMxlFboTFwOilnJBrUEOnD+fY1/uITLNARb+eFJk8ZskwqHaLd01xeqQs+JR4RH6/mrmX2s0 kk9CvZAFXR8Tz5KzwMRwgsaaZzr9GbL2y6Xubx90Uj2nV8Ae1lzgJm0/ayze/VelIBcY=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> From: Daniel Mendler Message-ID: <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> Date: Mon, 13 Dec 2021 19:35:37 +0100 MIME-Version: 1.0 In-Reply-To: <83tufcl5qd.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/13/21 7:28 PM, Eli Zaretskii wrote: > But in the debug UI use-case, you do want to see the text and be able > to read it, don't you? Which is why you said you don't want to have > escapes there, you want to see characters. Which means that use-case > is about _displaying_ these characters, not _replacing_ them with > something else. And you _cannot_ copy anything that only exists on > display, because that copies the actual codepoints, not their visual > representation. I want to escape *only control characters* not all multi byte characters. > The use case you describe with Marginalia is of a different kind -- > why do they use print-escape-multibyte in that case? Cyrillic text > doesn't use bidi controls, so what does that use case have to do with > your request? Of course Cyrillic text is not using bidi but it is still affected by print-escape-multibyte. If I have a debugging UI I want it to work with all kinds of languages, rtl and ltr. Multi-byte and single-byte. > What I'm suggesting is to use print-escape-multibyte when producing > strings for inclusion in the source code, and only for that purpose. > You, OTOH, are talking about case 2), where these strings are > presented in a UI. Then of course print-escape-multibyte is > inappropriate for that. This is not good enough, I want to produce strings which can be copied to the source and presented in the UI in the same form. I argue that this is not an unreasonable requirement. >> Once again - I propose the addition of configuration variables which >> configure `prin1-string` to produce output where all control characters >> are escaped. > > That could help in case 1), but not in case 2), because there prin1 is > not used, or not necessarily used. I am only taking about prin1. The issue is about prin1. My goal is to produce safely escaped string representations of Elisp values, including strings and other values. > But I already said that, and so it sounds like we have some grave > misunderstanding that I'm unable to resolve. So maybe it's time for > someone else to try. Sorry I couldn't be of more help. Yes. Maybe someone else can chime in with their opinion. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 13:52:23 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 18:52:23 +0000 Received: from localhost ([127.0.0.1]:56812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqQx-0001qD-6E for submit@debbugs.gnu.org; Mon, 13 Dec 2021 13:52:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56268) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqQt-0001py-9a for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 13:52:21 -0500 Received: from [2001:470:142:3::e] (port=40224 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwqQn-0001I7-Kl; Mon, 13 Dec 2021 13:52:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=3YyFb/a41JwiMgEXeqL1jnPFxb/al8XMmniO20TDmuE=; b=ZamVwh0t4mkb HIdGSFEbV6a2T6raohrQbVtW4kMl9v1cOY/Kmi2NF2tbrHXSh+U1WN3IwJZTcJsRGQgVk5Um2lukq BMczwYjatKOuNn5Ze7KwzBNVdO4bzD5U/PW5sHXZv5z6eU3YSEPYYDIb3eI3LU8rmq5kiNV2BDYCF UDhopUAeFHHkymckZ+HUDRxFZgrHExBEYlfAuapa/Y3Xv1KYXCn8kmTIQWTcXR6tzhRZtJO+g2Sax 0k2Ryu7SAZnKYfMzgc+GUi3h2dsTBN7tKtJhmBznHJ1ELO/COxsw1tfvMV7JxDjm1kM874c9VCVaJ j00YJaoMBjqzIrJFmU95LA==; Received: from [87.69.77.57] (port=4925 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwqQn-0003Ki-Bh; Mon, 13 Dec 2021 13:52:13 -0500 Date: Mon, 13 Dec 2021 20:52:08 +0200 Message-Id: <83sfuwl4mf.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> (message from Daniel Mendler on Mon, 13 Dec 2021 19:35:37 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Mon, 13 Dec 2021 19:35:37 +0100 > > > What I'm suggesting is to use print-escape-multibyte when producing > > strings for inclusion in the source code, and only for that purpose. > > You, OTOH, are talking about case 2), where these strings are > > presented in a UI. Then of course print-escape-multibyte is > > inappropriate for that. > > This is not good enough, I want to produce strings which can be copied > to the source and presented in the UI in the same form. I argue that > this is not an unreasonable requirement. Not only is it unreasonable, it is simply impossible. Representing characters _on_display_ and writing such a representation into a file, as in simple.el, are two different and incompatible goals. The solutions for them must be separate. I already explained why, and if my explanations still don't convince you, then I'm sorry, but I cannot help you more than that, because it means we don't have a common language and understanding to discuss this stuff. > >> Once again - I propose the addition of configuration variables which > >> configure `prin1-string` to produce output where all control characters > >> are escaped. > > > > That could help in case 1), but not in case 2), because there prin1 is > > not used, or not necessarily used. > > I am only taking about prin1. The issue is about prin1. My goal is to > produce safely escaped string representations of Elisp values, including > strings and other values. Once again: prin1 will not help with displaying these characters. Emacs doesn't use prin1 to display text. > > But I already said that, and so it sounds like we have some grave > > misunderstanding that I'm unable to resolve. So maybe it's time for > > someone else to try. Sorry I couldn't be of more help. > > Yes. Maybe someone else can chime in with their opinion. Yes, please. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 13:58:08 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 18:58:08 +0000 Received: from localhost ([127.0.0.1]:56816 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqWV-0001yI-Ut for submit@debbugs.gnu.org; Mon, 13 Dec 2021 13:58:08 -0500 Received: from server.qxqx.de ([178.63.65.180]:53835 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqWQ-0001xk-IK for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 13:58:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=HBGqlQN1YnxkcGk1a5fJD8RWj91OZpWcDfw12YyziaI=; b=XEqs7sli/y+I4Sq5vq9NOuNv09 Lp1qGhQDRdjCkFMoVcjOb9ad1xgGV4jzucdh+1vYrsJalIGlMQbq4QFt+0Dhm3SSxnjwvgPNESDUX q2olS5d1thA2Eg+v1Yfog53ZBfYoBuaxAED/CHJOCJAjGOo4QRWeCrAhXPBQTKWgVHFc=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> From: Daniel Mendler Message-ID: Date: Mon, 13 Dec 2021 19:57:54 +0100 MIME-Version: 1.0 In-Reply-To: <83sfuwl4mf.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/13/21 7:52 PM, Eli Zaretskii wrote: > Not only is it unreasonable, it is simply impossible. Representing > characters _on_display_ and writing such a representation into a file, > as in simple.el, are two different and incompatible goals. The > solutions for them must be separate. I already explained why, and if > my explanations still don't convince you, then I'm sorry, but I cannot > help you more than that, because it means we don't have a common > language and understanding to discuss this stuff. I produce strings from Elisp values using `prin1-to-string`. These strings should be escaped such that I can use copy them to source files as is. Furthermore when I display the string in the UI, the string should not mess up the display. This requires the string to not have control characters. You are seriously misunderstanding what I am proposing. Please consider my proposal: I propose the addition of a variable `print-escape-unicode-control-characters` which ensures that `prin1-to-string` returns a string where all control characters are escaped. This proposal is certainly not impossible. Currently `prin1-to-string` produces a string which contains bidi control characters despite `print-escape-control-characters=t`. >>>> Once again - I propose the addition of configuration variables which >>>> configure `prin1-string` to produce output where all control characters >>>> are escaped. >>> >>> That could help in case 1), but not in case 2), because there prin1 is >>> not used, or not necessarily used. >> >> I am only taking about prin1. The issue is about prin1. My goal is to >> produce safely escaped string representations of Elisp values, including >> strings and other values. > > Once again: prin1 will not help with displaying these characters. > Emacs doesn't use prin1 to display text. Of course not. I am using prin1 to create a string from a value which I can then copy to a source file or display somewhere in the UI. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 14:09:04 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 19:09:04 +0000 Received: from localhost ([127.0.0.1]:56832 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqh6-0002FY-3S for submit@debbugs.gnu.org; Mon, 13 Dec 2021 14:09:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:60098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqh3-0002En-79 for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 14:09:01 -0500 Received: from [2001:470:142:3::e] (port=40922 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwqgx-0003le-JE; Mon, 13 Dec 2021 14:08:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=R71J3QEpxsQEb6wrh187IyCr417NTa7Xg6Pugv2CwSg=; b=KwZMOgYHbkeF VLG1bfIbmtCB2U1kpwaeEsB+zXkqBxuYIOWyoEUOhZdxnWbXGbAl0GUFtfZOctWjoSGxUfRWNoUbv FXL/2Ujf7wk+l9Qz4utyvzm3x+31hXP1X4oFybbqIF9RpCkPFqhqebNF7b7w/8/yTsCy2+OMV7/Hx LeGrgYPvvAvg583z/69DKdErLPOG3nPw6EjfwYcIL1IPJDF1vwFfP9Sf82OQTDdbMwTZ+BK59pS7A I3dPbSgMz+SCJBi7YSVqWSwddhfHulKSO3kdpMBzEUWYiCjWEyVpL5+YzctyX8cXuYPzFJZtk1hvI 0MxU92D4HClN9qXFebCItQ==; Received: from [87.69.77.57] (port=1969 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwqgw-0002CY-IA; Mon, 13 Dec 2021 14:08:55 -0500 Date: Mon, 13 Dec 2021 21:08:49 +0200 Message-Id: <83r1agl3um.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: (message from Daniel Mendler on Mon, 13 Dec 2021 19:57:54 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Mon, 13 Dec 2021 19:57:54 +0100 > > > Once again: prin1 will not help with displaying these characters. > > Emacs doesn't use prin1 to display text. > > Of course not. I am using prin1 to create a string from a value which I > can then copy to a source file or display somewhere in the UI. That will solve only a small fraction of situations where these characters are displayed, because the vast majority of them don't use prin1 to produce a string to display. Most of the stuff displayed by Emacs doesn't come from strings produced by prin1, it comes from displaying the text of some buffer. So your proposal is incomplete at best. Again, I already tried to explain that several times. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 14:17:05 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 19:17:05 +0000 Received: from localhost ([127.0.0.1]:56846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqor-0002RX-AO for submit@debbugs.gnu.org; Mon, 13 Dec 2021 14:17:05 -0500 Received: from server.qxqx.de ([178.63.65.180]:50167 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqop-0002Qz-A1 for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 14:17:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=wqOrr0Y0MsiyO7W9aF2rQeeQIqhptZJAakL1qHG3zwI=; b=HD1L4AwIeF2rpTyUiXXIkpezX3 Yz/95sJJREwlVF/5YdXCP4A1NjxBxeEYWP1flXsvgl0b59qYOMPv4dv+IB+syMOx9ZJJUbMGwq2nI qnVRM568PS1bdWWMx7/m61Au1vOcWlSzR2EBIdcZZNFb2N1huE6EIkpzL3tIXIoRWUdk=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> From: Daniel Mendler Message-ID: <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> Date: Mon, 13 Dec 2021 20:16:54 +0100 MIME-Version: 1.0 In-Reply-To: <83r1agl3um.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/13/21 8:08 PM, Eli Zaretskii wrote: >> Cc: 52459@debbugs.gnu.org >> From: Daniel Mendler >> Date: Mon, 13 Dec 2021 19:57:54 +0100 >> >>> Once again: prin1 will not help with displaying these characters. >>> Emacs doesn't use prin1 to display text. >> >> Of course not. I am using prin1 to create a string from a value which I >> can then copy to a source file or display somewhere in the UI. > > That will solve only a small fraction of situations where these > characters are displayed, because the vast majority of them don't use > prin1 to produce a string to display. Most of the stuff displayed by > Emacs doesn't come from strings produced by prin1, it comes from > displaying the text of some buffer. Yes, it solves a small fraction of situations. This issue does not address a generic setting, it only addresses the behavior of `prin1`. As I described at length there are packages which are affected by this and which could improve from an improvement of `prin1`. To break it down once more: 1. We have the function `prin1-to-string` which can be used to produce a string representation for an Emacs lisp value. 2. The behavior of the function can be adjusted via configuration variables, in particular `print-escape-multibyte` and `print-escape-control-characters`. `print-escape-multibyte` is very aggressive, it escapes every multibyte character. `print-escape-control-characters` only escapes ASCII control characters. 3. I am asking for a way to configure `prin1-to-string` such that it escapes control and other glyphless characters but not all multibyte characters, such that text still stays somewhat readable. I want less aggressive escaping than `print-escape-multibyte`. If I set only `print-escape-control-characters=t` is not sufficient since it escapes only ASCII control characters. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 13 14:38:46 2021 Received: (at 52459) by debbugs.gnu.org; 13 Dec 2021 19:38:46 +0000 Received: from localhost ([127.0.0.1]:56882 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwr9p-0002yH-VO for submit@debbugs.gnu.org; Mon, 13 Dec 2021 14:38:46 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38962) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwr9m-0002y0-3O for 52459@debbugs.gnu.org; Mon, 13 Dec 2021 14:38:45 -0500 Received: from [2001:470:142:3::e] (port=42160 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwr9g-0008W4-Bs; Mon, 13 Dec 2021 14:38:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=N0AaKjVARnJ54nWMjZ7CD+xi++DHuUmMcs4+UrZII8U=; b=oOH4a1+PXZYZ IkVg5rvIDNul5OGtbOusHujvfexfG+GHA/vEliRRffV9xLd20TqAD23uW3v/d+vFyyLIaJsRaSEjS SH6b19KjsA/STfGad8DKnUMxfgKS9ycsb0tw22jtdlAdudht/+42JJLa6R+vOnwrqeFjDEyxK0Ota XXfNZ9dRQcVV0CKB5arGGJOQrqs+uNz0Q84oOiD5nNWcRD0TzXp8aykgiFrSe2JKz03zBpi+6v385 exFfqAhbkwyxjJiJmSLyDPL52tjeSmm2ILGMpUmR9/nT3s04MZs+FhGJc3yaqkClE8zKAu69mSpdH xyM3K7RGApFUOR6O/0NCSg==; Received: from [87.69.77.57] (port=3776 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwr9g-0002bq-6V; Mon, 13 Dec 2021 14:38:36 -0500 Date: Mon, 13 Dec 2021 21:38:30 +0200 Message-Id: <83pmq0l2h5.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> (message from Daniel Mendler on Mon, 13 Dec 2021 20:16:54 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Mon, 13 Dec 2021 20:16:54 +0100 > > > That will solve only a small fraction of situations where these > > characters are displayed, because the vast majority of them don't use > > prin1 to produce a string to display. Most of the stuff displayed by > > Emacs doesn't come from strings produced by prin1, it comes from > > displaying the text of some buffer. > > Yes, it solves a small fraction of situations. This issue does not > address a generic setting, it only addresses the behavior of `prin1`. As > I described at length there are packages which are affected by this and > which could improve from an improvement of `prin1`. > > To break it down once more: > > 1. We have the function `prin1-to-string` which can be used to produce a > string representation for an Emacs lisp value. > > 2. The behavior of the function can be adjusted via configuration > variables, in particular `print-escape-multibyte` and > `print-escape-control-characters`. `print-escape-multibyte` is very > aggressive, it escapes every multibyte character. > `print-escape-control-characters` only escapes ASCII control characters. > > 3. I am asking for a way to configure `prin1-to-string` such that it > escapes control and other glyphless characters but not all multibyte > characters, such that text still stays somewhat readable. I want less > aggressive escaping than `print-escape-multibyte`. If I set only > `print-escape-control-characters=t` is not sufficient since it escapes > only ASCII control characters. And, to reiterate once more, I'm against partial solutions that affect only some functions that produce strings, and don't affect at all any text displayed from a buffer. It would be a broken solution, because we will never be able to explain why 'prin1' produces escapes whereas 'format' and 'message' don't. It is unreasonable to require Lisp programs which will want to use something like that to use only 'prin1' and not the other functions routinely used for producing text for display. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:24:33 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:24:34 +0000 Received: from localhost ([127.0.0.1]:59519 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCTZ-0003mF-MX for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:24:33 -0500 Received: from mail-lj1-f171.google.com ([209.85.208.171]:46821) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCTY-0003m2-9L for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:24:32 -0500 Received: by mail-lj1-f171.google.com with SMTP id a37so28545237ljq.13 for <52459@debbugs.gnu.org>; Tue, 14 Dec 2021 10:24:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=uYbfIHJuhFX/pK5uPWJ/LZEclPvZkp2QFGxa9gQySzM=; b=NDM8acqstfD5y/59W58kxn4s5byfUCJoyrQXlOG4HKvCNBc70IPGXnaK/qvjyvjKiY 0Wxe7dOLppLCp6G+xaaZue3qfcEvERQ132z8E9y5jVr6X9jk6sgQnruZYnrIL8PTPwwA EGgLJZDIyMCWB2JGWMC0nEq00un/nsKy1PtL8JSixQt2s2MvVrFV+ov+nkSxP0Bn7tlW XmwmJYfGzQqoxwGw/TjTEpMdryi/mGcu++dKJcqZOll/CY0UOSihOYsAPpxo1oNwXRG4 gG5a1OQpase62OZTuxmbeqts82MvsAHLjCOecxKzHBjbnJqLkrRQS1tzk6u8k+VoZVjk /itg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=uYbfIHJuhFX/pK5uPWJ/LZEclPvZkp2QFGxa9gQySzM=; b=kZtcmukUpWRcOSnc+im3pR+Mu5JAavz8pPuy/LsDKlTlWmJ8dYBmdE36YQlGP7VhaG HH4VXAngY/1yrxM3MWrRyj11lyYWDOLUSzhLO4KnAktL6Zh0uGRciUIsZdvqQAvu1N3p eBMtGrvE5WasoeRtU0rnOAW5W5JXusqLRbRUmRVkiVVtyFVaXNevzE6c4u2dyN151CeW xnm8cZagEAZQ4CpbUIRMc2LnJkDh2oTmekMPP9nmj5SRNv3tN2h0Oesym/gfJ9hVZjSp TwhEBnc7O//d44thQKsGwOTfz+Q6g7kgegPkjnitkJI6vr88k2pxdAKTBzn6MNc9uVmr 5apA== X-Gm-Message-State: AOAM530f7DVOvDxtdNmuCn7iI1drj/ohpEI/h3QqJRDyy8FHuX5OVAET SugPUvwPRn64juD2e9BeNIntYvMJnpQ= X-Google-Smtp-Source: ABdhPJyT+mJZJoGHdqw1MNXQ1ZiKBRlE6YxcbXeKxOXqunPhERFL/uTUg/1ASSgG3Mtm2NBSBSHOsw== X-Received: by 2002:a2e:7a02:: with SMTP id v2mr6415971ljc.227.1639506266089; Tue, 14 Dec 2021 10:24:26 -0800 (PST) Received: from [192.168.0.103] ([5.18.233.185]) by smtp.googlemail.com with ESMTPSA id v2sm76745lfb.258.2021.12.14.10.24.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Dec 2021 10:24:25 -0800 (PST) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii , Daniel Mendler References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Tue, 14 Dec 2021 21:23:27 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <83pmq0l2h5.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.4 (/) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) On 13.12.2021 22:38, Eli Zaretskii wrote: >> To break it down once more: >> >> 1. We have the function `prin1-to-string` which can be used to produce a >> string representation for an Emacs lisp value. >> >> 2. The behavior of the function can be adjusted via configuration >> variables, in particular `print-escape-multibyte` and >> `print-escape-control-characters`. `print-escape-multibyte` is very >> aggressive, it escapes every multibyte character. >> `print-escape-control-characters` only escapes ASCII control characters. >> >> 3. I am asking for a way to configure `prin1-to-string` such that it >> escapes control and other glyphless characters but not all multibyte >> characters, such that text still stays somewhat readable. I want less >> aggressive escaping than `print-escape-multibyte`. If I set only >> `print-escape-control-characters=t` is not sufficient since it escapes >> only ASCII control characters. > And, to reiterate once more, I'm against partial solutions that affect > only some functions that produce strings, and don't affect at all any > text displayed from a buffer. It would be a broken solution, because > we will never be able to explain why 'prin1' produces escapes whereas > 'format' and 'message' don't. I just did a little testing, and it seems 'print-escape-control-characters' only affects 'prin1-to-string' and 'prin1' but not 'message' or 'format'. Is that a problem? If not, adding a new variable which makes the same distinction seems consistent with the current design. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:32:59 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:32:59 +0000 Received: from localhost ([127.0.0.1]:59524 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCbj-0003ys-IX for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:32:59 -0500 Received: from server.qxqx.de ([178.63.65.180]:34197 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCbh-0003yb-3Y for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:32:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=SEyHuTOouOvgeLS7rswsW0u/MLf6enr9YM1g51JDu5c=; b=kK5YxIaUjhY5IdnQNwA9NJfQbM 66dm0DONELzSUJ6M6SPZAe7sGOPe9rJcbv2H1tv6sViHEaRevNw9bpIQt6wfTR+hLBXtsjd2z6JGE 6zRH4jtS65TwlXr40wC/OuEBZ7y+RCBBtuTLZqZZ9oFCXPGqIOuUIyNX4IrdqOcvytyc=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Dmitry Gutov , Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> From: Daniel Mendler Message-ID: <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> Date: Tue, 14 Dec 2021 19:32:47 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/14/21 7:23 PM, Dmitry Gutov wrote: > I just did a little testing, and it seems > 'print-escape-control-characters' only affects 'prin1-to-string' and > 'prin1' but not 'message' or 'format'. No, `print-escape-multibyte` also applies to `format and `message`. Try the following: (let ((print-escape-multibyte t)) (format "%S" bidi-directional-controls-chars) (message "%S" bidi-directional-controls-chars)) Anyway Eli's criticism does not apply to my proposal of the addition of such a configuration variable. The variable would have the same scope as `print-escape-multiple` and the other `print-escape-*` variables. My proposal does not introduce an inconsistency or any other kind of incoherence. > Is that a problem? > > If not, adding a new variable which makes the same distinction seems > consistent with the current design. Exactly. My proposal is consistent with the current design. Daniel From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:39:52 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:39:52 +0000 Received: from localhost ([127.0.0.1]:59535 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCiN-00048U-Pk for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:39:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44824) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCiM-00048I-LC for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:39:51 -0500 Received: from [2001:470:142:3::e] (port=49896 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxCi7-00076u-JY; Tue, 14 Dec 2021 13:39:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=rSwfEVWKmKpueP9+0TwqBUGEEQZzAyJEYRIkx8U5oZE=; b=rX2Ri4HxO/tR aOuxlQtBPTXbxJGkd8LD+JmN32/tf/5Jra8CZSjf+PlESemH7IqNLSQkVKRmql4s2sIAPIOA8UtsF /lZWB2DFDWr3KbYJ1bePbH/jrPfWYcMKE2fBfC3Z+XsAJ1eIN/0LRZIj/sNx3heUB4b6S8ct+Ra/A AoAgUPsNvccU8U1F/x2j8nmQg1rar2EV/XZ7VseXS3A0qKw3t4ZOFmeBekyB3fh7fDr0T+SAGb1qz ri8wEyKiaV3ySbvRAhpwhuZqcpzprsbD5NrLYwIlLSGec9sZL0OpSVpycinM5E9PPX6HzGhEVBZc+ Nyrw0lze/5U8mgjh+v6eLg==; Received: from [87.69.77.57] (port=4860 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxCi4-00034R-4O; Tue, 14 Dec 2021 13:39:33 -0500 Date: Tue, 14 Dec 2021 20:39:29 +0200 Message-Id: <83pmpzjaji.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-Reply-To: (message from Dmitry Gutov on Tue, 14 Dec 2021 21:23:27 +0300) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: mail@daniel-mendler.de, 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 14 Dec 2021 21:23:27 +0300 > > > And, to reiterate once more, I'm against partial solutions that affect > > only some functions that produce strings, and don't affect at all any > > text displayed from a buffer. It would be a broken solution, because > > we will never be able to explain why 'prin1' produces escapes whereas > > 'format' and 'message' don't. > > I just did a little testing, and it seems > 'print-escape-control-characters' only affects 'prin1-to-string' and > 'prin1' but not 'message' or 'format'. > > Is that a problem? It could be. I guess the only reason no one complained about it is that those print functions are used in very specialized cases. But in this case, the requirement was to use it for displaying human-readable text in a UI, and I think in that context it would be highly surprising, to say the least, to have it supported by prin1, but not by formatted printing APIs. > If not, adding a new variable which makes the same distinction seems > consistent with the current design. The goal is explicitly different, and specifically targets the display of text to users. And IMO that is not consistent with the current design, because these variables definitely weren't meant to affect how text is presented to users. We have other similar features for that, like nobreak-char-display etc. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:41:56 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:41:56 +0000 Received: from localhost ([127.0.0.1]:59541 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCkO-0004Bs-74 for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:41:56 -0500 Received: from mail-lj1-f180.google.com ([209.85.208.180]:41767) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCkM-0004Bf-Sc for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:41:55 -0500 Received: by mail-lj1-f180.google.com with SMTP id bn20so29720815ljb.8 for <52459@debbugs.gnu.org>; Tue, 14 Dec 2021 10:41:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=R57c4azRfPKuSRJnP9cdyHCBMpP4Ak9+xFeQH8ajhK4=; b=l/5WY7IcpDhFQJqMgaYVkx2fJKfCmLijbw3afwoRwlJUQf/VY0lrCq2CHa81N+niKq 7KloP6Es9kEJfFP7q3wgY7sIt4mGU31cRCnsSFGQFMsJEFGAzngbAjNnLeEODu7UwPNv 1eOl5EZ8kCGulOSuMWyjYUHEfLFbcbQU8S8cWhYB9lAfzrEfdfNyq1qSk15n/izuNhW7 NrO7S9aUkVG2+XNIRY/V70XHMLHzhiMdVnK8b7syph4yDHOkyZQHoNsjzqsj/BjHQqFB 5yJQjgRYVtqP8bQGF+di6+R87JITqVLXA5S+2sL05yntH46Kx0jN5A/2mxsf7uicp7Ne HB3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=R57c4azRfPKuSRJnP9cdyHCBMpP4Ak9+xFeQH8ajhK4=; b=RhzeAtDcSWp0bxg3UVdjdC60sL3XuWC7aekPRk5xAbcpijIDB9CM6zxbeet/r0ZoaI xIDOrDyyYUt7JhAVlNUqETKa9ppDqm6n5/3LpTE3+rxQlqvtY74tDJwBTvRmH7a2/HDI nvMYEEuRLwbAUkbDPARiGDg1ir03Q2WeeFZyQlNq/3QvMhSFONNwahpmRyO82soiwxBC TBcrJubWSzBnIuWjY3ScFKNFyHIiNE7JBRqq7N65Pmzzxa9+hR0wPmXF3IOhM2SB2uqg q5pG6lpeL5DoVZzI+Tj2i8wxhAxDczzkjfK//9/DZ9KGDJxNPnSY7A/Cdk4caE6UmSOR rOaA== X-Gm-Message-State: AOAM533D5XRdWnw1MuE6lRdd5uZT+yz18xvc6YagbvyqAdY6H0AerY8o nCQSjnaaeeStf0th5dMiqTedaP1VHAE= X-Google-Smtp-Source: ABdhPJxpFeQ0qlu1rPtztmPZMiJtpdaBns0x1+b9XO4N7FbQuPyZVqnAjHqta+P+RbwsMDh3E++gKw== X-Received: by 2002:a05:651c:1031:: with SMTP id w17mr6489578ljm.112.1639507308684; Tue, 14 Dec 2021 10:41:48 -0800 (PST) Received: from [192.168.0.103] ([5.18.233.185]) by smtp.googlemail.com with ESMTPSA id bp15sm85211lfb.176.2021.12.14.10.41.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Dec 2021 10:41:48 -0800 (PST) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Daniel Mendler , Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> From: Dmitry Gutov Message-ID: Date: Tue, 14 Dec 2021 21:40:50 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Score: 0.4 (/) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) On 14.12.2021 21:32, Daniel Mendler wrote: > On 12/14/21 7:23 PM, Dmitry Gutov wrote: >> I just did a little testing, and it seems >> 'print-escape-control-characters' only affects 'prin1-to-string' and >> 'prin1' but not 'message' or 'format'. > No, `print-escape-multibyte` also applies to `format and `message`. Try > the following: > > (let ((print-escape-multibyte t)) > (format "%S" bidi-directional-controls-chars) > (message "%S" bidi-directional-controls-chars)) This is interesting, because print-escape-control-characters (which I mentioned) does not: ELISP> (let ((print-escape-control-characters t)) (prin1 "\b")) "\10" "" ELISP> (let ((print-escape-control-characters t)) (prin1-to-string "\b")) "\"\\10\"" ELISP> (let ((print-escape-control-characters t)) (format "\b")) "" ELISP> (let ((print-escape-control-characters t)) (message "\b")) "" >> Is that a problem? >> >> If not, adding a new variable which makes the same distinction seems >> consistent with the current design. > Exactly. My proposal is consistent with the current design. ...but indeed if the new variable has the same scope as either of the existing ones, it seems easy to justify. Maybe reconcile the scopes of the existing vars, too. 'print-escape-multibyte' is documented as "This affects only ‘prin1’", but it is the other var which makes the distinction. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:47:44 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:47:45 +0000 Received: from localhost ([127.0.0.1]:59587 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCq0-0004Ly-Kj for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:47:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCpy-0004Ll-EK for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:47:42 -0500 Received: from [2001:470:142:3::e] (port=50174 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxCps-0008Pi-IZ; Tue, 14 Dec 2021 13:47:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=EQr6IxMX/2n0QSWzC1KQNqtFWgNfVeyEIeWX5uPiQw8=; b=TJh2zSlw/C2m BqEjUkYUzY8IzRpD8MxqfOpF6TKpDP4WB6x9jMoAS6j4Q53cncd6+Bri369ybil4O4CoSXL1CVv5N yUdLgCK5JMDps4hnrCC6hkqdJIzLzcNCWOwJ4QVu+z2ejDaihhcGpGsLS69YQY/inEJkVAJbxwi/C nwx62FADFBHoNuUSw0BX1DddmuQqey55pcrykvEpg8eyUob31m7IVtdaW5O4+Uyix4AFnVt9hS92p XJrRFzqb3Kaes3M01ynZMTQD8iKQy4SP+Y2jAGad7D8dRy015rnUQMTwejTplCoIlfNgVWwp3hPG2 /prYEoGjdcDwtrelWrF6yw==; Received: from [87.69.77.57] (port=1380 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxCps-00040H-5F; Tue, 14 Dec 2021 13:47:36 -0500 Date: Tue, 14 Dec 2021 20:47:33 +0200 Message-Id: <83mtl3ja62.fsf@gnu.org> From: Eli Zaretskii To: Daniel Mendler In-Reply-To: <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> (message from Daniel Mendler on Tue, 14 Dec 2021 19:32:47 +0100) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 52459@debbugs.gnu.org > From: Daniel Mendler > Date: Tue, 14 Dec 2021 19:32:47 +0100 > > On 12/14/21 7:23 PM, Dmitry Gutov wrote: > > I just did a little testing, and it seems > > 'print-escape-control-characters' only affects 'prin1-to-string' and > > 'prin1' but not 'message' or 'format'. > > No, `print-escape-multibyte` also applies to `format and `message`. Try > the following: > > (let ((print-escape-multibyte t)) > (format "%S" bidi-directional-controls-chars) > (message "%S" bidi-directional-controls-chars)) Strings are not usually formatted using %S, they are formatted using %s. It would be unreasonable to expect Lisp programs to use %S for formatting strings. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:52:16 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:52:16 +0000 Received: from localhost ([127.0.0.1]:59596 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCuO-0004Ux-6T for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:52:16 -0500 Received: from server.qxqx.de ([178.63.65.180]:50291 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCu7-0004UE-PR for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:52:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=w7nudVINTnj9bJCT4+a/Oyp06yJIqFY9YMRYh7iYVng=; b=g+LMRElxd8bfzxCdPgrc12oBdP skc/Tf4U+mkBXVphWdVixvLxDBtI8SHR/zZz3t8jfvdrntjxI3tAa+v3L3bXYPf4PYcKcXNnHOywr yaFhvHxMSHgh8fLLCpGgbncgXPYYI6IYORl4nMyxqCFk3/GBYJwFUbGzPrHHaL7yTecA=; Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> <83mtl3ja62.fsf@gnu.org> From: Daniel Mendler Message-ID: <4c33ab33-6d07-3522-680a-4fbc7133d56e@daniel-mendler.de> Date: Tue, 14 Dec 2021 19:51:50 +0100 MIME-Version: 1.0 In-Reply-To: <83mtl3ja62.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 12/14/21 7:47 PM, Eli Zaretskii wrote: >> Cc: 52459@debbugs.gnu.org >> From: Daniel Mendler >> Date: Tue, 14 Dec 2021 19:32:47 +0100 >> >> On 12/14/21 7:23 PM, Dmitry Gutov [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 T_SPF_TEMPERROR SPF: test of record failed (temperror) -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 1.3 FORGED_SPF_HELO No description available. X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 12/14/21 7:47 PM, Eli Zaretskii wrote: >> Cc: 52459@debbugs.gnu.org >> From: Daniel Mendler >> Date: Tue, 14 Dec 2021 19:32:47 +0100 >> >> On 12/14/21 7:23 PM, Dmitry Gutov wrote: >>> I just did a little testing, and it seems >>> 'print-escape-control-characters' only affects 'prin1-to-string' and >>> 'prin1' but not 'message' or 'format'. >> >> No, `print-escape-multibyte` also applies to `format and `message`. Try >> the following: >> >> (let ((print-escape-multibyte t)) >> (format "%S" bidi-directional-controls-chars) >> (message "%S" bidi-directional-controls-chars)) > > Strings are not usually formatted using %S, they are formatted using > %s. It would be unreasonable to expect Lisp programs to use %S for > formatting strings. I am talking about creating string representations of Elisp values. I did not exclusively talk about strings. Strings are of course also Elisp values, but for a debugger UI it is useful to produce string representations of general Elisp values. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 13:57:22 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 18:57:22 +0000 Received: from localhost ([127.0.0.1]:59600 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCzK-0004c5-1S for submit@debbugs.gnu.org; Tue, 14 Dec 2021 13:57:22 -0500 Received: from mail-lf1-f51.google.com ([209.85.167.51]:40542) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxCzI-0004bp-C0 for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 13:57:21 -0500 Received: by mail-lf1-f51.google.com with SMTP id l22so38626197lfg.7 for <52459@debbugs.gnu.org>; Tue, 14 Dec 2021 10:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=3GaGLJzIIfL5MeJW6axnyzOfmDe2JjbQtd8LPJVzYag=; b=I5d/sRnYCKdC+7VA2JVTbfTmsl6L3armio6qI1Uy/qpf06L/Fe8dSTrdg+GqSURxpv mbUkWXJ8akylqbHEaWqUO8Wu2d3nuOhZps+KYCNwj8vpnH0hb+4FoXjn3neSG0+CugCO 7Ca3BRrkEb/Mdzpd1gxKt2KeH02ox2wwwfFjfRN8kmkayBlhX4vm1Q9G8kn4rDTt8ic9 Vx/inPuT9apRVSatdaUT4GgWhQjru5ZuMPLpspvIlff/+e6Uj+NWzj/2f77I3ePzUevs nDiWJrRjuXcYFkRrko0DuBeg9geCqrXjmzgiLjRemN62CAQPmPCySomAdEgywMXByLFI jkFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=3GaGLJzIIfL5MeJW6axnyzOfmDe2JjbQtd8LPJVzYag=; b=CqdD2QioWw0iftxTqxjeA1LlrJY9eHKCFV87kLIQaOmG3971rLyn3ZVf2sdk+NKgsW PgXiD08qrF6kYvZXs/OXTBq2mn+M6tm6YE98o8stUj5OhRtsUVQCJZUPCtjm0yu5dfbi vx8U1dWKVNi+0an2rrVEEum4RHODzNeHNcCRkbtOYYDSwDgmbNeX+EpxTFITJPy0+S+5 qiL8ss/Y7CqNuwORvjpybtvT6Qkfp6zG8r0XKPXmeaaTH7a+WPSMYvOZUGmdgBwO7sif oG9qLFGxIJkVm4zauerIdLVFhesDAv2oLC2eEaIu3yocUiYMX4H19ZkBwjCHNlVa3ccG UMxA== X-Gm-Message-State: AOAM533G70sk2J/4t+UclMq2xWEtuqbVlKxHjr6CRqyfpOwx+yDkhHFT aDHs4rJscCwxRqpx8ZmLKRsZGA9fBIA= X-Google-Smtp-Source: ABdhPJxSMqWuRKySYZTSHvtaNqCcZ9r5tswailDoESHuIxmUpuvSZSq3ekBrFW1YeLvdGDwAtRn4dw== X-Received: by 2002:a19:fc0a:: with SMTP id a10mr6228422lfi.507.1639508234054; Tue, 14 Dec 2021 10:57:14 -0800 (PST) Received: from [192.168.0.103] ([5.18.233.185]) by smtp.googlemail.com with ESMTPSA id d10sm92196lfv.114.2021.12.14.10.57.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Dec 2021 10:57:13 -0800 (PST) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <83pmpzjaji.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Tue, 14 Dec 2021 21:56:16 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <83pmpzjaji.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.4 (/) X-Debbugs-Envelope-To: 52459 Cc: mail@daniel-mendler.de, 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) On 14.12.2021 21:39, Eli Zaretskii wrote: >> Cc:52459@debbugs.gnu.org >> From: Dmitry Gutov >> Date: Tue, 14 Dec 2021 21:23:27 +0300 >> >>> And, to reiterate once more, I'm against partial solutions that affect >>> only some functions that produce strings, and don't affect at all any >>> text displayed from a buffer. It would be a broken solution, because >>> we will never be able to explain why 'prin1' produces escapes whereas >>> 'format' and 'message' don't. >> I just did a little testing, and it seems >> 'print-escape-control-characters' only affects 'prin1-to-string' and >> 'prin1' but not 'message' or 'format'. >> >> Is that a problem? > It could be. I guess the only reason no one complained about it is > that those print functions are used in very specialized cases. But in > this case, the requirement was to use it for displaying human-readable > text in a UI, and I think in that context it would be highly > surprising, to say the least, to have it supported by prin1, but not > by formatted printing APIs. I'm not sure it would be a problem if 'message' and 'format' also honor that new variable. Aside from inconsistency with existing vars, that is. >> If not, adding a new variable which makes the same distinction seems >> consistent with the current design. > The goal is explicitly different, and specifically targets the display > of text to users. And IMO that is not consistent with the current > design, because these variables definitely weren't meant to affect how > text is presented to users. We have other similar features for that, > like nobreak-char-display etc. It seems to me that you misunderstand the use case, or at least the approach that Daniel wants to take: Helpful, or Help buffers used by commands like 'describe-variable', use prin1 to output values which are not known in advance (like the value of the described variable). And the dynamic vars under discussion can make those printed values more predictable and easier to grok (and copy-paste, and etc). From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 14:21:08 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 19:21:08 +0000 Received: from localhost ([127.0.0.1]:59636 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDMJ-0005DW-J9 for submit@debbugs.gnu.org; Tue, 14 Dec 2021 14:21:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59900) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDME-0005Ce-DY for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 14:21:05 -0500 Received: from [2001:470:142:3::e] (port=51596 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxDM8-0005H7-0T; Tue, 14 Dec 2021 14:20:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=A7U05M3H5X9xDOMhiTsHezQANU4CygKFIBEkdGGh5dg=; b=cmFeMLnAnFtq ucmDGCQdZh7axGDul6aiwPpNepE8mLKs9ml/rshbOVS65wMIAr8i46XsLlI+1UidKmcKqQHcvtkR4 BsPTB3LF+hCk1K3jbAPUnlEpI5tFfEdEs1JoVJ74VEthqw9P62IR+VVIHGhuzvWvgzxEhRXDXsjdO TDGptaHNGB9sFnHARGGHgTIfsFYYQVq465Xn38qa70XUs5f830ucd7LeUiPN6Hf/O8mAC3Ed5egpm Xw9X6ntNOLvFrAThcjOalUPfTid7WsAgN2kjBr+/19l2hW/rAZ7e6ugZaBK+j3Tbz20iXwd+ghQaV U811IdPk0bn9q/jIg2CT+w==; Received: from [87.69.77.57] (port=3404 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxDM7-0004lW-Rj; Tue, 14 Dec 2021 14:20:56 -0500 Date: Tue, 14 Dec 2021 21:20:51 +0200 Message-Id: <83ilvrj8mk.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-Reply-To: (message from Dmitry Gutov on Tue, 14 Dec 2021 21:56:16 +0300) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <83pmpzjaji.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 52459 Cc: mail@daniel-mendler.de, 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: mail@daniel-mendler.de, 52459@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 14 Dec 2021 21:56:16 +0300 > > > It could be. I guess the only reason no one complained about it is > > that those print functions are used in very specialized cases. But in > > this case, the requirement was to use it for displaying human-readable > > text in a UI, and I think in that context it would be highly > > surprising, to say the least, to have it supported by prin1, but not > > by formatted printing APIs. > > I'm not sure it would be a problem if 'message' and 'format' also honor > that new variable. Aside from inconsistency with existing vars, that is. We agree. But that is not what is being requested by Daniel. > It seems to me that you misunderstand the use case, or at least the > approach that Daniel wants to take: > > Helpful, or Help buffers used by commands like 'describe-variable', use > prin1 to output values which are not known in advance (like the value of > the described variable). > > And the dynamic vars under discussion can make those printed values more > predictable and easier to grok (and copy-paste, and etc). The set of use cases that was described included more than just Helpful. It explicitly included displaying text not known in advance, such as in debugging UI, where one needs to display strings that come from the program being debugged. Hoever, you may be right that I have no good understanding of the use cases. I did try to understand them, but found it to be impossible, because every question I tried to ask, every concept of a solution I tried to propose was inevitably met with the equivalent of "why won't you give me my variable". I actually think that there are two distinct classes of use cases involved, but Daniel rejected that, insisting, AFAIU, that all the use cases are the same and need a single solution. Which I think is based on the wrong mental model of how the related stuff works in Emacs. So I'm sorry, but I cannot be of help in this discussion under those terms. That is not the way I'm used to approach a problem. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 14:22:16 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 19:22:16 +0000 Received: from localhost ([127.0.0.1]:59640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDNQ-0005FG-GB for submit@debbugs.gnu.org; Tue, 14 Dec 2021 14:22:16 -0500 Received: from mail-out.m-online.net ([212.18.0.10]:33175) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDNO-0005F6-1Y for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 14:22:14 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4JD7YJ3JqWz1sCwJ; Tue, 14 Dec 2021 20:22:12 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4JD7YJ2TGGz1qqkD; Tue, 14 Dec 2021 20:22:12 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 38ML6nP3Wa4I; Tue, 14 Dec 2021 20:22:11 +0100 (CET) X-Auth-Info: kSoOxX2Nva50SzLU86C7d2Jwqkf8SHY34yKVBM9OT3tS1cXLiRUqjJe1QF4F4DVN Received: from igel.home (ppp-46-244-174-98.dynamic.mnet-online.de [46.244.174.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Tue, 14 Dec 2021 20:22:11 +0100 (CET) Received: by igel.home (Postfix, from userid 1000) id 066382C398B; Tue, 14 Dec 2021 20:22:10 +0100 (CET) From: Andreas Schwab To: Daniel Mendler Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> X-Yow: I OWN six pink HIPPOS!! Date: Tue, 14 Dec 2021 20:22:10 +0100 In-Reply-To: <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> (Daniel Mendler's message of "Tue, 14 Dec 2021 19:32:47 +0100") Message-ID: <87o85j2dr1.fsf@igel.home> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 52459 Cc: Eli Zaretskii , 52459@debbugs.gnu.org, Dmitry Gutov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) On Dez 14 2021, Daniel Mendler wrote: > No, `print-escape-multibyte` also applies to `format and `message`. Try > the following: > > (let ((print-escape-multibyte t)) > (format "%S" bidi-directional-controls-chars) > (message "%S" bidi-directional-controls-chars)) %S is, in essence, prin1-to-string. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 14 14:42:15 2021 Received: (at 52459) by debbugs.gnu.org; 14 Dec 2021 19:42:15 +0000 Received: from localhost ([127.0.0.1]:59677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDgl-0005l0-Cx for submit@debbugs.gnu.org; Tue, 14 Dec 2021 14:42:15 -0500 Received: from mail-lf1-f41.google.com ([209.85.167.41]:42715) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxDgj-0005kl-P8 for 52459@debbugs.gnu.org; Tue, 14 Dec 2021 14:42:14 -0500 Received: by mail-lf1-f41.google.com with SMTP id t26so38877517lfk.9 for <52459@debbugs.gnu.org>; Tue, 14 Dec 2021 11:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=6ZD2RDYcidXwMW8XPqdUzPNg+nFWyjn7DR/iAZe3x+k=; b=mS1oRkXldQne3uyM3icrj/ITPLnAIZQL+de9ahYqVfCAptvoA022hI3AScaJQCbpl4 TxdN/JSpvEKcp5zl7Z8axiESQVuPc9BhZ3v9PvwCKDFyfZ+N5WQiXFQw6sflbuo55xEe LE85zoaR8i1Jk0jFyTzOuee6jlTU90f11B2ft1ne3TUhz8iv2tKWe4grQWDei+ggcFL7 bQouYwQQgLCuWhj4ucDAYV5QZ78BKu6YQUccclkEjyxix6M+CL70j+ZEjLQAC0uF2PBD OpsffvyC9KSVY3s+tPwYiinnooJvf1vBMtugDDN17fe4bnfnIa5L29bTYP5ccKJDY6U4 esyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=6ZD2RDYcidXwMW8XPqdUzPNg+nFWyjn7DR/iAZe3x+k=; b=HFMu5qerBa1rknb3pML7fLVhqPxWjP/bqO2K4oAJUixHpKJsKJbeFQj++tM7cYqwqP eAXJZyG8aHLE0gwxQmiAgzH6cKoPd3t8uHcD7swEvUZySBn4tIW/H8VtNLGJ2cj+SqPo 5l0Wgff02dtyJ7c1k9xJ9prY2/Kz3DjOL1qeHWfLnlW0cs/lxXBmPLNiuXLfoyNXsHFe QlGlJ7rzGwEsvop91tcjEhQ3XnvjJBZsvpS5/1CFHEx+4qIZtuW4BBENbcGboHji2APK vwYIMe2dCaW6LMwDwqAi+592DYnFKTwwSYC8kjJvWUUzPma5KAZRLqL9Zl/Msqf8jawb sRbA== X-Gm-Message-State: AOAM5330awNhi0c/LcpkMvZg6d/fyXIvl/8hMLEkqDXNu5B/UKO2pRBm fWArgUr9L/Ozw/V+bMVzXKOicLpDPAA= X-Google-Smtp-Source: ABdhPJxtgzzL41jkkB4optsWhejdZefdgZ8u49mNl60nCB5Q/Y6eQtFpL0eihmCYY865WxneTNU3yg== X-Received: by 2002:a05:6512:2204:: with SMTP id h4mr6624302lfu.687.1639510927729; Tue, 14 Dec 2021 11:42:07 -0800 (PST) Received: from [192.168.0.103] ([5.18.233.185]) by smtp.googlemail.com with ESMTPSA id s19sm121350lji.81.2021.12.14.11.42.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Dec 2021 11:42:07 -0800 (PST) Subject: Re: bug#52459: 28.0.90; prin1-to-string does not escape bidi control characters despite print-escape-control-characters=t To: Daniel Mendler , Eli Zaretskii References: <83v8ztmu75.fsf@gnu.org> <93d63756-f75d-c53e-de02-2e8270d07311@daniel-mendler.de> <83r1agn184.fsf@gnu.org> <0eabc668-ecb2-8f77-17cf-f9cb6dcf0626@daniel-mendler.de> <0504d4a8-1a4b-a451-d7d3-fea1c116b96d@daniel-mendler.de> <8335mwmssm.fsf@gnu.org> <7027aad4-d156-12ae-7356-4d55be5716b1@daniel-mendler.de> <83wnk8l9ha.fsf@gnu.org> <2511c498-1e36-07b2-b9b8-5849902cd416@daniel-mendler.de> <83tufcl5qd.fsf@gnu.org> <4ac4a4ce-2bb4-4120-d52d-665a029cab4e@daniel-mendler.de> <83sfuwl4mf.fsf@gnu.org> <83r1agl3um.fsf@gnu.org> <0491af1e-b57d-c5bd-c17d-b9bc1ef34929@daniel-mendler.de> <83pmq0l2h5.fsf@gnu.org> <86f4ea53-1f83-eb14-2f24-b50e9de6a80a@daniel-mendler.de> <83mtl3ja62.fsf@gnu.org> <4c33ab33-6d07-3522-680a-4fbc7133d56e@daniel-mendler.de> From: Dmitry Gutov Message-ID: <1a009779-9610-6f98-0122-3ec44f099e76@yandex.ru> Date: Tue, 14 Dec 2021 22:41:09 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <4c33ab33-6d07-3522-680a-4fbc7133d56e@daniel-mendler.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.4 (/) X-Debbugs-Envelope-To: 52459 Cc: 52459@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) On 14.12.2021 21:51, Daniel Mendler wrote: > I am talking about creating string representations of Elisp values. I > did not exclusively talk about strings. Strings are of course also Elisp > values, but for a debugger UI it is useful to produce string > representations of general Elisp values. Then, also with clarification from A. Schwab, we could say that perhaps we'd be fine with having all print-escape-* variables (the existing ones and the one under proposal) only affecting 'prin1', but not 'message' and 'format' in general (used without '%S').