Subject: Re: bug#51832: Piping unicode text in `shell-command'

From: Alan Third <[email-masked]>
Date: Sun, 14 Nov 2021 15:20:14 +0000

On Sun, Nov 14, 2021 at 03:28:02PM +0100, Lars Ingebrigtsen wrote:
> Philipp <[email-masked]> writes:
> 
> >> That didn't fix the issue for me, I'm afraid -- with that patch, LANG is
> >> still the invalid en_NO.UTF-8 for me.
> >
> > Maybe we should add similar logic as iTerm2

I tried to find how iTerm2 does it. Your search-fu is better than
mine, apparently. :)

> > (https://github.com/gnachman/iTerm2/blob/79aff4d59fd591e7628649bcabe5f27541740bf6/sources/PTYSession.m#L7107):
> > create the locale identifier from language code and country code
> > instead of the current locale identifier,
> 
> I think that's what's Macos is returning -- it's just concatenating
> those two codes to get a locale identifier.  (Which is wrong, of
> course.)

Yeah, I don't think there's any advantage to building them up
manually.

> > and use setlocale (or better, newlocale) to check whether it's valid,
> 
> Yes, that sounds good.
> 
> > and fall back to en_US.UTF-8 otherwise?
> 
> Hm...  I'd rather just leave LANG unset in that case -- it'll probably
> lead to fewer glitches, I think.

I proposed something similar before:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=51321#90

but it didn't look like we needed it then. We know better now.

New patch attached.
-- 
Alan Third
>From 3a2e20c659d8732b11d30cdb27e36610e87a0315 Mon Sep 17 00:00:00 2001
From: Alan Third <[email-masked]>
Date: Sun, 14 Nov 2021 15:09:43 +0000
Subject: [PATCH v2] Only set LANG if the ID is valid

* src/nsterm.m (ns_init_locale): Check the provided locale identifier
is available before trying to use it.
---
 src/nsterm.m | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/src/nsterm.m b/src/nsterm.m
index 1f17a30272..983e5eb8ac 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -535,21 +535,18 @@ - (NSColor *)colorUsingDefaultColorSpace
 
   NSTRACE ("ns_init_locale");
 
-  @try
+  if (!isatty (STDIN_FILENO))
     {
-      /* It seems macOS should probably use UTF-8 everywhere.
-         'localeIdentifier' does not specify the encoding, and I can't
-         find any way to get the OS to tell us which encoding to use,
-         so hard-code '.UTF-8'.  */
+      char *oldLocale = setlocale (LC_ALL, NULL);
       NSString *localeID = [NSString stringWithFormat:@"%@.UTF-8",
                                      [locale localeIdentifier]];
 
-      /* Set LANG to locale, but not if LANG is already set.  */
-      setenv("LANG", [localeID UTF8String], 0);
-    }
-  @catch (NSException *e)
-    {
-      NSLog (@"Locale detection failed: %@: %@", [e name], [e reason]);
+      /* Check the locale ID is valid and if so set LANG, but not if
+         it is already set.  */
+      if (setlocale (LC_ALL, [localeID UTF8String]))
+        setenv("LANG", [localeID UTF8String], 0);
+
+      setlocale (LC_ALL, oldLocale);
     }
 }
 
-- 
2.32.0