GNU bug report logs - #73660
[PATCH] gexp: Improve support of Unicode characters.

Previous Next

Full log

Message #38 received at 73660 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>,
 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Florian Pelz <pelzflorian <at> pelzflorian.de>, 73660 <at> debbugs.gnu.org,
 Christopher Baines <guix <at> cbaines.net>
Subject: Re: [bug#73660] [PATCH] gexp: Improve support of Unicode characters.
Date: Fri, 24 Jan 2025 15:16:19 +0100

Hi Tomas!

Tomas Volf <~@wolfsden.cz> skribis:

> Anyway, I followed the suggestion and v2 moves the LC_CTYPE setting to
> gexp->derivation.  The test script from the commit message still (after
> many hours of bootstrapping) works.

Sounds good!

>>> +          ;; Best effort.  The locale is not installed in all contexts.
>>> +          (false-if-exception (setlocale LC_ALL "C.UTF-8"))
>>
>> Sounds good.  I would make it a separate patch.
>
> Somewhat done.  I have made it a separate commit, but still included in
> v2.
>
>>
>> s/in all contexts/when cross-compiling/
>
> Interesting, I have modified the comment, however would you be willing
> to expand on this a bit?  Why is the C.UTF-8 locale not available when
> cross-compiling?  The Guile running this script runs on the build host,
> using build host's glibc and build host's locale definitions no?  So I
> assumed the locale *should* be available.  I feel like I am missing
> something fundamental about how Guix works here.

See the ‘install-utf8-c-locale’ phase of ‘glibc’: since there’s no
‘localedef’ program to use when cross-compiling, that phase is a no-op.

             ;; FIXME: When cross-compiling, attempt to use
             ;; 'localedef' from the same libc version.
             (invoke ,(if (%current-target-system)
                          "true"
                          '(string-append bin "/localedef"))
                     "--no-archive" "--prefix" locale
                     "-i" "C" "-f" "UTF-8"
                     (string-append locale "/C.UTF-8"))

It’s a problem because then we cannot really assume that C.UTF-8 is
*always* available, even though that was the goal (see
<https://issues.guix.gnu.org/67686>.)

AFAICS, the format is actually architecture-independent:

--8<---------------cut here---------------start------------->8---
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s i686-linux
/gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' 
/gnu/store/f3kpp3dh25893y79n32qlqqiwb5my4zg-glibc-utf8-locales-2.39
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/f3kpp3dh25893y79n32qlqqiwb5my4zg-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s aarch64-linux
/gnu/store/0i1brwncg9rpf7pvh9hs5vrchmfb3c9q-glibc-utf8-locales-2.39
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s armhf-linux
/gnu/store/8x26ik4jlvljcnx1jhfd83r5lyx04d15-glibc-utf8-locales-2.39
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/0i1brwncg9rpf7pvh9hs5vrchmfb3c9q-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/8x26ik4jlvljcnx1jhfd83r5lyx04d15-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
--8<---------------cut here---------------end--------------->8---

So it might work by replacing "true" by "localedef" in this phase (and
we could do that on ‘master’), under the assumption that the
cross-compiled libc and the one that provides the ‘localedef’ command
are the same version, which is usually the case.

We should do that.

>>>  (define* (text-file* name #:rest text)
>>>    "Return as a monadic value a derivation that builds a text file containing
>>> @@ -2108,6 +2119,7 @@ (define* (text-file* name #:rest text)
>>>    (define builder
>>>      (gexp (call-with-output-file (ungexp output "out")
>>>              (lambda (port)
>>> +              (set-port-encoding! port "UTF-8")
>>>                (display (string-append (ungexp-splicing text)) port)))))
>>
>> LGTM.  This can be moved to a separate file.
>
> By "separate file" you mean separate patch and/or commit?

Yes.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.