Subject: Re: [bug#73660] [PATCH] gexp: Improve support of Unicode characters.

From: Ludovic Courtès <[email-masked]>
Date: Fri, 24 Jan 2025 15:16:19 +0100

Hi Tomas!

Tomas Volf <[email-masked]> skribis:

> Anyway, I followed the suggestion and v2 moves the LC_CTYPE setting to
> gexp->derivation.  The test script from the commit message still (after
> many hours of bootstrapping) works.

Sounds good!

>>> +          ;; Best effort.  The locale is not installed in all contexts.
>>> +          (false-if-exception (setlocale LC_ALL "C.UTF-8"))
>>
>> Sounds good.  I would make it a separate patch.
>
> Somewhat done.  I have made it a separate commit, but still included in
> v2.
>
>>
>> s/in all contexts/when cross-compiling/
>
> Interesting, I have modified the comment, however would you be willing
> to expand on this a bit?  Why is the C.UTF-8 locale not available when
> cross-compiling?  The Guile running this script runs on the build host,
> using build host's glibc and build host's locale definitions no?  So I
> assumed the locale *should* be available.  I feel like I am missing
> something fundamental about how Guix works here.

See the ‘install-utf8-c-locale’ phase of ‘glibc’: since there’s no
‘localedef’ program to use when cross-compiling, that phase is a no-op.

             ;; FIXME: When cross-compiling, attempt to use
             ;; 'localedef' from the same libc version.
             (invoke ,(if (%current-target-system)
                          "true"
                          '(string-append bin "/localedef"))
                     "--no-archive" "--prefix" locale
                     "-i" "C" "-f" "UTF-8"
                     (string-append locale "/C.UTF-8"))

It’s a problem because then we cannot really assume that C.UTF-8 is
*always* available, even though that was the goal (see
<https://issues.guix.gnu.org/67686>.)

AFAICS, the format is actually architecture-independent:

--8<---------------cut here---------------start------------->8---
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s i686-linux
/gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' 
/gnu/store/f3kpp3dh25893y79n32qlqqiwb5my4zg-glibc-utf8-locales-2.39
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/f3kpp3dh25893y79n32qlqqiwb5my4zg-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s aarch64-linux
/gnu/store/0i1brwncg9rpf7pvh9hs5vrchmfb3c9q-glibc-utf8-locales-2.39
$ guix build -e '(@@ (gnu packages base) glibc-utf8-locales)' -s armhf-linux
/gnu/store/8x26ik4jlvljcnx1jhfd83r5lyx04d15-glibc-utf8-locales-2.39
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/0i1brwncg9rpf7pvh9hs5vrchmfb3c9q-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
$ diff -r /gnu/store/4v8xzpkkkzpkm6qmxjv1lcma69chdwkl-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8  /gnu/store/8x26ik4jlvljcnx1jhfd83r5lyx04d15-glibc-utf8-locales-2.39/lib/locale/2.39/C.utf8
$ echo $?
0
--8<---------------cut here---------------end--------------->8---

So it might work by replacing "true" by "localedef" in this phase (and
we could do that on ‘master’), under the assumption that the
cross-compiled libc and the one that provides the ‘localedef’ command
are the same version, which is usually the case.

We should do that.

>>>  (define* (text-file* name #:rest text)
>>>    "Return as a monadic value a derivation that builds a text file containing
>>> @@ -2108,6 +2119,7 @@ (define* (text-file* name #:rest text)
>>>    (define builder
>>>      (gexp (call-with-output-file (ungexp output "out")
>>>              (lambda (port)
>>> +              (set-port-encoding! port "UTF-8")
>>>                (display (string-append (ungexp-splicing text)) port)))))
>>
>> LGTM.  This can be moved to a separate file.
>
> By "separate file" you mean separate patch and/or commit?

Yes.

Thanks!

Ludo’.