Subject: 30.1; Test failure for fns-tests-collate-strings with musl libc
From: Ulrich Müller <[email-masked]>
To: [email-masked]
Date: Tue, 25 Feb 2025 12:49:49 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Forwarding Gentoo bug https://bugs.gentoo.org/906012.

On an x86_64-pc-linux-musl system (musl libc 1.2.5), we see one test
failure in fns-tests-collate-strings:

Running 81 tests (2025-02-25 10:06:35+0000, selector `(not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp)))')
   passed   1/81  fns--copy-alist (0.000116 sec)
   passed   2/81  fns--define-hash-table-test (0.000040 sec)
   passed   3/81  fns--string-to-unibyte-multibyte (0.000156 sec)
   passed   4/81  fns--take-ntake (0.000225 sec)
   passed   5/81  fns-append (0.000043 sec)
   passed   6/81  fns-concat (0.000062 sec)
   passed   7/81  fns-test-base64url-encode-region (0.000244 sec)
   passed   8/81  fns-test-base64url-encode-string (0.000083 sec)
   passed   9/81  fns-tests-base64-decode-string (0.000097 sec)
   passed  10/81  fns-tests-base64-encode-region (0.000090 sec)
   passed  11/81  fns-tests-base64-encode-string (0.000045 sec)
   passed  12/81  fns-tests-collate-sort (0.000180 sec)
Test fns-tests-collate-strings backtrace:
  signal(ert-test-failed (((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")) :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t :fail-reason "did not signal an error")))
  ert-fail(((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")) :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t :fail-reason "did not signal an error"))
  #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>)()
  #f(compiled-function () #<bytecode -0x1103712a176de548>)()
  handler-bind-1(#f(compiled-function () #<bytecode -0x1103712a176de548>) (error quit) #f(compiled-function (err) #<bytecode 0x1d814be4f72f192c>))
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test :name fns-tests-collate-strings :documentation nil :body #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>) :most-recent-result #s(ert-test-failed :messages "" :should-forms (... ... ... ... ... ...) :duration 9.6502e-05 :condition (ert-test-failed ...) :backtrace (... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) :infos nil) :expected-result-type :passed :tags nil :file-name "/var/tmp/portage/app-editors/emacs-30.1/work/emacs-30.1/test/src/fns-tests.el") :result #s(ert-test-failed :messages "" :should-forms ((... :form ... :value t) (... :form ... :value t) (... :form ... :value nil) (... :form ... :value t) (... :form ... :value t) (... :form ... :value t)) :duration 9.6502e-05 :condition (ert-test-failed (... :form ... :value t :fail-reason "did not signal an error")) :backtrace (#s(backtrace-frame :evald t :fun signal :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-fail :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0x1103712a176de548>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun handler-bind-1 :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert--run-test-internal :args #0 :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-test :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-or-rerun-test :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests-batch :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests-batch-and-exit :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun eval :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-line-1 :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-line :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun normal-top-level :args nil :flags nil :locals nil :buffer nil :pos nil)) :infos nil) :exit-continuation #f(compiled-function () #<bytecode 0x98f3573c9bd6c3>) :ert-debug-on-error nil))
  ert-run-test(#s(ert-test :name fns-tests-collate-strings :documentation nil :body #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>) :most-recent-result #s(ert-test-failed :messages "" :should-forms (((skip-unless (fns-tests--collate-enabled-p)) :form (fns-tests--collate-enabled-p) :value t) ((should (string-collate-equalp "xyzzy" "xyzzy")) :form (string-collate-equalp "xyzzy" "xyzzy") :value t) ((should-not (string-collate-equalp "xyzzy" "XYZZY")) :form (string-collate-equalp "xyzzy" "XYZZY") :value nil) ((should (string-collate-lessp "XYZZY" "xyzzy" "POSIX")) :form (string-collate-lessp "XYZZY" "xyzzy" "POSIX") :value t) ((should (string-collate-equalp "xyzzy" "XYZZY" nil t)) :form (string-collate-equalp "xyzzy" "XYZZY" nil t) :value t) ((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")) :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t)) :duration 9.6502e-05 :condition (ert-test-failed ((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")) :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t :fail-reason "did not signal an error")) :backtrace (#s(backtrace-frame :evald t :fun signal :args (ert-test-failed (...)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-fail :args ((... :form ... :value t :fail-reason "did not signal an error")) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0x1103712a176de548>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun handler-bind-1 :args (#f(compiled-function () #<bytecode -0x1103712a176de548>) (error quit) #f(compiled-function (err) #<bytecode 0x1d814be4f72f192c>)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert--run-test-internal :args (#s(ert--test-execution-info :test #1 :result #2 :exit-continuation #f(compiled-function () #<bytecode 0x98f3573c9bd6c3>) :ert-debug-on-error nil)) :flags nil :locals ((ert--infos) (lexical-binding . t)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-test :args #0 :flags nil :locals ((ert--running-tests #1) (message-log-max . t) (ert--should-execution-observer . #f(compiled-function (form-description) #<bytecode -0x68c27499cc3ea68>))) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-or-rerun-test :args (#s(ert--stats :selector ... :tests ... :test-map #<hash-table eql 81/81 0x1800022ce8f7 ...> :test-results ... :test-start-times ... :test-end-times ... :passed-expected 12 :passed-unexpected 0 :failed-expected 0 :failed-unexpected 1 :skipped 0 :start-time ... :end-time nil :aborted-p nil :current-test #1 :next-redisplay 0.0) #1 #f(compiled-function (event-type &rest event-args) #<bytecode 0x1073d527539a2e17>)) :flags nil :locals ((ert--current-run-stats . ...)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests :args ((not ...) #f(compiled-function (event-type &rest event-args) #<bytecode 0x1073d527539a2e17>) nil) :flags nil :locals ((ert--current-run-stats . ...)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests-batch :args ((not ...)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests-batch-and-exit :args ((not ...)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun eval :args ((ert-run-tests-batch-and-exit ...) t) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-line-1 :args (("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-load-path '(\"/var/tmp/portage/app-editors/emacs-30.1/homedir/.emacs.d/tree-sitter\"))" "--eval" "(setq ert-batch-backtrace-right-margin nil)" "-l" "src/fns-tests" "--eval" "(ert-run-tests-batch-and-exit (quote (not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp)))))")) :flags nil :locals ((command-line-args-left) (argi . "-eval")) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-line :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun normal-top-level :args nil :flags nil :locals nil :buffer nil :pos nil)) :infos nil) :expected-result-type :passed :tags nil :file-name "/var/tmp/portage/app-editors/emacs-30.1/work/emacs-30.1/test/src/fns-tests.el"))
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-map #<hash-table eql 81/81 0x1800022ce8f7 ...> :test-results ... :test-start-times ... :test-end-times ... :passed-expected 12 :passed-unexpected 0 :failed-expected 0 :failed-unexpected 1 :skipped 0 :start-time ... :end-time nil :aborted-p nil ...) #s(ert-test :name fns-tests-collate-strings :documentation nil :body #f(compiled-function () #<bytecode -0xf4a78f5ec213ec3>) :most-recent-result ... :expected-result-type :passed :tags nil :file-name "/var/tmp/portage/app-editors/emacs-30.1/work/emacs-30.1/test/src/fns-tests.el") #f(compiled-function (event-type &rest event-args) #<bytecode 0x1073d527539a2e17>))
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp))) #f(compiled-function (event-type &rest event-args) #<bytecode 0x1073d527539a2e17>) nil)
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp))))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp))))
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp)))) t)
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-load-path '(\"/var/tmp/portage/app-editors/emacs-30.1/homedir/.emacs.d/tree-sitter\"))" "--eval" "(setq ert-batch-backtrace-right-margin nil)" "-l" "src/fns-tests" "--eval" "(ert-run-tests-batch-and-exit (quote (not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp)))))"))
  command-line()
  normal-top-level()

Test fns-tests-collate-strings condition:
    (ert-test-failed
     ((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8"))
      :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t
      :fail-reason "did not signal an error"))
   FAILED  13/81  fns-tests-collate-strings (0.000097 sec) at src/fns-tests.el:264
   passed  14/81  fns-tests-compare-strings (0.000099 sec)
   passed  15/81  fns-tests-equal-including-properties (0.000083 sec)
   passed  16/81  fns-tests-equal-including-properties/string-prop-vals (0.000060 sec)
   passed  17/81  fns-tests-equal-symbols-with-position (0.000059 sec)
   passed  18/81  fns-tests-equality-nan (0.000041 sec)
   passed  19/81  fns-tests-func-arity (0.005018 sec)
   passed  20/81  fns-tests-hash-buffer (0.000075 sec)
   passed  21/81  fns-tests-identity (0.000042 sec)
   passed  22/81  fns-tests-length (0.000045 sec)
   passed  23/81  fns-tests-mapcan (0.000047 sec)
   passed  24/81  fns-tests-mapconcat (0.000053 sec)
   passed  25/81  fns-tests-nreverse (0.000056 sec)
   passed  26/81  fns-tests-nreverse-bool-vector (0.000041 sec)
   passed  27/81  fns-tests-random (0.000067 sec)
   passed  28/81  fns-tests-reverse (0.000077 sec)
   passed  29/81  fns-tests-reverse-bool-vector (0.000058 sec)
   passed  30/81  fns-tests-safe-length (0.000057 sec)
   passed  31/81  fns-tests-sort (0.827500 sec)
   passed  32/81  fns-tests-sort-gc (0.133975 sec)
   passed  33/81  fns-tests-sort-kw (0.027079 sec)
   passed  34/81  fns-tests-string-bytes (0.000046 sec)
   passed  35/81  fns-tests-string-lessp (0.000394 sec)
   passed  36/81  fns-tests-string-version-lessp (0.000054 sec)
   passed  37/81  fns-value<-bool-vector (0.003720 sec)
   passed  38/81  fns-value<-circle (0.000046 sec)
   passed  39/81  fns-value<-ordered (0.000362 sec)
   passed  40/81  fns-value<-symbol-with-pos (0.000084 sec)
   passed  41/81  fns-value<-type-mismatch (0.000129 sec)
   passed  42/81  fns-value<-unordered (0.000055 sec)
   passed  43/81  fns-vconcat (0.000054 sec)
   passed  44/81  length-equals-tests (0.000066 sec)
   passed  45/81  object-intervals (0.000058 sec)
   passed  46/81  plist-get/odd-number-of-elements (0.000038 sec)
   passed  47/81  plist-member/improper-list (0.000041 sec)
   passed  48/81  plist-put/odd-number-of-elements (0.000040 sec)
   passed  49/81  string-search (0.000099 sec)
   passed  50/81  test-assoc-testfn (0.000053 sec)
   passed  51/81  test-bignum-eql (0.000044 sec)
   passed  52/81  test-bignum-hash (0.000044 sec)
   passed  53/81  test-buffer-line-stats-gap (0.001859 sec)
   passed  54/81  test-buffer-line-stats-nogap (0.000082 sec)
   passed  55/81  test-cycle-assoc (0.000066 sec)
   passed  56/81  test-cycle-assq (0.000063 sec)
   passed  57/81  test-cycle-delete (0.000120 sec)
   passed  58/81  test-cycle-delq (0.000091 sec)
   passed  59/81  test-cycle-equal (0.000067 sec)
   passed  60/81  test-cycle-length (0.000053 sec)
   passed  61/81  test-cycle-member (0.000067 sec)
   passed  62/81  test-cycle-memq (0.000064 sec)
   passed  63/81  test-cycle-memql (0.000071 sec)
   passed  64/81  test-cycle-nconc (0.000051 sec)
   passed  65/81  test-cycle-plist-get (0.000058 sec)
   passed  66/81  test-cycle-plist-member (0.000064 sec)
   passed  67/81  test-cycle-plist-put (0.000066 sec)
   passed  68/81  test-cycle-rassoc (0.000098 sec)
   passed  69/81  test-cycle-rassq (0.000065 sec)
   passed  70/81  test-cycle-reverse (0.000064 sec)
   passed  71/81  test-cycle-safe-length (0.000069 sec)
   passed  72/81  test-hash-function-that-mutates-hash-table (0.000064 sec)
   passed  73/81  test-line-number-at-position (0.000051 sec)
   passed  74/81  test-nthcdr-circular (0.000284 sec)
   passed  75/81  test-nthcdr-simple (0.000041 sec)
   passed  76/81  test-plist (0.000061 sec)
   passed  77/81  test-proper-list-p (0.000063 sec)
   passed  78/81  test-secure-hash (0.000066 sec)
   passed  79/81  test-string-distance (0.000067 sec)
   passed  80/81  test-sxhash-equal (0.000058 sec)
   passed  81/81  test-vector-delete (0.000054 sec)

Ran 81 tests, 80 results as expected, 1 unexpected (2025-02-25 10:06:36+0000, 1.101394 sec)

1 unexpected results:
   FAILED  fns-tests-collate-strings  ((should-error (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")) :form (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8") :value t :fail-reason "did not signal an error")

make[3]: *** [Makefile:185: src/fns-tests.log] Error 1

The problem is that (string-collate-equalp "xyzzy" "xyzzy" "en_XY.UTF-8")
doesn't signal an error with musl libc. This is because newlocale(3)
always succeeds, regardless of the string passed as its second argument.

(Note that bug #62420 had addressed a failure of the same test, and the
fix was to replace "en_DE.UTF-8" with "en_XY.UTF-8". This definitely
isn't enough to fix the problem on musl.)