Subject: bug#68417: scm_from_stringn throws error regardless of failed conversion handler

From: Nate Whetsell <[email-masked]>
Date: Fri, 12 Jan 2024 14:11:33 -0500

Here is a test program that defines a Guile extension to convert invalid UTF-8 bytes to a string using each of the failed conversion handlers described at

https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom-C.html#index-scm_005ft_005fstring_005ffailed_005fconversion_005fhandler

```
#include <libguile.h>

static char str[] = {
  208, 151, 208, 178, 208, 184, 209, 135, 208, 176, 208, 185, 208, 189, 208, 184, 208, 102, 97, 109, 105, 108, 121
};

SCM
error_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", SCM_FAILED_CONVERSION_ERROR);
}

SCM
question_mark_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", SCM_FAILED_CONVERSION_QUESTION_MARK);
}

SCM
escape_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE);
}

void
init_test ()
{
  scm_c_define_gsubr ("conversion-error-test", 0, 0, 0, error_test);
  scm_c_define_gsubr ("conversion-question-mark-test", 0, 0, 0, question_mark_test);
  scm_c_define_gsubr ("conversion-escape-test", 0, 0, 0, escape_test);
}
```

If I compile this program using

```
gcc $(pkg-config --cflags --libs guile-3.0) -shared -o libguile-conversion-test.so -fPIC test.c
```

load the resulting extension into Guile, and run the functions, all of them throw errors. I would expect SCM_FAILED_CONVERSION_ERROR to throw an error, of course, and SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE is documented as being irrelevant to converting C strings to Scheme. However, I would not expect using SCM_FAILED_CONVERSION_QUESTION_MARK to throw a conversion error.

Here is the output I get from Guile:

```
GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load-extension "./libguile-conversion-test.so" "init_test")
scheme@(guile-user)> (conversion-error-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input locale conversion error" 0 #vu8(208 151 208 178 208 184 209 135 208 176 208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (conversion-question-mark-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_stringn" "input locale conversion error" 92 #vu8(208 151 208 178 208 184 209 135 208 176 208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (conversion-escape-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_stringn" "input locale conversion error" 92 #vu8(208 151 208 178 208 184 209 135 208 176 208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
```

Note that it is possible to convert those bytes to a string without an error using bytevector->string, which is documented as the Scheme interface to scm_from_stringn:

https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom-C.html#index-scm_005ffrom_005fstringn

```
scheme@(guile-user)> (use-modules (ice-9 iconv))
scheme@(guile-user)> (bytevector->string #vu8(208 151 208 178 208 184 209 135 208 176 208 185 208 189 208 184 208 102 97 109 105 108 121) "UTF-8" 'substitute)
$1 = "Звичайни�family"
```