Subject: Re: bug#48902: 28.0.50; Directory names containing apostrophes and backticks cause problems

From: Mattias Engdegård <[email-masked]>
Date: Tue, 8 Jun 2021 19:45:22 +0200

8 juni 2021 kl. 14.14 skrev Lars Ingebrigtsen <[email-masked]>:

>> It's always possible that stringWithLispString isn't doing the right
>> thing. It's implemented at nsfns.m:3026. I know almost nothing about
>> UTF8/UTF16 so while it looks like it's doing the right thing to me, I
>> could be entirely wrong.
> 
> Right -- and that was written by Mattias, so I've added him to the CCs.

Thank you, but stringWithLispString: is actually fine, unless you count its inability to produce useful results from wrong input!

The image code seems to be quite confused with respect to whether the file names being passed around are in encoded form. Until recently it seems to have worked by pure chance since as long as the file name encoding is UTF-8 and the low-level code accesses the raw string data we do get the same result, but at least since 747a923b9a35 that's no longer the case.

Concretely, we have:

1. image_find_image_file probably expects its `file` argument to be non-encoded, but the string it returns is always encoded.

2. native_image_load calls image_find_image_file and passes its return value to ns_load_image.

3. ns_load_image calls [EmacsImage allocInitFromFile:] with its file argument.

4. [EmacsImage allocInitFromFile: file] can apparently be called with both a non-encoded or an encoded `file` argument (clearly not ideal), and it does:

  found = image_find_image_file (file);
  // This is dubious when `file` is already encoded.

  found = ENCODE_FILE (found);
  // This is completely useless since `found` is already encoded! Apparently ENCODE_FILE is idempotent, at least on macOS...

  [NSString stringWithLispString: found]
  // This produces nonsense as `found` is a string of raw bytes, so any Unicode will be converted to stretches of U+FFFD REPLACEMENT CHAR.
  [NSString stringWithLispString: file]
  // Same problem again, with a different variable.

The quick fix of reverting to stringWithUTF8String: will work, but the real problem is that we have no control of the encodedness of lisp strings being passed around. Comments would help, and I'd even go as far to suggest

typedef struct { Lisp_Object string; } encoded_file_name_t;

with the appropriate constructors and accessors, to get C's static type checking to work for us.