Subject: bug#54591: 29.0.50; sqlite-select returns blob result as multibyte string

From: Eli Zaretskii <[email-masked]>
Date: Sat, 02 Apr 2022 09:52:31 +0300

> From: Johannes Grødem <[email-masked]>
> Date: Sat, 02 Apr 2022 08:33:55 +0200
> 
> > Does SQLite TEXT allow the superset of UTF-8 encoding Emacs uses
> > internally to store characters that are not in Unicode? If it does, we
> > could indeed assume that any BLOB is binary data and not attempt
> > encoding/decoding it.
> 
> SQLite documentation says this...
> 
>   TEXT. The value is a text string, stored using the database encoding
>   (UTF-8, UTF-16BE or UTF-16LE).
> 
> ...but it's still possible to store byte sequences that are not legal
> Unicode in there. This breaks the mentioned Python SQLite3 API, and
> possibly others, so maybe not great if someone wants to read tables from
> something else than Emacs.

This probably means we should reject text with raw bytes or characters
whose codepoints are beyond #x10FFFF, and document that those should
be encoded manually and stored as BLOBs.

Thanks.