Subject: Re: bug#77924: 31.0.50; [Feature branch] Change marker implementation

From: Gerd Möllmann <[email-masked]>
Date: Fri, 25 Apr 2025 07:40:16 +0200

Stefan Monnier <[email-masked]> writes:

> - Finally `smie-nonascii` is the same as `smie` except that it uses
>   a file that's identical to `xmenu.c` but where all the letters of
>   comments/strings/identifiers were replaced by non-ASCII ones.
>   Here, we see that TINDEX` is *much* slower than BASE.
>   This doesn't seem to be a fluke: I see the exact same performance
>   difference in my other pair of builds.
>
> This last one is a serious problem that we need to address before we can
> merge the branch.

ASCII means that the text index is basically not used.

In the non-ASCII case it is used. Re-indentation means text modification
which in turns leads to index invalidation and re-building the index as
needed. Index invalidation discards the index from the modification
position to the end of the text. Re-building goes from the last valid
position the index contains to the next position that is needed to do a
position conversion.