GNU bug report logs - #37036
[PATCH] Inconsistent ASCII and Latin char categories

Previous Next

Full log

Message #32 received at 37036 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 37036 <at> debbugs.gnu.org
Subject: Re: bug#37036: [PATCH] Inconsistent ASCII and Latin char categories
Date: Fri, 16 Aug 2019 00:19:43 +0200

15 aug. 2019 kl. 21.23 skrev Eli Zaretskii <eliz <at> gnu.org>:

> I think they should become entirely useless, i.e. we should stop using
> them.  We have the entire Unicode database with all the character
> properties for quite some time now, and should favor using that
> instead.  Categories are an old kludgey hack, which goes back to
> pre-Unicode Emacs; it can never be anything but arbitrary, and we will
> never be able to fix that anywhere near completely.

Thank you, I see what you mean, and I agree that Unicode properties probably are better for most purposes.
In any case, I wasn't aiming for perfection; that is indeed a fool's errand. It was just a discovery of a rather obvious mistake, and evidence of code that doesn't work properly because of it. I thought the patch would be rather uncontroversial.

>> Consider the function fill-polish-nobreak-p. It is clearly written with the assumption of a reasonable definition of the Latin category, and it doesn't work as expected because of that.
> 
> Can you tell the details of where this function doesn't work?  I'd
> like to understand why fixing it needs to change the categories.

Right: it attempts to match a single-character word before point, with the assumption that \cl would match any Latin(-script) letter. However, since that expression matches most of ASCII as well, the function incorrectly says that line-breaking would be disallowed after "In my dreams..." or "(She smiles!)" or "He died in 1951." (well, the equivalents in Polish).
Some details are in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20871 .

Of course it doesn't require the categories to be fixed. The point is that if there is some code that doesn't work because of the broken categories, there may very well be more.

> I don't think we should fix those mistakes, because that's an
> impossible goal.  We should instead gradually stop using categories
> for anything serious, certainly for any new code.  We should use the
> UCD properties and the various char-tables built upon that instead.

Perhaps, but categories still have one thing going for them: they have fairly good regexp support.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.