Subject: Re: bug#27193: 25.2; tmm should use completing-read-default

From: Ryan Thompson <[email-masked]>
Date: Fri, 02 Jun 2017 22:52:16 +0000

(Note: I think several more replies came in while I was writing this; I
haven't read those yet.)

Ok, so if I'm understanding correctly, the thrust of your argument is that
completing-read is a very general function that supports a very wide range
of behaviors/usages, to such a degree that tmm's usage does not seem like
an anomalous or unusual way to use it. I might not agree on what
constitutes a typical usage of completing-read, but I can definitely agree
that it is a very general function, and in practice no alternative value of
completing-read-function that works substantially differently from
completing-read-default is likely to be capable of replicating all of that
functionality.

So given your recommendation that alternative completion functions should
"punt" cases they can't handle back to completing-read-default, along with
the high likelihood that every alternate completion system is going to have
something that it can't handle and needs to punt on, would you be
interested in a patch that implements that punting mechanism within
completing-read itself, so that ido-ubiqutous, ivy, helm, and others don't
all have to invent their own punting code? I'm thinking along the lines of
wrapping the call to completing-read-function with a condition-case that
catches a "completing-read-fallback" signal, and calls
completing-read-default instead. So any completion system that wants to
punt just has to throw the appropriate signal.

Some replies to specific points, which may or may not be moot:

>
> Please think about providing a different implementation of tmm, which
> doesn't use `completing-read'. That will let us know whether
> `completing-read' is used only as an implementation detail.
>

Magit implements a very similar system of using single-key shortcuts to
activate items from a textual menu, with multiple levels of menus. It
doesn't use completing-read.

Of course it does completion.  Erase chars in the minibuffer from the right
> and then hit TAB - completion. It's not a completion to write home about,
> but it's completion.
>

I would say that the fact that tmm does completion on hitting TAB is an
internal implementation detail leaking out, i.e. an unintended side-effect
of the fact that it uses completing-read internally. The main functionality
of tmm is single-key shortcut access to menu items, not completion of
user-entered text with matching strings from a set of choices.

So pluggable completion backends don't make any sense for tmm,
>
>
>
> That too sounds overly broad. I think that you have too readily convinced
> yourself that this and that don't make *any possible* sense, and you are
> prone to nail things down to prevent what you see as the nonsensical. If
> such were truly nonsensical then they would likely never exist, and there
> would be no reason to forbid them.
>
>
>
> I sense that you think of "pluggable completion backends", that is, uses
> of `completing-read-function', as necessarily something similar to what
> your code does.
>

I don't think I'm making a broad statement. My claim is that settings that
affect how completion works should not affect tmm because tmm isn't doing
completion (or rather, tmm is doing completion only incidentally due to a
leaky abstraction). I still stand by this claim, regardless of whether
tmm's use of completing-read is considered typical or not. (If you accept
this claim, there's probably some other completion-related settings that
tmm should also ignore.)

[FWIW I don't think of `completing-read-function' as providing only a
> "pluggable completion backend". Do you think of
> `isearch-search-fun-function' as a "pluggable search backend"? Backend?
> frontend? There are many angles to something like completion or search
> or... There is not just a backend and a frontend. Likewise, what you call
> an "alternative completion system".
>

Well, I'm not sure exactly what to call it. I suppose "alternative values
of completing-read-function" is more technically correct, but it's a real
mouthful.

 Really, we're just talking about a completion function - a value for
> `completing-read-function'. That has wide scope - there are few constraints
> on what it *must* do. I don't see the justice in adding a constraint for
> tmm that it cannot do anything except `completing-read-default'.]
>
>
>
> and I can't imagine any other value of completing-read-function that would
> make sense for tmm besides completing-read-default.
>
>
>
> All you need to imagine is something similar to `completing-read-default'
> but slightly different in some respect. That's all you *need* to imagine,
> but nothing prevents more imagination.'
>

I was implicitly assuming that no one would bother to implement a new
completion function from scratch unless it differed substantially from an
existing one.

>
> And what you or I can imagine is not really the point. The question is
> whether tmm.el must limit itself to `completing-read-default'. I don't see
> why it should, just because we know of some values of
> `completing-read-function' that don't do the right thing for tmm.
>
>
>
> Looking at the "git blame" output for tmm, that call to completing-read
> has definitely not been updated since completing-read-function was
> introduced except for minor bugfixes,
>
>
>
> That's irrelevant.
>
>
>
> so it makes sense that tmm would be expecting one and only one
> implementation of completing-read.
>
>
>
> That does not follow at all.
>
>
>
> This kind of argument could (inappropriately, IMO) be
> applied to any number of completely normal uses of
> `completing-read'.
>
> I see no reason to impose a dichotomy of either a
> `completing-read-function' similar to yours or else
> `completing-read-default'.  There are likely other
> benign values of the variable, besides just
> `completing-read-default'.
>
>
>
> I'm not trying to set a general precedent here. tmm is the only code that
> I'm aware of that uses completing-read in this way.
>
>
>
> I agree that it is not a common way of using it. It doesn't follow that
> the only possible value of `completing-read-function' that is compatible
> with tmm is `completing-read-default'. Not at all.
>
>
>
> It sounds like (and no, I haven't looked into it;
> it just sounds like it) you have some particular
> `completing-read-function' values in mind, which
> you have found are incompatible with tmm's use of
> `completing-read'.
>
>
>
> The alternative completing-read-function providers that I am aware of are
> of are ido-ubiquitous (written by me), ivy, and helm. I'll also include
> icicles, even though uses some other mechanism besides modifying
> completing-read-function. ido-ubiquitous and ivy both have code to
> deactivate themselves when completing-read is called by tmm because
> otherwise their completion systems would break it, while icicles and helm
> simply break tmm when enabled. Thus, to my knowledge there is currently no
> other completing-read-function that doesn't break tmm (except for those
> that make exceptions specifically for tmm).
>
>
>
> Again, it is irrelevant that there are uses of `completing-read-function'
> that break tmm. And what you or I am aware of, or even what might exist
> anywhere today, does not define the scope of possibilities.
>
>
>
> [I drink values of the variable `liquid'. Some values, such as strong
> sulfuric acid, are quite incompatible with my proper functioning. That
> doesn't mean that the only *possible* value or the only *compatible*
> value for me is the default value, `water'.
>
>
>
> I drink blindly - I don't know what's in the glass. The only requirement
> my mouth imposes is that the variable value be a liquid. It is up to
> whoever fills my glass to DTRT.]
>
>
>
> And if those uses of `completing-read-function' are incompatible with tmm,
> and they thus deactivate themselves for tmm commands, that is exactly the
> *right* approach, IMO. It is exactly that which I suggested to you
> (without knowing that that is what you already do).
>
>
>
> [Tmm does work with Icicles, BTW. (But Icicles does not use
> `completing-read-function', among other reasons because it wants to work
> also with older Emacs versions.)]
>
>
>
> If so, that's not an argument for preventing the use of
> other values of `completing-read-function' with tmm.
> (Clearly the value `completing-read-default' is fine,
> for instance.)  That's not an argument for tmm to do
> something to prevent all use of `completing-read-function'.
>
> Instead, it's an argument for the code that defines and
> uses a particular `completing-read-function' to take
> care of the contexts where it makes sense, and to stop
> itself from being used in other contexts, where it might
> cause trouble.
>
> Only that code knows the kinds of context where its own
> `completing-read-function' makes sense and where it does
> not.  Code such as tmm should not try to guess what kinds
> of trouble different values of `completing-read-function'
> might cause.
>
> I don't think tmm should throw up its hands and say, "Gee,
> there might be some values of `completing-read-function'
> that are troublesome, so let's just prevent *all* use of
> that variable."  That doesn't make sense, to me.
>
>
>
> Based on my explanation above, that is precisely what I think tmm should
> do: avoid using completing-read-function entirely.
>
>
>
> I know you think that, but I don't see why. There are surely values of
> `completing-read-function' that do not bother tmm. We know of one already:
> `completing-read-default'. Why would you suppose that there can be *no*
> others?
>
>
>
> To look at it another way, tmm was originally written to use
> completing-read as an implementation detail, and later the function that
> used to be called completing-read was renamed to completing-read-default,
> but tmm was never updated to use the new name. This patch rectifies that.
>
>
>
> That's completely imagination. `completing-read-default' is not the new
> name of what was once `completing-read'. `completing-read' has never used
> code like `completing-read-default'. (It has always been written in C.)
>
>
>
> What I would say that perhaps you will think goes a bit in your direction
> is this: If you can rewrite `tmm.el' so that it has the *same behavior*
> without using `completing-read', and the code is at least as simple and
> easy to maintain, then I'd say go ahead and do that.
>
>
>
> That would support your idea that `completing-read' is only an
> implementation detail etc., and the question of `completing-read-function'
> would become moot.
>

I've provided Magit's popups as an example of similar functionality
implemented without completing-read. I'm not a user of tmm, I just want to
make sure my completion function doesn't break it.

>
>
> If you want additional suggestions, maybe describe just
> what the problem is that your completion function causes
> for tmm.  It's hard to offer suggestions if you only
> state that it is incompatible, without going into any
> detail.  (Not that you must ask for input about this.
> But if you would like some then giving more detail might
> help.)
>
> Please use your own judgment (as I said, I don't really
> care much about `tmm'), but a priori this sounds like
> overkill.
>
> It sounds a bit like trying to bend Emacs to fit your
> `completing-read-function'.  I can understand such a
> motivation, believe me; I don't ascribe a bad intention
> to you.  A guess is that you are not sure what to do,
> to prevent inappropriate application of your value of
> `completing-read-function' in this or that context.
>
> If you think it causes trouble in some contexts, or it
> is not able to handle some contexts properly, then
> I'd suggest you consider walling it off from those use
> cases.  It might take some time to discover which
> contexts it causes trouble for, but that's OK - you
> could add them as you discover them.  Tmm sounds like
> a start.
>
>
> The right approach, IMO, is to teach your code when to
> use its `completing-read-function' and when not to use
> it.  Put differently, consider teaching your
> `completing-read-function' when and where to hold back
> and just punt to the default behavior.
>
>
>
> This is exactly how ido-ubiquitous and ivy both currently work: they
> essentially have a blacklist of callers for which they revert to standard
> completion. tmm is on the blacklist for both packages.
>
>
>
> That's great. Problem solved. That's the right approach, IMO.
>
>
>
> Certainly, for any alternative completion system
>
>
>
> Any? Again with the superlatives. There is more to the world...
>
>
>
> there will be cases where it needs to fall back to standard completion. In
> my view, the completion system should be able to determine purely based on
> the calling context (i.e. its arguments and any relevant dynamically-bound
> variables) whether or not it needs to punt. Making this decision based on
> the name of the caller instead of the context to make this decision is
> admitting that not enough context was provided. I view it as a workaround,
> not a desirable design pattern, and someday in the future I hope to be able
> to remove the blacklist from ido-ubiquitous.
>
> In the case of tmm, the best heuristic I can think of would be to inspect
> the key bindings of all the letters and numbers. If any of them are locally
> rebound in the minibuffer to something other than self-insert-command, then
> punt.
>
>
>
> That would be silly, IMHO. There can be plenty of uses of
> `completing-read' where letter or number keys are bound to commands other
> than `self-insert-command'.
>
>
>
> [FWIW, though not directly relevant, when Icicle mode is on there are *no*
> keys bound to `self-insert-command' in any of the minibuffer completion
> keymaps. (But there are keys bound to `icicle-self-insert'.)]
>

My heuristic was meant specifically for ido-ubiquitous: the heuristic is
that if any letters or numbers are bound to something other than
self-insert-command, the caller is probably doing something fancy that
won't work well with ido completion. In any case, it was merely meant as
one example of how a completion function might infer from context whether
it should punt without having to identify callers by name.

Anyway, regardless of whether this patch is accepted or not, ido-ubiquitous
will still need a blacklist either way, since functions like
read-file-name that
are already covered by normal ido-mode will always be on that blacklist. So
I'm not exactly submitting this patch to make my implementation easier. I
just thought that having tmm ignore completing-read-function would make it
less fragile and mean one less obstacle for anyone implementing a new
complting-read-function.