Subject: Re: bug#62720: 29.0.60; Not easy at all to upgrade :core packages like Eglot

From: Dmitry Gutov <[email-masked]>
Date: Sat, 22 Apr 2023 13:48:41 +0300

On 22/04/2023 11:26, Eli Zaretskii wrote:

>> The above scenarios (package-install or (use-package eglot :ensure t))
>> when ~/.emacs.d/elpa is empty, result in the very latest Eglot being
>> installed when using Emacs 28. And will result in something different
>> with Emacs 29 (not the latest version). That's not nothing: the CI
>> scripts will have to be updated, and the user instructions for reporting
>> problems will have to be made more complicated as well. Some
>> possibilities will simply be gone (the user won't be able to upgrade
>> Eglot to the very latest by deleting it from ~/.emacs.d/elpa and calling
>> package-install, for example). This *is* a non-backward compatible
>> change from the perspective of Eglot's maintainer.
> 
> We were talking about users installing and updating packages.  The CI
> scenario doesn't belong here.  It is also much less important one
> (test suites are always required to chase the changes in development).
> 
> Let's not complicate an already complicated set of issues by bringing
> up unimportant secondary use cases.

You asked. I relayed what's already been said on the matter by Joao.

>> I, personally, don't really buy this kind of argument, but I figured you
>> might. After all, it's rather in line with reasoning we've seen voiced
>> around these parts many times ("X has worked this way for Y years, let's
>> never change it from now on"). Classic https://xkcd.com/1172/.
> 
> If you allude to my reasoning, then it is never that simplistic, and
> always considers each case separately, not "by analogy".

Of course that's a simplification. Hence the "might" anyway.

>> Either way would be fine, IMO, as long as the behavior is logical and
>> matches documentation. But having a separate command to upgrade now lets
>> us fix it separately without worry of breaking something more globally.
> 
> Except that, based on what we have (see below) ,we don't really have
> an "upgrade" operation, we only have "install" and "delete"
> (i.e. "uninstall").  So maybe we should preserve that, to minimize
> problems and user surprise/confusion.

We do. We have commands for upgrading, both in "list-packages", and used 
interactively. Which do the thing of installing the new version and 
removing the old one.

Which is what upgrading means in various tools, e.g. 'apt'.

>>> In interactive invocation, package-upgrade calls completing-read with
>>> its 4th argument non-nil, so you cannot select a package which is not
>>> in the collection returned by package--updateable-packages.  What I
>>> meant above is to allow that collection to include built-in packages
>>> as optional behavior.  I just tried invoking package-update for ElDoc,
>>> and I get "No match" after typing "eldoc" to its prompt, although
>>> eldoc version 1.14.0 is in the list presented by list-packages as
>>> "available".
>>
>> That's what I imagined: adding a new optional argument to
>> package--updateable-packages which would include builtins in the result.
>>
>> And only pass it when called from package-upgrade.
>>
>> Hopefully that's the kind of optional that you meant.
> 
> Yes, something like that.  Presumably activated by the same new option
> introduced for package-installed.

Okay, then it's something different, and you didn't answer my question.

>>>>>> 'package-update' is an
>>>>>> interactive function which itself it called from only one place:
>>>>>> 'package-update-all', and since the plan is to improve both, we can make
>>>>>> sure they only do what we ask of them: package-update will upgrade
>>>>>> builtins when invoked directly, and package-update-all will upgrade them
>>>>>> only when the builtin has been upgraded before (making it not a builtin
>>>>>> anymore), or a new user option is set.
>>>>>
>>>>> This is one possibility, and it might make sense to some users.  But I
>>>>> don't think we can be sure it will make sense to an overwhelming
>>>>> majority of the users.
>>>>
>>>> Hence the user option?
>>>
>>> Which one?  Are you suggesting to add a new one?  If so, why not use
>>> the one we already added, package-install-upgrade-built-in?
>>
>> The user option I was thinking of would probably be called a little
>> shorter: package-upgrade-built-in. And it would only affect the upgrader
>> commands.
> 
> We could rename the existing option, if the name is the problem.
> Otherwise, I don't see why we would need two separate options: they do
> the same job and have the same meaning from the user's POV.

The name is a problem, yes. What could also be a problem is a user that 
customizes this option to have package-update update builtin packages (a 
reasonable behavior that should be on by default anyway), will also 
automatically have change the behavior of package-install to be more 
surprising (install an already installed package).

Further, if we have a user option affect package-update, we'll have to 
alter package-update-all and package-manu-mark-for-update in the same 
patch (otherwise we'll have more nonsense on our hands). Whereas the 
first version I sent is more minimal.

>> All this is to say, the first step (upgrading Eglot to the version from
>> ELPA) will be less user-friendly compared to the other UIs we have. But
>> it's probably manageable, especially if documented well.
> 
> I don't see why it would be less user-friendly.

The same reason we do have commands with "upgrade" in their names, 
rather than force everybody to use the "install" and "delete" ones.

>> My point here, however, is that commit 580d8278c5f48 improved the
>> situation to a lesser degree that some people here might have expected.
> 
> One again: commit 580d8278c5f48 solved precisely the problem which
> opened this bug report, nothing more, nothing less.

It doesn't seem like the originator of the report agrees with that.

>> But we can conclude that we do have two different installation actions:
>>
>>    - package-install which will install the latest version of a package,
>> but only if it's not installed;
>>    - package-menu-mark-install, which will mark a specific version for
>> installation, disregarding whether some earlier version is already
>> installed; the previous version will remain installed still.
> 
> Which is again a breaking behavior change, AFAIU.  Is this a good idea
> so late in the development of Emacs 29?

The above is not a breaking change, it's how things work already. And 
have been working for quite a while.

>> But even if we decide that we want to eliminate that split, doing *that*
>> would really be a breaking change. I don't have a reasonable plan to
>> present for doing that in Emacs 29, so far.
> 
> There's no "split".  What I wanted to point out is that we don't seem
> to have a clear vision of these two commands, since they are confusing
> intertwined.  In fact, one could argue that package-upgrade in its
> current form is simply a convenience shortcut for "delete old and
> install new".

What should an upgrade command do, in your opinion, if not "delete old 
and install new"?