Subject: Re: bug#67331: 30.0.50; New Todo mode feature: changing item date style

From: Stephen Berman <[email-masked]>
Date: Thu, 23 Nov 2023 18:40:07 +0100

On Thu, 23 Nov 2023 10:06:46 +0200 Eli Zaretskii <[email-masked]> wrote:

>> From: Stephen Berman <[email-masked]>
>> Date: Tue, 21 Nov 2023 16:32:34 +0100
>>
>> This post is a followup to bug#66395, where I posted (and subsequently
>> installed) a patch to support the ISO date style in Todo mode item
>> headers.  As I pointed out in that bug thread, despite supporting
>> different date styles, Todo mode does not support interactively and
>> automatically changing between different date styles.  I have attempted
>> to add this functionality to Todo mode, and the attached patch contains
>> my work-in-progress implementation.  I'm still testing these changes, so
>> even if they are deemed acceptable I'm not yet ready to install them,
>> but they raise some questions and how I proceed will be influenced and
>> perhaps decided by the answers to these questions.
[...]
>
> Let me take a step or two back, and ask whether Todo mode needs to
> support different styles, let alone interactive changes between
> styles?  IOW, what bad things will happen if Todo mode supported only
> one fixed style?  I'm asking this because it sounds like adding this
> support raises quite a few issues whose resolution is hairy at best,
> and in some cases causes real user-visible problems.  Refusing to
> support more than one style will eliminate all these issues, with
> little or no effort.

Todo mode dates are essentially the same as Emacs Calendar/Diary dates,
which can be input in the three styles American, European and ISO.  Due
to negligence on my part (insufficient testing, forgetting about
bug#55284) only the first two styles worked in Todo mode until the fix I
installed for bug#66395.  So it's not inputting or displaying different
date styles that raises the issues, but switching between these styles.
And indeed, this is also a problem for the Diary (see bug#55286).
Admittedly, there's no need to switch styles that I'm aware of, but I
think it is a nice option to have if the cost isn't prohibitive, so...

> If you do have to support the various styles, my next question is: why
> not make the style a display-only feature?  That is, keep using a
> single style in the TODO files, but reformat the date to a
> user-specified style at display time.  This should at least avoid the
> complications of incompatible files, switching between file formats,
> whether and how should all the existing files be converted to the
> selected style, etc.  After all, why should users care about the
> internal format of how time stamps are recorded on a TODO file, if
> they use Todo mode to display and edit those files?

...yes, a display-only implementation is an alternative worth
considering.  Indeed, I believe this is what Org mode does, though it
doesn't seem to be entirely unproblematic (I don't use Org mode for
maintaining Todo lists, but that's what I gather from the Org manual
(info "(org) Custom time format")).  However, it seems to me less suited
to Todo mode for at least two reasons.

One is that, unlike Org mode, Todo mode, as just noted above, already
supports the input and display of the three Calendar/Diary styles, and
this is part of the file content, not a display-time-only attribute.
Hence, the display-time feature can't be a mapping from one underlying
date style to one of the three support styles but has to recognize any
of the three styles and from that produce an appropriate overlay or text
property, which seems more complex (though I haven't tried to implement
this idea yet, so maybe it isn't so complex).

The other problem with display-time date-style switching is the fact
that Todo mode allows manually editing the date header string, so if the
display is in one style and the actual string in another, then when
editing it manually there is a visual disconnect, which seems annoying.

But perhaps these issues aren't serious enough to reject the
display-time alternative.  I'll try to implement it and see how I find
using it.

> Failing these solutions that avoid the complications, here are my
> opinions about the issues you bumped into:
>
>> - Although the use and effect of this feature is confined to
>>   todo-mode.el, it depends on a making a change to calendar.el, as you
>>   can see in the above summary and in the patch.  The reason, as I
>>   already noted in bug#66395, is that Todo mode date styles are
>>   specified by `calendar-date-display-form', which you can change by
>>   customizing `calendar-date-style' or by executing
>>   `calendar-set-date-style'.  To keep this connection I added a hook to
>>   calendar.el callable from the latter function, that can be used to
>>   trigger the date header changes in Todo mode (but it's not a hook in
>>   the usual sense of being intended for user customization; rather, it's
>>   intended for use by packages).  There is precedence for this in commit
>>   a8f4bb8361, where I similarly added a hook to diary-lib.el for the
>>   benefit of todo-mode.el.  In addition, since the hook has the value
>>   nil by default, there is no change to the default behavior of the
>>   Emacs Calendar (or Diary).  So, is this change to calendar.el
>>   acceptable?
>
> I think this is acceptable, but we need commentary for this hook to
> explain the subtleties you mention above.

I'll add that if the present implementation is retained.

>> - My implementation of this feature crucially involves a change in the
>>   internal form of todo files.  Till now the first line of each todo
>>   file has contained a sexp listing the categories in the file (this
>>   line is hidden in todo-mode), but to allow switching date styles I
>>   have augmented this sexp to also contain a specification of the
>>   current date style used by the item headers in the file.  A number of
>>   existing todo-mode functions make use of this sexp, so they have also
>>   had to be adapted.  All these changes are strictly internal, so the
>>   Todo mode UI remains unchanged.  The implementation also includes
>>   automatically converting existing todo files to the augmented metadata
>>   format, so that users of Todo mode can continue using existing todo
>>   files, as well as the new functionality, without needing to be aware
>>   of the internal changes.  But once converted, the todo files will not
>>   be compatible with previous versions of todo-mode.el.  This would be a
>>   problem if someone wanted to use the same todo files on different
>>   systems, not all of which have an Emacs where todo-mode.el has these
>>   changes.
>>
>>   Is this problem considered unacceptable?
>
> It is certainly undesirable.  But we already did something similar,
> for example in the format of desktop files.  See commits c96983efef
> and 20defc5538.
>
>>   So is it acceptable to change the metadata format and the item header
>>   for each todo file on demand instead of for all todo files at once?
>
> I think so, yes.  But again, such changes are undesirable in the first
> place.

I agree, so if the new format is finally accepted, I'll provide an
escape hatch to revert to the earlier format.

>> - Another thorny issue is how to deal with unit tests.  Currently, there
>>   are a number of tests for todo-mode.el using ERT.  Since they all
>>   require a todo file as input data and many make use of the file
>>   metadata (currently in the old format), at least these tests will
>>   require adjusting.  So should there be parallel tests for todo-mode
>>   with the new and with the previous metadata format?  Or should the
>>   older format be declared obsolete?  (However, AFAIK there is no formal
>>   obsoletion mechanism for data formats as opposed to functions and
>>   variables.)  I think the decision about this is contingent on the
>>   decision about allowing reverting the format for backwards
>>   compatibility.
>
> If we keep supporting both formats, even if the old one is deprecated,
> we will need to keep the tests for that old format.  Deprecation is
> not removal, so removing the tests (which already exist) sounds like
> not the best idea to me.

Yeah, I'll do what I can to accommodate both formats (if the change is
made).

>> Finally, I want to point out an issue that I think is mainly orthogonal
>> to the new feature and the preceding considerations, but it helps
>> clarify the extent of the feature.  Currently, Todo mode supports item
>> date headers with either the American, European or ISO date style, but
>> all items can only be displayed with the same style, and that
>> restriction will persist if the changes discussed here are installed:
>> then you can switch date styles, but doing so changes all item headers
>> in a visited todo file.  In other words, there can be no mixing of date
>> styles within a todo file, or when switching between todo files.  This
>> restriction is justified by avoiding ambiguity if dates are displayed
>> simultaneously in the European and American styles with numbers for
>> months.  Admittedly, `calendar-date-display-form' uses month names by
>> default in these styles, which avoids ambiguity (and there's no
>> ambiguity between the ISO style and the other styles, unless years are
>> displayed with just the two least significant digits, which is common in
>> the American and European styles but AFAIK the ISO standard requires
>> four digits for years).  Hence, at least for such display forms, mixing
>> date display styles should be possible (though whether it's desirable is
>> questionable).  I haven't given serious thought to how to implement
>> that, nor to how it would interact with the proposed feature of
>> switching date styles without mixing, but it would likely require
>> changes to the implementation in the attached patch.
>
> I think it's entirely reasonable to support only one style for
> displaying TODO items.  I cannot think about a reason for displaying
> different items using different styles.

I can't either, so unless someone comes up with one, I won't pursue it.

> Thanks.

Thanks for the feedback.  I'll follow up again when I succeed in
implementating the display-time alternative.

Steve Berman