Package: emacs;
Reported by: Stephen Berman <stephen.berman <at> gmx.net>
Date: Tue, 21 Nov 2023 15:33:02 UTC
Severity: wishlist
Found in version 30.0.50
To reply to this bug, email your comments to 67331 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-gnu-emacs <at> gnu.org
:bug#67331
; Package emacs
.
(Tue, 21 Nov 2023 15:33:02 GMT) Full text and rfc822 format available.Stephen Berman <stephen.berman <at> gmx.net>
:bug-gnu-emacs <at> gnu.org
.
(Tue, 21 Nov 2023 15:33:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Stephen Berman <stephen.berman <at> gmx.net> To: bug-gnu-emacs <at> gnu.org Subject: 30.0.50; New Todo mode feature: changing item date style Date: Tue, 21 Nov 2023 16:32:34 +0100
[Message part 1 (text/plain, inline)]
This post is a followup to bug#66395, where I posted (and subsequently installed) a patch to support the ISO date style in Todo mode item headers. As I pointed out in that bug thread, despite supporting different date styles, Todo mode does not support interactively and automatically changing between different date styles. I have attempted to add this functionality to Todo mode, and the attached patch contains my work-in-progress implementation. I'm still testing these changes, so even if they are deemed acceptable I'm not yet ready to install them, but they raise some questions and how I proceed will be influenced and perhaps decided by the answers to these questions. The rest of this post is directed mainly to the Emacs maintainers and I apologize to them for its length, but unless they want to leave it entirely up to me how to proceed with these changes, I think they need to know in at least some detail what the issues are. As an aid to contextualizing these issues without having to carefully examine the code changes, here is a ChangeLog-style summary of the changes: New Todo mode feature: changing item date format This can be done both interactively by invoking a function to set the new date style, and also automatically when switching from a todo-mode buffer that uses a given date style to another one that uses a different one. To facilitate this functionality the internal todo-mode metadata sexp has been augmented to include the current setting of the date style in addition to the existing list of todo categories and their items counts. When a todo file containing the previous categories-only sexp is visited, the sexp is automatically changed to the new file metadata format. * lisp/calendar/calendar.el (calendar-after-set-date-style-function): New variable. (calendar-set-date-style): Call it when non-nil. * lisp/calendar/todo-mode.el (todo--file-metadata): New variable. (todo--date-pattern-groups): This is now a function instead of a defconst. Use it... (todo--make-date-pattern): ...in this new function. Use it to get the value of... (todo-date-pattern): ...this variable, which is now a defvar instead of a defconst. (todo--set-file-metadata, todo--get-file-metadata) (todo--update-file-metadata): New functions to set, access and update todo file metadata. (todo-show): Use `todo--get-file-metadata' to test for a new todo file. (todo-add-file): Improve doc string. Insert file metadata sexp. (todo-add-category, todo-rename-category): Update file metadata by calling `todo--update-file-metadata'. (todo-move-category): Insert file metadata sexp if moved to file is new. Simplify updating todo categories metadata in moved to file. (todo-edit-item--header): Use function `todo--date-pattern-groups'. Improve a comment. (todo-set-categories, todo-update-categories-sexp) (todo-check-format): Adjust to new file metadata format. (todo--maybe-update-date-style, todo--change-date-header-form): New functions to determine whether to alter the format of todo item date headers, and if so, to automatically do it. The latter function is added as the overriding value of the new variable `calendar-after-set-date-style-function' to enable interactively changing the date header format. (todo-category-completions): Adjust to new file metadata format and remove test for new todo file, since that is now done... (todo-read-category): ...here, using `todo--get-file-metadata'. (todo-key-bindings-t+a+f): Add binding for `calendar-set-date-style' to facilitate interactively changing the date header format. (todo-modes-set-3): Make `todo--file-metadata' buffer-local and evaluate it. Call `todo--maybe-update-date-style' both directly and as a hook on `window-buffer-change-functions', to adjust item date headers if necessary. The former is used when switching to another todo-mode buffer as a result of invoking a relevant todo-mode operation, the latter when switching from outside of todo-mode (e.g., when burying a todo-mode buffer makes another todo-mode buffer current). Here are the main issues these changes raise that I think should be resolved before proceeding: - Although the use and effect of this feature is confined to todo-mode.el, it depends on a making a change to calendar.el, as you can see in the above summary and in the patch. The reason, as I already noted in bug#66395, is that Todo mode date styles are specified by `calendar-date-display-form', which you can change by customizing `calendar-date-style' or by executing `calendar-set-date-style'. To keep this connection I added a hook to calendar.el callable from the latter function, that can be used to trigger the date header changes in Todo mode (but it's not a hook in the usual sense of being intended for user customization; rather, it's intended for use by packages). There is precedence for this in commit a8f4bb8361, where I similarly added a hook to diary-lib.el for the benefit of todo-mode.el. In addition, since the hook has the value nil by default, there is no change to the default behavior of the Emacs Calendar (or Diary). So, is this change to calendar.el acceptable? (In fact, diary-lib.el could probably also use this change in calendar.el to implement date-style switching in the Emacs Diary.) - My implementation of this feature crucially involves a change in the internal form of todo files. Till now the first line of each todo file has contained a sexp listing the categories in the file (this line is hidden in todo-mode), but to allow switching date styles I have augmented this sexp to also contain a specification of the current date style used by the item headers in the file. A number of existing todo-mode functions make use of this sexp, so they have also had to be adapted. All these changes are strictly internal, so the Todo mode UI remains unchanged. The implementation also includes automatically converting existing todo files to the augmented metadata format, so that users of Todo mode can continue using existing todo files, as well as the new functionality, without needing to be aware of the internal changes. But once converted, the todo files will not be compatible with previous versions of todo-mode.el. This would be a problem if someone wanted to use the same todo files on different systems, not all of which have an Emacs where todo-mode.el has these changes. Is this problem considered unacceptable? If so, the easiest solution I can think of is to add a function that would detect if the feature is available and if not, automatically convert todo files with the new metadata format to the previous format, i.e., undoing the automatic change the current implementation makes. I haven't implemented such an undoing function yet, because it wouldn't be needed if the problem is not considered unacceptable. (This kind of issue came up when I announced my rewrite of the original version of todo-mode.el years ago on emacs-devel, and at that time it was decided against keeping the older format available; but that rewrite involved significant incompatible UI changes, whereas the present extension of UI functionality makes only internal incompatible changes.) Another possible solution, if the problem is not acceptable, is to make it possible to install both the new and a previous version of todo-mode.el in parallel. AFAICS this would require making todo-mode.el an ELPA package. This may be a worthwhile alternative regardless of the question of backwards compatibility, since todo-mode.el is by no means an indispensable part of Emacs. (In fact, I thought about this alternative before my rewrite of todo-mode.el was added to Emacs, but for me keeping todo-mode.el in Emacs proper has been the path of least resistance.) - A related issue concerns the scope of the date-style switching operation. As implemented in the attached patch, date headers are changed only in the currently visited todo file, either interactively by invoking a command that changes the date style and then each item date header, or automatically when switching to a todo file whose metadata specifies a different date style from the current one. No changes are made to non-visited todo files. I did it this way because it struck me as wasteful to change files even if the change is unseen (and the date style and item headers could be repeatedly changed in a given file or selection of files, without visiting the majority of todo files), and in my testing, both interactive and automatic switching seemed practically instantaneous, even with my largest todo file with well over a thousand items. On the other hand, in the current implementation the automatic change of the metadata sexp to the new format is also only on demand, i.e., on visiting a todo file. But once that change is made, it is permanent, so that on revisiting the file it then already has the new format. Because of this it might seem more efficient to convert all todo files once and for all. I haven't done that for two reasons: one is that the change it practically imperceptible, so doing it on demand should cause no annoyance. And secondly, if it is decided to enable reverting to the old format for use with older versions of todo-mode.el, it will of course then be necessary to change back to the new format again on revisiting a file whose format has been reverted with the new version of todo-mode.el, so if all files have been changed, that might make a perceptible and possibly annoying difference (though I haven't made any measurements). So is it acceptable to change the metadata format and the item header for each todo file on demand instead of for all todo files at once? - Another thorny issue is how to deal with unit tests. Currently, there are a number of tests for todo-mode.el using ERT. Since they all require a todo file as input data and many make use of the file metadata (currently in the old format), at least these tests will require adjusting. So should there be parallel tests for todo-mode with the new and with the previous metadata format? Or should the older format be declared obsolete? (However, AFAIK there is no formal obsoletion mechanism for data formats as opposed to functions and variables.) I think the decision about this is contingent on the decision about allowing reverting the format for backwards compatibility. - The Tode mode manual will of course need to be updated to document the new feature. Currently, the manual does not document the internal structure of todo files, including the metadata sexp (it is only mentioned in passing in a couple of places). If reverting to the older metadata format is made an option, this would seem to invite giving at least some details of the incompatible metadata formats, but I'm not sure it's necessary. Finally, I want to point out an issue that I think is mainly orthogonal to the new feature and the preceding considerations, but it helps clarify the extent of the feature. Currently, Todo mode supports item date headers with either the American, European or ISO date style, but all items can only be displayed with the same style, and that restriction will persist if the changes discussed here are installed: then you can switch date styles, but doing so changes all item headers in a visited todo file. In other words, there can be no mixing of date styles within a todo file, or when switching between todo files. This restriction is justified by avoiding ambiguity if dates are displayed simultaneously in the European and American styles with numbers for months. Admittedly, `calendar-date-display-form' uses month names by default in these styles, which avoids ambiguity (and there's no ambiguity between the ISO style and the other styles, unless years are displayed with just the two least significant digits, which is common in the American and European styles but AFAIK the ISO standard requires four digits for years). Hence, at least for such display forms, mixing date display styles should be possible (though whether it's desirable is questionable). I haven't given serious thought to how to implement that, nor to how it would interact with the proposed feature of switching date styles without mixing, but it would likely require changes to the implementation in the attached patch. Steve Berman
[Message part 2 (text/x-patch, attachment)]
bug-gnu-emacs <at> gnu.org
:bug#67331
; Package emacs
.
(Thu, 23 Nov 2023 08:08:01 GMT) Full text and rfc822 format available.Message #8 received at 67331 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: Stephen Berman <stephen.berman <at> gmx.net> Cc: 67331 <at> debbugs.gnu.org Subject: Re: bug#67331: 30.0.50; New Todo mode feature: changing item date style Date: Thu, 23 Nov 2023 10:06:46 +0200
> From: Stephen Berman <stephen.berman <at> gmx.net> > Date: Tue, 21 Nov 2023 16:32:34 +0100 > > This post is a followup to bug#66395, where I posted (and subsequently > installed) a patch to support the ISO date style in Todo mode item > headers. As I pointed out in that bug thread, despite supporting > different date styles, Todo mode does not support interactively and > automatically changing between different date styles. I have attempted > to add this functionality to Todo mode, and the attached patch contains > my work-in-progress implementation. I'm still testing these changes, so > even if they are deemed acceptable I'm not yet ready to install them, > but they raise some questions and how I proceed will be influenced and > perhaps decided by the answers to these questions. > > The rest of this post is directed mainly to the Emacs maintainers and I > apologize to them for its length, but unless they want to leave it > entirely up to me how to proceed with these changes, I think they need > to know in at least some detail what the issues are. Let me take a step or two back, and ask whether Todo mode needs to support different styles, let alone interactive changes between styles? IOW, what bad things will happen if Todo mode supported only one fixed style? I'm asking this because it sounds like adding this support raises quite a few issues whose resolution is hairy at best, and in some cases causes real user-visible problems. Refusing to support more than one style will eliminate all these issues, with little or no effort. If you do have to support the various styles, my next question is: why not make the style a display-only feature? That is, keep using a single style in the TODO files, but reformat the date to a user-specified style at display time. This should at least avoid the complications of incompatible files, switching between file formats, whether and how should all the existing files be converted to the selected style, etc. After all, why should users care about the internal format of how time stamps are recorded on a TODO file, if they use Todo mode to display and edit those files? Failing these solutions that avoid the complications, here are my opinions about the issues you bumped into: > - Although the use and effect of this feature is confined to > todo-mode.el, it depends on a making a change to calendar.el, as you > can see in the above summary and in the patch. The reason, as I > already noted in bug#66395, is that Todo mode date styles are > specified by `calendar-date-display-form', which you can change by > customizing `calendar-date-style' or by executing > `calendar-set-date-style'. To keep this connection I added a hook to > calendar.el callable from the latter function, that can be used to > trigger the date header changes in Todo mode (but it's not a hook in > the usual sense of being intended for user customization; rather, it's > intended for use by packages). There is precedence for this in commit > a8f4bb8361, where I similarly added a hook to diary-lib.el for the > benefit of todo-mode.el. In addition, since the hook has the value > nil by default, there is no change to the default behavior of the > Emacs Calendar (or Diary). So, is this change to calendar.el > acceptable? I think this is acceptable, but we need commentary for this hook to explain the subtleties you mention above. > - My implementation of this feature crucially involves a change in the > internal form of todo files. Till now the first line of each todo > file has contained a sexp listing the categories in the file (this > line is hidden in todo-mode), but to allow switching date styles I > have augmented this sexp to also contain a specification of the > current date style used by the item headers in the file. A number of > existing todo-mode functions make use of this sexp, so they have also > had to be adapted. All these changes are strictly internal, so the > Todo mode UI remains unchanged. The implementation also includes > automatically converting existing todo files to the augmented metadata > format, so that users of Todo mode can continue using existing todo > files, as well as the new functionality, without needing to be aware > of the internal changes. But once converted, the todo files will not > be compatible with previous versions of todo-mode.el. This would be a > problem if someone wanted to use the same todo files on different > systems, not all of which have an Emacs where todo-mode.el has these > changes. > > Is this problem considered unacceptable? It is certainly undesirable. But we already did something similar, for example in the format of desktop files. See commits c96983efef and 20defc5538. > So is it acceptable to change the metadata format and the item header > for each todo file on demand instead of for all todo files at once? I think so, yes. But again, such changes are undesirable in the first place. > - Another thorny issue is how to deal with unit tests. Currently, there > are a number of tests for todo-mode.el using ERT. Since they all > require a todo file as input data and many make use of the file > metadata (currently in the old format), at least these tests will > require adjusting. So should there be parallel tests for todo-mode > with the new and with the previous metadata format? Or should the > older format be declared obsolete? (However, AFAIK there is no formal > obsoletion mechanism for data formats as opposed to functions and > variables.) I think the decision about this is contingent on the > decision about allowing reverting the format for backwards > compatibility. If we keep supporting both formats, even if the old one is deprecated, we will need to keep the tests for that old format. Deprecation is not removal, so removing the tests (which already exist) sounds like not the best idea to me. > Finally, I want to point out an issue that I think is mainly orthogonal > to the new feature and the preceding considerations, but it helps > clarify the extent of the feature. Currently, Todo mode supports item > date headers with either the American, European or ISO date style, but > all items can only be displayed with the same style, and that > restriction will persist if the changes discussed here are installed: > then you can switch date styles, but doing so changes all item headers > in a visited todo file. In other words, there can be no mixing of date > styles within a todo file, or when switching between todo files. This > restriction is justified by avoiding ambiguity if dates are displayed > simultaneously in the European and American styles with numbers for > months. Admittedly, `calendar-date-display-form' uses month names by > default in these styles, which avoids ambiguity (and there's no > ambiguity between the ISO style and the other styles, unless years are > displayed with just the two least significant digits, which is common in > the American and European styles but AFAIK the ISO standard requires > four digits for years). Hence, at least for such display forms, mixing > date display styles should be possible (though whether it's desirable is > questionable). I haven't given serious thought to how to implement > that, nor to how it would interact with the proposed feature of > switching date styles without mixing, but it would likely require > changes to the implementation in the attached patch. I think it's entirely reasonable to support only one style for displaying TODO items. I cannot think about a reason for displaying different items using different styles. Thanks.
bug-gnu-emacs <at> gnu.org
:bug#67331
; Package emacs
.
(Thu, 23 Nov 2023 17:41:01 GMT) Full text and rfc822 format available.Message #11 received at 67331 <at> debbugs.gnu.org (full text, mbox):
From: Stephen Berman <stephen.berman <at> gmx.net> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 67331 <at> debbugs.gnu.org Subject: Re: bug#67331: 30.0.50; New Todo mode feature: changing item date style Date: Thu, 23 Nov 2023 18:40:07 +0100
On Thu, 23 Nov 2023 10:06:46 +0200 Eli Zaretskii <eliz <at> gnu.org> wrote: >> From: Stephen Berman <stephen.berman <at> gmx.net> >> Date: Tue, 21 Nov 2023 16:32:34 +0100 >> >> This post is a followup to bug#66395, where I posted (and subsequently >> installed) a patch to support the ISO date style in Todo mode item >> headers. As I pointed out in that bug thread, despite supporting >> different date styles, Todo mode does not support interactively and >> automatically changing between different date styles. I have attempted >> to add this functionality to Todo mode, and the attached patch contains >> my work-in-progress implementation. I'm still testing these changes, so >> even if they are deemed acceptable I'm not yet ready to install them, >> but they raise some questions and how I proceed will be influenced and >> perhaps decided by the answers to these questions. [...] > > Let me take a step or two back, and ask whether Todo mode needs to > support different styles, let alone interactive changes between > styles? IOW, what bad things will happen if Todo mode supported only > one fixed style? I'm asking this because it sounds like adding this > support raises quite a few issues whose resolution is hairy at best, > and in some cases causes real user-visible problems. Refusing to > support more than one style will eliminate all these issues, with > little or no effort. Todo mode dates are essentially the same as Emacs Calendar/Diary dates, which can be input in the three styles American, European and ISO. Due to negligence on my part (insufficient testing, forgetting about bug#55284) only the first two styles worked in Todo mode until the fix I installed for bug#66395. So it's not inputting or displaying different date styles that raises the issues, but switching between these styles. And indeed, this is also a problem for the Diary (see bug#55286). Admittedly, there's no need to switch styles that I'm aware of, but I think it is a nice option to have if the cost isn't prohibitive, so... > If you do have to support the various styles, my next question is: why > not make the style a display-only feature? That is, keep using a > single style in the TODO files, but reformat the date to a > user-specified style at display time. This should at least avoid the > complications of incompatible files, switching between file formats, > whether and how should all the existing files be converted to the > selected style, etc. After all, why should users care about the > internal format of how time stamps are recorded on a TODO file, if > they use Todo mode to display and edit those files? ...yes, a display-only implementation is an alternative worth considering. Indeed, I believe this is what Org mode does, though it doesn't seem to be entirely unproblematic (I don't use Org mode for maintaining Todo lists, but that's what I gather from the Org manual (info "(org) Custom time format")). However, it seems to me less suited to Todo mode for at least two reasons. One is that, unlike Org mode, Todo mode, as just noted above, already supports the input and display of the three Calendar/Diary styles, and this is part of the file content, not a display-time-only attribute. Hence, the display-time feature can't be a mapping from one underlying date style to one of the three support styles but has to recognize any of the three styles and from that produce an appropriate overlay or text property, which seems more complex (though I haven't tried to implement this idea yet, so maybe it isn't so complex). The other problem with display-time date-style switching is the fact that Todo mode allows manually editing the date header string, so if the display is in one style and the actual string in another, then when editing it manually there is a visual disconnect, which seems annoying. But perhaps these issues aren't serious enough to reject the display-time alternative. I'll try to implement it and see how I find using it. > Failing these solutions that avoid the complications, here are my > opinions about the issues you bumped into: > >> - Although the use and effect of this feature is confined to >> todo-mode.el, it depends on a making a change to calendar.el, as you >> can see in the above summary and in the patch. The reason, as I >> already noted in bug#66395, is that Todo mode date styles are >> specified by `calendar-date-display-form', which you can change by >> customizing `calendar-date-style' or by executing >> `calendar-set-date-style'. To keep this connection I added a hook to >> calendar.el callable from the latter function, that can be used to >> trigger the date header changes in Todo mode (but it's not a hook in >> the usual sense of being intended for user customization; rather, it's >> intended for use by packages). There is precedence for this in commit >> a8f4bb8361, where I similarly added a hook to diary-lib.el for the >> benefit of todo-mode.el. In addition, since the hook has the value >> nil by default, there is no change to the default behavior of the >> Emacs Calendar (or Diary). So, is this change to calendar.el >> acceptable? > > I think this is acceptable, but we need commentary for this hook to > explain the subtleties you mention above. I'll add that if the present implementation is retained. >> - My implementation of this feature crucially involves a change in the >> internal form of todo files. Till now the first line of each todo >> file has contained a sexp listing the categories in the file (this >> line is hidden in todo-mode), but to allow switching date styles I >> have augmented this sexp to also contain a specification of the >> current date style used by the item headers in the file. A number of >> existing todo-mode functions make use of this sexp, so they have also >> had to be adapted. All these changes are strictly internal, so the >> Todo mode UI remains unchanged. The implementation also includes >> automatically converting existing todo files to the augmented metadata >> format, so that users of Todo mode can continue using existing todo >> files, as well as the new functionality, without needing to be aware >> of the internal changes. But once converted, the todo files will not >> be compatible with previous versions of todo-mode.el. This would be a >> problem if someone wanted to use the same todo files on different >> systems, not all of which have an Emacs where todo-mode.el has these >> changes. >> >> Is this problem considered unacceptable? > > It is certainly undesirable. But we already did something similar, > for example in the format of desktop files. See commits c96983efef > and 20defc5538. > >> So is it acceptable to change the metadata format and the item header >> for each todo file on demand instead of for all todo files at once? > > I think so, yes. But again, such changes are undesirable in the first > place. I agree, so if the new format is finally accepted, I'll provide an escape hatch to revert to the earlier format. >> - Another thorny issue is how to deal with unit tests. Currently, there >> are a number of tests for todo-mode.el using ERT. Since they all >> require a todo file as input data and many make use of the file >> metadata (currently in the old format), at least these tests will >> require adjusting. So should there be parallel tests for todo-mode >> with the new and with the previous metadata format? Or should the >> older format be declared obsolete? (However, AFAIK there is no formal >> obsoletion mechanism for data formats as opposed to functions and >> variables.) I think the decision about this is contingent on the >> decision about allowing reverting the format for backwards >> compatibility. > > If we keep supporting both formats, even if the old one is deprecated, > we will need to keep the tests for that old format. Deprecation is > not removal, so removing the tests (which already exist) sounds like > not the best idea to me. Yeah, I'll do what I can to accommodate both formats (if the change is made). >> Finally, I want to point out an issue that I think is mainly orthogonal >> to the new feature and the preceding considerations, but it helps >> clarify the extent of the feature. Currently, Todo mode supports item >> date headers with either the American, European or ISO date style, but >> all items can only be displayed with the same style, and that >> restriction will persist if the changes discussed here are installed: >> then you can switch date styles, but doing so changes all item headers >> in a visited todo file. In other words, there can be no mixing of date >> styles within a todo file, or when switching between todo files. This >> restriction is justified by avoiding ambiguity if dates are displayed >> simultaneously in the European and American styles with numbers for >> months. Admittedly, `calendar-date-display-form' uses month names by >> default in these styles, which avoids ambiguity (and there's no >> ambiguity between the ISO style and the other styles, unless years are >> displayed with just the two least significant digits, which is common in >> the American and European styles but AFAIK the ISO standard requires >> four digits for years). Hence, at least for such display forms, mixing >> date display styles should be possible (though whether it's desirable is >> questionable). I haven't given serious thought to how to implement >> that, nor to how it would interact with the proposed feature of >> switching date styles without mixing, but it would likely require >> changes to the implementation in the attached patch. > > I think it's entirely reasonable to support only one style for > displaying TODO items. I cannot think about a reason for displaying > different items using different styles. I can't either, so unless someone comes up with one, I won't pursue it. > Thanks. Thanks for the feedback. I'll follow up again when I succeed in implementating the display-time alternative. Steve Berman
Stefan Kangas <stefankangas <at> gmail.com>
to control <at> debbugs.gnu.org
.
(Wed, 10 Jan 2024 17:46:02 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.