GNU bug report logs - #54939
29.0.50; icalendar cannot infer the DTEND from DTSTART + DURATION

Previous Next

To reply to this bug, email your comments to 54939 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Hamelink <me <at> johnhame.link>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; icalendar cannot infer the DTEND from DTSTART + DURATION
Date: Thu, 14 Apr 2022 14:44:16 +0100

[Message part 1 (text/plain, inline)]

Hi there,

As part of my ongoing efforts to move entirely to Emacs for email, I've
discovered an incompatibility with calendar invites sent by Fastmail.

After triggering `mu4e-headers-view-message' from `mu4e-headers-mode', I
receive the following:

   Debugger entered--Lisp error: (wrong-type-argument consp nil)
   gnus-icalendar-event--decode-datefield((VEVENT nil ((UID nil
   "064fa1da-e6f1-4463-9f5d-97217cde2c17") (SEQUENCE nil "1") (DTSTAMP
   nil "20220414T130209Z") (CREATED nil "20220414T130209Z") (DTSTART
   (TZID "Europe/London") "20220415T090000") (DURATION nil "PT1H")
   (PRIORITY nil "0") (SUMMARY nil "Demo Event") (STATUS nil "CONFIRMED")
   (TRANSP nil "OPAQUE") (ORGANIZER (X-JMAP-ID "bWVAam9obmhhbWUubGluaw"
   CN "John Hamelink" EMAIL "me <at> johnhame.link")
   "mailto:me <at> johnhame.link") (ATTENDEE (X-JMAP-ID
   "bWVAam9obmhhbWUubGluaw" CN "John Hamelink" EMAIL "me <at> johnhame.link"
   CUTYPE "INDIVIDUAL" X-JMAP-ROLE "owner" X-JMAP-ROLE "attendee"
   PARTSTAT "ACCEPTED" RSVP "FALSE") "mailto:me <at> johnhame.link") (ATTENDEE
   (X-JMAP-ID "am9obkBtZWRpY2NyZWF0aW9ucy5jb20" CN "John Hamelink" EMAIL
   "john <at> work.com" CUTYPE "INDIVIDUAL" X-JMAP-ROLE "attendee" PARTSTAT
   "NEEDS-ACTION" RSVP "TRUE") "mailto:john <at> work.com")) nil) DTEND
   (("Europe/London"
   . "STD-00:00DST-01:00,M3.5.0/01:00:00,M10.5.0/02:00:00")))

This makes sense, because on inspection of the ics file, we see that
DTEND is not part of the file (I added indentation to clarify scope):

   BEGIN:VCALENDAR
   VERSION:2.0
   PRODID:-//CyrusIMAP.org/Cyrus=20
   3.7.0-alpha0-387-g7ea99c4045-fm-20220413.002-g7ea99c40//EN
   METHOD:REQUEST
   CALSCALE:GREGORIAN
   BEGIN:VTIMEZONE
       TZID:Europe/London
       LAST-MODIFIED:20210816T175139Z
       X-LIC-LOCATION:Europe/London
       TZUNTIL:20220415T090000Z
       BEGIN:DAYLIGHT
           TZNAME:BST
           TZOFFSETFROM:+0000
           TZOFFSETTO:+0100
           DTSTART:19810329T010000
           RRULE:FREQ=3DYEARLY;BYMONTH=3D3;BYDAY=3D-1SU
       END:DAYLIGHT
       BEGIN:STANDARD
           TZNAME:GMT
           TZOFFSETFROM:+0100
           TZOFFSETTO:+0000
           DTSTART:19961027T020000
           RRULE:FREQ=3DYEARLY;BYMONTH=3D10;BYDAY=3D-1SU
       END:STANDARD
   END:VTIMEZONE
   BEGIN:VEVENT
       UID:064fa1da-e6f1-4463-9f5d-97217cde2c17
       SEQUENCE:1
       DTSTAMP:20220414T130209Z
       CREATED:20220414T130209Z
       DTSTART;TZID=3DEurope/London:20220415T090000
       DURATION:PT1H
       PRIORITY:0
       SUMMARY:Demo Event
       STATUS:CONFIRMED
       TRANSP:OPAQUE
       ORGANIZER;X-JMAP-ID=3DbWVAam9obmhhbWUubGluaw;CN=3DJohn Hamelink;
       EMAIL=3Dme <at> johnhame.link:mailto:me <at> johnhame.link
       ATTENDEE;X-JMAP-ID=3DbWVAam9obmhhbWUubGluaw;CN=3DJohn Hamelink;
       EMAIL=3Dme <at> johnhame.link;CUTYPE=3DINDIVIDUAL;X-JMAP-ROLE=3Downer;
       X-JMAP-ROLE=3Dattendee;PARTSTAT=3DACCEPTED;RSVP=3DFALSE:mailto:
       me <at> johnhame.link
       ATTENDEE;X-JMAP-ID=3Dam9obkBtZWRpY2NyZWF0aW9ucy5jb20;CN=3DJohn Hamelink;
       EMAIL=3Djohn <at> work.com;CUTYPE=3DINDIVIDUAL;X-JMAP-ROLE=3Dattendee;
       PARTSTAT=3DNEEDS-ACTION;RSVP=3DTRUE:mailto:john <at> work.com
   END:VEVENT
   END:VCALENDAR

However, it seems that from the ics file we did receive, we should be
able to infer the DTEND value by simply adding DURATION to DTSTART.

I've been able to setup a minimal configuration which replicates the
problem using the icalendar file above (and without involving mu4e):

   (package-install 'gnus)
   (package-install 'org)

   (require 'gnus)
   (require 'gnus-icalendar)

   (setq-default gnus-icalendar-org-capture-file "/tmp/agenda.org"
               gnus-icalendar-org-capture-headline '("Calendar"))

   (gnus-icalendar-org-setup)

   (gnus-icalendar-event-from-ical
   '((VCALENDAR nil
           ((VERSION nil "2.0")
           (PRODID nil "-//CyrusIMAP.org/Cyrus 3.7.0-alpha0-387-g7ea99c4045-fm-20220413.002-g7ea99c40//EN")
           (METHOD nil "REQUEST")
           (CALSCALE nil "GREGORIAN"))
           ((VTIMEZONE nil
                       ((TZID nil "Europe/London")
                       (LAST-MODIFIED nil "20210816T175139Z")
                       (X-LIC-LOCATION nil "Europe/London")
                       (TZUNTIL nil "20220415T090000Z"))
                       ((DAYLIGHT nil (... ... ... ... ...) nil)
                       (STANDARD nil (... ... ... ... ...) nil)))
           (VEVENT nil (
                   (UID nil "064fa1da-e6f1-4463-9f5d-97217cde2c17")
                   (SEQUENCE nil "1")
                   (DTSTAMP nil "20220414T130209Z")
                   (CREATED nil "20220414T130209Z")
                   (DTSTART (TZID "Europe/London") "20220415T090000")
                   (DURATION nil "PT1H")
                   (PRIORITY nil "0")
                   (SUMMARY nil "Demo Event")
                   (STATUS nil "CONFIRMED")
                   (TRANSP nil "OPAQUE")
                   (ORGANIZER
                       (X-JMAP-ID "bWVAam9obmhhbWUubGluaw"
                        CN "John Hamelink"
                        EMAIL "me <at> johnhame.link") "mailto:me <at> johnhame.link")
                   (ATTENDEE
                       (X-JMAP-ID "bWVAam9obmhhbWUubGluaw"
                        CN "John Hamelink"
                        EMAIL "me <at> johnhame.link"
                        CUTYPE "INDIVIDUAL"
                        X-JMAP-ROLE "owner"
                        X-JMAP-ROLE "attendee"
                        PARTSTAT "ACCEPTED"
                        RSVP "FALSE") "mailto:me <at> johnhame.link")
                   (ATTENDEE
                       (X-JMAP-ID "am9obkBtZWRpY2NyZWF0aW9ucy5jb20"
                        CN "John Hamelink"
                        EMAIL "john <at> work.com"
                        CUTYPE "INDIVIDUAL"
                        X-JMAP-ROLE "attendee"
                        PARTSTAT "NEEDS-ACTION"
                        RSVP "TRUE") "mailto:john <at> work.com")) nil)))))


The part of the code which handles the end time is gnus-icalendar.el
line 246, where the code simply assumes that the DTEND property exists:

   :end-time (gnus-icalendar-event--decode-datefield event 'DTEND zone-map)

RFC5545 Section 3.6.1 (the iCalendar spec) says:

   ; Either 'dtend' or 'duration' MAY appear in
   ; a 'eventprop', but 'dtend' and 'duration'
   ; MUST NOT occur in the same 'eventprop'.

And further on:

   For cases where a "VEVENT" calendar component
   specifies a "DTSTART" property with a DATE value type but no
   "DTEND" nor "DURATION" property, the event's duration is taken to
   be one day.  For cases where a "VEVENT" calendar component
   specifies a "DTSTART" property with a DATE-TIME value type but no
   "DTEND" property, the event ends on the same calendar date and
   time of day specified by the "DTSTART" property.

So, I propose we build a function that either returns the end-time using
DTEND as is currently the case, or else to add DTSTART + DURATION and
use that as DTEND if one doesn't exist, or else to use DTSTART + 1 Day
if DTSTART is a date, or else to use DTSTART if DTSTART is a datetime.

How does that sound?

Best,
JH

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: John Hamelink <me <at> johnhame.link>
Cc: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from
 DTSTART + DURATION
Date: Thu, 14 Apr 2022 16:53:09 +0200

John Hamelink <me <at> johnhame.link> writes:

> So, I propose we build a function that either returns the end-time using
> DTEND as is currently the case, or else to add DTSTART + DURATION and
> use that as DTEND if one doesn't exist, or else to use DTSTART + 1 Day
> if DTSTART is a date, or else to use DTSTART if DTSTART is a datetime.
>
> How does that sound?

Makes sense to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #11 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: John Hamelink <me <at> johnhame.link>
To: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from
 DTSTART + DURATION
Date: Thu, 14 Apr 2022 20:09:24 +0100

[Message part 1 (text/plain, inline)]

>> How does that sound?
>
> Makes sense to me.

Great, then I'll have a crack at writing such a patch. I'm new to
Emacs lisp, but this seems to me like a manageable first task!

JH

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: John Hamelink <me <at> johnhame.link>
To: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from
 DTSTART + DURATION
Date: Fri, 15 Apr 2022 10:39:20 +0100

[Message part 1 (text/plain, inline)]

John Hamelink <me <at> johnhame.link> writes:
> Great, then I'll have a crack at writing such a patch. I'm new to
> Emacs lisp, but this seems to me like a manageable first task!

OK, so I've implemented something that's approaching spec - but not
ready for full review yet: particularly because I have a problem I
need some guidance on. I have introduced the following tests:

`gnus-icalendar-dtstart-only-date' (Failing)
- `DTSTART' is `DTSTART;TZID=Europe/Berlin:20200915'
- `DTEND' is undefined
- `DURATION' is undefined
- Fails because the `gnus-icalendar--datetimep' returns a truthy
  value.

`gnus-icalendar-dtstart-only-datetime' (Passing)
- `DTSTART' is `DTSTART;TZID=Europe/Berlin:20200915T140000'
- `DTEND' is undefined
- `DURATION' is undefined

`gnus-icalendar-dtstart-duration' (Passing)
- `DTSTART' is `DTSTART;TZID=Europe/Berlin:20200915T140000'
- `DTEND' is undefined
- `DURATION' is `PT3H'

The reason `gnus-icalendar-dtstart-only-date' currently fails is
because I haven't found a good way to differentiate between a date and
a datetime - as the spec requires.

In an attempt to unblock the rest of the implementation, I used the
following:

(defun gnus-icalendar--datep (date)
"return t if DATE matches a date list."
(and (length= date 9)
    (length=
        (seq-filter 'integerp (seq-take date 6))
        3)))

(defun gnus-icalendar--datetimep (datetime)
"Return t if DATETIME matches a date-time list."
(and (length= datetime 9)
(length=
    (seq-filter 'integerp (seq-take datetime 6))
    6)))

This strategy doesn't work. In `icalendar--decode-isodatetime', hours,
minutes and seconds are set to 0 by default, effectively normalising
date to datetime. I wonder if there's a function already implemented
for this that I don't know about yet?

I've included all my patches so far, but I do plan on refactoring more
and then checking my contribution against the contributing guidelines
before formally submitting a patch for review.

I've also sent an email to assign <at> gnu.org pre-emptively, in case that
is necessary.

Thanks!
JH

[0005-lisp-gnus-gnus-icalendar-Add-RFC5545-DTEND-calculati.patch (text/x-patch, attachment)]

[0004-lisp-gnus-gnus-icalendar-Add-gnus-icalendar-event-en.patch (text/x-patch, attachment)]

[0003-lisp-gnus-gnus-icalendar-Add-gnus-icalendar-event-de.patch (text/x-patch, attachment)]

[0003-lisp-gnus-gnus-icalendar-Add-gnus-icalendar-event-de.patch (text/x-patch, attachment)]

[0002-lisp-gnus-gnus-icalendar-Return-nil-if-datefield-cou.patch (text/x-patch, attachment)]

[0001-lisp-gnus-gnus-icalendar-Added-date-and-datetime-pre.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: John Hamelink <me <at> johnhame.link>
Cc: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from DTSTART
 + DURATION
Date: Sun, 17 Apr 2022 11:00:38 -0700

> In `icalendar--decode-isodatetime', hours,
> minutes and seconds are set to 0 by default, effectively normalising
> date to datetime. I wonder if there's a function already implemented
> for this that I don't know about yet?

I don't know the answer to your question. That being said, the 
"icalendar--" prefix means this is a private function, so you can change 
it to do what you want, so long as its callers in icalendar.el are 
adjusted accordingly.

Message #20 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: John Hamelink <me <at> johnhame.link>
To: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from
 DTSTART + DURATION
Date: Mon, 18 Apr 2022 20:50:39 +0100

[Message part 1 (text/plain, inline)]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

>> In `icalendar--decode-isodatetime', hours,
>> minutes and seconds are set to 0 by default, effectively normalising
>> date to datetime. I wonder if there's a function already implemented
>> for this that I don't know about yet?
>
> I don't know the answer to your question.

No problem :)

> That being said, the "icalendar--" prefix means this is a private
> function, so you can change it to do what you want, so long as its
> callers in icalendar.el are adjusted accordingly.

OK, in that case I'll try to fix the function responsibly, and I'll
try to find all usages of the function and change them accordingly.

I've also sent my side of the copy-assignment paperwork back, and I've
asked my employer to sign their side. I already have a clause in my
contract which says something similar to what is said in the
disclaimer, so things should move quickly on that front.

Best,
JH

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 54939 <at> debbugs.gnu.org (full text, mbox):

From: John Hamelink <me <at> johnhame.link>
To: 54939 <at> debbugs.gnu.org
Subject: Re: bug#54939: 29.0.50; icalendar cannot infer the DTEND from
 DTSTART + DURATION
Date: Mon, 25 Apr 2022 11:50:55 +0100

[Message part 1 (text/plain, inline)]

John Hamelink <me <at> johnhame.link> writes:

> OK, in that case I'll try to fix the function responsibly, and I'll
> try to find all usages of the function and change them accordingly.

Just wanted to send a holding-pattern update: I'm currently very busy
in the run-up to a local election on the 5th May here in Scotland, so
I probably won't be able to come back to this for a few weeks.

> I've also sent my side of the copy-assignment paperwork back, and I've
> asked my employer to sign their side. I already have a clause in my
> contract which says something similar to what is said in the
> disclaimer, so things should move quickly on that front.

I haven't heard back from FSF on this - which is fine since I'm so
busy - but FWIW I will try to be as responsive as I can if any further
requirements are discovered before the copyright assignment process
has completed.

Best
JH

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.