GNU bug report logs - #11125
date command calculations are not consistent

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11125 in the body.
You can then email your comments to 11125 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
To: bug-coreutils <at> gnu.org
Subject: date command calculations are not consistent
Date: Thu, 29 Mar 2012 10:43:39 -0700 (PDT)

[Message part 1 (text/plain, inline)]

Discovered this when I script I have cronned to run at 12:01AM gave some unexpected results.

After much picking at it I finally figured out that the date command itself was the source of my problem.  It isn't handling date calculations that span the daylight savings time change consistently..

[schaefer <at> fedora14 ~]$ date --version
date (GNU coreutils) 8.5
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.
[schaefer <at> fedora14 ~]$ date -d "last Thursday"
Thu Mar 22 00:00:00 CDT 2012
[schaefer <at> fedora14 ~]$ date -d "Thu Mar 22 00:00:00 CDT 2012"
Thu Mar 22 00:00:00 CDT 2012
[schaefer <at> fedora14 ~]$ date -d "last Thursday - 21 days"
Thu Mar  1 00:00:00 CST 2012
[schaefer <at> fedora14 ~]$ date -d "Thu Mar 22 00:00:00 CDT 2012 - 21 days"
Wed Feb 29 23:00:00 CST 2012

In my real world case I was running a script at 12:01AM on Sundays that did this..

RSD="$(date +%F -d "today - 35  days")" # Report Start Date

RSD was getting set to a Saturday date but ordinarily, when a DST change hasn't occurred in the past 35 days, it would be set to a Sunday date.  I expect and want the Sunday date regardless; although I realize an argument could be made for the correctness of the Saturday date too.  However the date command is going to make a calculation like that it should be consistent with its own self which if you look closely at my examples above regarding last Thursday you will see it is not consistent with itself.

Thank you,
Tom Schaefer

[Message part 2 (text/html, inline)]

Message #8 received at 11125 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
Cc: 11125 <at> debbugs.gnu.org
Subject: Re: bug#11125: date command calculations are not consistent
Date: Thu, 29 Mar 2012 12:01:47 -0600

[Message part 1 (text/plain, inline)]

forcemerge 11098 11125
thanks

On 03/29/2012 11:43 AM, Thomas R. Schaefer wrote:
> Discovered this when I script I have cronned to run at 12:01AM gave some unexpected results.
> 
> After much picking at it I finally figured out that the date command itself was the source of my problem.  It isn't handling date calculations that span the daylight savings time change consistently

Thanks for the report.  Join the club.  This has been reported twice
this week already, and the consensus is that date is behaving as
documented (relative date operations add or subtract in multiples of 24
hours, without regards to daylight savings boundaries), but that the
documented behavior could be improved if someone were to submit a patch.

https://lists.gnu.org/archive/html/bug-coreutils/2012-03/msg00102.html

As recommended in our FAQ,

https://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e

it's almost always better to base relative time computations off of noon
rather than midnight (as both 11 am and 1 pm fall in the same day, even
when your multiple-of-24-hours crosses a 23-hour or 25-hour day).

In your example, change:

RSD="$(date +%F -d "today - 35  days")" # Report Start Date

to:

RSD="$(date +%F -d "noon today - 35  days")" # Report Start Date

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #13 received at 11125 <at> debbugs.gnu.org (full text, mbox):

From: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 11125 <at> debbugs.gnu.org
Subject: Re: bug#11125: date command calculations are not consistent
Date: Thu, 29 Mar 2012 11:34:19 -0700 (PDT)

[Message part 1 (text/plain, inline)]

Thank you, I really appreciate your suggestion and will use it in my script.  That will solve my problem.

I can appreciate the consensus that "relative date operations add or subtract in multiples of 24 hours, without regards to daylight savings boundaries".  In fact I would be inclined to agree with it.

But if that is the consensus then I believe the date command should adhere to that consensus consistently which it does not.

In this case date is calculating with 24 hours days regardless of crossing a DST boundary..

[root <at> schaefer-test ~]# date -d "Thu Mar 22 00:00:00 CDT 2012 - 21 days"
Wed Feb 29 23:00:00 CST 2012

In this case date does take DST into account in a relative date operation..

[root <at> schaefer-test ~]# date -d "last Thursday - 21 days"
Thu Mar  1 00:00:00 CST 2012

If the date command where being consistent in following the consensus that "relative date operations add or subtract in multiples of 24 hours, without regards to daylight savings boundaries" then both of the above date commands would return Wed Feb 29 23:00:00 CST 2012.

Again thank you for your help,

Tom Schaefer










--- On Thu, 3/29/12, Eric Blake <eblake <at> redhat.com> wrote:

From: Eric Blake <eblake <at> redhat.com>
Subject: Re: bug#11125: date command calculations are not consistent
To: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
Cc: 11125 <at> debbugs.gnu.org
Date: Thursday, March 29, 2012, 1:01 PM

forcemerge 11098 11125
thanks

On 03/29/2012 11:43 AM, Thomas R. Schaefer wrote:
> Discovered this when I script I have cronned to run at 12:01AM gave some unexpected results.
> 
> After much picking at it I finally figured out that the date command itself was the source of my problem.  It isn't handling date calculations that span the daylight savings time change consistently

Thanks for the report.  Join the club.  This has been reported twice
this week already, and the consensus is that date is behaving as
documented (relative date operations add or subtract in multiples of 24
hours, without regards to daylight savings boundaries), but that the
documented behavior could be improved if someone were to submit a patch.

https://lists.gnu.org/archive/html/bug-coreutils/2012-03/msg00102.html

As recommended in our FAQ,

https://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e

it's almost always better to base relative time computations off of noon
rather than midnight (as both 11 am and 1 pm fall in the same day, even
when your multiple-of-24-hours crosses a 23-hour or 25-hour day).

In your example, change:

RSD="$(date +%F -d "today - 35  days")" # Report Start Date

to:

RSD="$(date +%F -d "noon today - 35  days")" # Report Start Date

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[Message part 2 (text/html, inline)]

Message #16 received at 11125 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
Cc: 11125 <at> debbugs.gnu.org
Subject: Re: bug#11125: date command calculations are not consistent
Date: Thu, 29 Mar 2012 13:06:43 -0600

[Message part 1 (text/plain, inline)]

On 03/29/2012 12:34 PM, Thomas R. Schaefer wrote:

> In this case date is calculating with 24 hours days regardless of crossing a DST boundary..
> 
> [root <at> schaefer-test ~]# date -d "Thu Mar 22 00:00:00 CDT 2012 - 21 days"
> Wed Feb 29 23:00:00 CST 2012

This started from one fixed point in time, relative to the 'CDT' time
zone, and subtracted 21 * 24 hours.

> 
> In this case date does take DST into account in a relative date operation..
> 
> [root <at> schaefer-test ~]# date -d "last Thursday - 21 days"
> Thu Mar  1 00:00:00 CST 2012

But notice what date -d "last Thursday" is:

$ TZ=CST date -d 'last Thursday CST'
Thu Mar 22 00:00:00 CST 2012

It's relative to the 'CST' timezone, which is an hour different from the
CDT timezone.

> 
> If the date command where being consistent in following the consensus that "relative date operations add or subtract in multiples of 24 hours, without regards to daylight savings boundaries" then both of the above date commands would return Wed Feb 29 23:00:00 CST 2012.

Only if you start from the same point in time in both commands, which
you didn't.

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #19 received at 11125 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: "Thomas R. Schaefer" <schaefert <at> yahoo.com>, 11125 <at> debbugs.gnu.org
Subject: Re: bug#11125: date command calculations are not consistent
Date: Thu, 29 Mar 2012 21:35:55 -0600

Eric Blake wrote:
> Thomas R. Schaefer wrote:
> > In this case date does take DST into account in a relative date operation..
> > 
> > [root <at> schaefer-test ~]# date -d "last Thursday - 21 days"
> > Thu Mar  1 00:00:00 CST 2012
> 
> But notice what date -d "last Thursday" is:
> 
> $ TZ=CST date -d 'last Thursday CST'
> Thu Mar 22 00:00:00 CST 2012
> 
> It's relative to the 'CST' timezone, which is an hour different from the
> CDT timezone.

Additionally it is working near the problematic midnight time when
crosing DST boundaries may cause problems.

> > If the date command where being consistent in following the
> > consensus that "relative date operations add or subtract in
> > multiples of 24 hours, without regards to daylight savings
> > boundaries" then both of the above date commands would return Wed
> > Feb 29 23:00:00 CST 2012.
> 
> Only if you start from the same point in time in both commands, which
> you didn't.

For an example always use a time in addition to a day.  Never default
to midnight.  Instead use 12:00 noon to avoid the DST problems.

  $ TZ=US/Mountain date -R -d 'last Thursday'
  Thu, 22 Mar 2012 00:00:00 -0600

That is problematic usage.  But specifying the time as 12:00 noon
solves the problem.

  $ TZ=US/Mountain date -R -d '12:00 last Thursday'
  Thu, 22 Mar 2012 12:00:00 -0600

Then doing calculations will be more reliable.

  $ TZ=US/Mountain date -R -d '12:00 last Thursday - 21 days'
  Thu, 01 Mar 2012 12:00:00 -0700

Note that I always use -R to obtain an unambiguous format.  Using the
default format isn't unambiguous as there are multiple timezones with
the same names.

Doing calculations on weeks and months have similar issues.  When
doing calculations on months I always work with the middle of the
month to avoid the difference in lenghts of months.

Generate a reference date in the middle of the month.

  $ date +%Y-%m-15
  2012-03-15

Use that date for month calculations.

  $ date --date="$(date +%Y-%m-15) -1 month" +'Last month was %B.'
  Last month was February.

Please see the FAQ for more examples:

  http://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e

Unfortunately the human date parsing that was added to date is a
rather simplistic implementation.

Bob

Message #22 received at 11125 <at> debbugs.gnu.org (full text, mbox):

From: "Thomas R. Schaefer" <schaefert <at> yahoo.com>
To: 11125 <at> debbugs.gnu.org, Bob Proulx <bob <at> proulx.com>
Subject: Re: bug#11125: date command calculations are not consistent
Date: Fri, 30 Mar 2012 08:24:48 -0700 (PDT)

[Message part 1 (text/plain, inline)]

Thanks Eric & Bob.  Good stuff.  I'm still slightly foggy on why I get different answers subtracting 35 days from seemingly the same point in time..

[tschaefer <at> schaefer-test ~]$ date  -d 'last Thursday'
Thu Mar 29 00:00:00 CDT 2012
[tschaefer <at> schaefer-test ~]$ date  -d 'Thu Mar 29 00:00:00 CDT 2012 - 35 days'
Wed Feb 22 23:00:00 CST 2012
[tschaefer <at> schaefer-test ~]$ date -d 'last Thursday - 35 days'
Thu Feb 23 00:00:00 CST 2012

But, I'm probably just to dense to get it.

Thanks again for the help.  The suggestion of using "noon today" when subtracting days from a date is an excellent one.

Tom Schaefer

--- On Thu, 3/29/12, Bob Proulx <bob <at> proulx.com> wrote:

From: Bob Proulx <bob <at> proulx.com>
Subject: Re: bug#11125: date command calculations are not consistent
To: "Thomas R. Schaefer" <schaefert <at> yahoo.com>, 11125 <at> debbugs.gnu.org
Date: Thursday, March 29, 2012, 10:35 PM

Eric Blake wrote:
> Thomas R. Schaefer wrote:
> > In this case date does take DST into account in a relative date operation..
> > 
> > [root <at> schaefer-test ~]# date -d "last Thursday - 21 days"
> > Thu Mar  1 00:00:00 CST 2012
> 
> But notice what date -d "last Thursday" is:
> 
> $ TZ=CST date -d 'last Thursday CST'
> Thu Mar 22 00:00:00 CST 2012
> 
> It's relative to the 'CST' timezone, which is an hour different from the
> CDT timezone.

Additionally it is working near the problematic midnight time when
crosing DST boundaries may cause problems.

> > If the date command where being consistent in following the
> > consensus that "relative date operations add or subtract in
> > multiples of 24 hours, without regards to daylight savings
> > boundaries" then both of the above date commands would return Wed
> > Feb 29 23:00:00 CST 2012.
> 
> Only if you start from the same point in time in both commands, which
> you didn't.

For an example always use a time in addition to a day.  Never default
to midnight.  Instead use 12:00 noon to avoid the DST problems.

  $ TZ=US/Mountain date -R -d 'last Thursday'
  Thu, 22 Mar 2012 00:00:00 -0600

That is problematic usage.  But specifying the time as 12:00 noon
solves the problem.

  $ TZ=US/Mountain date -R -d '12:00 last Thursday'
  Thu, 22 Mar 2012 12:00:00 -0600

Then doing calculations will be more reliable.

  $ TZ=US/Mountain date -R -d '12:00 last Thursday - 21 days'
  Thu, 01 Mar 2012 12:00:00 -0700

Note that I always use -R to obtain an unambiguous format.  Using the
default format isn't unambiguous as there are multiple timezones with
the same names.

Doing calculations on weeks and months have similar issues.  When
doing calculations on months I always work with the middle of the
month to avoid the difference in lenghts of months.

Generate a reference date in the middle of the month.

  $ date +%Y-%m-15
  2012-03-15

Use that date for month calculations.

  $ date --date="$(date +%Y-%m-15) -1 month" +'Last month was %B.'
  Last month was February.

Please see the FAQ for more examples:

  http://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e

Unfortunately the human date parsing that was added to date is a
rather simplistic implementation.

Bob

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.