GNU bug report logs - #20523
GNU coreutils 8.4 date: wrong day shift calculation at the spring daylight savings time cutover

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 20523 in the body.
You can then email your comments to 20523 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Markus Baur <mbaur <at> shutterfly.com>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: GNU coreutils 8.4 date: wrong day shift calculation at the spring
 daylight savings time cutover
Date: Thu, 7 May 2015 00:52:12 +0000

[Message part 1 (text/plain, inline)]

On one of my production systems I do daily database dumps between midnight and 1am every day. I noticed on March 9th this year is was dumping the wrong day. Digging further into this I found the shell wrapper script to be at fault and specifically the GNU date program. Here is a simplified version to reproduce the bug:

script:

#!/bin/sh

echo NOW is `date`
echo TODAY is `date +%Y%m%d`
echo YESTERDAY is `date -d 'yesterday' +%Y%m%d`
echo 30 DAYS AGO is `date -d '30 days ago' +%Y%m%d`


output:

root <at> yoyo-01-64-lv$ date 03090059; ./yesterday.sh
Mon Mar  9 00:59:00 PDT 2015
NOW is Mon Mar 9 00:59:00 PDT 2015
TODAY is 20150309
YESTERDAY is 20150307
30 DAYS AGO is 20150206

root <at> yoyo-01-64-lv$ date 03090100; ./yesterday.sh
Mon Mar  9 01:00:00 PDT 2015
NOW is Mon Mar 9 01:00:00 PDT 2015
TODAY is 20150309
YESTERDAY is 20150308
30 DAYS AGO is 20150207

GNU date version info:

root <at> yoyo-01-64-lv$ date --version
date (GNU coreutils) 8.4
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

As you can see, the “yesterday” as well as the “30 days ago” calculation are one day off at 00:59, but correct a minute later.

Cheers,
Markus

[Message part 2 (text/html, inline)]

Message #8 received at 20523 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Markus Baur <mbaur <at> shutterfly.com>, 20523 <at> debbugs.gnu.org
Subject: Re: bug#20523: GNU coreutils 8.4 date: wrong day shift calculation
 at the spring daylight savings time cutover
Date: Thu, 07 May 2015 09:45:00 +0100

forcemerge 18159 20523
stop

previously discussed at: http://bugs.gnu.org/18159

On 07/05/15 01:52, Markus Baur wrote:
> On one of my production systems I do daily database dumps between midnight and 1am every day. I noticed on March 9th this year is was dumping the wrong day. Digging further into this I found the shell wrapper script to be at fault and specifically the GNU date program. Here is a simplified version to reproduce the bug:
> 
> script:
> 
> #!/bin/sh
> 
> echo NOW is `date`
> echo TODAY is `date +%Y%m%d`
> echo YESTERDAY is `date -d 'yesterday' +%Y%m%d`
> echo 30 DAYS AGO is `date -d '30 days ago' +%Y%m%d`
> 
> 
> output:
> 
> root <at> yoyo-01-64-lv$ date 03090059; ./yesterday.sh 
> Mon Mar  9 00:59:00 PDT 2015
> NOW is Mon Mar 9 00:59:00 PDT 2015
> TODAY is 20150309
> YESTERDAY is 20150307
> 30 DAYS AGO is 20150206
> 
> root <at> yoyo-01-64-lv$ date 03090100; ./yesterday.sh 
> Mon Mar  9 01:00:00 PDT 2015
> NOW is Mon Mar 9 01:00:00 PDT 2015
> TODAY is 20150309
> YESTERDAY is 20150308
> 30 DAYS AGO is 20150207
> 
> GNU date version info:
> 
> root <at> yoyo-01-64-lv$ date --version
> date (GNU coreutils) 8.4
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> As you can see, the “yesterday” as well as the “30 days ago” calculation are one day off at 00:59, but correct a minute later.
> 
> Cheers,
> Markus
> 

Message #13 received at 20523 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Markus Baur <mbaur <at> shutterfly.com>
Cc: 20523 <at> debbugs.gnu.org
Subject: Re: bug#20523: GNU coreutils 8.4 date: wrong day shift calculation
 at the spring daylight savings time cutover
Date: Thu, 7 May 2015 15:00:47 -0600

Markus Baur wrote:
> On one of my production systems I do daily database dumps between
> midnight and 1am every day. I noticed on March 9th this year is was
> dumping the wrong day. Digging further into this I found the shell
> wrapper script to be at fault and specifically the GNU date
> program. Here is a simplified version to reproduce the bug:

Thank you for the report.  However this appears to be a usage issue.

> echo YESTERDAY is `date -d 'yesterday' +%Y%m%d`
> echo 30 DAYS AGO is `date -d '30 days ago' +%Y%m%d`

Both of those are problematic when used near Daylight Saving Time
changes.

  $ zdump -v US/Pacific |grep 2015
  US/Pacific  Sun Mar  8 09:59:59 2015 UT = Sun Mar  8 01:59:59 2015 PST isdst=0 gmtoff=-28800
  US/Pacific  Sun Mar  8 10:00:00 2015 UT = Sun Mar  8 03:00:00 2015 PDT isdst=1 gmtoff=-25200
  US/Pacific  Sun Nov  1 08:59:59 2015 UT = Sun Nov  1 01:59:59 2015 PDT isdst=1 gmtoff=-25200
  US/Pacific  Sun Nov  1 09:00:00 2015 UT = Sun Nov  1 01:00:00 2015 PST isdst=0 gmtoff=-28800

As you can see March 9th is right on top of the DST change.  Instead
use one of these.

Use UTC (with the UTC offset):

  echo YESTERDAY is `date -u -d 'yesterday' +%Y%m%d`
  echo 30 DAYS AGO is `date -u -d '30 days ago' +%Y%m%d`

Or use 12:00 noon localtime:

  echo YESTERDAY is `date -d 'yesterday 12:00' +%Y%m%d`
  echo 30 DAYS AGO is `date -d '12:00 30 days ago' +%Y%m%d`

> root <at> yoyo-01-64-lv$ date 03090059; ./yesterday.sh
> Mon Mar  9 00:59:00 PDT 2015
> NOW is Mon Mar 9 00:59:00 PDT 2015
> TODAY is 20150309
> YESTERDAY is 20150307
> 30 DAYS AGO is 20150206
> 
> root <at> yoyo-01-64-lv$ date 03090100; ./yesterday.sh
> Mon Mar  9 01:00:00 PDT 2015
> NOW is Mon Mar 9 01:00:00 PDT 2015
> TODAY is 20150309
> YESTERDAY is 20150308
> 30 DAYS AGO is 20150207

It is not necessary to set the system time.  Simply provide a full
time reference to date and then operate relative to it.

  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700' +%Y-%m-%d
  2015-03-09

  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700 yesterday' +%Y-%m-%d
  2015-03-07

  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700' '+%Y-%m-%d %T %z'
  2015-03-09 00:59:00 -0700

  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700 yesterday' '+%Y-%m-%d %T %z'
  2015-03-07 23:59:00 -0800

> As you can see, the "yesterday" as well as the "30 days ago"
> calculation are one day off at 00:59, but correct a minute later.

Actually not.  If you examine the times you will find that because of
DST the time springs forward in the Spring and falls back in the Fall.
In the Spring when time springs forward the hour is missing and the
time gap of one day ago (yesterday) lands on the day before.

The FAQ documents the issue in detail.

  http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#The-date-command-is-not-working-right_002e

In summary the usage is to either use UTC which avoids DST changes or
to specify a time that is away from DST changes such as 12:00 noon.
Use one or the other as described above and this problem is avoided.

Bob

Message #16 received at 20523 <at> debbugs.gnu.org (full text, mbox):

From: Markus Baur <mbaur <at> shutterfly.com>
To: Bob Proulx <bob <at> proulx.com>
Cc: "20523 <at> debbugs.gnu.org" <20523 <at> debbugs.gnu.org>
Subject: Re: bug#20523: GNU coreutils 8.4 date: wrong day shift calculation
 at the spring daylight savings time cutover
Date: Fri, 8 May 2015 01:04:08 +0000

Thanks Bob! 

Using 12:00 noon localtime is my workaround now since I discovered this problem / bug. It is also in line with what Pádraig suggested as a fix: Pick the middle of the unit before applying adjustments. It is also exactly what I’m doing inside my database dump perl script to avoid any daylight savings time hickups. It just never occured to me that “date” could be wrong and I thought running the cron job before the DST adjustment between 2 and 3 am should be safe. Alas this is not the case.

- Markus

> On May 7, 2015, at 14:00, Bob Proulx <bob <at> proulx.com> wrote:
> 
> Markus Baur wrote:
>> On one of my production systems I do daily database dumps between
>> midnight and 1am every day. I noticed on March 9th this year is was
>> dumping the wrong day. Digging further into this I found the shell
>> wrapper script to be at fault and specifically the GNU date
>> program. Here is a simplified version to reproduce the bug:
> 
> Thank you for the report.  However this appears to be a usage issue.
> 
>> echo YESTERDAY is `date -d 'yesterday' +%Y%m%d`
>> echo 30 DAYS AGO is `date -d '30 days ago' +%Y%m%d`
> 
> Both of those are problematic when used near Daylight Saving Time
> changes.
> 
>  $ zdump -v US/Pacific |grep 2015
>  US/Pacific  Sun Mar  8 09:59:59 2015 UT = Sun Mar  8 01:59:59 2015 PST isdst=0 gmtoff=-28800
>  US/Pacific  Sun Mar  8 10:00:00 2015 UT = Sun Mar  8 03:00:00 2015 PDT isdst=1 gmtoff=-25200
>  US/Pacific  Sun Nov  1 08:59:59 2015 UT = Sun Nov  1 01:59:59 2015 PDT isdst=1 gmtoff=-25200
>  US/Pacific  Sun Nov  1 09:00:00 2015 UT = Sun Nov  1 01:00:00 2015 PST isdst=0 gmtoff=-28800
> 
> As you can see March 9th is right on top of the DST change.  Instead
> use one of these.
> 
> Use UTC (with the UTC offset):
> 
>  echo YESTERDAY is `date -u -d 'yesterday' +%Y%m%d`
>  echo 30 DAYS AGO is `date -u -d '30 days ago' +%Y%m%d`
> 
> Or use 12:00 noon localtime:
> 
>  echo YESTERDAY is `date -d 'yesterday 12:00' +%Y%m%d`
>  echo 30 DAYS AGO is `date -d '12:00 30 days ago' +%Y%m%d`
> 
>> root <at> yoyo-01-64-lv$ date 03090059; ./yesterday.sh
>> Mon Mar  9 00:59:00 PDT 2015
>> NOW is Mon Mar 9 00:59:00 PDT 2015
>> TODAY is 20150309
>> YESTERDAY is 20150307
>> 30 DAYS AGO is 20150206
>> 
>> root <at> yoyo-01-64-lv$ date 03090100; ./yesterday.sh
>> Mon Mar  9 01:00:00 PDT 2015
>> NOW is Mon Mar 9 01:00:00 PDT 2015
>> TODAY is 20150309
>> YESTERDAY is 20150308
>> 30 DAYS AGO is 20150207
> 
> It is not necessary to set the system time.  Simply provide a full
> time reference to date and then operate relative to it.
> 
>  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700' +%Y-%m-%d
>  2015-03-09
> 
>  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700 yesterday' +%Y-%m-%d
>  2015-03-07
> 
>  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700' '+%Y-%m-%d %T %z'
>  2015-03-09 00:59:00 -0700
> 
>  $ TZ=US/Pacific date -d '2015-03-09 00:59:00 -0700 yesterday' '+%Y-%m-%d %T %z'
>  2015-03-07 23:59:00 -0800
> 
>> As you can see, the "yesterday" as well as the "30 days ago"
>> calculation are one day off at 00:59, but correct a minute later.
> 
> Actually not.  If you examine the times you will find that because of
> DST the time springs forward in the Spring and falls back in the Fall.
> In the Spring when time springs forward the hour is missing and the
> time gap of one day ago (yesterday) lands on the day before.
> 
> The FAQ documents the issue in detail.
> 
>  http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#The-date-command-is-not-working-right_002e
> 
> In summary the usage is to either use UTC which avoids DST changes or
> to specify a time that is away from DST changes such as 12:00 noon.
> Use one or the other as described above and this problem is avoided.
> 
> Bob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.