GNU bug report logs - #22768
Crash safety

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#22768: closed (Crash safety)
Date: Tue, 23 Feb 2016 07:29:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 22 Feb 2016 23:28:07 -0800
with message-id <56CC0A07.1080403 <at> cs.ucla.edu>
and subject line Re: bug#22768: Crash safety
has caused the debbugs.gnu.org bug report #22768,
regarding Crash safety
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
22768: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22768
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Yanyan Jiang <jiangyy <at> outlook.com>
To: bug-gzip <at> gnu.org
Subject: Crash safety
Date: Mon, 22 Feb 2016 03:57:47 -0500

[Message part 3 (text/plain, inline)]

Hi gzip developers,

Gzip version: 1.6

I am developing a tool to validate crash safety of application software. I have just found that the file deletion has a potential safety venerability: if only a prefix of I/O operations are flushed to disk, after reboot, the file-system would only contain a 0-byte file (the data is not reached to disk yet).

A paper FYI: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/17780-osdi14-paper-pillai.pdf <http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/17780-osdi14-paper-pillai.pdf> (Table 1 on Page 440). Data append can be (virtually) reordered with any operation at default ext3 and ext4 settings. I recommend to use fsync() to persist the .gz file before deletion.

— strace log —

 36 open("a", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW) = 3
 37 fstat64(3, {st_mode=S_IFREG|0664, st_size=19730, ...}) = 0
 38 rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE TERM XCPU XFSZ], [], 8) = 0
 39 open("a.gz", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 4
 40 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 41 read(3, "10017649652034232324895361757801"..., 65536) = 19730
 42 read(3, "", 45806)                      = 0
 43 write(4, "\37\213\10\10\24\312\312V\0\3a\0-\334m\226\234\274\22\3\340\377Y\r    \30l\354\375o,z\324"..., 9954) = 9954
 44 close(3)                                = 0
 45 utimensat(4, NULL, {{1456130580, 76955623}, {1456130580, 128955620}}, 0) = 0
 46 fchown32(4, 1000, 1000)                 = 0
 47 fchmod(4, 0664)                         = 0
 48 close(4)                                = 0
 49 rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE TERM XCPU XFSZ], [], 8) = 0
 50 unlink("a")                             = 0

Thank you for your attention!

Regards,
Yanyan Jiang 蒋炎岩
Institute of Computer Software,
Dept. of Computer Science, Nanjing University

[Message part 4 (text/html, inline)]

[Message part 5 (message/rfc822, inline)]

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Yanyan Jiang <jiangyy <at> outlook.com>, 22768-done <at> debbugs.gnu.org
Subject: Re: bug#22768: Crash safety
Date: Mon, 22 Feb 2016 23:28:07 -0800

[Message part 6 (text/plain, inline)]

Thanks for reporting the problem. It's annoying that gzip must invoke fsync, as 
that's way overkill compared to the write-ordering that is needed and fsync will 
slow gzip down, but I don't see any safe and reasonably portable alternative so 
I installed the attached patch on Savannah, here:

http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=22aac8f8a616a72dbbe0e4119db8ddda0f076c04

[0001-fsync-output-file-before-closing.txt (text/plain, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.