GNU bug report logs - #7649
sparse files and commands

Previous Next

Full log

View this message in rfc822 format

From: Eric Blake <eblake <at> redhat.com>
To: support <at> sigmagames.com, 7649 <at> debbugs.gnu.org
Subject: bug#7649: sparse files and commands
Date: Wed, 15 Dec 2010 15:12:15 -0700

[Message part 1 (text/plain, inline)]

[please keep the list in the loop, and don't top-post on technical lists]

On 12/15/2010 03:20 PM, support <at> sigmagames.com wrote:
> THANKS, I see what you mean.
> Also food for thought, IF OS supports it, skip file blocks that are all null in a sparse file.
> This way less IO and less processing.

That's what I was alluding to - we are working on adding fiemap support
here:

http://git.savannah.gnu.org/cgit/coreutils.git/log/?h=fiemap-copy

as well as a proposal for a hole iterator on OSs that support it (Linux
supports it for ext4 and btrfs via fiemap ioctl; Solaris supports it via
lseek(,SEEK_HOLE)

http://lists.gnu.org/archive/html/bug-coreutils/2010-07/msg00117.html

Note that an explicit block of all 0s cannot be efficiently skipped;
this only works for sparse files (where the block is completely
represented in the metadata of the file, and is implicitly all 0s).
Also, there has been movement to get Linux to add support for punching
holes into an existing file (right now, the only way to make a file more
sparse is to create a new file with the same contents while leaving
holes where the source had all-0 blocks, but that's obviously not as
efficient as modifying a file in-place to add a hole).

Once hole iteration is implemented (and right now, cp is our guinea
pig), then we can port that effort to a number of other programs (tr,
cmp, tar, ...) to make for more efficient I/O on files where we can
behave differently if we know that a block is sparse.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.