GNU bug report logs - #6268
Suggestion: truncate should allow -r and -s options together

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 6268 in the body.
You can then email your comments to 6268 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Richard W.M. Jones" <rjones <at> redhat.com>
To: bug-coreutils <at> gnu.org
Subject: Suggestion: truncate should allow -r and -s options together
Date: Tue, 25 May 2010 15:32:05 +0100

The manpage for truncate says:

  Note that the -r and -s options are mutually exclusive.

and indeed you cannot have both:

  $ truncate -r /tmp/ref -s +10M /tmp/new
  truncate: you must specify one of `--size' or `--reference'

We suggest using the 'truncate' command in the 'virt-resize'
documentation[1].  We want users to create a file which is the size of
an existing file + some extra amount.  Naturally this would be:

  truncate -r old-disk.img -s +1G new-disk.img

but this does not work and we have to suggest that users use two
separate 'truncate' commands[2].

Related to this feature request, it would be nice if you could suffix
a size with a % sign to indicate that you want to increase a file by a
certain percentage of the old or reference size, eg:

  truncate -r old-disk.img -s +10% new-disk.img

  truncate -r old-disk.img -s 110% new-disk.img

Neither of these forms work at the moment.

Rich.

[1] http://libguestfs.org/virt-resize.1.html#expanding_a_virtual_machine_disk
[2] http://git.annexia.org/?p=libguestfs.git;a=commitdiff;h=0e28e4104d96bf0bf5b88fb07bb7e5f9f6e6f41f#patch1

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora

Message #8 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: "Richard W.M. Jones" <rjones <at> redhat.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Wed, 26 May 2010 02:01:25 +0100

On 25/05/10 15:32, Richard W.M. Jones wrote:
> The manpage for truncate says:
> 
>   Note that the -r and -s options are mutually exclusive.
> 
> and indeed you cannot have both:
> 
>   $ truncate -r /tmp/ref -s +10M /tmp/new
>   truncate: you must specify one of `--size' or `--reference'
> 
> We suggest using the 'truncate' command in the 'virt-resize'
> documentation[1].  We want users to create a file which is the size of
> an existing file + some extra amount.  Naturally this would be:
> 
>   truncate -r old-disk.img -s +1G new-disk.img
> 
> but this does not work and we have to suggest that users use two
> separate 'truncate' commands[2].

That seems like a worthwhile change.
I.E. allow relative --size with -r.
I'll cook up a patch in the morning.

> Related to this feature request, it would be nice if you could suffix
> a size with a % sign to indicate that you want to increase a file by a
> certain percentage of the old or reference size, eg:
> 
>   truncate -r old-disk.img -s +10% new-disk.img
> 
>   truncate -r old-disk.img -s 110% new-disk.img

I'm a little less enthusiastic about the need for that.
Given how awkward it is to achieve then I might do it
(as a separate patch). Here's how you'd might do it
handling overflow currently:

truncate -s$(echo "$(stat --format %s file) * 1.1" | bc | cut -d. -f1) file

I guess it would be OK to truncate the result
rather than round to nearest?

cheers,
Pádraig.

Message #11 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: "Richard W.M. Jones" <rjones <at> redhat.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Wed, 26 May 2010 09:54:47 +0100

[Message part 1 (text/plain, inline)]

On 26/05/10 02:01, Pádraig Brady wrote:
> On 25/05/10 15:32, Richard W.M. Jones wrote:
>> The manpage for truncate says:
>>
>>   Note that the -r and -s options are mutually exclusive.
>>
>> and indeed you cannot have both:
>>
>>   $ truncate -r /tmp/ref -s +10M /tmp/new
>>   truncate: you must specify one of `--size' or `--reference'
>>
>> We suggest using the 'truncate' command in the 'virt-resize'
>> documentation[1].  We want users to create a file which is the size of
>> an existing file + some extra amount.  Naturally this would be:
>>
>>   truncate -r old-disk.img -s +1G new-disk.img
>>
>> but this does not work and we have to suggest that users use two
>> separate 'truncate' commands[2].
> 
> That seems like a worthwhile change.
> I.E. allow relative --size with -r.
> I'll cook up a patch in the morning

Attached.

>> Related to this feature request, it would be nice if you could suffix
>> a size with a % sign to indicate that you want to increase a file by a
>> certain percentage of the old or reference size, eg:
>>
>>   truncate -r old-disk.img -s +10% new-disk.img
>>
>>   truncate -r old-disk.img -s 110% new-disk.img
> 
> I'm a little less enthusiastic about the need for that.
> Given how awkward it is to achieve then I might do it
> (as a separate patch). Here's how you'd might do it
> handling overflow currently:
> 
> truncate -s$(echo "$(stat --format %s file) * 1.1" | bc | cut -d. -f1) file
> 
> I guess it would be OK to truncate the result
> rather than round to nearest?

With multi-precision expr now in fedora one could simplify to:

truncate -s$(expr $(stat --format %s file) '*' 110 / 100) file

I'm still not convinced it's worth bringing % calculations within truncate.

cheers,
Pádraig.

[truncate-relative-to-file.diff (text/x-patch, attachment)]

Message #14 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: "Richard W.M. Jones" <rjones <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Wed, 26 May 2010 09:48:17 +0100

On Wed, May 26, 2010 at 02:01:25AM +0100, Pádraig Brady wrote:
> On 25/05/10 15:32, Richard W.M. Jones wrote:
> >   truncate -r old-disk.img -s +10% new-disk.img
> > 
> >   truncate -r old-disk.img -s 110% new-disk.img
> 
> I'm a little less enthusiastic about the need for that.
> Given how awkward it is to achieve then I might do it
> (as a separate patch). Here's how you'd might do it
> handling overflow currently:
[...]
> I guess it would be OK to truncate the result
> rather than round to nearest?

We offer a similar feature in virt-resize, and there we just truncate
the result.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Message #17 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: "Richard W.M. Jones" <rjones <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Wed, 26 May 2010 10:08:23 +0100

On Wed, May 26, 2010 at 09:54:47AM +0100, Pádraig Brady wrote:
> Attached.

The patch looks good to me.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://et.redhat.com/~rjones/libguestfs/
See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html

Message #20 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6268 <at> debbugs.gnu.org, "Richard W.M. Jones" <rjones <at> redhat.com>
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Fri, 28 May 2010 09:59:39 +0200

Pádraig Brady wrote:
> Subject: [PATCH] truncate: support sizes relative to an existing file
>
> * doc/coreutils.texi (truncate invocation): Mention that --reference
> bases the --size rather than just setting it.
> * src/truncate.c (usage): Likewise. Also remove the clause
> describing --size and --reference as being mutually exclusive.
> (do_truncate): Add an extra parameter to hold the size
> of a referenced file, and use it if positive.
> (main): Pass the size of a referenced file to do_truncate().
> * tests/misc/truncate-parameters: Adjust for the new combinations.
> * NEWS: Mention the change
> Suggested by Richard W.M. Jones

Nice.  Thanks!

Not new with this change, but what about --ref=non-regular-file ?

Perhaps truncate should refuse to use st_size info from
a non-regular file.

Here's a pathological example:

  $ echo abcdefgh > bar
  $ strace -e ftruncate ./truncate --ref=/dev/tty bar
  ftruncate(3, 0)                         = 0
  $ wc -c bar
  0 bar

It's obvious that such an example is not likely in practice, but since
currently truncate's --ref uses stat, it would follow a symlink, too.

For a directory you'd get unportable and probably surprising results.
Using such a reference size deserves at least a warning.

Message #23 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Fri, 28 May 2010 14:17:58 +0100

> what about --ref=non-regular-file ?
> 
> Perhaps truncate should refuse to use st_size info from
> a non-regular file.
> 
> Here's a pathological example:
> 
>   $ echo abcdefgh > bar
>   $ strace -e ftruncate ./truncate --ref=/dev/tty bar
>   ftruncate(3, 0)                         = 0
>   $ wc -c bar
>   0 bar
> 
> It's obvious that such an example is not likely in practice, but since
> currently truncate's --ref uses stat, it would follow a symlink, too.
> 
> For a directory you'd get unportable and probably surprising results.
> Using such a reference size deserves at least a warning.

Yes that's safer. st_size is only defined for regular files
(or shared mem), so I'll only allow regular files.
I'll push a separate patch soon.

Since we're only referencing the size, there is the argument
that we should be trying harder to get the size of block devices
for example.  There are portability issues with that though,
and doing outside of truncate is easy enough, so I'll leave that for now.

cheers,
Pádraig.

Message #26 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Sat, 29 May 2010 01:03:08 +0100

[Message part 1 (text/plain, inline)]

On 28/05/10 14:17, Pádraig Brady wrote:
>> what about --ref=non-regular-file ?
>>
>> Perhaps truncate should refuse to use st_size info from
>> a non-regular file.
>>
>> Here's a pathological example:
>>
>>   $ echo abcdefgh > bar
>>   $ strace -e ftruncate ./truncate --ref=/dev/tty bar
>>   ftruncate(3, 0)                         = 0
>>   $ wc -c bar
>>   0 bar
>>
>> It's obvious that such an example is not likely in practice, but since
>> currently truncate's --ref uses stat, it would follow a symlink, too.
>>
>> For a directory you'd get unportable and probably surprising results.
>> Using such a reference size deserves at least a warning.
> 
> Yes that's safer. st_size is only defined for regular files
> (or shared mem), so I'll only allow regular files.
> I'll push a separate patch soon.
> 
> Since we're only referencing the size, there is the argument
> that we should be trying harder to get the size of block devices
> for example.  There are portability issues with that though,
> and doing outside of truncate is easy enough, so I'll leave that for now.

I actually removed all ignoring of errors for
non regular files, as well as not allowing
one to reference st_size for non regular files.

I'll push the attached tomorrow sometime.

cheers,
Pádraig.

[truncate-non-files.diff (text/x-patch, attachment)]

Message #29 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Sat, 29 May 2010 10:21:46 +0200

Pádraig Brady wrote:
> Subject: [PATCH] truncate: improve handling of non regular files
>
> Previously we copied `dd` and suppressed error messages
> when truncating neither regular files or shared mem objects.
> This was valid for `dd`, as truncation is ancillary to copying
> it may also do, but for `truncate` we should display all errors.
> Also we used the st_size from non regular files which is undefined,
> so we display an error when the user tries this.
>
> * src/truncate (do_truncate):  Error when referencing the size
> of non regular files or non shared memory objects.  Display all
> errors returned by ftruncate().
> (main): Error when referencing the size of non regular files or
> non shared memory objects.  Don't suppress error messages for
> any file types that can't be opened for writing.
> * tests/misc/truncate-dir-fail: Check that referencing the
> size of a directory is not supported.
> * tests/misc/truncate-fifo: Ensure the test doesn't hang
> by using the `timeout` command.  Don't test the return from
> running ftruncate on the fifo as it's system dependent as
> to whether this fails or not.

Thanks again.
This all looks fine.

> -        }
> -      else if (S_ISREG (sb.st_mode) || S_ISDIR (sb.st_mode)
> -               || S_TYPEISSHM (&sb))
> -        {
> -          error (0, ftruncate_errno,
> -                 _("truncating %s at %" PRIdMAX " bytes"), quote (fname),
> -                 (intmax_t) nsize);
> -          return 1;
> -        }
> -      return 0;
> +      error (0, errno,
> +             _("truncating %s at %" PRIdMAX " bytes"), quote (fname),
> +             (intmax_t) nsize);

It's slightly clearer to say "failed ...":

                _("failed to truncate %s at %" PRIdMAX " bytes"), quote (fname),

Otherwise, e.g., truncating FILE ...: permission denied
might leave a user wondering whether or not the truncation
actually succeeded.  Sure, *we* know, but if they are not
as familiar with how command-line diagnostics work and/or
interpret the ":" as a "." or ";", then it looks like the
truncation succeeded.

...
> +++ b/tests/misc/truncate-fifo
> @@ -1,5 +1,5 @@
>  #!/bin/sh
> -# Make sure truncate works on fifos without hanging or errors
> +# Make sure truncate works on fifos without hanging
>
>  # Copyright (C) 2008-2010 Free Software Foundation, Inc.
>
> @@ -25,7 +25,7 @@ fi
>
>  mkfifo_or_skip_ "fifo"
>
> -
> -truncate -s0 "fifo" || fail=1
> +timeout 10 truncate -s0 "fifo"

Please remove the three sets of unneeded double quotes: s/"fifo"/fifo/.

Message #34 received at 6268-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: 6268-done <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Sat, 29 May 2010 10:44:18 +0100

On 29/05/10 09:21, Jim Meyering wrote:
> Pádraig Brady wrote:
>> Subject: [PATCH] truncate: improve handling of non regular files
>>

> Thanks again.
> This all looks fine.

OK I pushed that with an addition to NEWS.

cheers,
Pádraig.

Message #37 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: "Voelker, Bernhard" <bernhard.voelker <at> siemens-enterprise.com>
To: Pádraig Brady <P <at> draigBrady.com>,
	Jim Meyering <jim <at> meyering.net>
Cc: "6268 <at> debbugs.gnu.org" <6268 <at> debbugs.gnu.org>
Subject: RE: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Mon, 31 May 2010 08:31:49 +0200

Pádraig Brady wrote:
> Jim Meyering wrote:
>> what about --ref=non-regular-file ?

> Yes that's safer. st_size is only defined for regular files
> (or shared mem), so I'll only allow regular files.
> I'll push a separate patch soon.

what about --ref=- ?

one wants to have a file with the size
of a partition plus 5G, e.g.
	truncate --ref=/dev/sda5 -s +5G myfile
This will fail and will be rejected with such a patch.
So the user could use:
	truncate --ref=- -s +5G myfile < /dev/sda5

Just a thought ...

Have a nice day,
Berny

Message #40 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: "Voelker, Bernhard" <bernhard.voelker <at> siemens-enterprise.com>
Cc: 6268 <at> debbugs.gnu.org
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Mon, 31 May 2010 09:50:44 +0100

On 31/05/10 07:31, Voelker, Bernhard wrote:
> Pádraig Brady wrote:
>> Jim Meyering wrote:
>>> what about --ref=non-regular-file ?
> 
>> Yes that's safer. st_size is only defined for regular files
>> (or shared mem), so I'll only allow regular files.
>> I'll push a separate patch soon.
> 
> what about --ref=- ?
> 
> one wants to have a file with the size
> of a partition plus 5G, e.g.
> 	truncate --ref=/dev/sda5 -s +5G myfile
> This will fail and will be rejected with such a patch.
> So the user could use:
> 	truncate --ref=- -s +5G myfile < /dev/sda5
> 
> Just a thought ...

Well we never supported referencing stdin as
we stat() the name. Also one needs to use a separate
operation to get the size of a block device,
so before the patch we just returned 0 which was invalid.

I wrote a comment in the code to say referencing
the size of a block device is probably useful,
but we'll need to worry about portability of that.

cheers,
Pádraig.

Message #43 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: "Richard W.M. Jones" <rjones <at> redhat.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 6268 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Tue, 1 Jun 2010 10:32:40 +0100

On Fri, May 28, 2010 at 09:59:39AM +0200, Jim Meyering wrote:
> Pádraig Brady wrote:
> > Subject: [PATCH] truncate: support sizes relative to an existing file
> >
> > * doc/coreutils.texi (truncate invocation): Mention that --reference
> > bases the --size rather than just setting it.
> > * src/truncate.c (usage): Likewise. Also remove the clause
> > describing --size and --reference as being mutually exclusive.
> > (do_truncate): Add an extra parameter to hold the size
> > of a referenced file, and use it if positive.
> > (main): Pass the size of a referenced file to do_truncate().
> > * tests/misc/truncate-parameters: Adjust for the new combinations.
> > * NEWS: Mention the change
> > Suggested by Richard W.M. Jones
> 
> Nice.  Thanks!
> 
> Not new with this change, but what about --ref=non-regular-file ?
> 
> Perhaps truncate should refuse to use st_size info from
> a non-regular file.
> 
> Here's a pathological example:
> 
>   $ echo abcdefgh > bar
>   $ strace -e ftruncate ./truncate --ref=/dev/tty bar
>   ftruncate(3, 0)                         = 0
>   $ wc -c bar
>   0 bar
> 
> It's obvious that such an example is not likely in practice, but since
> currently truncate's --ref uses stat, it would follow a symlink, too.
> 
> For a directory you'd get unportable and probably surprising results.
> Using such a reference size deserves at least a warning.

And of course the right behaviour for block devices might be to get
their size (ie. blockdev --getsize64 or the equivalent set of ioctls).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

Message #46 received at 6268 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6268 <at> debbugs.gnu.org, "Voelker,
	Bernhard" <bernhard.voelker <at> siemens-enterprise.com>
Subject: Re: bug#6268: Suggestion: truncate should allow -r and -s options
	together
Date: Tue, 01 Jun 2010 09:53:48 -0600

[Message part 1 (text/plain, inline)]

On 05/31/2010 02:50 AM, Pádraig Brady wrote:
>> So the user could use:
>> 	truncate --ref=- -s +5G myfile < /dev/sda5
>>
>> Just a thought ...
> 
> Well we never supported referencing stdin as
> we stat() the name. Also one needs to use a separate
> operation to get the size of a block device,
> so before the patch we just returned 0 which was invalid.
> 
> I wrote a comment in the code to say referencing
> the size of a block device is probably useful,
> but we'll need to worry about portability of that.

The only portable way to get the size of a block device is to open() it,
then do lseek(fd,0,SEEK_END).  Other more efficient methods, like an
ioctl, may exist for some platforms, but then it could become a
maintenance nightmare to figure out where the shortcuts are available.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.