GNU bug report logs - #50940
how df utility displays sizes - GB vs GiB

Previous Next

To reply to this bug, email your comments to 50940 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Danie de Jager <danie.dejager <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 15:28:10 +0200

[Message part 1 (text/plain, inline)]

Hi,

The output from df -h and df -H is always G or M. Depending on who sends me
usage stats I have to ask how the command was run to make sure I calculate
usage correctly. Systems like Amazon EC2 use the explicit GiB suffix.
Making it easier to know what sizes you are looking at.

Can a future release of df not be improved to print out the GB or GiB?

Regards,
Danie

[Message part 2 (text/html, inline)]

Message #8 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 17:18:04 +0100

On 01/10/2021 14:28, Danie de Jager wrote:
> Hi,
> 
> The output from df -h and df -H is always G or M. Depending on who sends me
> usage stats I have to ask how the command was run to make sure I calculate
> usage correctly. Systems like Amazon EC2 use the explicit GiB suffix.
> Making it easier to know what sizes you are looking at.
> 
> Can a future release of df not be improved to print out the GB or GiB?

So you're suggesting that `df -h` changes to using IEC suffixes like Gi, Ti etc.
and -H would use the current suffix format to indicate SI units.
That would be the right thing to do if we were adding these options now,
but at this stage I'm not sure it's worth it, because you wouldn't
be sure which version of df you were getting the output from.

For completeness one can get unambiguous output using numfmt like:

  $ df -B1 | numfmt --field - --invalid=ignore --to=iec-i
  Filesystem        1B-blocks        Used  Available Use% Mounted on
  devtmpfs              3.9Gi           0      3.9Gi   0% /dev

cheers,
Pádraig

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Glenn Golden <gdg <at> zplane.com>
To: Danie de Jager <danie.dejager <at> gmail.com>
Cc: bug-coreutils <at> gnu.org, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 10:28:04 -0600

Danie de Jager <danie.dejager <at> gmail.com> [2021-10-01 15:28:10 +0200]:
> 
> The output from df -h and df -H is always G or M. Depending on who sends me
> usage stats I have to ask how the command was run to make sure I calculate
> usage correctly. Systems like Amazon EC2 use the explicit GiB suffix.
> Making it easier to know what sizes you are looking at.
> 
> Can a future release of df not be improved to print out the GB or GiB?
> 

A patchset submitted last year

    https://lists.gnu.org/archive/html/coreutils/2020-09/msg00001.html

would partially address this (for df, du, and ls) by consistently enforcing
the semantics given in Section 2.3 of coreutils.info (8.32): If that patch
were adopted, units suffixed with "B" (e.g. kB, MB, GB, etc.) would always
imply base-2 units, and B-less suffixes (e.g. k, M, G) would always imply
base-10 units, with no exceptions. ("iB" suffixes would not be used.)

However, the overall issue is more complicated than this, because those
semantics in Section 2.3 are directly contradicted by statements appearing
elsewhere in 8.32 coreutils.info that invert the 2.3 semantics. 

See the above posting (and follow-ups in that thread) for all the gory
details and historical background.

NOTE: I do not know whether the program behavior and documentation described
in the above post is still extant in coreutils release 9.

Glenn Golden

Message #17 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Danie de Jager <danie.dejager <at> gmail.com>
Cc: 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 09:33:13 -0700

On 10/1/21 6:28 AM, Danie de Jager wrote:
> Systems like Amazon EC2 use the explicit GiB suffix.

Are you saying that Amazon EC2's 'df' utility behaves differently from 
coreutils' 'df'? If so, could you send us documentation or source code 
indicating exactly what Amazon EC2's 'df' does?

As Pádraig wrote, there are compatibility arguments for 'df' to not 
change its behavior here.

As Glenn wrote, there was a longstanding issue with coreutils 
documentation disagreeing with behavior. In 
<https://lists.gnu.org/archive/html/coreutils/2020-09/msg00002.html> 
Kamil Dudka suggested that the documentation be fixed; I don't know 
whether that happened.

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Danie de Jager <danie.dejager <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 22:30:19 +0200

[Message part 1 (text/plain, inline)]

Thank you for sharing. After reading I agree that changing
existing features could break processes for users.

It would be easy to make mistakes when in a hurry and looking at the
following output:
$ df -h | grep /$
/dev/nvme0n1p1   12G  8.5G  3.6G  71% /

$ df -H | grep /$
/dev/nvme0n1p1   13G  9.1G  3.8G  71% /

I'm happy with the calculations done but as stated in the path set
mentioned I would've preferred if the output was:
$ df -h | grep /$
/dev/nvme0n1p1   12GiB  8.5GiB  3.6GiB   71% /

$ df -H | grep /$
/dev/nvme0n1p1   13GB  9.1GB  3.8GB  71% /

I propose that the options -h and -H not be changed, for compatibility, but
that it becomes possible to make a distinction between the two formats
outputs.

Can we use the same options, but to trigger the longer annotation, we
double the characters used to -hh and -HH?

therefore
$ df -hh | grep /$
/dev/nvme0n1p1   12GiB  8.5GiB  3.6GiB   71% /
$ df -HH | grep /$
/dev/nvme0n1p1   13GB  9.1GB  3.8GB  71% /

or for ls

ls -lhh *.crt
-rw-r--r--. 1 danie.dejager danie.dejager 2.6KB Sep 29 13:07 1.crt

Regards,
Danie

On Fri, 1 Oct 2021 at 18:28, Glenn Golden <gdg <at> zplane.com> wrote:

>
> A patchset submitted last year
>
>     https://lists.gnu.org/archive/html/coreutils/2020-09/msg00001.html
>
> would partially address this (for df, du, and ls) by consistently enforcing
> the semantics given in Section 2.3 of coreutils.info (8.32): If that patch
> were adopted, units suffixed with "B" (e.g. kB, MB, GB, etc.) would always
> imply base-2 units, and B-less suffixes (e.g. k, M, G) would always imply
> base-10 units, with no exceptions. ("iB" suffixes would not be used.)
>
> However, the overall issue is more complicated than this, because those
> semantics in Section 2.3 are directly contradicted by statements appearing
> elsewhere in 8.32 coreutils.info that invert the 2.3 semantics.
>
> See the above posting (and follow-ups in that thread) for all the gory
> details and historical background.
>
> NOTE: I do not know whether the program behavior and documentation
> described
> in the above post is still extant in coreutils release 9.
>
> Glenn Golden
>

[Message part 2 (text/html, inline)]

Message #23 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 14:01:14 -0700

On 10/1/21 1:30 PM, Danie de Jager wrote:
> Can we use the same options, but to trigger the longer annotation, we
> double the characters used to -hh and -HH?

Interesting idea. Normally, later options override earlier, so 'df -h 
-H' is equivalent to 'df -H'. This is so that one can alias 'df' to 'df 
-h' and then type plain 'df' to get the same behavior as 'df -h', while 
being able to type 'df -H' to get the other behavior. With that in mind 
if we made the change you suggest, such an alias would mean that if you 
typed 'df' you would get the behavior of ordinary 'df -h' while if you 
typed 'df -h' you'd get the behavior of ordinary 'df -hh'. So there 
might be an opportunity for confusion there.

We could of course use a different option letter (unfortunately -B is 
already taken...).

Message #26 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Golden <gdg <at> zplane.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 15:16:24 -0600

Paul Eggert <eggert <at> cs.ucla.edu> [2021-10-01 14:01:14 -0700]:
> 
> On 10/1/21 1:30 PM, Danie de Jager wrote:
> > Can we use the same options, but to trigger the longer annotation, we
> > double the characters used to -hh and -HH?
> 
> Interesting idea. Normally, later options override earlier, so 'df -h -H' is
> equivalent to 'df -H'. This is so that one can alias 'df' to 'df -h' and
> then type plain 'df' to get the same behavior as 'df -h', while being able
> to type 'df -H' to get the other behavior. With that in mind if we made the
> change you suggest, such an alias would mean that if you typed 'df' you
> would get the behavior of ordinary 'df -h' while if you typed 'df -h' you'd
> get the behavior of ordinary 'df -hh'. So there might be an opportunity for
> confusion there.
> 
> We could of course use a different option letter (unfortunately -B is
> already taken...).
> 

Might it be possible to finesse the already existing envars (BLOCK_SIZE,
DF_BLOCK_SIZE, etc.) to accomplish the desired suffixing mods?  Or perhaps
even a new envar(s) dedicated to simply specifying the suffixing, and
defaulting to present behavior for backcompat...?

Haven't thought this thru, just tossing it out as a possibility to think on.
Might be cleaner in some respects than trying to wedge it into the options.

Message #29 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: gdg <at> zplane.com
Cc: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Fri, 1 Oct 2021 14:26:24 -0700

On 10/1/21 2:16 PM, Glenn Golden wrote:
> Might it be possible to finesse the already existing envars (BLOCK_SIZE,
> DF_BLOCK_SIZE, etc.) to accomplish the desired suffixing mods?

We've been moving away from using those environment variables, for 
security and reproducibility reasons.

Message #32 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Danie de Jager <danie.dejager <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Mon, 4 Oct 2021 21:38:52 +0200

[Message part 1 (text/plain, inline)]

Hi Paul,

Thanks for your consideration. I understand your concern about causing
confusion with set aliases. In that case I agree, create a new option
instead.

Regards,
Danie

On Fri, 1 Oct 2021 at 23:01, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 10/1/21 1:30 PM, Danie de Jager wrote:
> > Can we use the same options, but to trigger the longer annotation, we
> > double the characters used to -hh and -HH?
>
> Interesting idea. Normally, later options override earlier, so 'df -h
> -H' is equivalent to 'df -H'. This is so that one can alias 'df' to 'df
> -h' and then type plain 'df' to get the same behavior as 'df -h', while
> being able to type 'df -H' to get the other behavior. With that in mind
> if we made the change you suggest, such an alias would mean that if you
> typed 'df' you would get the behavior of ordinary 'df -h' while if you
> typed 'df -h' you'd get the behavior of ordinary 'df -hh'. So there
> might be an opportunity for confusion there.
>
> We could of course use a different option letter (unfortunately -B is
> already taken...).
>

[Message part 2 (text/html, inline)]

Message #35 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Chris Murphy <lists <at> colorremedies.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Mon, 4 Oct 2021 16:31:46 -0400

On Fri, Oct 1, 2021 at 12:19 PM Pádraig Brady <P <at> draigbrady.com> wrote:
>
> On 01/10/2021 14:28, Danie de Jager wrote:
> > Hi,
> >
> > The output from df -h and df -H is always G or M. Depending on who sends me
> > usage stats I have to ask how the command was run to make sure I calculate
> > usage correctly. Systems like Amazon EC2 use the explicit GiB suffix.
> > Making it easier to know what sizes you are looking at.
> >
> > Can a future release of df not be improved to print out the GB or GiB?
>
> So you're suggesting that `df -h` changes to using IEC suffixes like Gi, Ti etc.
> and -H would use the current suffix format to indicate SI units.
> That would be the right thing to do if we were adding these options now,
> but at this stage I'm not sure it's worth it, because you wouldn't
> be sure which version of df you were getting the output from.
>
> For completeness one can get unambiguous output using numfmt like:
>
>    $ df -B1 | numfmt --field - --invalid=ignore --to=iec-i
>    Filesystem        1B-blocks        Used  Available Use% Mounted on
>    devtmpfs              3.9Gi           0      3.9Gi   0% /dev


I'm pretty biased toward showing units correctly, no matter the
ensuing compatibility arguments. The distillation of the compatibility
argument is: changing this will be painful for scripts. While true,
it's still worse to have the wrong units reported. Is the primary
target audience for human-readable values humans? Or scripts?

If it's really such a problem, introduce a computer/script friendly
output format and only ever append to the output when introducing new
values.


-- 
Chris Murphy

Message #38 received at 50940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Chris Murphy <lists <at> colorremedies.com>, Pádraig Brady
 <P <at> draigbrady.com>
Cc: Danie de Jager <danie.dejager <at> gmail.com>, 50940 <at> debbugs.gnu.org
Subject: Re: bug#50940: how df utility displays sizes - GB vs GiB
Date: Mon, 4 Oct 2021 13:42:13 -0700

On 10/4/21 13:31, Chris Murphy wrote:
> Is the primary
> target audience for human-readable values humans? Or scripts

Both.

Output columns are at a premium, so there is some advantage to omitting 
the units suffix (plus a lot of tradition for omission, outside of 
coreutils).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.