GNU bug report logs - #49994
upgrade: have "-s" of 'ls' display alloc'd sizes for specified "--block-size"

Previous Next

To reply to this bug, email your comments to 49994 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Coreutils <bug-coreutils <at> gnu.org>
Subject: upgrade: have "-s" of 'ls' display alloc'd sizes for specified
 "--block-size"
Date: Tue, 10 Aug 2021 23:56:08 -0700

How can I find all perl source files under the
perl source tree sorted by size + displayed 
w/binary-metric prefixes.  I wanted to look for
a good sample for _some_ specific size (not sure
of the size until I'd looked through a few).

I got close using:
find -iname \*.pl |sed 's!./!!' |xargs /usr/bin/ls --block-size=1 -1dhsS

That gave 577 lines of output, but only displaying the 1st entry
of same-size, consecutive entries that only has 15 unique sizes:

312K lib/perl5db.pl
200K lib/Unicode/Collate/Locale/ja.pl
100K lib/Unicode/Collate/Locale/zh_pin.pl
96K ext/XS-APItest/t/utf8_warn_base.pl
36K regen/opcode.pl
32K ext/DynaLoader/DynaLoader_pm.PL
28K regen/feature.pl
24K ext/XS-APItest/t/handy_base.pl
20K Porting/checkURL.pl
16K Porting/corelist.pl
12K t/lib/dbmt_common.pl
8.0K dist/lib/lib_pm.PL
4.0K cpan/Pod-Simple/t/ascii_order.pl
4.0K t/run/dtrace.pl

Unfortunately, the "-s" param on 'ls' drops 99.98% of 
the size info.  If the -s parameter displayed allocated
space for a file on a device with the block-size 
as specified on the command line (useful for determining
how much space the files will take on some device
with a finer block-size).

Using the block size as specified by the user gives
279 uniquely-sized entries losing no information (0%)
on the sizes of the files, with only my elision
of names for the sake of less verbosity


310K  lib/perl5db.pl
197K  cpan/Unicode-Collate/Collate/Locale/ja.pl
138K  regen/mk_invlists.pl
126K  Porting/bisect-runner.pl
...
53.3K win32/bin/search.pl
52.1K cpan/IO-Compress/t/compress/generic.pl
49.9K t/test.pl
46.4K utils/perlbug.PL
...
17.4K Porting/pod_lib.pl
17.1K Porting/add-package.pl
....
6.7K  dist/Devel-PPPort/Makefile.PL
6.6K  cpan/Sys-Syslog/Makefile.PL
6.5K  cpan/podlators/Makefile.PL
5.5K  cpan/Unicode-Collate/Collate/Locale/da.pl
5.4K  symbian/makesis.pl
...
1.2K  ext/Hash-Util/Makefile.PL
1.1K  win32/bin/perlglob.pl
1.0K  ext/File-Glob/Makefile.PL
997  cpan/Unicode-Collate/Makefile.PL
983  cpan/Unicode-Collate/Collate/Locale/mk.pl
...
125  dist/Time-HiRes/hints/sco.pl
110  ext/VMS-DCLsym/Makefile.PL
104  dist/Time-HiRes/hints/dynixptx.pl
101  cpan/DB_File/hints/dynixptx.pl
 97  ext/POSIX/hints/mint.pl
 94  dist/IO/hints/sco.pl
....
 11  symbian/uid.pl
  6  t/op/svleak.pl
  3  t/run/dtrace.pl
  2  cpan/Module-Load-Conditional/t/to_load/LoadMe.pl

--------------------------


Perhaps the '-s' option in ls could pay attention to
the user-specified block-size?

Message #8 received at 49994 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>, 49994 <at> debbugs.gnu.org
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Wed, 11 Aug 2021 00:40:56 -0700

On 8/10/21 11:56 PM, L A Walsh wrote:
> Perhaps the '-s' option in ls could pay attention to
> the user-specified block-size?

Doesn't it already do that?

$ ls -s libc.so
12192 libc.so
$ ls -s --block-size=1 libc.so
12484608 libc.so

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Sun, 15 Aug 2021 16:13:12 -0700

(reposting w/list+bugdb)

Hmm...  for libc.so, ls -l shows 253 bytes
for -s, it shows '4' 
and w/block-size=1, it shows
4096...

My LS_OPTIONS is
LS_OPTIONS=-N --color=tty -T 0

-N for quoting and -T for no tabs

my version: rpm -qf /usr/bin/ls
coreutils-8.23-2.3.2.x86_64

may not be as current as yours, but it would be odd if it changed and
you didn't know...


Ishtar:/usr/lib64> command ls -lgG libc.so
-rw-r--r-- 1 253 Mar 11 11:22 libc.so
Ishtar:/usr/lib64> command ls -s libc.so
4 libc.so
Ishtar:/usr/lib64> command ls --block-size=1 -s libc.so
4096 libc.so


On 2021/08/11 00:40, Paul Eggert wrote:
> On 8/10/21 11:56 PM, L A Walsh wrote:
>> Perhaps the '-s' option in ls could pay attention to
>> the user-specified block-size?
> 
> Doesn't it already do that?
> 
> $ ls -s libc.so
> 12192 libc.so
> $ ls -s --block-size=1 libc.so
> 12484608 libc.so


On 2021/08/11 00:40, Paul Eggert wrote:
> On 8/10/21 11:56 PM, L A Walsh wrote:
>> Perhaps the '-s' option in ls could pay attention to
>> the user-specified block-size?
> 
> Doesn't it already do that?
> 
> $ ls -s libc.so
> 12192 libc.so
> $ ls -s --block-size=1 libc.so
> 12484608 libc.so

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Sun, 15 Aug 2021 21:00:51 -0700

On 8/15/21 4:13 PM, L A Walsh wrote:
> Hmm...  for libc.so, ls -l shows 253 bytes
> for -s, it shows '4' and w/block-size=1, it shows
> 4096...

Right. That is, those numbers look right.

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Sun, 15 Aug 2021 21:26:26 -0700

On 2021/08/15 21:00, Paul Eggert wrote:
> On 8/15/21 4:13 PM, L A Walsh wrote:
>> Hmm...  for libc.so, ls -l shows 253 bytes
>> for -s, it shows '4' and w/block-size=1, it shows
>> 4096...
> 
> Right. That is, those numbers look right.
----
	But I'm specifying a block size of 1 byte, to determine
what size it would take up on a device that allocates in 'bytes'.
I don't want to display the allocated size on the current device,
but the size that would be allocated on a device with block-size=1-byte.

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Sun, 15 Aug 2021 21:36:13 -0700

On 8/15/21 9:26 PM, L A Walsh wrote:
> I don't want to display the allocated size on the current device,
> but the size that would be allocated on a device with block-size=1-byte.

There's no practical way for 'ls' to calculate that, as the kernel 
doesn't reveal that information.

Message #25 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Tue, 17 Aug 2021 03:24:06 -0700

On 2021/08/15 21:36, Paul Eggert wrote:
> On 8/15/21 9:26 PM, L A Walsh wrote:
>> I don't want to display the allocated size on the current device,
>> but the size that would be allocated on a device with block-size=1-byte.
> 
> There's no practical way for 'ls' to calculate that, as the kernel 
> doesn't reveal that information.

'du' can display sizes using and arbitrary block size with
no need for kernel support, but it doesn't have the format options
that ls has.

For determining the size with an arbitrary block size, the code
could just be dupped from 'du'.

Message #28 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Tue, 17 Aug 2021 16:00:08 -0700

On 8/17/21 3:24 AM, L A Walsh wrote:
> 
> 
> On 2021/08/15 21:36, Paul Eggert wrote:
>> On 8/15/21 9:26 PM, L A Walsh wrote:
>>> I don't want to display the allocated size on the current device,
>>> but the size that would be allocated on a device with block-size=1-byte.
>>
>> There's no practical way for 'ls' to calculate that, as the kernel 
>> doesn't reveal that information.
> 
> 'du' can display sizes using and arbitrary block size with
> no need for kernel support

That's different from displaying the size that would have been allocated 
on a device with a block size of 1 byte. 'du' can't do what you 
requested, any more than 'ls' can.

Message #31 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Tue, 17 Aug 2021 16:03:40 -0700

On 2021/08/17 16:00, Paul Eggert wrote:
> That's different from displaying the size that would have been allocated 
> on a device with a block size of 1 byte. 'du' can't do what you 
> requested, any more than 'ls' can.

----
However you want to phrase it, with:

   --block-size=SIZE

I want to have 'ls'

   'scale sizes by SIZE before printing them'

Message #34 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Wed, 18 Aug 2021 08:39:26 -0700

On 8/17/21 4:03 PM, L A Walsh wrote:
> However you want to phrase it, with:
> 
>     --block-size=SIZE
> 
> I want to have 'ls'
> 
>     'scale sizes by SIZE before printing them'

But that's what --block-size does.

Perhaps your request would be more clear if you gave a specific example 
of what you want and why --block-size doesn't do what you want.

Message #37 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 00:02:46 -0700

On 2021/08/18 08:39, Paul Eggert wrote:
> Perhaps your request would be more clear if you gave a specific example 
> of what you want and why --block-size doesn't do what you want.
----

I wanted to use ls to list the size of files in bytes, and use
Binary prefixes K=1024, M=1024**2, G=1024**3 to display
the the numeric portion with a maximum 3 digits or
use a floating point format with up to 1 digit to the right
of the decimal point: similar and as rounded by use of "%.1f"
as in printf.

Using a list of 577 files in file "files.txt", the existing 'ls'
using:

cat files.txt|xargs /usr/bin/ls --block-size=1 -1dhsS

Instead of displaying the file sizes in bytes, it showed them
rounded to the nearest 4K, so sorting by size and displaying
only the 1st of a given size, I only ended up with 13 unique 
file-sizes in the list:

312K lib/perl5db.pl
200K lib/Unicode/Collate/Locale/ja.pl
100K lib/Unicode/Collate/Locale/zh_pin.pl
96K ext/XS-APItest/t/utf8_warn_base.pl
36K regen/opcode.pl
32K ext/DynaLoader/DynaLoader_pm.PL
28K regen/feature.pl
24K ext/XS-APItest/t/handy_base.pl
20K Porting/checkURL.pl
16K Porting/corelist.pl
12K t/lib/dbmt_common.pl
8.0K dist/lib/lib_pm.PL
4.0K cpan/Pod-Simple/t/ascii_order.pl


I expected closer to 100 or more files displayed with size under
1000 bytes.

If 'ls' used the user-specified 'block-size' of 1 byte, the number
of files displayed is actually 279, or over 20 times the number
of files with some names elided for sake of brevity.

310K  lib/perl5db.pl
197K  cpan/Unicode-Collate/Collate/Locale/ja.pl
138K  regen/mk_invlists.pl
126K  Porting/bisect-runner.pl
...
53.3K win32/bin/search.pl
52.1K cpan/IO-Compress/t/compress/generic.pl
49.9K t/test.pl
46.4K utils/perlbug.PL
...
17.4K Porting/pod_lib.pl
17.1K Porting/add-package.pl
....
6.7K  dist/Devel-PPPort/Makefile.PL
6.6K  cpan/Sys-Syslog/Makefile.PL
6.5K  cpan/podlators/Makefile.PL
5.5K  cpan/Unicode-Collate/Collate/Locale/da.pl
5.4K  symbian/makesis.pl
...
1.2K  ext/Hash-Util/Makefile.PL
1.1K  win32/bin/perlglob.pl
1.0K  ext/File-Glob/Makefile.PL
997  cpan/Unicode-Collate/Makefile.PL
983  cpan/Unicode-Collate/Collate/Locale/mk.pl
...
125  dist/Time-HiRes/hints/sco.pl
110  ext/VMS-DCLsym/Makefile.PL
104  dist/Time-HiRes/hints/dynixptx.pl
101  cpan/DB_File/hints/dynixptx.pl
97  ext/POSIX/hints/mint.pl
94  dist/IO/hints/sco.pl
....
11  symbian/uid.pl
 6  t/op/svleak.pl
 3  t/run/dtrace.pl
 2  cpan/Module-Load-Conditional/t/to_load/LoadMe.pl

-----------------------------

Instead 'ls' displays 4.0K for a file that is only 2 bytes.

Clearly it isn't using the block-size specified by the user
in the same way 'du' does:
cd /tmp
echo "1" >out.pl
du -b out.pl
2 out.pl
Ishtar:/tmp> /bin/ls --block-size=1 -1dhsS  /tmp/out.pl
4.0K /tmp/out.pl
---

When I use "-b" in du (equivalent to '--apparent-size --block-size=1),
I get a size of '2' which one would expect for a file with 2 chars in it.
however, ls displays 4.0K.  Clearly it isn't using the
--block-size=1 that 'du'  is using.

Message #40 received at 49994 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: 49994 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 10:30:39 +0200

There is stat and numfmt.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #43 received at 49994 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: 49994 <at> debbugs.gnu.org
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 08:08:13 -0700

On 8/20/21 12:02 AM, L A Walsh wrote:

> When I use "-b" in du (equivalent to '--apparent-size --block-size=1)

Previously in this thread you asked for "allocated space for a file on a 
device", which is why I was talking about that number.

Now you're asking for apparent size, which is a different number. If you 
want apparent size, you can use 'ls -l', or follow Andreas's suggestion 
and use something like 'stat -c "%s %n"'. However, apparent size might 
be quite different from allocated space, as can be seen in the 1st and 
6th columns of the last two lines of the following:

$ echo '' >a
$ truncate -s 1TB b; echo '' >>b
$ ls -ls --block-size=1 a b
4096 -rw-rw-r-- 1 eggert eggert             1 Aug 20 08:07 a
4096 -rw-rw-r-- 1 eggert eggert 1000000000001 Aug 20 08:07 b

Message #46 received at 49994 <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 49994 <at> debbugs.gnu.org
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 10:40:02 -0700

On 2021/08/20 08:08, Paul Eggert wrote:
> On 8/20/21 12:02 AM, L A Walsh wrote:
> 
>> When I use "-b" in du (equivalent to '--apparent-size --block-size=1)
> 
> Previously in this thread you asked for "allocated space for a file on a 
> device", which is why I was talking about that number.
---
	No, I wanted space that "would" be allocated on a device if the 
device allocated size in bytes -- which is what 'du' does.

	I even said I wanted the output of ls's block-size output to be
enhanced to pay attention to what users specify via that parameter.
I wanted a simple multi-column output like 'ls' provides.  

	I wanted it just to work -- not require a program or figuring it out
but just have it be user-friendly -- something the tools designed by gnu
definitely are not.

> 

Message #49 received at 49994 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: 49994 <at> debbugs.gnu.org
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 11:31:33 -0700

On 8/20/21 10:40 AM, L A Walsh wrote:
>      I wanted it just to work

So use 'stat -c "%s %n"'. That works just fine. 'stat' is a good tool 
for this job, and there's no need for yet another 'ls' option for this 
rare use case.

Message #52 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 49994 <at> debbugs.gnu.org, Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#49994: upgrade: have "-s" of 'ls' display alloc'd sizes for
 specified "--block-size"
Date: Fri, 20 Aug 2021 12:57:03 -0700

On 2021/08/20 11:31, Paul Eggert wrote:
> On 8/20/21 10:40 AM, L A Walsh wrote:
>>      I wanted it just to work
> 
> So use 'stat -c "%s %n"'. That works just fine. 'stat' is a good tool 
> for this job, and there's no need for yet another 'ls' option for this 
> rare use case.

stat doesn't come close.
I wanted the automatic-multi column output of 'ls' like "ls -s" gives, 
but instead of the disk-space-allocated number as 'du' would be
sufficient for, I wanted 'byte' output -- like the default output of 'ls'
gives.

By default, ls gives you names, or with -l gives you bytes.  If you want
a short form that gives you bytes+name, the obvious attempt would be
use of -s with a flag that would say to display bytes.

This rare use case? Having a file-list command that 
displays filename+size in a compact form seems one of the
top use use-cases.  Yet that seems to be missing.

The default of ls -s should be to display size, not allocated blocks, since
if you want that, 'du' is what first comes to mind to find the
diskspace-used.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.