GNU bug report logs - #22363
Bug of stat in bash

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22363 in the body.
You can then email your comments to 22363 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Arturo <artblas <at> yahoo.es>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: Bug of stat in bash
Date: Wed, 13 Jan 2016 15:14:04 +0000 (UTC)

[Message part 1 (text/plain, inline)]

Hi.
I've needed isolate the size of a directory writing a script. I've used "stat -c %s directory", but the result is a great surprise. You can view the image stat.jpg: the size of /var/cache/apt/archives is 9804 KB (10 MB aprox.), but stat shows "159744". This number is the same if the directory have 60 MB, 10 MB or 0 MB. In other equal instalation (Ubuntu 14.04) this number showed 4096 with more than 300 MB in the same directory.
In the stat2.jpg image you can view the same situation with the directory empty.
It's a bug? The way to obtain my goal (only the number) is another? I've read the manual of stat and I can't discover another way. Reading pages in Internet, this is the way explained to achieve my goal...My system: Ubuntu 14.04; Terminal: Gnome 3.6.2.
Thanks for your attention.                                           Arturo

[Message part 2 (text/html, inline)]

[stat.jpg (image/jpeg, attachment)]

[stat2.jpg (image/jpeg, attachment)]

Message #12 received at 22363-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Arturo <artblas <at> yahoo.es>, 22363-done <at> debbugs.gnu.org
Subject: Re: bug#22363: Bug of stat in bash
Date: Wed, 13 Jan 2016 10:24:20 -0700

[Message part 1 (text/plain, inline)]

tag 22363 notabug
thanks

On 01/13/2016 08:14 AM, Arturo wrote:
> Hi.
> I've needed isolate the size of a directory writing a script. I've used "stat -c %s directory", but the result is a great surprise. You can view the image stat.jpg: the size of /var/cache/apt/archives is 9804 KB (10 MB aprox.), but stat shows "159744". This number is the same if the directory have 60 MB, 10 MB or 0 MB. In other equal instalation (Ubuntu 14.04) this number showed 4096 with more than 300 MB in the same directory.
> In the stat2.jpg image you can view the same situation with the directory empty.

In the future, please don't spam the list with a .jpg screenshot of a
terminal, when a simpler text-based copy-and-paste of its contents will
convey the same information with a lot less bandwidth.  Remember, ever
100kb you send in a .jpg turns into megabytes of outgoing traffic when
it is multiplied to all list subscribers.

> It's a bug?

No. The size of a directory is heavily dependent on the file system in
use, but in general is a multiple of the file system sector size (for
example, if the minimum size you can allocate in a file system is 64k,
but the information required to describe a single entry in a directory
occupies only 512 bytes, then a directory in that file system would show
up as 64k in size if it has anywhere between 1 and 128 entries, and
would jump to 128k in size if it has between 129 and 256 entries).

The 'stat' program is merely reporting to you the information reported
by the operating system, which in turn is reporting what the file system
is using.  It is NOT coreutils' fault that the number reported is not
the number you are looking for.  In other words, when you are asking
stat for the size of the directory, you are getting how much space the
directory itself occupies on the disk, and NOT recursing into how much
space its contents occupy.

> The way to obtain my goal (only the number) is another?

It sounds like what you are asking for is the cumulative size of all
files within a directory, added together, and NOT how much space the
directory itself occupies on disk.  The program for that is 'du'
(mnemonic for 'disk usage'), not 'stat'.

Maybe we are missing a link on the 'stat' manual page pointing to 'du',
but other than that, I see no problem here, so I'm closing the report.
However, feel free to reply with any further related questions.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #15 received at 22363 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Arturo <artblas <at> yahoo.es>, 22363 <at> debbugs.gnu.org
Subject: Re: bug#22363: Bug of stat in bash
Date: Tue, 19 Jan 2016 09:40:52 -0700

[Message part 1 (text/plain, inline)]

[re-adding the list, with permission]
[top-posting makes it harder to follow conversations on a technical
list; I'm reformatting your message]

On 01/16/2016 04:14 AM, Arturo wrote:
> Eric, thank you very much for your response. I don't know the question of .jpg added, excuse me.

Basically, I was trying to point out that a screenshot of text is less
efficient than the actual text itself, when it comes to the size of the
email sent describing a problem.  It's a common first-timer mistake
(you're not the first one who has sent a .jpg instead of
copying-and-pasting text), and I was merely trying to help you avoid the
problem in the future.

> I've read your response and I don't understand "almost nothing" (I'm beginner in this field). What means so the number 159744 in my case, despite the real size of the files contained? I can't obtain their origin (blocks x number of blocks...?)

The 'st_size' of a directory is how much disk space is occupied by that
directory, and is dependent on the choice of file system used.  And it
has nothing to do with how much space is occupied by the files within
that directory.

A common page size is 4k; and 159744 happens to be 39 * 4k.  So one
could assume that on that particular file system, your directory
occupies 39 pages of disk space.  But without knowing which file system
you are using, nor how many files are in that directory, that's all the
more I can tell you.

Maybe some examples from my system will help:

$ ls -ld /etc
drwxr-xr-x. 154 root root 12288 Jan 18 07:41 /etc/
$ stat -f /etc
  File: "/etc"
    ID: a18b6460f08a5036 Namelen: 255     Type: ext2/ext3
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 12868767   Free: 7633334    Available: 6973878
Inodes: Total: 3276800    Free: 3029863
$ ls -a /etc | wc
    292     292    2966

That says that my /etc resides on an ext[234] file system (actually
ext4, but the kernel folks made it rather hard for 'stat -f' to tell the
difference between the three versions).  It says that my file system was
formatted to allocate in pages (or "Fundamental block size") of 4k bytes
each.  So my /etc occupies 12288 bytes, or 3 pages.  Then I counted how
many actual entries there are, and 'wc' reports 292 lines.  Knowing what
I do about powers of 2, that means I could assume that 1 4k page will
allow me to create 128 files (or put another way, the directory only
needs 32 bytes per file to record its name and attributes, for the sizes
of names I have used), and because I have between 256 and 384 files, I
needed three pages in the directory to hold all 292 files.  But of
course, this is all guessing; I could create really long file names
(such as a file name longer than 32 bytes) to throw off all the math I
did above.  Ultimately, at the end of the day, I don't care how many
pages my directory required, as long as it is able to hold all the files
I created within it.

> What is their practical sense?I commented you the first time the different result obtained in two "identicals" Ubuntu 14.04, this 159744 and 4096. Why the file system "assign" 159744 to the directory in my laptop and only 4096 in other laptop? It is a "reserved" space, for example?

Most likely, it is because one system has more files in the directory
than the other, and those additional files required allocating
additional pages to the directory in order to track those file names.
Or maybe because the two systems are NOT identical, but are using
different filesystems, where each file system has different rules on how
much space will be occupied by a directory.

But the whole art of file systems is a very involved concept, so it is
probably sufficient to leave it to the file system experts (I am not
one), and to just remember that 'st_size' of a directory is usually not
the information you are looking for.

> 
> Otherwise, finally I had resolved my script with du.Best regards. 

I'm glad to hear that 'du' is indeed what you were looking for.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #18 received at 22363 <at> debbugs.gnu.org (full text, mbox):

From: Arturo <artblas <at> yahoo.es>
To: "22363 <at> debbugs.gnu.org" <22363 <at> debbugs.gnu.org>
Subject: Agreement with the last explanation
Date: Wed, 20 Jan 2016 12:59:34 +0000 (UTC)

[Message part 1 (text/plain, inline)]

Hi. The last explanation is enough for me. Only a suggestion: in manual for bash says: "%s     total size, in bytes". This expression, for a beginner, i.e., produces a mistake of perception about what stat offers exactly. Can be a option modifying the manual?Thanks for Eric. 
                                          Arturo

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.