GNU bug report logs - #19969
problem: wc -c doesn't read actual # of bytes in file

Previous Next

Full log

View this message in rfc822 format

From: Linda Walsh <coreutils <at> tlinx.org>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 19969 <at> debbugs.gnu.org
Subject: bug#19969: problem: wc -c doesn't read actual # of bytes in file
Date: Sun, 01 Mar 2015 13:33:51 -0800

Bernhard Voelker wrote:
> On 02/28/2015 09:59 AM, Linda Walsh wrote:
>> (coreutils-8.21-7.7.7)
>>
>> wc -c(bytes) doesn't seem to reliably read the number
>> of bytes in a file.
>>
>> I was wanting to find out what the largest data-source
>> files in '/proc' and '/sys' (didn't get around to trying
>> /sys, since all the files under /proc/sys return 0 bytes.
>>
>> Note -- "wc -l" doesn't return '0' on the /proc/sys files.
>>
>> As root:
>> # cd /proc
>> # find -H [^0-9]* -name self -prune -o -name thread-self -prune -o -type 
>> f ! -name kmsg ! -name kcore ! -name kpagecount ! -name kpageflags 
>> -print0|wc -c --files0-from=- |sort -n
> 
> Thanks for the report.
> However, I'm not 100% sure what the problem is - as you didn't narrow
> the case down to a certain file, nor did you show us the output and
> what you expected.
---
	Yes.   I did.  Anything under /proc/sys is affected.
There is more than 1 'certain file' that is affected.

	What I would _expect_ is for it to show the actual # of bytes
in the file rather than a size of 0.

	That's why I included the command to duplicate the problem.
Are you saying you tried it and it doesn't do the same thing on
your system?

	Without me debugging the source, how much more do you
want me to narrow it down?  If you change the "wc -c" => "wc "-l",
you will see non-zero numbers for files under /sys/proc.

	BTW .. it seems many files under /sys are also affected.

----
	As for files under /proc, the 'net' directory seems to work:
Are you wanting a list like this:
Ishtar:/proc> find -H sys net -name self -prune -o -name thread-self -prune -o -type f ! -name kmsg ! -name kcore ! -name kpagecount ! -name kpageflags -print0|wc -c --files0-from=- |sort -n       
wc: sys/fs/protected_hardlinks: Permission denied
wc: sys/fs/protected_symlinks: Permission denied
wc: sys/kernel/cad_pid: Permission denied
wc: sys/kernel/usermodehelper/bset: Permission denied
wc: sys/kernel/usermodehelper/inheritable: Permission denied
wc: sys/net/ipv4/route/flush: Permission denied
wc: sys/net/ipv4/tcp_fastopen_key: Permission denied
wc: sys/vm/compact_memory: Permission denied
0 net/netfilter/nfnetlink_log
0 sys/abi/vsyscall32
0 sys/debug/exception-trace
0 sys/dev/hpet/max-user-freq
0 sys/dev/raid/speed_limit_max
0 sys/dev/raid/speed_limit_min
0 sys/dev/scsi/logging_level
0 sys/fs/aio-max-nr
0 sys/fs/aio-nr
0 sys/fs/dentry-state
0 sys/fs/dir-notify-enable
0 sys/fs/epoll/max_user_watches
0 sys/fs/file-max
0 sys/fs/file-nr
0 sys/fs/inode-nr
0 sys/fs/inode-state
0 sys/fs/inotify/max_queued_events
0 sys/fs/inotify/max_user_instances
0 sys/fs/inotify/max_user_watches
0 sys/fs/lease-break-time
... (pattern is /proc/sys/* => size==0)
vs. directory /proc/net:

11 net/ip_tables_names
17 net/ip_tables_targets
36 net/psched
39 net/connector
47 net/mcfilter
54 net/ip_mr_cache
55 net/ip_tables_matches
71 net/ip_mr_vif
128 net/icmp
128 net/rt_cache
128 net/udplite
142 net/sockstat
171 net/ptype
233 net/raw
321 net/netfilter/nf_log
381 net/packet
644 net/dev_mcast
733 net/fib_triestat
822 net/igmp
896 net/route
1062 net/protocols
1090 net/dev
1188 net/softnet_stat
1234 net/arp
1385 net/snmp
1506 net/stat/arp_cache
1678 net/netlink
1718 net/fib_trie
2077 net/stat/rt_cache
2412 net/netstat
4096 net/rt_acct
6528 net/udp
20000 net/unix
41550 net/tcp
92581 total

---
Sorry to be unclear, but I would expect the numbers under /proc/sys
to look something like the numbers under /proc/net.

FWIW, the 'prune' and exceptions in the find are to filter
out some degenerate cases (like byte counts in /proc/kmem)
as well as too much verbosity (skip the processes, including self and 
thread-self).  I also used the -H switch to not skip initial symlinks,
but ignore lower ones.

For the numbers under /sys, most seem to return a wrong size.a size of 0, but 
doing a 'cat FILENAME|wc -c' shows the correct number.

I didn't give much data about /sys, as getting a test case that 
would run just for /proc, w/o hanging or looping took too long
(like trying to read 'kmsg' takes forever and will basically
hang the test case).

> A quick shot is
>   http://bugs.gnu.org/18621
---
	Certainly looks like what I'm seeing.  HOWEVER, 
why do I get numbers in "/proc/net" but not under "/sys"?

	Seems like there is something else going on here.



> and the corresponding commit (after the latest v8.23 release)
>   http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=2662702b
> 
> Does this - i.e., a wc(1) built from git - solve your problem?
---
	It may work *around* the problem, but doesn't explain
the inconsistencies between, for example, /proc/sys and /proc/net.
If the patch forces a read, there's a good chance it will work, but
the inconsistent returns from /proc/sys, /proc/net AND /sys(seems
to often show 4K as noted in the other bug report), would seem
to indicted a kernel bug somewhere.  If they were ALL 0 or all 4K,
then I'd say the kernel doesn't return size for either of those
interfaces (and that may be as far as that goes and the workaround
is the only solution), but that the kernel interfaces are giving 
back inconsistent answers seems like different users not using
the same kernel interfaces to return the 'inode' size.

	Did this provide enough info?

	Am a bit leery of trying a new version of core utils on
short notice as I need to make sure new breakages having occurred in
other utils and w/my luck, I might accidently install the newer
version over a working version (Murphy** visits, here, often).

**-as in Murphy's Law..

Cheers!
Linda

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.