GNU bug report logs - #19218
Inconsistent spacing of output of "ls --full-time [file argument]"

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19218 in the body.
You can then email your comments to 19218 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Michael Salem" <mgs <at> michaels.demon.co.uk>
To: bug-coreutils <at> gnu.org
Subject: Inconsistent spacing of output of "ls --full-time [file argument]"
Date: Sat, 29 Nov 2014 14:02:28 +0000

Summary: the output format of "ls --full-time" with a file argument 
seems to be inconsistent (within the same version of ls) for 
different arguments.

I have either a question, or a bug report. on ls. The question: is 
the output of 
ls --full-time
intended to be consistent within one version of ls?
I was trying to use it to extract file timestamps using "cut", and 
was finding that for different file arguments the output was 
different, and there was no unique way to extract the timestamp. This 
always with the sane version of ls; different versions have 
significantly different output spacing, which is just the way it is, 
not a problem. If the answer is "the horizontal spacing is not 
specified and not consistent", I express criticism, but say no more.

Otherwise, I have found what appears to be a consistent bug in 
several versions of ls, both in Linux and three different versioned 
Windows ports. In the first cases I tested there was an extra space 
character in the output depending upon the file argument. I later 
found, but did not investigate further, MAJOR differences in spacing 
in output from the Linux 8.21 version (examples below, look for 
pam.conf).

The problem manifests in the same way in "ls" in (GNU coreutils) 8.21 
in Linux Mint, ls 4.4.231 from the UnxUtils Windows port; ls 5.3.0 
from GnuWin32; and ls 4.5.287 for Windows from 
https://u-tools.com/msls.asp. It is the no surprise that the 
horizontal output spacing of DIFFERENT versions are different from 
each other; but within EACH SINGLE version they all omit a space 
before the file size for some, but not all, file arguments with the 
--full-time option. (I first discovered the variability using "cut" 
to process the output, finding that what worked for a full listing 
didn't work in some other cases). I also found an example of much 
worse variability in 8.21 Gnu. This last variability is so great that 
I would say that the output from "ls --full-time" is essentially 
randomly, unpredictably spaced, unsuitable for piping. Or have I 
misunderstood something?

In all cases I invoked, for files in the working directory,
ls --full-time [file argument without path]

In Windows (with cut.exe in the working directory) I used file 
arguments cut.exe, c*.exe *.exe, and no argument (all files). In 
Linux I worked in the bin directory, and used file arguments cut, c*, 
*, and no argument. With the more restrictive argument all the lines 
of output had one less space before the file size. I don't know 
exactly what happens; in Windows cut.exe, c*.exe came out one way, 
*.exe, and no argument the other. At least in the Windows versions 
there were no tab characters, only spaces; this isn't a tab spacing 
issue. (I later found much worse in the Linux /etc directory, see 
examples.)

Anyway, astonishing as it sounds, there may be a very long-standing 
bug in something as basic as ls.

All examples given are generated by "ls --fulltime [argument] > 
report" and copied and pasted, not hand-copied; they contained no tab 
characters.

SAMPLES OF INCONSISTENT OUTPUT FROM ls --full-time
--------------------------------------------------
(Lines contain no unexpected hard breaks; visibly broken lines are 
due to viewer and transmission issues.)
MS Windows
----------
ls 5.3.0 
ls --full-time (with no argument; same for ls --full-time *.exe)
...
-rw-rw-rw-  1 Michael 0      0 2014-11-29 12:28:00.000000000 +0000 
cccc.txt
-rwxrwxrwx  1 Michael 0  17920 2003-06-20 16:57:42.000000000 +0100 
cut.exe
...

ls --full-time cut.exe (same for c*.exe)
-rwxrwxrwx  1 Michael 0 17920 2003-06-20 16:57:42.000000000 +0100 
cut.exe

ls 4.4.231 and 4.5.287 
ls --full-time (with no argument; same layout for ls --full-time 
*.exe)
-rw-rw-rwa  1 Everyone              0 2014-11-29 12:28:00 cccc.txt
-rwxrwxrwa  1 Everyone          17920 2003-06-20 16:57:42 cut.exe
ls --full-time cut.exe (same for c*.exe)
-rwxrwxrwa  1 Everyone         17920 2003-06-20 16:57:42 cut.exe

Linux Mint, ls 8.21
----------
/bin directory
...
-rwxr-xr-x 1 root root  124932 2014-03-24 07:37:58.000000000 +0000 cp
-rwxr-xr-x 1 root root  138900 2013-06-02 13:07:47.000000000 +0100 
cpio
...
ls --full-time cp (same layout with c*)
...
-rwxr-xr-x 1 root root 124932 2014-03-24 07:37:58.000000000 +0000 cp
-rwxr-xr-x 1 root root 138900 2013-06-02 13:07:47.000000000 +0100 
cpio
...
/etc directory - MASSIVE variability found here. The output contained 
no tab characters - this is not an issue of tab expansion
ls --full-time (with no argument)
...
-rw-r--r--  1 root    root      249 2014-07-22 16:34:41.000000000 
+0100 os-release
-rw-r--r--  1 root    root      552 2014-01-31 22:19:46.000000000 
+0000 pam.conf
...
ls --full-time pam.conf
-rw-r--r-- 1 root root 552 2014-01-31 22:19:46.000000000 +0000 
pam.conf

In case the spacing gets messed up by the display software, I append 
four lines, repeated from the Linux examples, with spaces globally 
replaced by 
underlines. i consider that the first two should be identical to each 
other, as 
should the last two.
-rwxr-xr-x_1_root_root__124932_2014-03-24_07:37:58.000000000_+0000_cp
-rwxr-xr-x_1_root_root_124932_2014-03-24_07:37:58.000000000_+0000_cp
-rw-r--r--__1_root____root______552_2014-01-31_22:19:46.000000000_+000
0_pam.conf
-rw-r--r--_1_root_root_552_2014-01-31_22:19:46.000000000_+0000_pam.con
f

I don't know how communication is handled; this email is sent with a 
legitimate email address, hopefully not for publication. I'm not 
subscribed to the list, but mail will also reach me for the time 
being at the temporary address 8ec94ac4 <at> opayq.com. Text is not ideal 
and lines break, but I don't know if attaching a file is supported.

Best wishes, Michael Salem

Message #8 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: "Michael Salem" <mgs <at> michaels.demon.co.uk>
Cc: 19218 <at> debbugs.gnu.org, michaels <at> michaels.demon.co.uk
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Sat, 29 Nov 2014 21:53:13 +0100

"Michael Salem" <mgs <at> michaels.demon.co.uk> writes:

> I was trying to use it to extract file timestamps using "cut",

Don't do that.  Use stat(1) instead.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #11 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: michaels <at> michaels.demon.co.uk, 19218 <at> debbugs.gnu.org
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Sat, 29 Nov 2014 13:14:33 -0800

I don't see a bug in the cases you mention.  First, 'ls' dynamically adjusts 
column widths to fit the data, and this is considered to be a feature.  Second, 
different platforms have different time stamp resolutions.  The idea that all 
dates should use the same width is doomed anyway, since file time stamps can 
exceed the year 9999:

$ touch -d'10000-01-01 00:00:00' far-in-future
$ touch now
$ ls -l --full-time
-rw-r--r-- 1 eggert eggert 0 10000-01-01 00:00:00.000000000 -0800 far-in-future
-rw-r--r-- 1 eggert eggert 0 2014-11-29 13:07:55.182466680 -0800 now

Arguably this last example *is* a bug in 'ls', as dates should line up even when 
they're outlandish.  But it's not likely to be a bug one runs into with real 
files, at least, not for another 7985 years or so.

> this email is sent with a
> legitimate email address, hopefully not for publication.

GNU bug reports are public, I'm afraid.

Message #14 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: "Michael Salem" <michaels <at> michaels.demon.co.uk>
To: 19218 <at> debbugs.gnu.org
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Sun, 30 Nov 2014 00:36:46 +0000

Thanks for response to my bug report. I'm not familiar with this 
tracking system, and hope the response is organised right, threading 
appropriately without too much duplication or long-windedness (I 
won;t quote massively in detail). I reported an issue whereby several 
versions of ls all manifested what I describe as inappropriate 
behaviour when used with the --full-time option. This made ls 
inappropriate for my purpose, using cut to extract parts of the line.

I gave specific examples where ls --full-time gave different results 
on the same directory at the same time for different file arguments 
producing the same output line plus others (cp, cp*, c*, no 
argument). BTW, I don't actually have a problem to resolve any more, 
my report was simply for information (I have used stat, and finally 
ended up writing and compiling a simple program).

> Andreas Shcwab kindly suggested that ls is the wrong tool, 
recommending stat. That is absolutely right, and I had later been 
using stat (which happened not to be available to me originally, but 
I learnt about it later) after giving up on ls.

> Paul Eggert says "I don't see a bug in the cases you mention.  
First, 'ls' dynamically adjusts column widths to fit the data, and 
this is considered to be a feature.  Second, different platforms have 
different time stamp resolutions."

This clarifies matters for me, and I now agree insofar as one can't 
expect ls to produce the same output in all circumstances (though 
it's perhaps reasonable to expect it to be consistent ON THE SAME 
PLATFORM and using the same version, just needing small script 
changes if using "cut" to extract subtrings on different platforms).

However, I maintain that the behaviour I described, and repeat below, 
is bad enough to be called a bug. Whether it's considered worth 
correcting is another matter, which I won't push.

ls --full-time on the SAME file on the SAME directory at the SAME 
time (within a couple of minutes) produces differently-formatted 
output depending upon the file argument. The copied and pasted 
example I gave illustarted this.

I gave several examples in my original posting (which can be referred 
to if redundant examples are wanted), but one blatant one is enough, 
with ls 8.21, Linux Mint, /etc directory. In the following I've 
replaced each space by an underline; the output continaed no "real" 
underlines or tab characters. This is a particularly bad example, but 
many others produced output differing by one space between using a 
full filename or no argument. (In /etc try cp, no argumjent, c*.)

ls --full-time (with no argument) produces, amongst other lines:
-rw-r--r--__1_root____root______552_2014-01-31_22:19:46.000000000_+000
0_pam.conf

ls --full-time pam.conf produces:
-rw-r--r--_1_root_root_552_2014-01-31_22:19:46.000000000_+0000_pam.con
f

There is NO WAY that this is anything other than a bug. Maybe not 
worth fixing though?

I repeat, ls --full-time on the SAME file on the SAME directory at 
the SAME time on the SAME platform with the SAME ls version produces 
significantly differently-spaced output formats (the actual non-space 
content is identical, as expected)

Best wishes,
Michael Salem

Message #17 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: michaels <at> michaels.demon.co.uk, 19218 <at> debbugs.gnu.org
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Sun, 30 Nov 2014 02:09:19 +0000

tag 19218 notabug
close 19218
stop

On 29/11/14 21:14, Paul Eggert wrote:
> I don't see a bug in the cases you mention.  First, 'ls' dynamically adjusts 
> column widths to fit the data, and this is considered to be a feature.

Right. This is a limitation of cut, rather than anything wrong with ls.
See the awk usage at http://www.gnu.org/software/coreutils/cut

These examples might help:

  ls --full-time | tr -s ' ' | cut -d' ' -f6-
  ls --full-time | awk '{ print substr($0, index($0,$6)) }'

> Second, different platforms have different time stamp resolutions.  The idea that
> all dates should use the same width is doomed anyway, since file time stamps can 
> exceed the year 9999:
> 
> $ touch -d'10000-01-01 00:00:00' far-in-future
> $ touch now
> $ ls -l --full-time
> -rw-r--r-- 1 eggert eggert 0 10000-01-01 00:00:00.000000000 -0800 far-in-future
> -rw-r--r-- 1 eggert eggert 0 2014-11-29 13:07:55.182466680 -0800 now
> 
> Arguably this last example *is* a bug in 'ls', as dates should line up even when 
> they're outlandish.  But it's not likely to be a bug one runs into with real 
> files, at least, not for another 7985 years or so.

:)

thanks,
Pádraig.

Message #24 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: mgs <at> michaels.demon.co.uk, 19218 <at> debbugs.gnu.org
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Sat, 29 Nov 2014 20:42:17 -0800

Michael Salem wrote:
> There is NO WAY that this is anything other than a bug.

It's not a bug.  The first listing produces several lines of output, with wider 
columns, enough to accommodate every entry in every row.  The second listing 
does the same with just one line of output, so the columns can be narrower.

Message #27 received at 19218 <at> debbugs.gnu.org (full text, mbox):

From: "Michael Salem" <mgs <at> michaels.demon.co.uk>
To: 19218 <at> debbugs.gnu.org
Subject: Re: bug#19218: Inconsistent spacing of output of "ls --full-time
 [file argument]"
Date: Wed, 03 Dec 2014 12:49:02 +0000

To wrap up: I reported inconsistent output format from ls as a bug; 
others have made informative comments, to the effect that the output 
format of ls is not specified in detail so there is no program error; 
it is user error. I think the essential points are, going from the 
general to the particular:

1. Don't rely on undocumented behaviour in general.
2. If you do rely on undocumented output format, you're less likely 
to have trouble using whitespace as a delimiter rather than rely on 
absolute character positions. (In my particular case whitespace would 
have been OK, although it's been pointed out that ls date format, 
which I was assuming, is not guaranteed anyway, and ls is not the 
appropriate tool.)
3. The output format of ls depends upon its content, in particular it 
adjusts the lengths of all lines according to the longest one, so it 
is not incorrect for runs of ls which would produce overlong lines to 
eliminate some space characters. Thus, the line for file cp in /etc, 
in the output of <ls --full-time cp> can legitimately be spaced 
differently than for a listing of all the files in the directory, 
some of them with long names.

I wouldn't have distributed a script like the one I was trying to use 
without more care, but had thought it would be safe when using the 
same version of ls on the same system, and always in the same 
directory!

Thanks to all, and I hope this conversation might be of use to anyone 
repeating my mistake.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.