GNU bug report logs - #21130
test fail: 'tests/ls/stat-failed'

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21130 in the body.
You can then email your comments to 21130 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: test fail: 'tests/ls/stat-failed'
Date: Fri, 24 Jul 2015 17:46:36 -0400

[Message part 1 (text/plain, inline)]

Hello,

checking coreutils 8.24, running on NFS, the test 'ls/stat-failed' fails (log attached).

If I understand correctly,
The test creates a symlink to a directory then removes execute permissions:
   mkdir d
   ln -s / d/s
   chmod 600 d

Then tries to dereference it:
    $ ls -Log d
    ls: cannot access d/s: Permission denied
    total 0
    d????????? ? ?            ? s

The test expect 's' to have 'l' type, on my system it is 'd'.

The attached patch avoids the failure, though I don't know if this is correct or not (perhaps this failure should not be avoided?).

The system is:
    $ uname -a
    Linux XXX 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
    $ gcc --version
     gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)

The file system is NFS.


regards,
 - assaf

[ls-stat-failed-test-suite.log (text/x-log, attachment)]

[ls-stat-failed.patch (text/x-diff, attachment)]

Message #8 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Assaf Gordon <assafgordon <at> gmail.com>, 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 06:07:21 +0100

On 24/07/15 22:46, Assaf Gordon wrote:
> Hello,
> 
> checking coreutils 8.24, running on NFS, the test 'ls/stat-failed' fails (log attached).
> 
> If I understand correctly,
> The test creates a symlink to a directory then removes execute permissions:
>     mkdir d
>     ln -s / d/s
>     chmod 600 d
> 
> Then tries to dereference it:
>      $ ls -Log d
>      ls: cannot access d/s: Permission denied
>      total 0
>      d????????? ? ?            ? s
> 
> The test expect 's' to have 'l' type, on my system it is 'd'.
> 
> The attached patch avoids the failure, though I don't know if this is correct or not (perhaps this failure should not be avoided?).
> 
> The system is:
>      $ uname -a
>      Linux XXX 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>      $ gcc --version
>       gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
> 
> The file system is NFS.

Interesting. Is the NFS server RHEL6 too?

With normal perms on 'd':
 On ext4 I get ^d
  With syscalls being:
    getdents(3, /* 3 entries */, 32768)     = 72
    stat("d/s", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0

With -x perms on 'd':
 On nfsv4 here I get ^?
 On ext4 I get ^l
  With syscalls being:
    getdents(3, /* 3 entries */, 32768)     = 72
    stat("d/s", 0x7f634b9594a0)             = -1 EACCES (Permission denied)

So it seems you're getting next->d_type==DR_DIR in that case?
That would be wrong, but a system issue rather than a problem in ls I think.

The test adjustment looks OK but would benefit from a comment
stating the first letter represents d_type and that on buggy
systems may be 'd'.

thanks!
Pádraig

Message #11 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Assaf Gordon <assafgordon <at> gmail.com>, 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 06:25:05 +0100

On 25/07/15 06:07, Pádraig Brady wrote:
> On 24/07/15 22:46, Assaf Gordon wrote:
>> Hello,
>>
>> checking coreutils 8.24, running on NFS, the test 'ls/stat-failed' fails (log attached).
>>
>> If I understand correctly,
>> The test creates a symlink to a directory then removes execute permissions:
>>     mkdir d
>>     ln -s / d/s
>>     chmod 600 d
>>
>> Then tries to dereference it:
>>      $ ls -Log d
>>      ls: cannot access d/s: Permission denied
>>      total 0
>>      d????????? ? ?            ? s
>>
>> The test expect 's' to have 'l' type, on my system it is 'd'.
>>
>> The attached patch avoids the failure, though I don't know if this is correct or not (perhaps this failure should not be avoided?).
>>
>> The system is:
>>      $ uname -a
>>      Linux XXX 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>>      $ gcc --version
>>       gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
>>
>> The file system is NFS.
> 
> Interesting. Is the NFS server RHEL6 too?
> 
> With normal perms on 'd':
>  On ext4 I get ^d
>   With syscalls being:
>     getdents(3, /* 3 entries */, 32768)     = 72
>     stat("d/s", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
> 
> With -x perms on 'd':
>  On nfsv4 here I get ^?
>  On ext4 I get ^l
>   With syscalls being:
>     getdents(3, /* 3 entries */, 32768)     = 72
>     stat("d/s", 0x7f634b9594a0)             = -1 EACCES (Permission denied)
> 
> So it seems you're getting next->d_type==DR_DIR in that case?
> That would be wrong, but a system issue rather than a problem in ls I think.
> 
> The test adjustment looks OK but would benefit from a comment
> stating the first letter represents d_type and that on buggy
> systems may be 'd'.

Another possibility might be that the chmod(1) and stat(2) are racy
thus allowing the stat() to succeed? If that was the case then
a stat d/s && skip_ ... would avoid the false failure?

Message #14 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>, 
 Assaf Gordon <assafgordon <at> gmail.com>, 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 08:05:52 -0700

Pádraig Brady wrote:
>> On 24/07/15 22:46, Assaf Gordon wrote:
>>> If I understand correctly,
>>> The test creates a symlink to a directory then removes execute permissions:
>>>      mkdir d
>>>      ln -s / d/s
>>>      chmod 600 d
>>>
>>> Then tries to dereference it:
>>>       $ ls -Log d
>>>       ls: cannot access d/s: Permission denied
>>>       total 0
>>>       d????????? ? ?            ? s

> Another possibility might be that the chmod(1) and stat(2) are racy
> thus allowing the stat() to succeed? If that was the case then
> a stat d/s && skip_ ... would avoid the false failure?

Sorry, I don't understand the scenario here.  If the stat succeeds, why would ls 
output '????'?  The '????' means that the stat failed.

Message #17 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Assaf Gordon <assafgordon <at> gmail.com>,
 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 18:13:12 +0100

On 25/07/15 16:05, Paul Eggert wrote:
> Pádraig Brady wrote:
>>> On 24/07/15 22:46, Assaf Gordon wrote:
>>>> If I understand correctly,
>>>> The test creates a symlink to a directory then removes execute permissions:
>>>>      mkdir d
>>>>      ln -s / d/s
>>>>      chmod 600 d
>>>>
>>>> Then tries to dereference it:
>>>>       $ ls -Log d
>>>>       ls: cannot access d/s: Permission denied
>>>>       total 0
>>>>       d????????? ? ?            ? s
> 
>> Another possibility might be that the chmod(1) and stat(2) are racy
>> thus allowing the stat() to succeed? If that was the case then
>> a stat d/s && skip_ ... would avoid the false failure?
> 
> Sorry, I don't understand the scenario here.  If the stat succeeds, why would ls 
> output '????'?  The '????' means that the stat failed.

Yes good point. So it must be that d_type is set to 'd' erroneously.
What is the file system and operating system on the NFS server?

Message #20 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 21130 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 18:43:03 -0400

Hello,

> On Jul 25, 2015, at 13:13, Pádraig Brady <P <at> draigBrady.com> wrote:

>>>> On 24/07/15 22:46, Assaf Gordon wrote:
>>>>> Then tries to dereference it:
>>>>>      $ ls -Log d
>>>>>      ls: cannot access d/s: Permission denied
>>>>>      total 0
>>>>>      d????????? ? ?            ? s
> 
<...>

> Yes good point. So it must be that d_type is set to 'd' erroneously.
> What is the file system and operating system on the NFS server?

The NFS server is an Isilon OneFS (not sure which model/version).

strace gives:
  stat("d", {st_mode=S_IFDIR|0600, st_size=19, ...}) = 0
  open("d", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
  fcntl(3, F_GETFD) = 0x1 (flags FD_CLOEXEC)
  getdents(3, /* 3 entries */, 32768) = 72
  stat("d/s", 0x1fc9690) = -1 EACCES (Permission denied) 

Breaking in 'print_dir()' (ls.c:2612) gives:

---
##
## The code was "next = readdir(dirp)"
##
(gdb) p *next
$9 = {d_ino = 6682814318, d_off = 3, d_reclen = 24, d_type = 4 '\004',
  d_name = "s\000\000\000\004", '\000' <repeats 250 times>}
----
So it seems the filesystem reports d_type==DT_DIR.


For completeness, I've tested with other file types, and it seems that the returned type is always 'directory':
---
$ chmod 700 d
$ ls -log d
total 8
lrwxrwxrwx 1  9 Jul 25 18:29 blockdev -> /dev/sda1
lrwxrwxrwx 1  9 Jul 25 18:29 chardev -> /dev/tty0
lrwxrwxrwx 1 11 Jul 25 18:29 file -> /etc/passwd
lrwxrwxrwx 1  1 Jul 24 17:41 s -> /

$ ls -Log d
total 8
brw-rw----  1 8, 1 Dec 16  2014 blockdev
crw--w----  1 4, 0 Dec 16  2014 chardev
-rw-r--r--  1 1808 Jun 26 12:00 file
dr-xr-xr-x 26 4096 Jan 29 15:06 s

$ chmod 600 d
$ ls -Log d
ls: cannot access d/file: Permission denied
ls: cannot access d/chardev: Permission denied
ls: cannot access d/blockdev: Permission denied
ls: cannot access d/s: Permission denied
total 0
d????????? ? ?            ? blockdev
d????????? ? ?            ? chardev
d????????? ? ?            ? file
d????????? ? ?            ? s
---

regards,
 - assaf

Message #23 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 16:57:14 -0700

Assaf Gordon wrote:
> it seems that the returned type is always 'directory':

Wow,that's quite a bug.  I imagine it affects programs other than 'ls'.  Sounds 
like it needs to be fixed in the file system or kernel, as it's not realistic to 
install workarounds in every application that uses readdir.

Is the bug reproducible on a more-modern Linux kernel?  2.6.32 is projected for 
end-of-life around now, according to 
<https://www.kernel.org/category/releases.html>.  If the bug doesn't appear in 
3.2 or newer, I wouldn't worry about it.

I found a machine with an old kernel (RHEL 6.6 with kernel 
2.6.32-504.12.2.el6.x86_64) and could not reproduce the bug with a Network 
Appliance NFS server.  So perhaps the bug is Isilon-specific rather than 
kernel-specific.  If so, Isilon should fix it.

Message #26 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 21:09:11 -0400

On 07/25/2015 07:57 PM, Paul Eggert wrote:
> Assaf Gordon wrote:
>> it seems that the returned type is always 'directory':
<...>
> Is the bug reproducible on a more-modern Linux kernel?
<...>
> If the bug doesn't appear in 3.2 or newer, I wouldn't worry about it.

Thanks for the pointer - on another machine with newer kernel (mounting the same NFS share) the reported type is 'unknown'.

===
$  uname -a
Linux xxxx 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 GNU/Linux

$ ./src/ls -Log d
./src/ls: cannot access d/file: Permission denied
./src/ls: cannot access d/chardev: Permission denied
./src/ls: cannot access d/blockdev: Permission denied
./src/ls: cannot access d/s: Permission denied
total 0
?????????? ? ?            ? blockdev
?????????? ? ?            ? chardev
?????????? ? ?            ? file
?????????? ? ?            ? s
===

Message #29 received at 21130 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: 21130 <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 25 Jul 2015 18:26:47 -0700

Assaf Gordon wrote:
> $ ./src/ls -Log d
> ./src/ls: cannot access d/file: Permission denied
> ./src/ls: cannot access d/chardev: Permission denied
> ./src/ls: cannot access d/blockdev: Permission denied
> ./src/ls: cannot access d/s: Permission denied
> total 0
> ?????????? ? ?            ? blockdev

Arguably this is a better result than what we normally get, and our test 
framework should allow it.

The argument is that if a directory is not searchable, an application should be 
able to discover its files' names (as they belong to the directory and not to 
the files), but not the files' attributes (as they belong to the files, not to 
the directory).  Admittedly this is a fine distinction.

Message #34 received at 21130-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: 21130-done <at> debbugs.gnu.org
Subject: Re: bug#21130: test fail: 'tests/ls/stat-failed'
Date: Sat, 21 Nov 2015 11:05:29 +0000

[Message part 1 (text/plain, inline)]

On 24/07/15 22:46, Assaf Gordon wrote:
> Hello,
> 
> checking coreutils 8.24, running on NFS, the test 'ls/stat-failed' fails (log attached).
> 
> If I understand correctly,
> The test creates a symlink to a directory then removes execute permissions:
>     mkdir d
>     ln -s / d/s
>     chmod 600 d
> 
> Then tries to dereference it:
>      $ ls -Log d
>      ls: cannot access d/s: Permission denied
>      total 0
>      d????????? ? ?            ? s
> 
> The test expect 's' to have 'l' type, on my system it is 'd'.
> 
> The attached patch avoids the failure, though I don't know if this is correct or not (perhaps this failure should not be avoided?).
> 
> The system is:
>      $ uname -a
>      Linux XXX 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>      $ gcc --version
>       gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
> 
> The file system is NFS.

False failure documented and avoided in the attached.

thanks,
Pádraig.

[stat-false-fail.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.