GNU bug report logs -
#20826
SEEK_HOLE not supported for ext4 for kernel < 3.1
Previous Next
Reported by: Johannes Meixner <jsmeix <at> suse.de>
Date: Tue, 16 Jun 2015 10:40:03 UTC
Severity: normal
Tags: wontfix
Done: Paul Eggert <eggert <at> cs.ucla.edu>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 20826 in the body.
You can then email your comments to 20826 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 10:40:04 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Johannes Meixner <jsmeix <at> suse.de>:
New bug report received and forwarded. Copy sent to
bug-grep <at> gnu.org.
(Tue, 16 Jun 2015 10:40:06 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hello,
newer grep versions do "lseek (fd, cur, SEEK_HOLE)"
but that does not always work as expected,
at least not for ext4 for kernel < 3.1.
This can cause weird results when using a newer grep
for files on ext4 with kernel < 3.1.
Our (SUSE) kernel developers found and analyzed that issue.
Here some excerpts from their mails:
----------------------------------------------------------------------
> homer:/usr/local/src/kernel/linux-3.x.git # grep master xx
> Binary file xx matches
> homer:/usr/local/src/kernel/linux-3.x.git # split -n 2 xx
> homer:/usr/local/src/kernel/linux-3.x.git # grep master xaa
> CONFIG_LOCALVERSION="-master"
> homer:/usr/local/src/kernel/linux-3.x.git # grep master xab
> homer:/usr/local/src/kernel/linux-3.x.git #
>
> it doesn't like the first 2000 lines of .config (xx),
> but does like both halves.
>
> Made sure by adding that line to xab.
>
> homer:/usr/local/src/kernel/linux-3.x.git # grep master xaa
> CONFIG_LOCALVERSION="-master"
> homer:/usr/local/src/kernel/linux-3.x.git # grep master xab
> CONFIG_LOCALVERSION="-master"
I could reproduce it.
Now grepping the grep code, I found that it checks the binary
also by looking at the hole in a file:
/* Look for a hole after the current location. */
off_t hole_start = lseek (fd, cur, SEEK_HOLE);
if (0 <= hole_start)
{
if (lseek (fd, cur, SEEK_SET) < 0)
suppressible_error (filename, errno);
if (hole_start < st->st_size)
return TEXTBIN_BINARY;
}
And hole_start returns 32768 instead of an error or the end of file.
The reason is that ext4 of SLE11 doesn't support SEEK_HOLE,
it just ignores the flag and returns the current position as is.
----------------------------------------------------------------------
"SLE11" is "SUSE Linux Enterprise 11" which has kernel 3.0.
Kind Regards
Johannes Meixner
--
SUSE LINUX GmbH - GF: Felix Imendoerffer, Jane Smithard,
Dilip Upmanyu, Graham Norton - HRB 21284 (AG Nuernberg)
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 11:51:03 GMT)
Full text and
rfc822 format available.
Message #8 received at 20826 <at> debbugs.gnu.org (full text, mbox):
An addendum FYI:
From our (SUSE) kernel developers I got the information that
only ext4 has this problem and that kernel git commit c334b1138bd4
(handle SEEK_HOLE/SEEK_DATA generically in /fs/ext4/file.c)
fixes it.
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 12:29:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 20826 <at> debbugs.gnu.org (full text, mbox):
From one of our (SUSE) kernel developers
I even got a proposal for a workaround in grep:
--- a/src/grep.c
+++ b/src/grep.c
@@ -575,6 +575,17 @@ file_textbin (char *buf, size_t size, in
off_t hole_start = lseek (fd, cur, SEEK_HOLE);
if (0 <= hole_start)
{
+ /* if hole_start is identical with cur, it's more likely a buggy
+ * lseek(SEEK_HOLE) implementation in kernel filesystem;
+ * check whether it really reads a null byte.
+ */
+ if (hole_start == cur)
+ {
+ char c;
+ if (read (fd, &c, 1) != 1 || c != 0)
+ hole_start = st->st_size; /* no hole */
+ }
+
if (lseek (fd, cur, SEEK_SET) < 0)
suppressible_error (filename, errno);
if (hole_start < st->st_size)
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 12:48:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 20826 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
[adding bug-gnulib]
On 06/16/2015 06:28 AM, Johannes Meixner wrote:
>
>> From one of our (SUSE) kernel developers
> I even got a proposal for a workaround in grep:
>
> --- a/src/grep.c
> +++ b/src/grep.c
> @@ -575,6 +575,17 @@ file_textbin (char *buf, size_t size, in
> off_t hole_start = lseek (fd, cur, SEEK_HOLE);
> if (0 <= hole_start)
> {
> + /* if hole_start is identical with cur, it's more likely a
> buggy
> + * lseek(SEEK_HOLE) implementation in kernel filesystem;
> + * check whether it really reads a null byte.
> + */
> + if (hole_start == cur)
> + {
> + char c;
> + if (read (fd, &c, 1) != 1 || c != 0)
> + hole_start = st->st_size; /* no hole */
> + }
> +
Sounds like we need to update the gnulib lseek module to work around
bugs like this in the wild.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 13:07:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 20826 <at> debbugs.gnu.org (full text, mbox):
On 16/06/15 13:47, Eric Blake wrote:
> [adding bug-gnulib]
>
> On 06/16/2015 06:28 AM, Johannes Meixner wrote:
>>
>>> From one of our (SUSE) kernel developers
>> I even got a proposal for a workaround in grep:
>>
>> --- a/src/grep.c
>> +++ b/src/grep.c
>> @@ -575,6 +575,17 @@ file_textbin (char *buf, size_t size, in
>> off_t hole_start = lseek (fd, cur, SEEK_HOLE);
>> if (0 <= hole_start)
>> {
>> + /* if hole_start is identical with cur, it's more likely a
>> buggy
>> + * lseek(SEEK_HOLE) implementation in kernel filesystem;
>> + * check whether it really reads a null byte.
>> + */
>> + if (hole_start == cur)
>> + {
>> + char c;
>> + if (read (fd, &c, 1) != 1 || c != 0)
>> + hole_start = st->st_size; /* no hole */
>> + }
>> +
>
> Sounds like we need to update the gnulib lseek module to work around
> bugs like this in the wild.
Interesting. The above runtime check wouldn't be appropriate I think
for such an old fleeting problem. Also the read() as a side effect
would be very much best avoided. How about a linux kernel version check
at build time?
As a general point, gnulib would really benefit from SEEK_HOLE, SEEK_DATA
wrappers that degenerated to SEEK_END,0 and SEEK_SET,offset respectively.
cheers,
Pádraig.
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 19:08:02 GMT)
Full text and
rfc822 format available.
Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady <P <at> draigBrady.com> wrote:
>> How about a linux kernel version check at build time?
Binaries are not always executed on the same kernel on which they are built.
--
Paul Jackson
pj <at> usa.net
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Tue, 16 Jun 2015 19:18:03 GMT)
Full text and
rfc822 format available.
Message #23 received at 20826 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/16/2015 07:06 AM, Pádraig Brady wrote:
> On 16/06/15 13:47, Eric Blake wrote:
>> [adding bug-gnulib]
>>
>> On 06/16/2015 06:28 AM, Johannes Meixner wrote:
>>>
>>>> From one of our (SUSE) kernel developers
>>> I even got a proposal for a workaround in grep:
>>>
>>> --- a/src/grep.c
>>> +++ b/src/grep.c
>>> @@ -575,6 +575,17 @@ file_textbin (char *buf, size_t size, in
>>> off_t hole_start = lseek (fd, cur, SEEK_HOLE);
>>> if (0 <= hole_start)
>>> {
>>> + /* if hole_start is identical with cur, it's more likely a
>>> buggy
>>> + * lseek(SEEK_HOLE) implementation in kernel filesystem;
>>> + * check whether it really reads a null byte.
>>> + */
>>> + if (hole_start == cur)
>>> + {
>>> + char c;
>>> + if (read (fd, &c, 1) != 1 || c != 0)
>>> + hole_start = st->st_size; /* no hole */
You MUST lseek() back to the original spot (or use pread() to avoid
changing offset) for the workaround to work. There might also be ways to
cache whether the workaround has ever been needed, so that subsequent
calls aren't penalized (then again, when working around the bug, the
likely behavior is to just treat the file as all data, so you'll only
ever encounter the bug once per file, unless you are reading the file's
contents multiple times); and as the call is filesystem related, we
can't guarantee that it works or fails on all possible files served by
the kernel, when crossing device boundaries. So yeah, we're probably
stuck with a per-file probing, minimized to the cases where a reported
hole sounds suspicious.
>>> + }
>>> +
>>
>> Sounds like we need to update the gnulib lseek module to work around
>> bugs like this in the wild.
>
> Interesting. The above runtime check wouldn't be appropriate I think
> for such an old fleeting problem. Also the read() as a side effect
> would be very much best avoided. How about a linux kernel version check
> at build time?
>
> As a general point, gnulib would really benefit from SEEK_HOLE, SEEK_DATA
> wrappers that degenerated to SEEK_END,0 and SEEK_SET,offset respectively.
Yeah, I've wanted those for several years now (I even mentioned it as a
"nice to have" in my Linux Plumber's Convention talk in 2012), but
haven't had time to work on it myself yet.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-grep <at> gnu.org:
bug#20826; Package
grep.
(Sun, 19 Jul 2015 00:39:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 20826 <at> debbugs.gnu.org (full text, mbox):
As I understand from <https://www.kernel.org/category/releases.html>, Linux
releases before 3.1 are no longer supported as of "mid-2015", which is right
around now. Also, OpenSUSE 11.4 (based on kernel 2.6.37 -- does this have the
kernel bug in question?) is seeing its support end this month, and all other
supported openSUSE systems have kernels newer than 3.1.
Given all this, I'm inclined to mark this bug as "wontfix" for Grep. This case
(though borderline) seems to fall more on the museum-piece side, and we don't
want to take on the burden of supporting museum pieces.
Added tag(s) wontfix.
Request was from
Paul Eggert <eggert <at> cs.ucla.edu>
to
control <at> debbugs.gnu.org.
(Sun, 19 Jul 2015 00:39:02 GMT)
Full text and
rfc822 format available.
bug closed, send any further explanations to
20826 <at> debbugs.gnu.org and Johannes Meixner <jsmeix <at> suse.de>
Request was from
Paul Eggert <eggert <at> cs.ucla.edu>
to
control <at> debbugs.gnu.org.
(Sun, 10 Apr 2016 22:02:03 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Mon, 09 May 2016 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 9 years and 103 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.