GNU bug report logs - #25149
md5sum fails with message "Invalid argument" on 4,294,967,295-byte files in FAT32

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25149 in the body.
You can then email your comments to 25149 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jaime Gaspar <mail <at> jaimegaspar.com>
To: bug-coreutils <at> gnu.org
Subject: md5sum fails with message "Invalid argument" on 4,294,967,295-byte
 files in FAT32
Date: Fri, 9 Dec 2016 07:45:00 -0800

--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one byte less than the maximum file size) it succeeds, but if one runs md5sum on a 4,294,967,295-byte file (the maximum file size) it fails with error message "Invalid argument".

--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
   truncate -s 9G tmp.fs
   mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
   sudo mkdir /tmp/mounted_tmp/
   sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294 bytes and file "file_2" with 4,294,967,295 bytes:
   cd /tmp/mounted_tmp/
   truncate -s 4294967294 file_1
   truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
   md5sum file_1
   md5sum file_2
The outputs should be respectively (notice that the second output is an error message):
   541249e3205af07b4a03f891185f64a0  file_1
   md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
   cd ..
   sudo umount /tmp/mounted_tmp/
   sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".

--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel 4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum, but not crc32.

Message #8 received at 25149 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jaime Gaspar <mail <at> jaimegaspar.com>, 25149 <at> debbugs.gnu.org
Subject: Re: bug#25149: md5sum fails with message "Invalid argument" on 4,
 294, 967, 295-byte files in FAT32
Date: Fri, 9 Dec 2016 19:22:20 +0000

tag 25149 notabug
close 25149
stop

On 09/12/16 18:28, Jaime Gaspar wrote:
>    read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32767

OK we've read all we can, but to verify md5sum will do:
  fread(buffer + 32767, 1, 1, stream)

Then the stdio stream will issue the underlying read()s
I'm not too sure where there are two reads here,
but they shouldn't be returning EINVAL, but just
returning 0 to indicate EOF.

>    read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
>    read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)

So it's a kernel bug as suspected.

thanks,
Pádraig

Message #11 received at 25149 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jaime Gaspar <mail <at> jaimegaspar.com>, 25149 <at> debbugs.gnu.org
Subject: Re: bug#25149: md5sum fails with message "Invalid argument" on 4,
 294, 967, 295-byte files in FAT32
Date: Fri, 9 Dec 2016 18:11:42 +0000

On 09/12/16 15:45, Jaime Gaspar wrote:
> --- Bug ---
> In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one byte less than the maximum file size) it succeeds, but if one runs md5sum on a 4,294,967,295-byte file (the maximum file size) it fails with error message "Invalid argument".
> 
> --- How to reproduce the bug ---
> Create a FAT32 file system in a file "tmp.fs":
>    truncate -s 9G tmp.fs
>    mkfs.vfat -F 32 tmp.fs
> Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
>    sudo mkdir /tmp/mounted_tmp/
>    sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
> Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294 bytes and file "file_2" with 4,294,967,295 bytes:
>    cd /tmp/mounted_tmp/
>    truncate -s 4294967294 file_1
>    truncate -s 4294967295 file_2
> Run md5sum on the two files "file_1" and "file_2":
>    md5sum file_1
>    md5sum file_2
> The outputs should be respectively (notice that the second output is an error message):
>    541249e3205af07b4a03f891185f64a0  file_1
>    md5sum: file_2: Invalid argument
> Unmount the file system at "/tmp/mounted_tmp/":
>    cd ..
>    sudo umount /tmp/mounted_tmp/
>    sudo rmdir /tmp/mounted_tmp/
> Remove the file "tmp.fs".
> 
> --- Notes ---
> Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel 4.8.0-30-generic.
> The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum, but not crc32.

I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning EINVAL?
In any case I'd direct the issue towards the kernel folks.

thanks,
Pádraig

Message #14 received at 25149 <at> debbugs.gnu.org (full text, mbox):

From: Jaime Gaspar <mail <at> jaimegaspar.com>
To: Pádraig Brady <p <at> draigbrady.com>, 25149 <at> debbugs.gnu.org
Subject: Re: bug#25149: md5sum fails with message "Invalid argument" on 4,
 294, 967, 295-byte files in FAT32
Date: Fri, 9 Dec 2016 10:28:38 -0800

> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?

It is the seventh and eighth lines below:

      ...
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32767
   read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
   read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
   write(2, "md5sum: ", 8)                 = 8
   write(2, "file_2", 6)                   = 6
   open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
   fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
   read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
   read(4, "", 4096)                       = 0
   close(4)                                = 0
   open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
   fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
   mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
   close(4)                                = 0
   open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   write(2, ": Invalid argument", 18)      = 18
   write(2, "\n", 1)                       = 1
   lseek(3, 0, SEEK_CUR)                   = 4294967295
   close(3)                                = 0
   close(1)                                = 0
   close(2)                                = 0
   exit_group(1)                           = ?
   +++ exited with 1 +++

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.