GNU bug report logs - #10493
coreutils shred fat32

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10493 in the body.
You can then email your comments to 10493 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leontiev Danil <YaTree <at> yandex.ru>
To: bug-coreutils <at> gnu.org
Subject: coreutils shred fat32
Date: Fri, 13 Jan 2012 11:43:26 +0400

Good Day.

I have problem with using shred util.

On usb-flash fs - FAT32
Files are not deleted from the first time. 
The important point: file's size < 12kb  

[root <at> localhost re]# shred -uvf /media/usb/test 
shred: /media/usb/test: проход 1/3 (random)...
shred: /media/usb/test: проход 2/3 (random)...
shred: /media/usb/test: error writing at offset 0: Invalid argument             <---- error

[root <at> localhost re]# shred -uvf /media/usb/test 
shred: /media/usb/test: проход 1/3 (random)...
shred: /media/usb/test: проход 2/3 (random)...
shred: /media/usb/test: проход 3/3 (random)...
shred: /media/usb/test: удаление
shred: /media/usb/test: переименован в /media/usb/0000
shred: /media/usb/0000: переименован в /media/usb/000
shred: /media/usb/000: переименован в /media/usb/00
shred: /media/usb/00: переименован в /media/usb/0
shred: /media/usb/test: удалён


Best regards,
Leontiev Danila.

Message #8 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: g.esp <at> free.fr
To: Leontiev Danil <YaTree <at> yandex.ru>
Cc: 10493 <at> debbugs.gnu.org
Subject: Re: bug#10493: coreutils shred fat32
Date: Fri, 13 Jan 2012 10:13:06 +0100 (CET)

----- Mail original -----
> De: "Leontiev Danil" <YaTree <at> yandex.ru>
> À: 10493 <at> debbugs.gnu.org
> Envoyé: Vendredi 13 Janvier 2012 08:43:26
> Objet: bug#10493: coreutils shred fat32
>
> Good Day.
>
> I have problem with using shred util.
>
> On usb-flash fs - FAT32
> Files are not deleted from the first time.
> The important point: file's size < 12kb
>
> [root <at> localhost re]# shred -uvf /media/usb/test
> shred: /media/usb/test: проход 1/3 (random)...
> shred: /media/usb/test: проход 2/3 (random)...
> shred: /media/usb/test: error writing at offset 0: Invalid argument
>             <---- error
>
> [root <at> localhost re]# shred -uvf /media/usb/test
> shred: /media/usb/test: проход 1/3 (random)...
> shred: /media/usb/test: проход 2/3 (random)...
> shred: /media/usb/test: проход 3/3 (random)...
> shred: /media/usb/test: удаление
> shred: /media/usb/test: переименован в /media/usb/0000
> shred: /media/usb/0000: переименован в /media/usb/000
> shred: /media/usb/000: переименован в /media/usb/00
> shred: /media/usb/00: переименован в /media/usb/0
> shred: /media/usb/test: удалён
>
>
> Best regards,
> Leontiev Danila.
>
>
Could you run the 2 commands with LANG=C for those that can read the messages above

LANG=C shred -uvf /media/usb/test

Gilles

Message #11 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: Leontiev Danil <YaTree <at> yandex.ru>
To: g.esp <at> free.fr,10493 <at> debbugs.gnu.org
Subject: Re: bug#10493: coreutils shred fat32
Date: Fri, 13 Jan 2012 13:49:46 +0400

Thx for the answer.

first start:
[root <at> localhost usb]$ LANG=C shred -uvf /media/usb/test 
shred: /media/usb/test: pass 1/3 (random)...
shred: /media/usb/test: pass 2/3 (random)...
shred: /media/usb/test: error writing at offset 0: Invalid argument

second start: 
[root <at> localhost usb]$ LANG=C shred -uvf /media/usb/test 
shred: /media/usb/test: pass 1/3 (random)...
shred: /media/usb/test: pass 2/3 (random)...
shred: /media/usb/test: pass 3/3 (random)...
shred: /media/usb/test: removing
shred: /media/usb/test: renamed to /media/usb/0000
shred: /media/usb/0000: renamed to /media/usb/000
shred: /media/usb/000: renamed to /media/usb/00
shred: /media/usb/00: renamed to /media/usb/1
shred: /media/usb/test: removed


I did insert debugging information in a source, I can send, and this log.

Obscure point is related to the variable "ssize", in which the first "run" to 4096. If you call the utility shred on device with a non-fat32 fs (i use ext2 i.e.), the value will -1.

Sorry for my english.


13.01.2012, 13:13, g.esp <at> free.fr:
> ----- Mail original -----
>
>>  De: "Leontiev Danil" <YaTree <at> yandex.ru>
>>  À: 10493 <at> debbugs.gnu.org
>>  Envoyé: Vendredi 13 Janvier 2012 08:43:26
>>  Objet: bug#10493: coreutils shred fat32
>>
>>  Good Day.
>>
>>  I have problem with using shred util.
>>
>>  On usb-flash fs - FAT32
>>  Files are not deleted from the first time.
>>  The important point: file's size < 12kb
>>
>>  [root <at> localhost re]# shred -uvf /media/usb/test
>>  shred: /media/usb/test: проход 1/3 (random)...
>>  shred: /media/usb/test: проход 2/3 (random)...
>>  shred: /media/usb/test: error writing at offset 0: Invalid argument
>>              <---- error
>>
>>  [root <at> localhost re]# shred -uvf /media/usb/test
>>  shred: /media/usb/test: проход 1/3 (random)...
>>  shred: /media/usb/test: проход 2/3 (random)...
>>  shred: /media/usb/test: проход 3/3 (random)...
>>  shred: /media/usb/test: удаление
>>  shred: /media/usb/test: переименован в /media/usb/0000
>>  shred: /media/usb/0000: переименован в /media/usb/000
>>  shred: /media/usb/000: переименован в /media/usb/00
>>  shred: /media/usb/00: переименован в /media/usb/0
>>  shred: /media/usb/test: удалён
>>
>>  Best regards,
>>  Leontiev Danila.
>
> Could you run the 2 commands with LANG=C for those that can read the messages above
>
> LANG=C shred -uvf /media/usb/test
>
> Gilles

Message #14 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Leontiev Danil <YaTree <at> yandex.ru>
Cc: g.esp <at> free.fr, 10493 <at> debbugs.gnu.org
Subject: Re: bug#10493: coreutils shred fat32
Date: Fri, 13 Jan 2012 14:57:39 +0100

Leontiev Danil wrote:
> Thx for the answer.
>
> first start:
> [root <at> localhost usb]$ LANG=C shred -uvf /media/usb/test
> shred: /media/usb/test: pass 1/3 (random)...
> shred: /media/usb/test: pass 2/3 (random)...
> shred: /media/usb/test: error writing at offset 0: Invalid argument
>
> second start:
> [root <at> localhost usb]$ LANG=C shred -uvf /media/usb/test
> shred: /media/usb/test: pass 1/3 (random)...
> shred: /media/usb/test: pass 2/3 (random)...
> shred: /media/usb/test: pass 3/3 (random)...
> shred: /media/usb/test: removing
> shred: /media/usb/test: renamed to /media/usb/0000
> shred: /media/usb/0000: renamed to /media/usb/000
> shred: /media/usb/000: renamed to /media/usb/00
> shred: /media/usb/00: renamed to /media/usb/1
> shred: /media/usb/test: removed
>
>
> I did insert debugging information in a source, I can send, and this log.
>
> Obscure point is related to the variable "ssize", in which the first
> "run" to 4096. If you call the utility shred on device with a
> non-fat32 fs (i use ext2 i.e.), the value will -1.
>
> Sorry for my english.

Thanks for the report.
What version of shred are you using?
Run "shred --version" to see that.
It's 7.1 or greater, since that's when we switched the default
from 25 to 3 overwrites, but not much has changed in shred since then.

What type of system are you using?
I.e., what type/version kernel and C library?
For the kernel, run uname -a.

Please run this command and send us the resulting LOG file, too:

  strace -o LOG shred -uvf /media/usb/test

Message #17 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: Leontiev Danil <YaTree <at> yandex.ru>
To: 10493 <at> debbugs.gnu.org
Date: Mon, 16 Jan 2012 07:43:05 +0400

[Message part 1 (text/plain, inline)]

Good day.

Sorry for delay.

Log for  "strace -o LOG shred -uvf /media/usb/test" in attachment.



Operation system info:

[root <at> localhost]$ uname -a
Linux localhost 2.6.33.7-desktop586-1mnb #1 SMP Fri Aug 27 20:24:16 UTC 2010 i686 i686 i386 GNU/Linux

[root <at> localhost]# LANG=C shred --version
shred (GNU coreutils) 8.5
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Colin Plumb.

[LOG (application/octet-stream, attachment)]

Message #20 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Leontiev Danil <YaTree <at> yandex.ru>
Cc: 10493 <at> debbugs.gnu.org
Subject: Re: bug#10493: (no subject)
Date: Mon, 16 Jan 2012 09:27:59 +0100

Leontiev Danil wrote:
> Log for  "strace -o LOG shred -uvf /media/usb/test" in attachment.
>
> Operation system info:
>
> [root <at> localhost]$ uname -a
> Linux localhost 2.6.33.7-desktop586-1mnb #1 SMP Fri Aug 27 20:24:16
> UTC 2010 i686 i686 i386 GNU/Linux
>
> [root <at> localhost]# LANG=C shred --version
> shred (GNU coreutils) 8.5
...

Thanks for the strace log.
It suggests a problem with your file system or the
underlying device, or maybe a kernel bug.

> execve("/usr/bin/shred", ["shred", "-uvf", "/media/netbook-ia32-x86_64-20110"...], [/* 65 vars */]) = 0
> brk(0)                                  = 0x9385000
...
> open("/media/netbook-ia32-x86_64-201105181448/0", O_WRONLY|O_NOCTTY|O_LARGEFILE) = 3
...
> fcntl64(3, F_SETFL, O_WRONLY|O_DIRECT|O_LARGEFILE) = 0
...
> write(3, "\214~\372\267\214C\230\242,\372\357\333\331\34\257{\327\6{\362\363o\256\212\211\204\242\235\200\265\200\320"..., 4096) = 4096
> time(NULL)                              = 1326713309
> fdatasync(3)                            = 0
> _llseek(3, 0, [0], SEEK_SET)            = 0
> write(2, "shred: ", 7)                  = 7
> write(2, "/media/netbook-ia32-x86_64-20110"..., 71) = 71
> write(2, "\n", 1)                       = 1
> time(NULL)                              = 1326713309
> write(3, "TI\245\352\217)\3541\f\234\260\272h\330\246{\303\200\317A\372\217\374\21\25\17\376\32\34\320\34N"..., 4096) = -1 EINVAL (Invalid argument)

Note how the first 4KiB write succeeds,
yet the second one fails with EINVAL.

shred.c is designed to accommodate (and recover from) an EINVAL failure
when that failure happens on the very first write syscall.  However, when
it happens
after that, the code presumes there really is an error.

For the record, the diagnostic you're seeing is the one from the
end of this excerpt:

  while (true)
    {
      /* How much to write this time? */
      lim = sizeof r;
      if (0 <= size && size - offset < sizeof_r)
        {
          if (size < offset)
            break;
          lim = size - offset;
          if (!lim)
            break;
        }
      if (type < 0)
        randread (s, &r, lim);
      /* Loop to retry partial writes. */
      for (soff = 0; soff < lim; soff += ssize, first_write = false)
        {
          ssize = write (fd, r.c + soff, lim - soff);
          if (ssize <= 0)
            {
              if (size < 0 && (ssize == 0 || errno == ENOSPC))
                {
                  /* Ah, we have found the end of the file */
                  *sizep = size = offset + soff;
                  break;
                }
              else
                {
                  int errnum = errno;
                  char buf[INT_BUFSIZE_BOUND (uintmax_t)];

                  /* If the first write of the first pass for a given file
                     has just failed with EINVAL, turn off direct mode I/O
                     and try again.  This works around a bug in Linux kernel
                     2.4 whereby opening with O_DIRECT would succeed for some
                     file system types (e.g., ext3), but any attempt to
                     access a file through the resulting descriptor would
                     fail with EINVAL.  */
                  if (k == 1 && first_write && errno == EINVAL)
                    {
                      direct_mode (fd, false);
                      ssize = 0;
                      continue;
                    }
                  error (0, errnum, _("%s: error writing at offset %s"),
                         qname, umaxtostr (offset + soff, buf));


If you're ambitious, you can find out if the kernel bug mentioned above
affects 2.6.x and is now triggered only on the *second* write (with 2.4,
it was triggered on the first).  To do that, could change this line in
src/shred.c,

-                 if (k == 1 && first_write && errno == EINVAL)
+                 if (k == 1 && errno == EINVAL)

and build shred from source.

Note however: that induces the risk of an infinite loop.
It will work fine if write fails with EINVAL only on the second
attempt, but then always succeeds after turning off O_DIRECT.

Message #23 received at 10493 <at> debbugs.gnu.org (full text, mbox):

From: Leontiev Danil <YaTree <at> yandex.ru>
To: 10493 <at> debbugs.gnu.org
Date: Mon, 16 Jan 2012 20:07:37 +0400

Thx for the answer.

But may be you can specify "target" where to look for bug in kernel and specify some "anomiles" while diagnostic\debug?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.