GNU bug report logs -
#17149
shred - tape data
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 17149 in the body.
You can then email your comments to 17149 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Mon, 31 Mar 2014 15:45:04 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Rupert Russell <Rupert.Russell <at> ilmor.co.uk>
:
New bug report received and forwarded. Copy sent to
bug-coreutils <at> gnu.org
.
(Mon, 31 Mar 2014 15:45:05 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
shred -v /dev/st0 # where /dev/st0 is the drive in a multi tape carousel.
This starts well with.
Pass 1/4 (random) 15MiB
Pass 1/4 (random) 41MiB
Pass 1/4 (random) 70MiB
Pass 1/4 (random).....etc
But when the end of a 200GiB tape is reached in pass 1, 'shred' doesn't appear to trigger a rewind or maybe wait for the tape to be rewound before attempting to overwrite the data.
The final messages before and around the error are:-
Pass 1/4 (random) 190GiB/190GiB 100%
Pass 2/4 (random)
Error writing at offset 1288. No space left on device.
(The next line in my script then runs and unloads the tape.)
Is this not rewinding the tape a bug or my mis-use of the 'shred' command?
Thanks
Rupert Russell
OPERATIONS MANAGER
[60-Ilmor-logo-72dpi]
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk<http://www.ilmor.co.uk>
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
[Message part 2 (text/html, inline)]
[image001.jpg (image/jpeg, inline)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Mon, 31 Mar 2014 17:33:03 GMT)
Full text and
rfc822 format available.
Message #8 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 03/31/2014 12:00 PM, Rupert Russell wrote:
> Hi,
>
> shred -v /dev/st0 # where /dev/st0 is the drive in a multi tape carousel.
>
> This starts well with.
> Pass 1/4 (random) 15MiB
> Pass 1/4 (random) 41MiB
> Pass 1/4 (random) 70MiB
> Pass 1/4 (random).....etc
>
> But when the end of a 200GiB tape is reached in pass 1, 'shred' doesn't appear to trigger a rewind or maybe wait for the tape to be rewound before attempting to overwrite the data.
>
>
> The final messages before and around the error are:-
>
> Pass 1/4 (random) 190GiB/190GiB 100%
>
> Pass 2/4 (random)
>
> Error writing at offset 1288. No space left on device.
>
> (The next line in my script then runs and unloads the tape.)
>
> Is this not rewinding the tape a bug or my mis-use of the 'shred' command?
shred(1) doesn't rewind the tape.
It could issue an MTREW ioctl or something,
though I suspect there are all sorts of ioctls to consider depending on plaftorm.
I also see the `mt erase` command/ioctl which might suffice?
If not then explicitly handling the tape like this might suffice?
for i in $(seq 3); do
mt -f /dev/st0 rewind
shred -n1 /dev/st0
done
thanks,
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 01:21:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady wrote:
> shred(1) doesn't rewind the tape.
But shred is supposed to rewind the tape, using lseek.
Can you find out what system calls are executed? Use the command
shred -o tracefile -v /dev/st0
Warning: 'tracefile' will get quite large. The point is to see what
happens near the start of execution, and around the time that the write
files.
My guess is that 'shred' is incorrectly calculating the tape size; it
does that by seeking to the end of the tape, and perhaps this is going awry.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 07:23:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Thanks Paul,
However, my distribution doesn't appear to have a -o option.
Tried, -o, -O and ,-0 but all not recognised options.
--version says 7.1
courtesy one Colin Plumb.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Paul Eggert [mailto:eggert <at> cs.ucla.edu]
Sent: 01 April 2014 02:21
To: Pádraig Brady; Rupert Russell
Cc: 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
Pádraig Brady wrote:
> shred(1) doesn't rewind the tape.
But shred is supposed to rewind the tape, using lseek.
Can you find out what system calls are executed? Use the command
shred -o tracefile -v /dev/st0
Warning: 'tracefile' will get quite large. The point is to see what happens near the start of execution, and around the time that the write files.
My guess is that 'shred' is incorrectly calculating the tape size; it does that by seeking to the end of the tape, and perhaps this is going awry.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 12:19:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig,
This seems to be getting my task done.
for i in $(seq 1 $NUMTAPES) ; do
mtx -f $LOADER load $i
for j in $(seq 1 $NUMWIPES); do
shred -v -n 1 $TAPE
mt -f $TAPE rewind
done
shred -v -z -n 0 $TAPE
mt -f $TAPE rewind
mtx -f $LOADER unload $i
done
By the way, is there a difference between what these two write to the media?
mt -f $TAPE erase and
shred -z -n 0 $TAPE
Regards
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Pádraig Brady [mailto:P <at> draigBrady.com]
Sent: 31 March 2014 18:33
To: Rupert Russell
Cc: 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
On 03/31/2014 12:00 PM, Rupert Russell wrote:
> Hi,
>
> shred -v /dev/st0 # where /dev/st0 is the drive in a multi tape carousel.
>
> This starts well with.
> Pass 1/4 (random) 15MiB
> Pass 1/4 (random) 41MiB
> Pass 1/4 (random) 70MiB
> Pass 1/4 (random).....etc
>
> But when the end of a 200GiB tape is reached in pass 1, 'shred' doesn't appear to trigger a rewind or maybe wait for the tape to be rewound before attempting to overwrite the data.
>
>
> The final messages before and around the error are:-
>
> Pass 1/4 (random) 190GiB/190GiB 100%
>
> Pass 2/4 (random)
>
> Error writing at offset 1288. No space left on device.
>
> (The next line in my script then runs and unloads the tape.)
>
> Is this not rewinding the tape a bug or my mis-use of the 'shred' command?
shred(1) doesn't rewind the tape.
It could issue an MTREW ioctl or something, though I suspect there are all sorts of ioctls to consider depending on plaftorm.
I also see the `mt erase` command/ioctl which might suffice?
If not then explicitly handling the tape like this might suffice?
for i in $(seq 3); do
mt -f /dev/st0 rewind
shred -n1 /dev/st0
done
thanks,
Pádraig.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 12:39:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/01/2014 01:18 PM, Rupert Russell wrote:
> Pádraig,
>
> This seems to be getting my task done.
>
> for i in $(seq 1 $NUMTAPES) ; do
> mtx -f $LOADER load $i
> for j in $(seq 1 $NUMWIPES); do
> shred -v -n 1 $TAPE
> mt -f $TAPE rewind
> done
> shred -v -z -n 0 $TAPE
> mt -f $TAPE rewind
> mtx -f $LOADER unload $i
> done
>
> By the way, is there a difference between what these two write to the media?
> mt -f $TAPE erase and
> shred -z -n 0 $TAPE
It depends I suppose.
`mt erase` might trigger a command to the hardware
to activate a degausser or something, rather than writing zeros,
though from a very quick google it defaults to writing zeros.
BTW I find this a handy way to see what shred is doing
rather than reading the docs:
strace -e write shred -s 1K /dev/null
thanks,
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 13:52:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Rupert Russell wrote:
> shred -o tracefile -v /dev/st0
Ouch, sorry, I meant "strace -o tracefile shred -v /dev/st0".
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 14:40:03 GMT)
Full text and
rfc822 format available.
Message #26 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Aha,
Ok, I've aborted my script which had only been running a short while and I'll run this overnight and see what happens at the end.
Just ran a short test for a few minutes (90mb worth on the tape) and the tracefile was under 3meg, so I am hoping that my 140gig available diskspace will be enough...
Regards
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Paul Eggert [mailto:eggert <at> cs.ucla.edu]
Sent: 01 April 2014 14:51
To: Rupert Russell
Cc: 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
Rupert Russell wrote:
> shred -o tracefile -v /dev/st0
Ouch, sorry, I meant "strace -o tracefile shred -v /dev/st0".
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Tue, 01 Apr 2014 15:32:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/01/2014 03:38 PM, Rupert Russell wrote:
> Aha,
>
> Ok, I've aborted my script which had only been running a short while and I'll run this overnight and see what happens at the end.
> Just ran a short test for a few minutes (90mb worth on the tape) and the tracefile was under 3meg, so I am hoping that my 140gig available diskspace will be enough...
To save some space you could:
strace -e lseek ....
Though TBH I think what may be happening is that the lseek()
does not rewind the tape at all and only resets the file offset.
I also just noticed the `man 4 st` information which suggests
that the mt command to rewind the tape might be redundant
and the tape would auto rewind anyway when each device was closed.
So you could also hack it like:
shred -n1 /dev/st0 /dev/st0 /dev/st0
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 08:04:01 GMT)
Full text and
rfc822 format available.
Message #32 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Paul,
Interesting challenge you set extracting the last few lines from a 4 gig file, but I've passed that test I think. (Used Less..)
read(3, "\374\204\234\317t\\\216)\224\225d\244\372\3741\265P\203}\177\226\226\273\213V9\205\331v6\331\315"..., 12288) = 12288
write(4, "\374\204\234\317t\\\216)\224\225d\244\372\3741\265P\203}\177\226\226\273\213V9\205\331v6\331\315"..., 12288) = 12288
read(3, "\335\331|\27\26\r*.\270k@>&\365\315*\362'\321b0\2240KQ!\212\2\336\25\276\5"..., 12288) = 12288
write(4, "\335\331|\27\26\r*.\270k@>&\365\315*\362'\321b0\2240KQ!\212\2\336\25\276\5"..., 12288) = 12288
read(3, "\321`\346>\360\276R\250{\1\371\37\2553\341\357\21\247\24=~*g\270\235\204\301-\256zQ\261"..., 12288) = 12288
write(4, "\321`\346>\360\276R\250{\1\371\37\2553\341\357\21\247\24=~*g\270\235\204\301-\256zQ\261"..., 12288) = 12288
read(3, "\316\177\377\254\347\231\226\233_v;$\211\330~Y\345\230\27\17\231\352A\177'\351\31\326\3135\220\372"..., 12288) = 12288
write(4, "\316\177\377\254\347\231\226\233_v;$\211\330~Y\345\230\27\17\231\352A\177'\351\31\326\3135\220\372"..., 12288) = -1 ENOSPC (No space left on device)
write(2, "shred: ", 7) = 7
write(2, "/dev/st0: pass 1/3 (random)...19"..., 48) = 48
write(2, "\n", 1) = 1
fdatasync(4) = -1 EINVAL (Invalid argument)
fsync(4) = -1 EINVAL (Invalid argument)
sync() = 0
fdatasync(4) = -1 EINVAL (Invalid argument)
fsync(4) = -1 EINVAL (Invalid argument)
sync() = 0
lseek(4, 0, SEEK_SET) = 0
write(2, "shred: ", 7) = 7
write(2, "/dev/st0: pass 2/3 (random)...", 30) = 30
write(2, "\n", 1) = 1
read(3, ";\214\272\247\374S\261\312]=\306\r\215\355F\24\32\204!l9KD\337\227\273=\376s\361^\272"..., 12288) = 12288
write(4, ";\214\272\247\374S\261\312]=\306\r\215\355F\24\32\204!l9KD\337\227\273=\376s\361^\272"..., 12288) = 12288
read(3, "d\30\203*\363\230\356I\210\363\304\227\235\327\257\302\307RYg\363\343\230\300\354\315i\f\220\241oF"..., 12288) = 12288
write(4, "d\30\203*\363\230\356I\210\363\304\227\235\327\257\302\307RYg\363\343\230\300\354\315i\f\220\241oF"..., 12288) = -1 ENOSPC (No space left on device)
write(2, "shred: ", 7) = 7
write(2, "/dev/st0: error writing at offse"..., 39) = 39
open("/usr/share/locale-langpack/en_GB.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-bundle/en_GB.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-bundle/en_GB.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=1474, ...}) = 0
mmap(NULL, 1474, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7f3e20dc8000
close(5) = 0
open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-bundle/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-bundle/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-bundle/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": No space left on device", 25) = 25
write(2, "\n", 1) = 1
close(4) = 0
close(3) = 0
close(1) = 0
close(2) = 0
exit_group(1) = ?
Hopefully it'll make some sense.
Regards
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Paul Eggert [mailto:eggert <at> cs.ucla.edu]
Sent: 01 April 2014 14:51
To: Rupert Russell
Cc: 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
Rupert Russell wrote:
> shred -o tracefile -v /dev/st0
Ouch, sorry, I meant "strace -o tracefile shred -v /dev/st0".
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 08:15:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig.
"Though TBH I think what may be happening is that the lseek() does not rewind the tape at all and only resets the file offset."
Looking through the trace file I see only this line with 'lseek'.
lseek(4, 0, SEEK_SET) = 0
So I am guessing that what you suggest is correct?
" shred -n1 /dev/st0 /dev/st0 /dev/st0"
Goodness, interesting stuff.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 13:13:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Sorry, but curiosity got the better of me and I found myself Googling for lseek(4, 0, SEEK_SET)
And somehow I arrived here,
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=69630
Seems a chap called Paul Eggert has discussed lseek and zero's with tape drives once before.
I was also interested that dd is mentioned in the same breath, and it was a failing script using dd, that led me to shred in the 1st place....although that failure was probably unrelated.
What busy chaps you've been.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 14:56:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/02/2014 09:03 AM, Rupert Russell wrote:
> Paul,
>
> Interesting challenge you set extracting the last few lines from a 4 gig file, but I've passed that test I think. (Used Less..)
>
>
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = -1 ENOSPC (No space left on device)
So we got ENOSPC and didn't warn about that,
but rather just used that info to determine the end of the device.
That also implies that the initial lseek (fd, 0, SEEK_END)
on the device did return a failure (which we ignored by design).
> write(2, "/dev/st0: pass 1/3 (random)...19"..., 48) = 48
> sync() = 0
> lseek(4, 0, SEEK_SET) = 0
We ask here to "rewind", which returns "successfully"
> write(2, "/dev/st0: pass 2/3 (random)...", 30) = 30
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = 12288
I'm a bit surprised this first write passes
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = -1 ENOSPC (No space left on device)
> write(2, "/dev/st0: error writing at offse"..., 39) = 39
> write(2, ": No space left on device", 25) = 25
> close(4) = 0
So now we know the size of the device, we correctly warn about this error.
The key point is the lseek(4, 0, SEEK_SET) returns success here for the tape device.
I guessed that this was by design, but let's just delve into the code a little...
http://lxr.linux.no/linux+v3.13.5/drivers/scsi/st.c#L1176
static int st_open(struct inode *inode, struct file *filp) {
/*
* We really want to do nonseekable_open(inode, filp); here, but some
* versions of tar incorrectly call lseek on tapes and bail out if that
* fails. So we disallow pread() and pwrite(), but permit lseeks.
*/
}
And the lseek() maps to a noop that just returns the current file pointer.
thanks,
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 16:00:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig,
So, I think this means there isn't a bug. The behaviour I observe is from a decision made to work in this way and so my scripted workaround is the way forward.
Regards
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Pádraig Brady [mailto:P <at> draigBrady.com]
Sent: 02 April 2014 15:55
To: Rupert Russell
Cc: 'Paul Eggert'; 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
On 04/02/2014 09:03 AM, Rupert Russell wrote:
> Paul,
>
> Interesting challenge you set extracting the last few lines from a 4
> gig file, but I've passed that test I think. (Used Less..)
>
>
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = -1 ENOSPC (No space left on device)
So we got ENOSPC and didn't warn about that, but rather just used that info to determine the end of the device.
That also implies that the initial lseek (fd, 0, SEEK_END) on the device did return a failure (which we ignored by design).
> write(2, "/dev/st0: pass 1/3 (random)...19"..., 48) = 48
> sync() = 0
> lseek(4, 0, SEEK_SET) = 0
We ask here to "rewind", which returns "successfully"
> write(2, "/dev/st0: pass 2/3 (random)...", 30) = 30
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = 12288
I'm a bit surprised this first write passes
> read(3, "..., 12288) = 12288
> write(4, "..., 12288) = -1 ENOSPC (No space left on device) write(2,
> "/dev/st0: error writing at offse"..., 39) = 39 write(2, ": No space
> left on device", 25) = 25
> close(4) = 0
So now we know the size of the device, we correctly warn about this error.
The key point is the lseek(4, 0, SEEK_SET) returns success here for the tape device.
I guessed that this was by design, but let's just delve into the code a little...
http://lxr.linux.no/linux+v3.13.5/drivers/scsi/st.c#L1176
static int st_open(struct inode *inode, struct file *filp) {
/*
* We really want to do nonseekable_open(inode, filp); here, but some
* versions of tar incorrectly call lseek on tapes and bail out if that
* fails. So we disallow pread() and pwrite(), but permit lseeks.
*/
}
And the lseek() maps to a noop that just returns the current file pointer.
thanks,
Pádraig.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 16:32:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 17149 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Rupert Russell wrote:
> I think this means there isn't a bug.
Well, it is a bug in the sense that Linux lseek doesn't work on tapes
(which POSIX allows), and that 'shred' doesn't work with this.
Does the attached patch work for you?
[shred.diff (text/plain, attachment)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 18:00:03 GMT)
Full text and
rfc822 format available.
Message #50 received at 17149 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/02/2014 09:31 AM, Paul Eggert wrote:
> Does the attached patch work for you?
Oops. that had an incorrect integer overflow check. Attached is an
improved version, which uses INT_ADD_OVERFLOW instead of attempting to
do it by hand.
[shred1.diff (text/x-patch, attachment)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Wed, 02 Apr 2014 18:25:01 GMT)
Full text and
rfc822 format available.
Message #53 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/02/2014 06:59 PM, Paul Eggert wrote:
> On 04/02/2014 09:31 AM, Paul Eggert wrote:
>> Does the attached patch work for you?
> Oops. that had an incorrect integer overflow check. Attached is an improved version, which uses INT_ADD_OVERFLOW instead of attempting to do it by hand.
I was thinking initially the MTIOCTOP code would be too specific
to add to shred, but searching a bit more it seems generic
enough to be useful. The code looks good.
thanks!
Pádraig.
Reply sent
to
Paul Eggert <eggert <at> cs.ucla.edu>
:
You have taken responsibility.
(Thu, 03 Apr 2014 06:31:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Rupert Russell <Rupert.Russell <at> ilmor.co.uk>
:
bug acknowledged by developer.
(Thu, 03 Apr 2014 06:31:04 GMT)
Full text and
rfc822 format available.
Message #58 received at 17149-done <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady wrote:
> The code looks good.
Thanks for the review. I installed the patch and am marking this bug as
done.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Thu, 03 Apr 2014 08:02:02 GMT)
Full text and
rfc822 format available.
Message #61 received at 17149-done <at> debbugs.gnu.org (full text, mbox):
Gents,
Thank you so much for getting to the bottom of this knotty problem and providing me with a solution.
Really impressed.
Please accept hero status.
regards
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Paul Eggert [mailto:eggert <at> cs.ucla.edu]
Sent: 03 April 2014 07:30
To: Pádraig Brady
Cc: Rupert Russell; 17149-done <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
Pádraig Brady wrote:
> The code looks good.
Thanks for the review. I installed the patch and am marking this bug as done.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Thu, 03 Apr 2014 13:40:04 GMT)
Full text and
rfc822 format available.
Message #64 received at 17149 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/03/2014 07:29 AM, Paul Eggert wrote:
> Pádraig Brady wrote:
>> The code looks good.
>
> Thanks for the review. I installed the patch and am marking this bug as done.
A test needs an adjustment also due to assuming that
writing to an empty file was a valid thing for shred to be doing.
The attached should fix it up.
thanks,
Pádraig.
[shred-fix-test.patch (text/x-patch, attachment)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Thu, 03 Apr 2014 16:45:01 GMT)
Full text and
rfc822 format available.
Message #67 received at 17149 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I found a glitch in that previous patch, and installed the attached fixup.
[0001-shred-shred-one-block-even-for-empty-files.patch (text/x-patch, attachment)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 04:41:03 GMT)
Full text and
rfc822 format available.
Message #70 received at 17149 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/03/2014 05:44 PM, Paul Eggert wrote:
> I found a glitch in that previous patch, and installed the attached fixup.
Actually we shouldn't be doing a data pass for empty files at all.
The original version of this utility didn't do that, nor do
I see a reason for doing that. So for correctness/consistency reasons,
I'm applying the attached.
thanks,
Pádraig.
[shred-no-empty.patch (text/x-patch, attachment)]
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 04:54:02 GMT)
Full text and
rfc822 format available.
Message #73 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady wrote:
> if (! INT_ADD_OVERFLOW (size, size_incr))
> size += size_incr;
> + else
> + size = OFF_T_MAX;
A clearer way to write that would be:
size += MIN (size_incr, OFF_T_MAX - size);
Otherwise it looks good.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 10:50:02 GMT)
Full text and
rfc822 format available.
Message #76 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Paul,
I have a confession to make and that is I'm not sure really how to deal with the diff file and now the fixup for the patch.
Googling around, I can see that
patch -p1 < shred1.diff
needs to be run from where the source resides.
But then it's not immediately obvious to me, at my relatively light level of Linux knowledge, if this requires a remake or compilation of the kernel.
If it does, then it's not something I've done before, and depending on where I read, the methods for doing this aren't universally the same.
Might I have a few pointers please?
Thank you.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Paul Eggert [mailto:eggert <at> cs.ucla.edu]
Sent: 03 April 2014 17:45
To: Pádraig Brady
Cc: Rupert Russell; 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
I found a glitch in that previous patch, and installed the attached fixup.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 11:04:02 GMT)
Full text and
rfc822 format available.
Message #79 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/04/2014 11:49 AM, Rupert Russell wrote:
> Paul,
>
> I have a confession to make and that is I'm not sure really how to deal with the diff file and now the fixup for the patch.
> Googling around, I can see that
> patch -p1 < shred1.diff
> needs to be run from where the source resides.
> But then it's not immediately obvious to me, at my relatively light level of Linux knowledge, if this requires a remake or compilation of the kernel.
> If it does, then it's not something I've done before, and depending on where I read, the methods for doing this aren't universally the same.
>
> Might I have a few pointers please?
The patch works around the kernel behavior and so no changes are needed there.
To compile a new shred binary for testing:
mkdir shred-test && cd shred-test
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.22.tar.xz
tar -xf coreutils-8.22.tar.xz
cd coreutils-8.22
wget 'http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=patch;h=88545ad0' -O- | patch -p1
./configure --quiet && make -j $(nproc)
Now the new shred is available in src/shred.
You can run it directly from there or copy to somewhere like /usr/local/bin.
cheers,
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 12:08:02 GMT)
Full text and
rfc822 format available.
Message #82 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig,
Confession number 2, the Linux box I am using for this task isn't connected to the internet.
Could run this on another Linux box that is connected and transfer the resultant shred file across on a stick?
So much to learn.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
The patch works around the kernel behavior and so no changes are needed there.
To compile a new shred binary for testing:
mkdir shred-test && cd shred-test
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.22.tar.xz
tar -xf coreutils-8.22.tar.xz
cd coreutils-8.22
wget 'http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=patch;h=88545ad0' -O- | patch -p1
./configure --quiet && make -j $(nproc)
Now the new shred is available in src/shred.
You can run it directly from there or copy to somewhere like /usr/local/bin.
cheers,
Pádraig.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 12:41:02 GMT)
Full text and
rfc822 format available.
Message #85 received at 17149 <at> debbugs.gnu.org (full text, mbox):
On 04/04/2014 01:07 PM, Rupert Russell wrote:
> Pádraig,
>
> Confession number 2, the Linux box I am using for this task isn't connected to the internet.
> Could run this on another Linux box that is connected and transfer the resultant shred file across on a stick?
Probably if the libc's are compat.
Otherwise if the tape box has a compiler, you could transfer the patch and tarball and compile there.
Otherwise you could look at static compilation on the orig box.
Pádraig.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#17149
; Package
coreutils
.
(Fri, 04 Apr 2014 14:51:01 GMT)
Full text and
rfc822 format available.
Message #88 received at 17149 <at> debbugs.gnu.org (full text, mbox):
Pádraig.
I hadn't realised that there was a network connection in where the tape box is, so now connected to the internet.
I've got the tar file and untarred it.
Now I guess I don't completely follow this line:-
wget 'http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=patch;h=88545ad0' -O- | patch -p1
I've two files from Paul.
shred1.diff
0001-shred-shred-one-block-even-for-empty-files.patch
Do I edit the text in the line where is see the word patch, and insert in both places the shred1.diff file name?
Then do the same with the 0001-shred-shred-one-block-even-for-empty-files.patch after?
Or ..blinding flash of light.. maybe I need to add < shred1.diff to the end of the line
Sorry to be so dim, must be my time of life.
Rupert Russell
OPERATIONS MANAGER
Ilmor Engineering Ltd
Quarry Road, Brixworth, Northamptonshire, NN6 9UB, UK
Direct: +44 (0)1604 799105
Main: +44 (0)1604 799100
Web: www.ilmor.co.uk
-----Original Message-----
From: Pádraig Brady [mailto:P <at> draigBrady.com]
Sent: 04 April 2014 13:41
To: Rupert Russell
Cc: 17149 <at> debbugs.gnu.org
Subject: Re: bug#17149: shred - tape data
On 04/04/2014 01:07 PM, Rupert Russell wrote:
> Pádraig,
>
> Confession number 2, the Linux box I am using for this task isn't connected to the internet.
> Could run this on another Linux box that is connected and transfer the resultant shred file across on a stick?
Probably if the libc's are compat.
Otherwise if the tape box has a compiler, you could transfer the patch and tarball and compile there.
Otherwise you could look at static compilation on the orig box.
Pádraig.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This e-mail may contain confidential and /or privileged information. If you
are not the intended recipient (or have received this e-mail in error)please
notify the sender immediately and destroy this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this email is strictly forbidden.
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Sat, 03 May 2014 11:24:03 GMT)
Full text and
rfc822 format available.
This bug report was last modified 11 years and 43 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.