GNU bug report logs - #19072
cp does not work with davfs2 (fuse) under GNU/Linux; coreutils-8.21

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19072 in the body.
You can then email your comments to 19072 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
To: bug-coreutils <at> gnu.org
Subject: cp does not work with davfs2 (fuse) under GNU/Linux; coreutils-8.21
Date: Sun, 16 Nov 2014 11:22:25 +0100

[Message part 1 (text/plain, inline)]

With my cloud space provider, I have observed that copying to a davfs2
mounted cloud space does only work if using tar, e.g.

tar -cf - sourcefile |tar -xf - -C clouddir

Doing the same (copying) with cp or mv results in the error messages:

cp: error writing ‘/home/kj/MyDrive/sourcefile’: Input/output error
cp: failed to extend ‘/home/kj/MyDrive/sourcefile’: Input/output error

I suppose cp and mv use a file creation method not appropriate for this
kind of device. Either coreutils should detect the device and use a method
better suited, or coreutils should restrain to be plain and simple and make
use of a simple method working everywhere. (IMHO)

[Message part 2 (text/html, inline)]

Message #8 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>, 
 19072 <at> debbugs.gnu.org
Cc: werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 01:15:22 +0000

On 16/11/14 10:22, Klaus-Jürgen Wolf wrote:
> With my cloud space provider, I have observed that copying to a davfs2
> mounted cloud space does only work if using tar, e.g.
> 
> tar -cf - sourcefile |tar -xf - -C clouddir
> 
> Doing the same (copying) with cp or mv results in the error messages:
> 
> cp: error writing ‘/home/kj/MyDrive/sourcefile’: Input/output error
> cp: failed to extend ‘/home/kj/MyDrive/sourcefile’: Input/output error
> 
> I suppose cp and mv use a file creation method not appropriate for this
> kind of device. Either coreutils should detect the device and use a method
> better suited, or coreutils should restrain to be plain and simple and make
> use of a simple method working everywhere. (IMHO)

You seem to be getting an EIO error from ftruncate().
That's used when creating sparse files which is automatically enabled
when the source file is sparse.

Does passing --sparse=never avoid the issue for you?

BTW it's surprising that EIO is returned by davfs2 rather than ENOSYS or ENOTSUP.
Conceivably if davfs2 did return one of those error codes, then cp could fall back
to writing zeros in this unusual case (though it doesn't do that now).

For confirmation of the issue, could you provide the output of this command:

  for i in truncate seek; do
    dd conv=notrunc count=0 seek=1 of=/home/kj/MyDrive/coreutils.test
  done

thanks,
Pádraig.

Message #11 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>, 
 19072 <at> debbugs.gnu.org
Cc: werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 01:23:38 +0000

On 17/11/14 01:15, Pádraig Brady wrote:
> On 16/11/14 10:22, Klaus-Jürgen Wolf wrote:
>> With my cloud space provider, I have observed that copying to a davfs2
>> mounted cloud space does only work if using tar, e.g.
>>
>> tar -cf - sourcefile |tar -xf - -C clouddir
>>
>> Doing the same (copying) with cp or mv results in the error messages:
>>
>> cp: error writing ‘/home/kj/MyDrive/sourcefile’: Input/output error
>> cp: failed to extend ‘/home/kj/MyDrive/sourcefile’: Input/output error
>>
>> I suppose cp and mv use a file creation method not appropriate for this
>> kind of device. Either coreutils should detect the device and use a method
>> better suited, or coreutils should restrain to be plain and simple and make
>> use of a simple method working everywhere. (IMHO)
> 
> You seem to be getting an EIO error from ftruncate().
> That's used when creating sparse files which is automatically enabled
> when the source file is sparse.
> 
> Does passing --sparse=never avoid the issue for you?
> 
> BTW it's surprising that EIO is returned by davfs2 rather than ENOSYS or ENOTSUP.
> Conceivably if davfs2 did return one of those error codes, then cp could fall back
> to writing zeros in this unusual case (though it doesn't do that now).
> 
> For confirmation of the issue, could you provide the output of this command:
> 
>   for i in truncate seek; do
>     dd conv=notrunc count=0 seek=1 of=/home/kj/MyDrive/coreutils.test
>   done

I missed the "error writing" error message above,
which is given by EIO error from a plain old write().
That suggests a communications failure that cp is just reporting.
Is this a repeatable error?
Is any of the file written by cp?
Is the md5sum of the source and destination matching after tar writes the file?

thanks,
Pádraig.

Message #14 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 09:23:36 +0100

[Message part 1 (text/plain, inline)]

Pádraig, wow super, "cp --sparse=never" works though the file I tried with
was in no way sparse. I checked (after umount & mount again) the checksums,
and it's a real success.

The error (before) was absolutely repeatable, it happened always, without
exception, it created a file with 0 bytes length. Same effect when
directories copied recursively, each file was created with zero length.

Thanks a lot,

Yours,
Klaus-Jürgen

[Message part 2 (text/html, inline)]

Message #17 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 09:53:44 +0000

On 17/11/14 08:23, Klaus-Jürgen Wolf wrote:
> Pádraig, wow super, "cp --sparse=never" works though the file I tried with
> was in no way sparse. I checked (after umount & mount again) the checksums,
> and it's a real success.
> 
> The error (before) was absolutely repeatable, it happened always, without
> exception, it created a file with 0 bytes length. Same effect when
> directories copied recursively, each file was created with zero length.

OK, so the "failed to extend" error was extraneous.
BTW I've recently fixed up that issue with:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commit;h=84616da8

Therefore it seems that davfs2 doesn't deal with lseek() well.
The lseek() ran without error, but subsequent write() gives EIO?
There is nothing extra that cp could do in that particular case.

It would be good for comfirmation to get the output from these two commands:

  stat /your/sourcefile
  dd if=/dev/zero conv=notrunc count=1 seek=1 of=/home/kj/MyDrive/eio.test

thanks,
Pádraig.

Message #20 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 15:36:24 +0100

[Message part 1 (text/plain, inline)]

I have to correct my former comment: cp --sparse=never only works with very
small files, e.g. 542 bytes.

(PWD=/home/kj/MyDrive)

$ /bin/cp --sparse=never ~/prj/kaese/cpp/Makefile .
$ stat Makefile
  File: ‘Makefile’
  Size: 542             Blocks: 2          IO Block: 4096   regular file
Device: 28h/40d Inode: 18825536    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      kj)   Gid: ( 1000/      kj)
Access: 2014-11-17 15:27:05.000000000 +0100
Modify: 2014-11-17 15:27:05.000000000 +0100
Change: 2014-11-17 15:27:05.000000000 +0100
 Birth: -

$ dd if=/dev/zero conv=notrunc count=1 seek=1 of=/home/kj/MyDrive/eio.test
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.00248944 s, 206 kB/s

But, with longer files:

$ /bin/cp --sparse=never ~/Tc7200.bin .
/bin/cp: error writing ‘./Tc7200.bin’: Input/output error
/bin/cp: failed to extend ‘./Tc7200.bin’: Input/output error

$ ll ~/Tc7200.bin
-rw-r--r-- 1 kj kj 26296 Sep  8 21:45 /home/kj/Tc7200.bin

I need to say that using tar, I can produce much longer files:

$ stat kaese20141116.7z
  File: ‘kaese20141116.7z’
  Size: 981305          Blocks: 1917       IO Block: 4096   regular file
Device: 28h/40d Inode: 18802464    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      kj)   Gid: ( 1000/      kj)
Access: 2014-11-16 11:13:20.000000000 +0100
Modify: 2014-11-16 11:12:48.000000000 +0100
Change: 2014-11-16 11:12:55.000000000 +0100
 Birth: -

[Message part 2 (text/html, inline)]

Message #23 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 16:11:16 +0000

On 17/11/14 14:36, Klaus-Jürgen Wolf wrote:
> I have to correct my former comment: cp --sparse=never only works with very small files, e.g. 542 bytes.
> 
> (PWD=/home/kj/MyDrive)
> 
> $ /bin/cp --sparse=never ~/prj/kaese/cpp/Makefile .
> $ stat Makefile
>   File: ‘Makefile’
>   Size: 542             Blocks: 2          IO Block: 4096   regular file
> Device: 28h/40d Inode: 18825536    Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/      kj)   Gid: ( 1000/      kj)
> Access: 2014-11-17 15:27:05.000000000 +0100
> Modify: 2014-11-17 15:27:05.000000000 +0100
> Change: 2014-11-17 15:27:05.000000000 +0100
>  Birth: -
> 
> $ dd if=/dev/zero conv=notrunc count=1 seek=1 of=/home/kj/MyDrive/eio.test
> 1+0 records in
> 1+0 records out
> 512 bytes (512 B) copied, 0.00248944 s, 206 kB/s
> 
> But, with longer files:
> 
> $ /bin/cp --sparse=never ~/Tc7200.bin .
> /bin/cp: error writing ‘./Tc7200.bin’: Input/output error
> /bin/cp: failed to extend ‘./Tc7200.bin’: Input/output error
> 
> $ ll ~/Tc7200.bin
> -rw-r--r-- 1 kj kj 26296 Sep  8 21:45 /home/kj/Tc7200.bin
> 
> I need to say that using tar, I can produce much longer files:
> 
> $ stat kaese20141116.7z 
>   File: ‘kaese20141116.7z’
>   Size: 981305          Blocks: 1917       IO Block: 4096   regular file
> Device: 28h/40d Inode: 18802464    Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/      kj)   Gid: ( 1000/      kj)
> Access: 2014-11-16 11:13:20.000000000 +0100
> Modify: 2014-11-16 11:12:48.000000000 +0100
> Change: 2014-11-16 11:12:55.000000000 +0100
>  Birth: -
> 

With --sparse=never we really should only be doing write()s to the destination.
That could be confirmed with:

  strace cp --sparse=never ~/Tc7200.bin .

The only thing left that I can think of is the size of the writes is causing a problem.
cp will read and write 64KiB by default. tar may be using smaller than this and so
not triggering the issue?

I searched for that issue and noticed this fuse issue:
http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/14095

thanks,
Pádraig.

Message #26 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 17:48:08 +0100

[Message part 1 (text/plain, inline)]

execve("/bin/cp", ["cp", "--sparse=never", "/home/kj/Tc7200.bin",
"MyDrive"], [/* 96 vars */]) = 0
brk(0)                                  = 0x9eb000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2fc02ed000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=319914, ...}) = 0
mmap(NULL, 319914, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2fc0298000
close(3)                                = 0
open("/lib64/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\0\0\0\0\0\0"..., 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size=35272, ...}) = 0
mmap(NULL, 2130592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7f2fbfeb8000
mprotect(0x7f2fbfec0000, 2093056, PROT_NONE) = 0
mmap(0x7f2fc00bf000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7f2fc00bf000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 O\2\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1728864, ...}) = 0
mmap(NULL, 3841112, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7f2fbfb08000
mprotect(0x7f2fbfca9000, 2093056, PROT_NONE) = 0
mmap(0x7f2fbfea8000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a0000) = 0x7f2fbfea8000
mmap(0x7f2fbfeae000, 15448, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2fbfeae000
close(3)                                = 0
open("/lib64/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\26\0\0\0\0\0\0"..., 832) =
832
fstat(3, {st_mode=S_IFREG|0755, st_size=18616, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2fc02ec000
mmap(NULL, 2113904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7f2fbf900000
mprotect(0x7f2fbf904000, 2093056, PROT_NONE) = 0
mmap(0x7f2fbfb03000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f2fbfb03000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2fc02eb000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2fc02e7000
arch_prctl(ARCH_SET_FS, 0x7f2fc02eb700) = 0
mprotect(0x7f2fbfea8000, 16384, PROT_READ) = 0
mprotect(0x7f2fbfb03000, 4096, PROT_READ) = 0
mprotect(0x7f2fc00bf000, 4096, PROT_READ) = 0
mprotect(0x61a000, 4096, PROT_READ)     = 0
mprotect(0x7f2fc02e8000, 4096, PROT_READ) = 0
munmap(0x7f2fc0298000, 319914)          = 0
brk(0)                                  = 0x9eb000
brk(0xa0c000)                           = 0xa0c000
open("/usr/lib64/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3847568, ...}) = 0
mmap(NULL, 3847568, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2fbf550000
close(3)                                = 0
geteuid()                               = 1000
stat("MyDrive", {st_mode=S_IFDIR|0755, st_size=496, ...}) = 0
stat("/home/kj/Tc7200.bin", {st_mode=S_IFREG|0644, st_size=26296, ...}) = 0
stat("MyDrive/Tc7200.bin", 0x7fff14fec6e0) = -1 ENOENT (No such file or
directory)
open("/home/kj/Tc7200.bin", O_RDONLY)   = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=26296, ...}) = 0
open("MyDrive/Tc7200.bin", O_WRONLY|O_CREAT|O_EXCL, 0644) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
read(3, "<\2401\344kT*\3\347\233\316Oz[\255\3756u9E9eWF0bt9Y8Rw"..., 65536)
= 26296
write(4, "<\2401\344kT*\3\347\233\316Oz[\255\3756u9E9eWF0bt9Y8Rw"...,
26296) = -1 EIO (Input/output error)
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GETFD)                       = 0x1 (flags FD_CLOEXEC)
fstat(5, {st_mode=S_IFREG|0644, st_size=2492, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2fc02e6000
read(5, "# Locale name alias data base.\n#"..., 4096) = 2492
read(5, "", 4096)                       = 0
close(5)                                = 0
munmap(0x7f2fc02e6000, 4096)            = 0
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/lib64/charset.alias", O_RDONLY|O_NOFOLLOW) = -1 ENOENT (No such
file or directory)
write(2, "cp: ", 4)                     = 4
write(2, "error writing \342\200\230MyDrive/Tc7200."..., 38) = 38
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
write(2, ": Input/output error", 20)    = 20
write(2, "\n", 1)                       = 1
write(2, "cp: ", 4)                     = 4
write(2, "failed to extend \342\200\230MyDrive/Tc72"..., 41) = 41
write(2, ": Input/output error", 20)    = 20
write(2, "\n", 1)                       = 1
close(4)                                = 0
close(3)                                = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

[Message part 2 (text/html, inline)]

Message #29 received at 19072 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Klaus-Jürgen Wolf <kjwolf <at> arcor.de>
Cc: 19072 <at> debbugs.gnu.org, werner.baumann <at> onlinehome.de
Subject: Re: bug#19072: cp does not work with davfs2 (fuse) under GNU/Linux;
 coreutils-8.21
Date: Mon, 17 Nov 2014 17:12:58 +0000

tag 19072 notabug
close 19072
stop

On 17/11/14 16:48, Klaus-Jürgen Wolf wrote:
> read(3, "<\2401\344kT*\3\347\233\316Oz[\255\3756u9E9eWF0bt9Y8Rw"..., 65536) = 26296
> write(4, "<\2401\344kT*\3\347\233\316Oz[\255\3756u9E9eWF0bt9Y8Rw"..., 26296) = -1 EIO (Input/output error)

Looks to be that kernel fuse bug I referenced,
so closing here.

thanks,
Pádraig

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.