GNU bug report logs - #8636
cp -x flag does not work

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8636 in the body.
You can then email your comments to 8636 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: obb33 <at> verizon.net
To: bug-coreutils <at> gnu.org
Subject: cp -x flag does not work
Date: Sat, 07 May 2011 12:51:31 -0400

coreutils 8.12, i don't know if this problem existed in previous versions.


copying the root directory ( / ) of a running linux system using cp -xdvpR
cp followed a directory that was mounted via NFS under / . the copy was
stopped before any other directories on other filesystems were copied so i
can't speak about how cp would handle those mounts.



fractoid

Message #8 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: obb33 <at> verizon.net
Cc: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: cp -x flag does not work
Date: Sun, 08 May 2011 19:32:08 +0200

obb33 <at> verizon.net wrote:
> coreutils 8.12, i don't know if this problem existed in previous versions.
>
> copying the root directory ( / ) of a running linux system using cp -xdvpR
> cp followed a directory that was mounted via NFS under / . the copy was
> stopped before any other directories on other filesystems were copied so i
> can't speak about how cp would handle those mounts.

tags 8636 + moreinfo
thanks

Thanks for the report.

AFAIK, cp's -x (--one-file-system) option does work.
If it's not working for you, perhaps there is something
unusual about your situation.  What does "df" say about
the NFS-mounted directory that was mistakenly copied?

To test it without copying data unnecessarily, I ran this:

    mkdir junk && cp -srx / junk

That makes symlinks to the same files that a one-file-system "cp" would
have copied.  The only nonempty directories in junk/ correspond to ones
in "/" that reside on the same file system as "/".  So at least for me,
it works as advertised.  Note that copying each top-level mount point
directory is expected -- those are all empty in the destination (junk/
in my case).

Please provide more details, including
  - the precise command you ran
  - a sample of the output showing it copying a file that it shouldn't
  - what this prints: stat --format=%04D:%n / DIR
      where DIR is an NFS-mounted directory that was mistakenly copied

Message #11 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: obb33 <at> verizon.net
To: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: cp -x flag does not work
Date: Mon, 09 May 2011 06:45:34 -0400

context: copy a linux system to a directory where, after chrooting to that
directory, i could safely  torture and mangle the duplicate system.

the command used was cp -xvdpR /*   /newdirectory

where / contains mounted other(not on the drive / is on) filesystems. these
are nfs mounts(direct and via ssh), mounted hard drives, and mounted loop
drives.

the guilty party is apparently the wildcard

the command as given above copies everything and descends all directories on
all mounted file systems under / no matter what they are.

the command:  cp -xvdp /<otherfs-dir>/*   /newdirectory

descends into the top level of the  otherfs dir and copies the files
therein, but does not descend into other directories.

so the wildcard essentially negates -x and this was not anticipated
behavior. checking the docs i did not find any mention of this.

if this is WAD, then it should be documented, however, i don't think that
someone using the -x wants a wildcard to undo that spec especially in a dir
with several mounts on subdirectories.

however, it is also probably the case that most people wouldn't put a
wildcard in there to do the job anyway(given a -R)...but i'm weird :-)
(well, actually, i thought that it would copy the / itself which would be a
mess, but there may be other situations where it's use is warranted ).

the latter part of the above comment suggests that i have to test what
happens when using ? wildcards.

the command:   cp -xvdpR /mnt1? /newdir

 where there are mounted drives on /mnt1, /mnt11,/mnt12, descended into and
copied everything on all 3 fs. without the -R flag the 3 directories were
excluded.

i didn't provide the items you requested because in the context of the above
discoveries i didn't think it was relevant, this happens no matter what the
drives are.

any other info you need or testing done, let me know, and thanks

fractoid



On Sun, May 08, 2011 at 07:32:08PM +0200, Jim Meyering wrote:
> obb33 <at> verizon.net wrote:
> > coreutils 8.12, i don't know if this problem existed in previous versions.
> >
> > copying the root directory ( / ) of a running linux system using cp -xdvpR
> > cp followed a directory that was mounted via NFS under / . the copy was
> > stopped before any other directories on other filesystems were copied so i
> > can't speak about how cp would handle those mounts.
> 
> tags 8636 + moreinfo
> thanks
> 
> Thanks for the report.
> 
> AFAIK, cp's -x (--one-file-system) option does work.
> If it's not working for you, perhaps there is something
> unusual about your situation.  What does "df" say about
> the NFS-mounted directory that was mistakenly copied?
> 
> To test it without copying data unnecessarily, I ran this:
> 
>     mkdir junk && cp -srx / junk
> 
> That makes symlinks to the same files that a one-file-system "cp" would
> have copied.  The only nonempty directories in junk/ correspond to ones
> in "/" that reside on the same file system as "/".  So at least for me,
> it works as advertised.  Note that copying each top-level mount point
> directory is expected -- those are all empty in the destination (junk/
> in my case).
> 
> Please provide more details, including
>   - the precise command you ran
>   - a sample of the output showing it copying a file that it shouldn't
>   - what this prints: stat --format=%04D:%n / DIR
>       where DIR is an NFS-mounted directory that was mistakenly copied

Message #14 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: obb33 <at> verizon.net
Cc: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: cp -x flag does not work
Date: Mon, 09 May 2011 14:07:49 +0200

tags 8636 + notabug
thanks

obb33 <at> verizon.net wrote:
> context: copy a linux system to a directory where, after chrooting to that
> directory, i could safely  torture and mangle the duplicate system.
>
> the command used was cp -xvdpR /*   /newdirectory
>
> where / contains mounted other(not on the drive / is on) filesystems. these
> are nfs mounts(direct and via ssh), mounted hard drives, and mounted loop
> drives.
>
> the guilty party is apparently the wildcard
>
> the command as given above copies everything and descends all directories on
> all mounted file systems under / no matter what they are.
>
> the command:  cp -xvdp /<otherfs-dir>/*   /newdirectory

Thanks for explaining.
With -x, cp considers each specified source directory when applying this rule:

    `-x'
    `--one-file-system'
         Skip subdirectories that are on different file systems from the
         one that the copy started on.  However, mount point directories
         _are_ copied.

So this might have been what you intended, assuming that /newdirectory
is not on the same file system as "/":

    cp -xvdp / /newdirectory

Even if it is on the same file system, it should work,
but you should get a diagnostic about cp refusing to copy
/newdirectory into itself.

> descends into the top level of the  otherfs dir and copies the files
> therein, but does not descend into other directories.
>
> so the wildcard essentially negates -x and this was not anticipated
> behavior. checking the docs i did not find any mention of this.
>
> if this is WAD, then it should be documented, however, i don't think that
> someone using the -x wants a wildcard to undo that spec especially in a dir
> with several mounts on subdirectories.

I think it even works as _documented_ (in the "info" documentation) ;-)
Yes, the five-word description in --help (and hence in "man cp") is too
succinct, but at the end of that man page, you should see a note explaining
that "info cp" will give you the real documentation.

Would you like to suggest wording that would make it clearer?

Message #19 received at 8636-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: 8636-done <at> debbugs.gnu.org
Subject: Re: bug#8636: cp -x flag does not work
Date: Tue, 13 Sep 2011 14:22:28 +0200

tags 8636 + notabug
thanks

Jim Meyering wrote:
> tags 8636 + notabug
> thanks
...

Marking this as done and (again) notabug.

Message #22 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: peek <at> kutek.info
To: Jim Meyering <jim <at> meyering.net>
Cc: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: closed (Re: bug#8636: cp -x flag does not work)
Date: Thu, 15 Sep 2011 08:32:42 -0400

Hi Jim,

I would ask you again to reconsider your view on this issue  because it is
irrational. 

you are claiming functionality that you do not provide (stay on one file
system during a potentially recursive copy ) while allowing recursion into
other file systems if they are mounted beneath the directory you are
copying. 

why you think that this constitutes staying  on one filesystem eludes me.

find does this correctly ( -xdev or -mount switches) and coreutil's cp and
mv can and should too.

thanks for all the great work on coreutils and other of your work. if i had
programming skills, i would fix this myself and provide a patch, but i
don't.


best regards,

fractoid



On Tue, Sep 13, 2011 at 12:27:02PM +0000, GNU bug Tracking System wrote:
> Your bug report
> 
> #8636: cp -x flag does not work
> 
> which was filed against the coreutils package, has been closed.
> 
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 8636 <at> debbugs.gnu.org.
> 
> -- 
> 8636: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8636
> GNU Bug Tracking System
> Contact help-debbugs <at> gnu.org with problems

> Date: Tue, 13 Sep 2011 14:22:28 +0200
> From: Jim Meyering <jim <at> meyering.net>
> Subject: Re: bug#8636: cp -x flag does not work
> To: 8636-done <at> debbugs.gnu.org
> 
> tags 8636 + notabug
> thanks
> 
> Jim Meyering wrote:
> > tags 8636 + notabug
> > thanks
> ...
> 
> Marking this as done and (again) notabug.
> 

> Date: Sat, 07 May 2011 12:51:31 -0400
> From: obb33 <at> verizon.net
> Subject: cp -x flag does not work
> To: bug-coreutils <at> gnu.org
> 
> coreutils 8.12, i don't know if this problem existed in previous versions.
> 
> 
> copying the root directory ( / ) of a running linux system using cp -xdvpR
> cp followed a directory that was mounted via NFS under / . the copy was
> stopped before any other directories on other filesystems were copied so i
> can't speak about how cp would handle those mounts.
> 
> 
> 
> fractoid
> 
> 

Message #25 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: peek <at> kutek.info
Cc: 8636 <at> debbugs.gnu.org, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#8636: closed (Re: bug#8636: cp -x flag does not work)
Date: Thu, 15 Sep 2011 17:14:42 +0100

On 09/15/2011 01:32 PM, peek <at> kutek.info wrote:
> Hi Jim,
> 
> I would ask you again to reconsider your view on this issue  because it is
> irrational.

Reading back over the thread, I think it's a usage error.
Note that cp does not process the wildcard, so that when you do:

  cp -x /*

what cp is really getting from the shell is:

  cp -x /dir1 /dir2 ...

cp is right to treat each of those independently.

cheers,
Pádraig.

Message #28 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: peek <at> kutek.info
Cc: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: closed (Re: bug#8636: cp -x flag does not work)
Date: Thu, 15 Sep 2011 18:18:48 +0200

peek <at> kutek.info wrote:
> I would ask you again to reconsider your view on this issue  because it is
> irrational.
>
> you are claiming functionality that you do not provide (stay on one file
> system during a potentially recursive copy ) while allowing recursion into
> other file systems if they are mounted beneath the directory you are
> copying.
>
> why you think that this constitutes staying  on one filesystem eludes me.
>
> find does this correctly ( -xdev or -mount switches) and coreutil's cp and
> mv can and should too.
>
> thanks for all the great work on coreutils and other of your work. if i had
> programming skills, i would fix this myself and provide a patch, but i
> don't.

You used this command:

    cp -xvdpR /* /newdirectory

That tells cp explicitly to copy every top-level directory under /.
i.e., for each directory one level under /, cp copies it to
newdirectory, but without traversing into a directory residing
on a device different than the command-line-specified directory.
That is how cp -x is documented to work.

To get the behavior you want, run this command:

    cp -xvdpR / /newdirectory

(note there's no "*" here, as you surmised in
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8636#11
your use of the "*" wildcard is what caused problems)

I hope you see that there is nothing to reconsider
and that cp -x can do precisely what you want.

Message #31 received at 8636 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: peek <at> kutek.info
Cc: 8636 <at> debbugs.gnu.org
Subject: Re: bug#8636: cp -x flag does not work
Date: Thu, 15 Sep 2011 10:33:24 -0600

peek <at> kutek.info wrote:
> you are claiming functionality that you do not provide (stay on one file
> system during a potentially recursive copy ) while allowing recursion into
> other file systems if they are mounted beneath the directory you are
> copying. 
> 
> why you think that this constitutes staying  on one filesystem eludes me.

Let me help with this.  You said the command you were using was:

>> copying the root directory ( / ) of a running linux system using cp -xdvpR
>> cp followed a directory that was mounted via NFS under / .

This failure was not reproducible.  More information was requested.
In that follow-up you said the actual command you used was:

>> the command used was cp -xvdpR /*   /newdirectory
>> the guilty party is apparently the wildcard
>> the command as given above copies everything and descends all directories on
>> all mounted file systems under / no matter what they are.

That is correct.  And that is the reason cp is doing the right thing
and this is not a bug in cp.  You have specifically requested that the
mount point be copied.

You may be misunderstanding how how wildcards work.  The shell expands
wildcards before invoking the command.  You can see this using the
echo command.

  $ echo somecommand /*
  
The 'somecommand' does not see the '*' at all.  The shell sees the '*'
and expands it to match files.  The names of the files replace the
string on the command line.  The modified list of arguments is passed
to the command when it is invoked.  The command does not know that a
wildcard was involved.

When you call

  cp -xvdpR /*   /newdirectory

The cp command sees something like (using ... to indicate more files):

  cp -xvdpR /bin /boot /dev /etc ... /mnt ...   /newdirectory

Since cp was asked to recursively copy /mnt it will copy /mnt and that
is regardless of the -x option.  The documentation for -x is:

         Skip subdirectories that are on different file systems from
         the one that the copy started on.

The cp command was told to copy /mnt and relating to this
documentation that is where the copy starts.  If there is a mount
point below that directory then the -x will instruct cp to skip it.
But since /mnt was explicitly stated in the cp command it is not below
anything and will not be skipped.

If you were to remove the wildcard and specify only the root directory
then the -x would skip any mount points below it.

  cp -xvdpR /  /newdirectory

There cp would be given only the / directory and the /mnt mount point
below it would be skipped.

> find does this correctly ( -xdev or -mount switches) and coreutil's cp and
> mv can and should too.

No it does not.  Try 'find' on your mount point.  I will use the
"-maxdepth 2" to limit the amount of runtime only.  But if run find
using the same "/*" you used with cp then find will have the same
behavior.

  find /* -xdev -maxdepth 2 -print

Hope this helps.

Bob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.