GNU bug report logs - #20603
Possible bug in cp

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 20603 in the body.
You can then email your comments to 20603 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Chris Puttick <cputtick <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: Possible bug in cp
Date: Mon, 18 May 2015 14:04:20 +0100

Hi

In moment of tiredness issued a cp command similar to

cp /path/to/files/*

neglecting to add a destination (should have been ./)

The files in the directory were 2 vdisks both ending .qcow2.

No error message was generated and the apparent result of running the
command is that the file whose name came first alphabetically was
copied over the other.

The installed OS on the machine was Ubuntu 12.04 and cp version 7.4.

I can provide more details on request.

Regards

Chris

-- 
@putt1ck
putt1ck.blogspot.com
http://twoten.is
skype: putt1ck

Opinions in this email are my own and may not reflect that of my
clients, past employers, associates, friends, family, pets etc..

Message #8 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Chris Puttick <cputtick <at> gmail.com>, 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Mon, 18 May 2015 16:55:40 +0100

tag 20603 notabug
close 20603
stop

On 18/05/15 14:04, Chris Puttick wrote:
> Hi
> 
> In moment of tiredness issued a cp command similar to
> 
> cp /path/to/files/*
> 
> neglecting to add a destination (should have been ./)
> 
> The files in the directory were 2 vdisks both ending .qcow2.
> 
> No error message was generated and the apparent result of running the
> command is that the file whose name came first alphabetically was
> copied over the other.
> 
> The installed OS on the machine was Ubuntu 12.04 and cp version 7.4.
> 
> I can provide more details on request.

You were unlucky that the glob expanded to two files.
With more files you would have gotten an error.
However the expansion is done before cp executes,
and so there is no opportunity to behave differently here.

thanks,
Pádraig

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Chris Puttick <cputtick <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: Re: Possible bug in cp
Date: Mon, 18 May 2015 15:49:43 +0100

Follow up, reproducing:

tested on openSuSE Factory

Do:

# mkdir test
# echo "this is file 1" > test/file1
# touch test/file2
# cp test/*
# cat test/file2

If there is a 3rd file in the directory you get an error:

"cp: target ‘test/file3’ is not a directory"



On 18 May 2015 at 14:04, Chris Puttick <cputtick <at> gmail.com> wrote:
> Hi
>
> In moment of tiredness issued a cp command similar to
>
> cp /path/to/files/*
>
> neglecting to add a destination (should have been ./)
>
> The files in the directory were 2 vdisks both ending .qcow2.
>
> No error message was generated and the apparent result of running the
> command is that the file whose name came first alphabetically was
> copied over the other.
>
> The installed OS on the machine was Ubuntu 12.04 and cp version 7.4.
>
> I can provide more details on request.
>
> Regards
>
> Chris
>
> --
> @putt1ck
> putt1ck.blogspot.com
> http://twoten.is
> skype: putt1ck
>
> Opinions in this email are my own and may not reflect that of my
> clients, past employers, associates, friends, family, pets etc..



-- 
@putt1ck
putt1ck.blogspot.com
http://twoten.is
skype: putt1ck

Opinions in this email are my own and may not reflect that of my
clients, past employers, associates, friends, family, pets etc..

Message #14 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Chris Puttick <cputtick <at> gmail.com>, 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Mon, 18 May 2015 18:00:33 +0100

On 18/05/15 15:49, Chris Puttick wrote:
> Follow up, reproducing:
> 
> tested on openSuSE Factory
> 
> Do:
> 
> # mkdir test
> # echo "this is file 1" > test/file1
> # touch test/file2
> # cp test/*
> # cat test/file2
> 
> If there is a 3rd file in the directory you get an error:
> 
> "cp: target ‘test/file3’ is not a directory"

Note you can prompt before overwriting existing files like:

cp -i file1 file2

cheers,
Pádraig

Message #17 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Chris Puttick <cputtick <at> gmail.com>
Cc: 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Mon, 18 May 2015 17:24:17 -0600

Chris Puttick wrote:
> In moment of tiredness issued a cp command similar to
> cp /path/to/files/*
> neglecting to add a destination (should have been ./)
> The files in the directory were 2 vdisks both ending .qcow2.

Ouch!  I am sorry for your loss.  I hope you had backups. :-(

> No error message was generated and the apparent result of running the
> command is that the file whose name came first alphabetically was
> copied over the other.

There was no error message because as far as the computer was
concerned it did exactly as you instructed it and no error occurred.

You are apparently not aware that the shell reads your command line,
parses it, expands it, and then the shell executes the resulting
command.  Many command line characters are "shell metacharacters".
Search for that and you will find many references.  When I say shell
here I will assume the bash shell however this part applies to all of
the Unix-like command line shells such as ksh, zsh, csh, sh and so forth.

One of the file glob characters is the "*" character.  (It is called a
file glob because the star is expanded to match a glob of files.)
Whenever you use a '*' in a command line that is an instruction to the
shell.  It tells the shell to list the files and match them and
replace the star character with the list of matching file names.  Try
this exercise to understand a little bit about the '*' character and
what the shell does.

  $ mkdir /tmp/junk
  $ cd /tmp/junk
  $ touch file1
  $ echo *
  file1
  $ touch file2
  $ echo *
  file1 file2
  $ touch file 3
  $ echo *
  file1 file2 file3
  $ echo *1
  file1
  $ echo *[23]
  file2 file3

As you can see the shell is replacing the '*' character with the files
that were expanded which match it.  And I threw in another couple of
file expansions too just to help push the concept home.

By this point you should know that your cp command had a '*' in the
command line.  The shell expanded that star.  There were only two
expansions.  Therefore the shell invoked cp with two arguments.

  cp /path/to/files/file1 /path/to/files/file2

That is the command that cp was invoked with after the shell expanded
the file globs on the command line.  As far as the cp command is
concerned it was given a command, cp executed the command, and the
command was completed without error.

The cp command has no way of knowing that you wanted to execute this
command instead.

  cp /path/to/files/file1 /path/to/files/file2 ./

How could it?  It can't.  One command looks just the same as the other
one to the cp command.  None of the commands ever see the '*'
character because it is expanded by the shell and replaced before the
utility is invoked.

Hope that expansion helps explain things.

Bob

Message #20 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Chris Puttick <cputtick <at> gmail.com>
To: Bob Proulx <bob <at> proulx.com>
Cc: 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Tue, 19 May 2015 06:21:03 +0100

The expansion & consequences of my typo understood! However, given the
risks inherent in this edge case (directory only has 2 files in it)
and the unlikelihood of someone wanting to change a directory
containing 2 different files into a directory containing 2 identical
but differently named files, it would be great if the cp command to
check, when the source and destination directories are the same, the
file count in that directory and issue a warning before continuing if
the file count =2.

I guess this would apply to others where the consequences of a similar
typo would be bad, like mv.

On 19 May 2015 at 00:24, Bob Proulx <bob <at> proulx.com> wrote:
> Chris Puttick wrote:
>> In moment of tiredness issued a cp command similar to
>> cp /path/to/files/*
>> neglecting to add a destination (should have been ./)
>> The files in the directory were 2 vdisks both ending .qcow2.
>
> Ouch!  I am sorry for your loss.  I hope you had backups. :-(
>
>> No error message was generated and the apparent result of running the
>> command is that the file whose name came first alphabetically was
>> copied over the other.
>
> There was no error message because as far as the computer was
> concerned it did exactly as you instructed it and no error occurred.
>
> You are apparently not aware that the shell reads your command line,
> parses it, expands it, and then the shell executes the resulting
> command.  Many command line characters are "shell metacharacters".
> Search for that and you will find many references.  When I say shell
> here I will assume the bash shell however this part applies to all of
> the Unix-like command line shells such as ksh, zsh, csh, sh and so forth.
>
> One of the file glob characters is the "*" character.  (It is called a
> file glob because the star is expanded to match a glob of files.)
> Whenever you use a '*' in a command line that is an instruction to the
> shell.  It tells the shell to list the files and match them and
> replace the star character with the list of matching file names.  Try
> this exercise to understand a little bit about the '*' character and
> what the shell does.
>
>   $ mkdir /tmp/junk
>   $ cd /tmp/junk
>   $ touch file1
>   $ echo *
>   file1
>   $ touch file2
>   $ echo *
>   file1 file2
>   $ touch file 3
>   $ echo *
>   file1 file2 file3
>   $ echo *1
>   file1
>   $ echo *[23]
>   file2 file3
>
> As you can see the shell is replacing the '*' character with the files
> that were expanded which match it.  And I threw in another couple of
> file expansions too just to help push the concept home.
>
> By this point you should know that your cp command had a '*' in the
> command line.  The shell expanded that star.  There were only two
> expansions.  Therefore the shell invoked cp with two arguments.
>
>   cp /path/to/files/file1 /path/to/files/file2
>
> That is the command that cp was invoked with after the shell expanded
> the file globs on the command line.  As far as the cp command is
> concerned it was given a command, cp executed the command, and the
> command was completed without error.
>
> The cp command has no way of knowing that you wanted to execute this
> command instead.
>
>   cp /path/to/files/file1 /path/to/files/file2 ./
>
> How could it?  It can't.  One command looks just the same as the other
> one to the cp command.  None of the commands ever see the '*'
> character because it is expanded by the shell and replaced before the
> utility is invoked.
>
> Hope that expansion helps explain things.
>
> Bob



-- 
@putt1ck
putt1ck.blogspot.com
http://twoten.is
skype: putt1ck

Opinions in this email are my own and may not reflect that of my
clients, past employers, associates, friends, family, pets etc..

Message #23 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Chris Puttick <cputtick <at> gmail.com>, Bob Proulx <bob <at> proulx.com>
Cc: 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Tue, 19 May 2015 08:02:50 +0200

On 05/19/2015 07:21 AM, Chris Puttick wrote:
> The expansion & consequences of my typo understood! However, given the
> risks inherent in this edge case (directory only has 2 files in it)
> and the unlikelihood of someone wanting to change a directory
> containing 2 different files into a directory containing 2 identical
> but differently named files, it would be great if the cp command to
> check, when the source and destination directories are the same, the
> file count in that directory and issue a warning before continuing if
> the file count =2.

If there was such a check, a warning wouldn't have helped you - as
the command would have proceeded with overwriting the file anyway.
Second, how should cp know when the user really wants to do that?
We'd have to special case this, too.  Third, such a check would be
racy.  After all, we won't add such a construct.  cp(1) performed
exactly the option you told it to do.

But there is other help:
I rather suggest that you to consider using the -n or -i option:

  -i, --interactive            prompt before overwrite (overrides a previous -n
                                  option)

  -n, --no-clobber             do not overwrite an existing file (overrides
                                 a previous -i option)

You can even set an alias for it like this in your profile:
  alias cp='\cp -n'

Or, if in doubt, pass the command line to echo before executing,
by which you would see what would happen:

  $ echo cp /some/dir/file*
  cp /some/dir/file1 /some/dir/file2

Have a nice day,
Berny

Message #26 received at 20603 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Chris Puttick <cputtick <at> gmail.com>
Cc: 20603 <at> debbugs.gnu.org
Subject: Re: bug#20603: Possible bug in cp
Date: Tue, 19 May 2015 01:13:54 -0600

Chris Puttick wrote:
> The expansion & consequences of my typo understood! However, given the
> risks inherent in this edge case (directory only has 2 files in it)
> and the unlikelihood of someone wanting to change a directory
> containing 2 different files into a directory containing 2 identical
> but differently named files, it would be great if the cp command to
> check, when the source and destination directories are the same, the
> file count in that directory and issue a warning before continuing if
> the file count =2.

So I think you are telling me that if I were to do this:

  mkdir /tmp/junk
  touch /tmp/junk/file1 /tmp/junk/file2
  cp /tmp/junk/file1 /tmp/junk/file2

You are suggesting that the cp command above would fail in the above
situation?  I am sorry but that is much to specific of something to
single out as a special case in the code.  And it would fail in most
perfectly valid situations.  Like the above.  The above is perfectly
valid and should not fail.

Additionally I often do things like this for example:

  cp foo.conf foo.conf.bak
  edit foo.conf
  cp foo.conf.bak foo.conf

It would be wrong if the above were to fail.

Additionally what about this case:

  mkdir /tmp/junk && cd /tmp/junk
  touch one.txt two.txt
  touch README
  cp *.txt

The shell will expand "cp *.txt" to be "cp one.txt two.txt".  The file
count for the directory will be three and therefore the suggested
special case rule would not be triggered.

I daresay that the suggested special case rule would only very rarely
be triggered when it would help and would be a false positive for most
valid cases.

I am sorry that you had a bad experience with file glob expansion.
But so far the suggestions for improvement hasn't made a compelling
argument yet.

Bob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.