GNU bug report logs - #24011
parallel find error

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24011 in the body.
You can then email your comments to 24011 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: jklowden <jklowden <at> schemamania.org>
To: bug-coreutils <at> gnu.org
Subject: parallel find error
Date: Sat, 16 Jul 2016 15:42:15 -0400 (EDT)

There appears to be an error in the parallel execution of find.  I
created a minimal example below.

I realize the software is old.  I was unable to find a bug report
related to it.

	--jkl

basename (GNU coreutils) 8.21
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

$ find .. -type f -name lx[tr]\*.1
../appl/xslt/lxt.1
../appl/xslt/lxreplace.1
../appl/fsgmatch/lxtransduce.1

$ find .. -type f -name lx[tr]\*.1 -exec basename {} \;
lxt.1
lxreplace.1
lxtransduce.1

$ find .. -type f -name lx[tr]\*.1 -exec basename {} +
basename: extra operand ‘../appl/fsgmatch/lxtransduce.1’
Try 'basename --help' for more information.

Message #8 received at 24011 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: jklowden <jklowden <at> schemamania.org>
Cc: 24011 <at> debbugs.gnu.org
Subject: Re: bug#24011: parallel find error
Date: Sat, 16 Jul 2016 19:55:03 -0400

tag 24011 notabug
close 24011
stop

Hello,

> On Jul 16, 2016, at 15:42, jklowden <jklowden <at> schemamania.org> wrote:
> 
> There appears to be an error in the parallel execution of find.  I
> created a minimal example below.
> 
> [...]
> 
> $ find .. -type f -name lx[tr]\*.1 -exec basename {} +
> basename: extra operand ‘../appl/fsgmatch/lxtransduce.1’
> Try 'basename --help' for more information.

There are several issues here, though none of them are bugs.

First,
The difference between "find -exec basename {} \;" and "find -exec basename {} +" :
The former will execute the program 'basename' once per file,
and the latter will execute it fewer times with multiple files as parameters.

The following will demonstrate:

   $ mkdir c
   $ touch c/1 c/2 c/3 c/4
   $ find c -type f
   c/1
   c/2
   c/3
   c/4

Compare:

   $ find c -type f -exec echo basename {} \;
   basename c/1
   basename c/2
   basename c/3
   basename c/4

   $ find c -type f -exec echo basename {} +
   basename c/1 c/2 c/3 c/4


Second,
the "basename" command by default accepts a single parameter (pathname) or two parameters (pathname and optional extension).
Running it with three or more parameters will cause the error you were experiencing:

  $ basename c/a.txt 
  a.txt

  $ basename c/a.txt .txt
  a

  $ basename c/a.txt .txt c
  basename: extra operand ‘c’
  Try 'basename --help' for more information.

That explains your error: using the "+" syntax, "find" executed 'basename' with multiple filenames, and basename rejected it as invalid syntax.

However,
basename from coreutils version 8.16 and newer (released in 2012) supports the "-a" option (gnu extension) that handles multiple filenames on the same command line.
Compare:

   $ basename c/a.txt c/b.txt c/c.txt
   basename: extra operand ‘c/c.txt’
   Try 'basename --help' for more information.

   $ basename -a c/a.txt c/b.txt c/c.txt
   a.txt
   b.txt
   c.txt

Continuing the example above, the following syntax should work and do what you wanted:

   $ find c -type f -exec basename -a {} +
   1
   2
   3
   4


As such, I'm closing this bug, but discussion can continue by replying to this thread.

regards,
- assaf

Message #11 received at 24011 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: jklowden <jklowden <at> schemamania.org>
Cc: 24011 <at> debbugs.gnu.org
Subject: Re: bug#24011: parallel find error
Date: Sat, 16 Jul 2016 19:55:52 -0400

tag 24011 notabug
close 24011
stop

Hello,

> On Jul 16, 2016, at 15:42, jklowden <jklowden <at> schemamania.org> wrote:
> 
> There appears to be an error in the parallel execution of find.  I
> created a minimal example below.
> 
> [...]
> 
> $ find .. -type f -name lx[tr]\*.1 -exec basename {} +
> basename: extra operand ‘../appl/fsgmatch/lxtransduce.1’
> Try 'basename --help' for more information.

There are several issues here, though none of them are bugs.

First,
The difference between "find -exec basename {} \;" and "find -exec basename {} +" :
The former will execute the program 'basename' once per file,
and the latter will execute it fewer times with multiple files as parameters.

The following will demonstrate:

   $ mkdir c
   $ touch c/1 c/2 c/3 c/4
   $ find c -type f
   c/1
   c/2
   c/3
   c/4

Compare:

   $ find c -type f -exec echo basename {} \;
   basename c/1
   basename c/2
   basename c/3
   basename c/4

   $ find c -type f -exec echo basename {} +
   basename c/1 c/2 c/3 c/4


Second,
the "basename" command by default accepts a single parameter (pathname) or two parameters (pathname and optional extension).
Running it with three or more parameters will cause the error you were experiencing:

  $ basename c/a.txt 
  a.txt

  $ basename c/a.txt .txt
  a

  $ basename c/a.txt .txt c
  basename: extra operand ‘c’
  Try 'basename --help' for more information.

That explains your error: using the "+" syntax, "find" executed 'basename' with multiple filenames, and basename rejected it as invalid syntax.

However,
basename from coreutils version 8.16 and newer (released in 2012) supports the "-a" option (gnu extension) that handles multiple filenames on the same command line.
Compare:

   $ basename c/a.txt c/b.txt c/c.txt
   basename: extra operand ‘c/c.txt’
   Try 'basename --help' for more information.

   $ basename -a c/a.txt c/b.txt c/c.txt
   a.txt
   b.txt
   c.txt

Continuing the example above, the following syntax should work and do what you wanted:

   $ find c -type f -exec basename -a {} +
   1
   2
   3
   4


As such, I'm closing this bug, but discussion can continue by replying to this thread.

regards,
- assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.