GNU bug report logs -
#60506
feature: parallel grep --recursive
Previous Next
Full log
View this message in rfc822 format
[Message part 1 (text/plain, inline)]
xargs enhancement: I collect new args while the last set is running, use a fixed common buffer for input, and vary the arg count down for long args.
dgp <at> dgp-p6803w:~$ fxargs2
Usage: fxargs2 [ -n <args_per_exec> ] [ -v ] [ -p ] <cmd> [ <cmd_arg> ... ]
Reads arguments as lines from standard input and executes: <cmd> [ <cmd_args> ... ] <args_from_stdin>Each line becomes one argument. The number of args per command is limitedby <args_per_exec> (default 1024). The command is executed when either: - the total number of args from standard input is <args_per_exec>, or - the buffer has ( 80 * <args_per_exec> ) unexecuted bytes of input, or - stdin EOF is detected with any args from standard input.The <cmd> [ <cmd_args> ... ] is never executed alone.The buffer is fixed in size at 80 * <args_per_exec>, so long args can forcefewer <args_per_exec> for any pass.While a command is executing, reading resumes, but before another commandis executed, the prior command must return a status.With -v, any abnormal child state returned is reported.With -p, any child terminating on SIGPIPE causes a normal exit.
dgp <at> dgp-p6803w:~$
I was tempted to exec more often if stdin was temporarily dry, but better is the enemy of good enough!
-----Original Message-----
From: Paul Jackson <pj <at> usa.net>
To: David G. Pickett <dgpickett <at> aol.com>; eggert <at> cs.ucla.edu <eggert <at> cs.ucla.edu>; 60506 <at> debbugs.gnu.org <60506 <at> debbugs.gnu.org>
Sent: Tue, Jan 3, 2023 5:32 pm
Subject: Re: bug#60506: feature: parallel grep --recursive
#yiv4580765374 p.yiv4580765374MsoNormal, #yiv4580765374 p.yiv4580765374MsoNoSpacing{margin:0;}#yiv4580765374 p.yiv4580765374MsoNormal, #yiv4580765374 p.yiv4580765374MsoNoSpacing{margin:0;}David Pickett wrote:<< I also wrote a simpler, line oriented, faster xargs, fxargs! >>
I've been quite pleased with an xargs wrapper I wrote that basically
converts newlines to nuls, and then invokes either "xargs" or, if asked
to run multiple threads, "parallel --xargs", passing all the "xargs" arguments
to "xargs --null".
I got all the exit status's and such just right, and preferred having all thexargs options available, once this hack worked around the confused
space character handling of xargs without the --null option.
I call my wrapper "x", a short name since I use it a lot, having been a regularxargs user since it was added to Version 7 Unix, inside Bell Labs, back around
1978.
You can find my wrapper at:
http://thepythoniccow.us/x.c
By the way, even the original author of xargs, Herb Gellis, agrees that its
interface is somewhat borked. See a note Gellis posted a decade after writing
xargs, which I include in the above "x.c" source. An amusing bit of history ...
--
Paul Jackson
pj <at> usa.net
[Message part 2 (text/html, inline)]
This bug report was last modified 2 years and 159 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.