Subject: bug#60506: feature: parallel grep --recursive

From: "David G. Pickett" <[email-masked]>
Date: Wed, 4 Jan 2023 16:42:56 +0000 (UTC)

xargs enhancement: I collect new args while the last set is running, use a fixed common buffer for input, and vary the arg count down for long args.

dgp@dgp-p6803w:~$ fxargs2
Usage: fxargs2 [ -n <args_per_exec> ] [ -v ] [ -p ] <cmd> [ <cmd_arg> ... ]
Reads arguments as lines from standard input and executes: <cmd> [ <cmd_args> ... ] <args_from_stdin>Each line becomes one argument.  The number of args per command is limitedby <args_per_exec> (default 1024).  The command is executed when either: - the total number of args from standard input is <args_per_exec>, or - the buffer has ( 80 * <args_per_exec> ) unexecuted bytes of input, or - stdin EOF is detected with any args from standard input.The <cmd> [ <cmd_args> ... ] is never executed alone.The buffer is fixed in size at 80 * <args_per_exec>, so long args can forcefewer <args_per_exec> for any pass.While a command is executing, reading resumes, but before another commandis executed, the prior command must return a status.With -v, any abnormal child state returned is reported.With -p, any child terminating on SIGPIPE causes a normal exit.
dgp@dgp-p6803w:~$ 

I was tempted to exec more often if stdin was temporarily dry, but better is the enemy of good enough!

-----Original Message-----
From: Paul Jackson <[email-masked]>
To: David G. Pickett <[email-masked]>; [email-masked] <[email-masked]>; [email-masked] <[email-masked]>
Sent: Tue, Jan 3, 2023 5:32 pm
Subject: Re: bug#60506: feature: parallel grep --recursive

#yiv4580765374 p.yiv4580765374MsoNormal, #yiv4580765374 p.yiv4580765374MsoNoSpacing{margin:0;}#yiv4580765374 p.yiv4580765374MsoNormal, #yiv4580765374 p.yiv4580765374MsoNoSpacing{margin:0;}David Pickett wrote:<< I also wrote a simpler, line oriented, faster xargs, fxargs!  >>

I've been quite pleased with an xargs wrapper I wrote that basically
converts newlines to nuls, and then invokes either "xargs" or, if asked
to run multiple threads, "parallel --xargs", passing all the "xargs" arguments
to "xargs --null".

I got all the exit status's and such just right, and preferred having all thexargs options available, once this hack worked around the confused
space character handling of xargs without the --null option.

I call my wrapper "x", a short name since  I use it a lot, having been a regularxargs user since it was added to Version 7 Unix, inside Bell Labs, back around
1978.

You can find my wrapper at:
http://thepythoniccow.us/x.c

By the way, even the original author of xargs, Herb Gellis, agrees that its
interface is somewhat borked.  See a note Gellis posted a decade after writing
xargs, which I include in the above "x.c" source.  An amusing bit of history ...

-- 
                Paul Jackson
                [email-masked]