GNU bug report logs - #20006
Get rid of excessive sed forks

Previous Next

Full log

View this message in rfc822 format

From: Pavel Raiskup <praiskup <at> redhat.com>
To: libtool <at> gnu.org
Cc: Eric Blake <eblake <at> redhat.com>, 20006 <at> debbugs.gnu.org, Richard Purdie <richard.purdie <at> linuxfoundation.org>, Mike Frysinger <vapier <at> gentoo.org>
Subject: bug#20006: Bash-specific performance by avoiding sed
Date: Mon, 05 Oct 2015 15:28:56 +0200

[Message part 1 (text/plain, inline)]

On Monday 05 of October 2015 09:47:05 Pavel Raiskup wrote:
> On Monday 05 of October 2015 01:25:24 Pavel Raiskup wrote:
> > On Monday 05 of October 2015 00:45:50 Pavel Raiskup wrote:
> > > > should we test the size of the string first ?  i've written such raw shell 
> > > > string parsing functions before, and once you hit a certain size (like 1k+ 
> > > > iirc), forking out to sed is way faster, especially when running in multibyte 
> > > > locales (like UTF8) which most people are doing nowadays.
> > > > -mike
> > > 
> > > Well, that optimization would require (fast) strlen()-like construct.
> > > Anyway, the vast majority of calls to func_quote () function will have
> > > short ARG, and its complexity is still "just" linear.  We could optimize
> > > later if that was a real issue.
> > > 
> > > I would like to propose solution based on Eric's one, without using of
> > > '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even
> > > more portable while it keeps almost the same speed (if we can use += its
> > > even faster).
> > > 
> > > I have yet a another patch trying to minimize option-parser overhead
> > > (that is focused on the POV of Richard, but that needs to be cleaned up a
> > > bit, I'll post hopefully tomorrow).
> > > 
> > > Any comment is welcome!
> > 
> > Re-attached (fixes for 'make syntax-check' and fixed one comment).
> 
> Hmm, one might-be-a-problem with this (catched by testsuite), when you
> have:
> 
>   $ cat build-aux/test-quoting
>   . `echo "$0" |${SED-sed} 's|[^/]*$||'`/funclib.sh
>   # source this for "GNU m4" detection methods
>   . `echo "$0" |${SED-sed} 's|[^/]*$||'`/extract-trace
> 
>   func_quote_for_eval "$@"
>   echo "$func_quote_for_eval_result"
> 
> Then:
> 
>   $ ./build-aux/test-quoting '"a b"' # fine
>   "\"a b\""
> 
>   $ ./build-aux/test-quoting '"*tool"' # broken
>   ./build-aux/test-quoting '"*tool"'
>   \"libtool\"
> 
> We would like to have an output \"*\".  I'm not aware of portable way
> how to disable wildcard expansion in shell, and autoconf 'Shellology'
> section haven't helped me.  In particular, the problem is here:
> 
>   x='a"[a-z]*"c'
>   IFS='"'
>   for i in $x; do # Here we wan't to disable wildcard expansion
>     echo $i
>   done
> 
> Any idea other than fallback to $sed_quote_subst in case of '*' or '['
> exists in ARG?

Attaching two (yet to be cleaned) patches doing the optimization.  Is
anybody able to test/comment on this particular solution?  That would be
really appreciated.

Pavel

[0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch (text/x-patch, attachment)]

[0002-libbtool-optimize-parse-options-calls.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.