From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 05 03:58:32 2015 Received: (at submit) by debbugs.gnu.org; 5 Mar 2015 08:58:32 +0000 Received: from localhost ([127.0.0.1]:35598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YTRc0-0006yA-F3 for submit@debbugs.gnu.org; Thu, 05 Mar 2015 03:58:32 -0500 Received: from eggs.gnu.org ([208.118.235.92]:38118) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YTRW1-0006pA-JL for submit@debbugs.gnu.org; Thu, 05 Mar 2015 03:52:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YTRVv-0007UH-BU for submit@debbugs.gnu.org; Thu, 05 Mar 2015 03:52:16 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:33103) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTRVv-0007UB-8W for submit@debbugs.gnu.org; Thu, 05 Mar 2015 03:52:15 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39836) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTRVu-0000EB-3T for bug-libtool@gnu.org; Thu, 05 Mar 2015 03:52:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YTRVo-0007TQ-TI for bug-libtool@gnu.org; Thu, 05 Mar 2015 03:52:14 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53078) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTRVo-0007TG-K3 for bug-libtool@gnu.org; Thu, 05 Mar 2015 03:52:08 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t258q78S018432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Thu, 5 Mar 2015 03:52:07 -0500 Received: from lenovo.fritz.box (vpn1-4-98.ams2.redhat.com [10.36.4.98]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t258q5np003445 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 5 Mar 2015 03:52:07 -0500 Message-ID: <54F81935.2000904@redhat.com> Date: Thu, 05 Mar 2015 09:52:05 +0100 From: Harald Hoyer User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: bug-libtool@gnu.org Subject: Re: Get rid of excessive sed forks References: <54F813FD.7080409@redhat.com> In-Reply-To: <54F813FD.7080409@redhat.com> Content-Type: multipart/mixed; boundary="------------050305020305000909030108" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Thu, 05 Mar 2015 03:58:31 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --------------050305020305000909030108 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 05.03.2015 09:29, Harald Hoyer wrote: > See > https://harald.hoyer.xyz/2015/03/05/libtool-getting-rid-of-180000-sed-forks/ > > Patch attached. > Corrected patch attached --------------050305020305000909030108 Content-Type: text/x-diff; name="libtool-sed.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="libtool-sed.patch" --- libtool.orig 2015-03-04 15:44:39.632703654 +0100 +++ libtool.fixed 2015-03-05 09:51:17.321333343 +0100 @@ -1607,7 +1607,15 @@ while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + if test set = "${BASH_VERSION+set}"; then + _G_unquoted_arg=${1//\\/\\\\} + _G_unquoted_arg=${_G_unquoted_arg//\"/\\\"} + _G_unquoted_arg=${_G_unquoted_arg//\$/\\\$} + _G_unquoted_arg=${_G_unquoted_arg//\`/\\\`} + else + _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` + fi + ;; *) _G_unquoted_arg=$1 ;; esac @@ -5756,7 +5764,11 @@ if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + if test set = "${BASH_VERSION+set}"; then + qECHO=$(_G=$($ECHO "$ECHO");_G=${_G//\\/\\\\};_G=${_G//\"/\\\"};_G=${_G//\$/\\\$};_G=${_G//\`/\\\`};printf '%s\n' "$_G") + else + qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + fi $ECHO "\ # A function that is used when there is no print builtin or printf. --------------050305020305000909030108-- From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 04 18:46:00 2015 Received: (at 20006) by debbugs.gnu.org; 4 Oct 2015 22:46:00 +0000 Received: from localhost ([127.0.0.1]:54711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zis2Z-0000e1-33 for submit@debbugs.gnu.org; Sun, 04 Oct 2015 18:45:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55969) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zis2W-0000dp-5t for 20006@debbugs.gnu.org; Sun, 04 Oct 2015 18:45:57 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 79FAEC0B1B4D; Sun, 4 Oct 2015 22:45:54 +0000 (UTC) Received: from nb.usersys.redhat.com (ovpn-200-16.brq.redhat.com [10.40.200.16]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t94Mjphe015393 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Sun, 4 Oct 2015 18:45:53 -0400 From: Pavel Raiskup To: libtool@gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 05 Oct 2015 00:45:50 +0200 Message-ID: <1594368.rU6RqJ2yht@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.1-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <20150309220434.GA9455@vapier> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <54FE0727.4080305@redhat.com> <20150309220434.GA9455@vapier> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart5352205.9MGm54S4aO" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Eric Blake , 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart5352205.9MGm54S4aO Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" forcemerge 20006 20005 thanks On Monday 09 of March 2015 18:04:34 Mike Frysinger wrote: > On 09 Mar 2015 14:48, Eric Blake wrote: > > On 03/09/2015 01:50 PM, Bob Friesenhahn wrote: > > > On Mon, 9 Mar 2015, Mike Gran wrote: > > >> I don't know if y'all saw this blogpost where a guy pushed > > >> the sed regular expression handling into bash-specific > > >> regular expressions when bash was available. He claims > > >> there's a significant performance improvement because of > > >> reduced forking. > > >> > > >> http://harald.hoyer.xyz/2015/03/05/libtool-getting-rid-of-180000-sed-forks/ > > > > > > There is an issue in the libtool bug tracker regarding this. > > > > > > This solution only works with GNU bash. It would be good if volunteers > > > could research to see if there are similar solutions which can work with > > > other common shells (e.g. dash, ksh, zsh). > > > > For context, we're trying to speed up: > > > > sed_quote_subst='s|\([`"$\\]\)|\\\1|g' > > _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` > > > > How about this, which should be completely portable to XSI shells (alas, > > it still uses ${a#b} and ${a%b} at the end, so it is not portable to > > ancient Solaris /bin/sh): > > > > # func_quote STRING > > # Escapes all \`"$ in STRING with another \, and stores that in $quoted > > func_quote () { > > case $1 in > > *[\\\`\"\$]*) > > save_IFS=$IFS pre=.$1. > > for char in '\' '`' '"' '$'; do > > post= IFS=$char > > for part in $pre; do > > post=${post:+$post\\$char}$part > > done > > pre=$post > > done > > should we test the size of the string first ? i've written such raw shell > string parsing functions before, and once you hit a certain size (like 1k+ > iirc), forking out to sed is way faster, especially when running in multibyte > locales (like UTF8) which most people are doing nowadays. > -mike Well, that optimization would require (fast) strlen()-like construct. Anyway, the vast majority of calls to func_quote () function will have short ARG, and its complexity is still "just" linear. We could optimize later if that was a real issue. I would like to propose solution based on Eric's one, without using of '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even more portable while it keeps almost the same speed (if we can use += its even faster). I have yet a another patch trying to minimize option-parser overhead (that is focused on the POV of Richard, but that needs to be cleaned up a bit, I'll post hopefully tomorrow). Any comment is welcome! Pave --nextPart5352205.9MGm54S4aO Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From aa988d0a49f2d2b419519b09fef62fc993a6169f Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH] libtool: mitigate the $sed_quote_subst slowdown References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substition for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- build-aux/ltmain.in | 10 ++++++---- gl/build-aux/funclib.sh | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 103 insertions(+), 8 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..0c73a49 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(N) too). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result="dummy${_G_char}${func_quote_result}${_G_char}dummy" + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\${_G_char}" + ;; + start) + set first "" "" + func_quote_result="" + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1275,7 +1319,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..d2f2bff 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(N) too). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result="dummy${_G_char}${func_quote_result}${_G_char}dummy" + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\${_G_char}" + ;; + start) + set first "" "" + func_quote_result="" + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1044,7 +1088,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac -- 2.5.0 --nextPart5352205.9MGm54S4aO-- From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 04 18:46:50 2015 Received: (at control) by debbugs.gnu.org; 4 Oct 2015 22:46:50 +0000 Received: from localhost ([127.0.0.1]:54715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zis3N-0000fV-3b for submit@debbugs.gnu.org; Sun, 04 Oct 2015 18:46:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33657) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zis3L-0000fO-Al for control@debbugs.gnu.org; Sun, 04 Oct 2015 18:46:48 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id C85533B3C2 for ; Sun, 4 Oct 2015 22:46:46 +0000 (UTC) Received: from nb.usersys.redhat.com (ovpn-200-16.brq.redhat.com [10.40.200.16]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t94MkiGA017079 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 4 Oct 2015 18:46:46 -0400 From: Pavel Raiskup To: control@debbugs.gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 05 Oct 2015 00:46:44 +0200 Message-ID: <4177468.1OKFKNd7KT@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.1-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <20150309220434.GA9455@vapier> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <54FE0727.4080305@redhat.com> <20150309220434.GA9455@vapier> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart4118018.8iC5YniOVb" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart4118018.8iC5YniOVb Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" forcemerge 20006 20005 thanks On Monday 09 of March 2015 18:04:34 Mike Frysinger wrote: > On 09 Mar 2015 14:48, Eric Blake wrote: > > On 03/09/2015 01:50 PM, Bob Friesenhahn wrote: > > > On Mon, 9 Mar 2015, Mike Gran wrote: > > >> I don't know if y'all saw this blogpost where a guy pushed > > >> the sed regular expression handling into bash-specific > > >> regular expressions when bash was available. He claims > > >> there's a significant performance improvement because of > > >> reduced forking. > > >> > > >> http://harald.hoyer.xyz/2015/03/05/libtool-getting-rid-of-180000-sed-forks/ > > > > > > There is an issue in the libtool bug tracker regarding this. > > > > > > This solution only works with GNU bash. It would be good if volunteers > > > could research to see if there are similar solutions which can work with > > > other common shells (e.g. dash, ksh, zsh). > > > > For context, we're trying to speed up: > > > > sed_quote_subst='s|\([`"$\\]\)|\\\1|g' > > _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` > > > > How about this, which should be completely portable to XSI shells (alas, > > it still uses ${a#b} and ${a%b} at the end, so it is not portable to > > ancient Solaris /bin/sh): > > > > # func_quote STRING > > # Escapes all \`"$ in STRING with another \, and stores that in $quoted > > func_quote () { > > case $1 in > > *[\\\`\"\$]*) > > save_IFS=$IFS pre=.$1. > > for char in '\' '`' '"' '$'; do > > post= IFS=$char > > for part in $pre; do > > post=${post:+$post\\$char}$part > > done > > pre=$post > > done > > should we test the size of the string first ? i've written such raw shell > string parsing functions before, and once you hit a certain size (like 1k+ > iirc), forking out to sed is way faster, especially when running in multibyte > locales (like UTF8) which most people are doing nowadays. > -mike Well, that optimization would require (fast) strlen()-like construct. Anyway, the vast majority of calls to func_quote () function will have short ARG, and its complexity is still "just" linear. We could optimize later if that was a real issue. I would like to propose solution based on Eric's one, without using of '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even more portable while it keeps almost the same speed (if we can use += its even faster). I have yet a another patch trying to minimize option-parser overhead (that is focused on the POV of Richard, but that needs to be cleaned up a bit, I'll post hopefully tomorrow). Any comment is welcome! Pave --nextPart4118018.8iC5YniOVb Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="utf-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From aa988d0a49f2d2b419519b09fef62fc993a6169f Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH] libtool: mitigate the $sed_quote_subst slowdown References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substition for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- build-aux/ltmain.in | 10 ++++++---- gl/build-aux/funclib.sh | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 103 insertions(+), 8 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..0c73a49 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(N) too). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result="dummy${_G_char}${func_quote_result}${_G_char}dummy" + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\${_G_char}" + ;; + start) + set first "" "" + func_quote_result="" + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1275,7 +1319,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..d2f2bff 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(N) too). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result="dummy${_G_char}${func_quote_result}${_G_char}dummy" + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\${_G_char}" + ;; + start) + set first "" "" + func_quote_result="" + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1044,7 +1088,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac -- 2.5.0 --nextPart4118018.8iC5YniOVb-- From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 04 19:25:32 2015 Received: (at 20006) by debbugs.gnu.org; 4 Oct 2015 23:25:32 +0000 Received: from localhost ([127.0.0.1]:54722 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zisep-00036Z-D6 for submit@debbugs.gnu.org; Sun, 04 Oct 2015 19:25:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59039) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zisem-00036Q-7P for 20006@debbugs.gnu.org; Sun, 04 Oct 2015 19:25:29 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 77C76C0B1B22; Sun, 4 Oct 2015 23:25:27 +0000 (UTC) Received: from nb.usersys.redhat.com (ovpn-200-16.brq.redhat.com [10.40.200.16]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t94NPPoU024871 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Sun, 4 Oct 2015 19:25:26 -0400 From: Pavel Raiskup To: libtool@gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 05 Oct 2015 01:25:24 +0200 Message-ID: <18282557.LaRZhCTEKb@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.1-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <1594368.rU6RqJ2yht@nb.usersys.redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <20150309220434.GA9455@vapier> <1594368.rU6RqJ2yht@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart5432637.PaE8YzkZ7t" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Eric Blake , 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart5432637.PaE8YzkZ7t Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Monday 05 of October 2015 00:45:50 Pavel Raiskup wrote: > > should we test the size of the string first ? i've written such raw shell > > string parsing functions before, and once you hit a certain size (like 1k+ > > iirc), forking out to sed is way faster, especially when running in multibyte > > locales (like UTF8) which most people are doing nowadays. > > -mike > > Well, that optimization would require (fast) strlen()-like construct. > Anyway, the vast majority of calls to func_quote () function will have > short ARG, and its complexity is still "just" linear. We could optimize > later if that was a real issue. > > I would like to propose solution based on Eric's one, without using of > '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even > more portable while it keeps almost the same speed (if we can use += its > even faster). > > I have yet a another patch trying to minimize option-parser overhead > (that is focused on the POV of Richard, but that needs to be cleaned up a > bit, I'll post hopefully tomorrow). > > Any comment is welcome! Re-attached (fixes for 'make syntax-check' and fixed one comment). Pavel --nextPart5432637.PaE8YzkZ7t Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From f6935309a07b2a4797ff5612bff00ced4842c3e5 Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH] libtool: mitigate the $sed_quote_subst slowdown References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substition for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- build-aux/ltmain.in | 10 ++++++---- gl/build-aux/funclib.sh | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 103 insertions(+), 8 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..c2f6545 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1275,7 +1319,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..1a406cd 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,50 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + func_quote_old_IFS=$IFS + + case $func_quote_result in + *[\\\`\"\$]*) + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1044,7 +1088,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac -- 2.5.0 --nextPart5432637.PaE8YzkZ7t-- From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 05 03:47:13 2015 Received: (at 20006) by debbugs.gnu.org; 5 Oct 2015 07:47:13 +0000 Received: from localhost ([127.0.0.1]:54824 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj0UL-0006BD-4w for submit@debbugs.gnu.org; Mon, 05 Oct 2015 03:47:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41470) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj0UI-0006B4-6c for 20006@debbugs.gnu.org; Mon, 05 Oct 2015 03:47:10 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 7B9E88E693; Mon, 5 Oct 2015 07:47:08 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-141.brq.redhat.com [10.34.4.141]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t957l5Kv003250 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Mon, 5 Oct 2015 03:47:07 -0400 From: Pavel Raiskup To: libtool@gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 05 Oct 2015 09:47:05 +0200 Message-ID: <1541901.1iDc6jCD0b@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.1-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <18282557.LaRZhCTEKb@nb.usersys.redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <1594368.rU6RqJ2yht@nb.usersys.redhat.com> <18282557.LaRZhCTEKb@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Eric Blake , 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On Monday 05 of October 2015 01:25:24 Pavel Raiskup wrote: > On Monday 05 of October 2015 00:45:50 Pavel Raiskup wrote: > > > should we test the size of the string first ? i've written such raw shell > > > string parsing functions before, and once you hit a certain size (like 1k+ > > > iirc), forking out to sed is way faster, especially when running in multibyte > > > locales (like UTF8) which most people are doing nowadays. > > > -mike > > > > Well, that optimization would require (fast) strlen()-like construct. > > Anyway, the vast majority of calls to func_quote () function will have > > short ARG, and its complexity is still "just" linear. We could optimize > > later if that was a real issue. > > > > I would like to propose solution based on Eric's one, without using of > > '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even > > more portable while it keeps almost the same speed (if we can use += its > > even faster). > > > > I have yet a another patch trying to minimize option-parser overhead > > (that is focused on the POV of Richard, but that needs to be cleaned up a > > bit, I'll post hopefully tomorrow). > > > > Any comment is welcome! > > Re-attached (fixes for 'make syntax-check' and fixed one comment). Hmm, one might-be-a-problem with this (catched by testsuite), when you have: $ cat build-aux/test-quoting . `echo "$0" |${SED-sed} 's|[^/]*$||'`/funclib.sh # source this for "GNU m4" detection methods . `echo "$0" |${SED-sed} 's|[^/]*$||'`/extract-trace func_quote_for_eval "$@" echo "$func_quote_for_eval_result" Then: $ ./build-aux/test-quoting '"a b"' # fine "\"a b\"" $ ./build-aux/test-quoting '"*tool"' # broken ./build-aux/test-quoting '"*tool"' \"libtool\" We would like to have an output \"*\". I'm not aware of portable way how to disable wildcard expansion in shell, and autoconf 'Shellology' section haven't helped me. In particular, the problem is here: x='a"[a-z]*"c' IFS='"' for i in $x; do # Here we wan't to disable wildcard expansion echo $i done Any idea other than fallback to $sed_quote_subst in case of '*' or '[' exists in ARG? Pavel From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 05 09:29:12 2015 Received: (at 20006) by debbugs.gnu.org; 5 Oct 2015 13:29:12 +0000 Received: from localhost ([127.0.0.1]:54924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj5pF-0007HC-LI for submit@debbugs.gnu.org; Mon, 05 Oct 2015 09:29:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56890) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj5p6-0007GV-M8 for 20006@debbugs.gnu.org; Mon, 05 Oct 2015 09:29:03 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id C0C80C0B1B4D; Mon, 5 Oct 2015 13:28:59 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-141.brq.redhat.com [10.34.4.141]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t95DSu9H005727 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Mon, 5 Oct 2015 09:28:58 -0400 From: Pavel Raiskup To: libtool@gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 05 Oct 2015 15:28:56 +0200 Message-ID: <2468314.8kiGoXDBPl@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.1-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <1541901.1iDc6jCD0b@nb.usersys.redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <18282557.LaRZhCTEKb@nb.usersys.redhat.com> <1541901.1iDc6jCD0b@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart2208721.7tWzOsEx4c" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Eric Blake , 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart2208721.7tWzOsEx4c Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Monday 05 of October 2015 09:47:05 Pavel Raiskup wrote: > On Monday 05 of October 2015 01:25:24 Pavel Raiskup wrote: > > On Monday 05 of October 2015 00:45:50 Pavel Raiskup wrote: > > > > should we test the size of the string first ? i've written such raw shell > > > > string parsing functions before, and once you hit a certain size (like 1k+ > > > > iirc), forking out to sed is way faster, especially when running in multibyte > > > > locales (like UTF8) which most people are doing nowadays. > > > > -mike > > > > > > Well, that optimization would require (fast) strlen()-like construct. > > > Anyway, the vast majority of calls to func_quote () function will have > > > short ARG, and its complexity is still "just" linear. We could optimize > > > later if that was a real issue. > > > > > > I would like to propose solution based on Eric's one, without using of > > > '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even > > > more portable while it keeps almost the same speed (if we can use += its > > > even faster). > > > > > > I have yet a another patch trying to minimize option-parser overhead > > > (that is focused on the POV of Richard, but that needs to be cleaned up a > > > bit, I'll post hopefully tomorrow). > > > > > > Any comment is welcome! > > > > Re-attached (fixes for 'make syntax-check' and fixed one comment). > > Hmm, one might-be-a-problem with this (catched by testsuite), when you > have: > > $ cat build-aux/test-quoting > . `echo "$0" |${SED-sed} 's|[^/]*$||'`/funclib.sh > # source this for "GNU m4" detection methods > . `echo "$0" |${SED-sed} 's|[^/]*$||'`/extract-trace > > func_quote_for_eval "$@" > echo "$func_quote_for_eval_result" > > Then: > > $ ./build-aux/test-quoting '"a b"' # fine > "\"a b\"" > > $ ./build-aux/test-quoting '"*tool"' # broken > ./build-aux/test-quoting '"*tool"' > \"libtool\" > > We would like to have an output \"*\". I'm not aware of portable way > how to disable wildcard expansion in shell, and autoconf 'Shellology' > section haven't helped me. In particular, the problem is here: > > x='a"[a-z]*"c' > IFS='"' > for i in $x; do # Here we wan't to disable wildcard expansion > echo $i > done > > Any idea other than fallback to $sed_quote_subst in case of '*' or '[' > exists in ARG? Attaching two (yet to be cleaned) patches doing the optimization. Is anybody able to test/comment on this particular solution? That would be really appreciated. Pavel --nextPart2208721.7tWzOsEx4c Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From 1b89ae66f88c3822b2afec128f288d409287780f Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH 1/2] libtool: mitigate the $sed_quote_subst slowdown When it is reasonably possible, use shell implementation for quoting. References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substitution for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 56 +++++++++++++++++++++++++++++++++++++++++++++++-- build-aux/ltmain.in | 10 +++++---- gl/build-aux/funclib.sh | 56 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 117 insertions(+), 8 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..2649478 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *'*'*|*'['*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1275,7 +1326,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..8032e6b 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *'*'*|*'['*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1044,7 +1095,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac -- 2.5.0 --nextPart2208721.7tWzOsEx4c Content-Disposition: attachment; filename="0002-libbtool-optimize-parse-options-calls.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0002-libbtool-optimize-parse-options-calls.patch" >From c283b8a8e7540b65fd7a7d32fceec6ed42b5b2a7 Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Mon, 5 Oct 2015 13:16:08 +0200 Subject: [PATCH 2/2] libbtool: optimize parse-options calls Its not necessary to re-escape-for-eval in each function in the hook-able function tree -- usually is enough if the leaf function executes func_quote_for_eval() and its caller just re-uses the CALLEE_return value. * gl/build-aux/options-parser (func_run_hooks): Propagate 0 return code down to caller if any $_G_hook succeeded. Don't re-quote the result as that has already been done by the succeeding hook itself. (func_parse_options): Quote only if we changed something and use appropriate return code. (func_validate_options): Likewise. (func_options_prep): Likewise. (func_options_finish): New hook-caller for 'func_options' hooks. (func_options): Propagate return value down to top-level caller, but pay attention we have always set $func_options_result. * build-aux/ltmain.in (libtool_options_prep): Quote only if we changed something and use appropriate return code. (libtool_parse_options): Likewise. * bootstrap: Sync gl/build-aux/with option-parser. --- bootstrap | 128 +++++++++++++++++++++++++++++++++----------- build-aux/ltmain.in | 33 +++++++++--- gl/build-aux/options-parser | 128 +++++++++++++++++++++++++++++++++----------- 3 files changed, 220 insertions(+), 69 deletions(-) diff --git a/bootstrap b/bootstrap index 2649478..c4de711 100755 --- a/bootstrap +++ b/bootstrap @@ -1744,10 +1744,17 @@ func_remove_hook () # than a whitespace-delimited list of legal shell function names, and # no effort is wasted trying to catch shell meta-characters or preserve # whitespace. +# If at least one hook returns 0 (exit success) upon execution, +# func_run_hooks () returns zero too. Otherwise 1 is returned. +# All hooks are expected to correctly return $HOOKNAME_result variable +# upon success exection. The output variable $func_run_hooks_result +# is set to latest (successful) $HOOKNAME_result. func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -1756,16 +1763,24 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' + _G_hook_unchanged=${_G_hook}_unchanged + eval $_G_hook_unchanged=false - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=true + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + if $_G_rc_run_hooks; then + # No need to re-quote by func_quoteh + func_run_hooks_result=$_G_hook_result + fi + + $_G_rc_run_hooks } @@ -1839,6 +1854,24 @@ func_run_hooks () # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# All finishing suff required after the option parse loop. This executes +# all 'func_options' hooks. +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -1848,17 +1881,26 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + if $_G_rc_options; then + func_options_result=$_G_res_var + else + # As a top-level function, we need *always* set the *_result + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -1869,7 +1911,9 @@ func_options () # positional parameters. If a hook function modifies that list, and # needs to propogate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning and 0 returned. If no changes were done, script might return +# 1 and caller of the hook will not read the 'func_run_hooks_result' +# value. func_hookable func_options_prep func_options_prep () { @@ -1879,10 +1923,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -1896,18 +1944,21 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + # Adjust func_parse_options positional parameters to match + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -1977,13 +2028,23 @@ func_parse_options () --) break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"} ; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -1996,16 +2057,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 24acefd..e57042a 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -358,6 +358,8 @@ libtool_options_prep () nonopt= preserve_args= + _G_rc_libtool_options_prep=: + # Shorthand for --mode=foo, only valid as the first argument case $1 in clean|clea|cle|cl) @@ -381,11 +383,18 @@ libtool_options_prep () uninstall|uninstal|uninsta|uninst|unins|unin|uni|un|u) shift; set dummy --mode uninstall ${1+"$@"}; shift ;; + *) + _G_rc_libtool_options_prep=false + ;; esac - # Pass back the list of options. - func_quote_for_eval ${1+"$@"} - libtool_options_prep_result=$func_quote_for_eval_result + if $_G_rc_libtool_options_prep; then + # Pass back the list of options. + func_quote_for_eval ${1+"$@"} + libtool_options_prep_result=$func_quote_for_eval_result + fi + + $_G_rc_libtool_options_prep } func_add_hook func_options_prep libtool_options_prep @@ -397,9 +406,12 @@ libtool_parse_options () { $debug_cmd + _G_rc_lt_parse_options=false + # Perform our own loop to consume as many options as possible in # each iteration. while test $# -gt 0; do + _G_match_lt_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -475,14 +487,21 @@ libtool_parse_options () ;; # An option not handled by this hook function: - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"} ; shift + _G_match_lt_parse_options=false + break + ;; esac + $_G_match_lt_parse_options && _G_rc_lt_parse_options=: done + if $_G_rc_lt_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + libtool_parse_options_result=$func_quote_for_eval_result + fi - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - libtool_parse_options_result=$func_quote_for_eval_result + $_G_rc_lt_parse_options } func_add_hook func_parse_options libtool_parse_options diff --git a/gl/build-aux/options-parser b/gl/build-aux/options-parser index d651f1d..73db1e3 100644 --- a/gl/build-aux/options-parser +++ b/gl/build-aux/options-parser @@ -157,10 +157,17 @@ func_remove_hook () # than a whitespace-delimited list of legal shell function names, and # no effort is wasted trying to catch shell meta-characters or preserve # whitespace. +# If at least one hook returns 0 (exit success) upon execution, +# func_run_hooks () returns zero too. Otherwise 1 is returned. +# All hooks are expected to correctly return $HOOKNAME_result variable +# upon success exection. The output variable $func_run_hooks_result +# is set to latest (successful) $HOOKNAME_result. func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -169,16 +176,24 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' + _G_hook_unchanged=${_G_hook}_unchanged + eval $_G_hook_unchanged=false - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=true + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + if $_G_rc_run_hooks; then + # No need to re-quote by func_quoteh + func_run_hooks_result=$_G_hook_result + fi + + $_G_rc_run_hooks } @@ -252,6 +267,24 @@ func_run_hooks () # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# All finishing suff required after the option parse loop. This executes +# all 'func_options' hooks. +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -261,17 +294,26 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + if $_G_rc_options; then + func_options_result=$_G_res_var + else + # As a top-level function, we need *always* set the *_result + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -282,7 +324,9 @@ func_options () # positional parameters. If a hook function modifies that list, and # needs to propogate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning and 0 returned. If no changes were done, script might return +# 1 and caller of the hook will not read the 'func_run_hooks_result' +# value. func_hookable func_options_prep func_options_prep () { @@ -292,10 +336,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -309,18 +357,21 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + # Adjust func_parse_options positional parameters to match + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -390,13 +441,23 @@ func_parse_options () --) break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"} ; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -409,16 +470,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } -- 2.5.0 --nextPart2208721.7tWzOsEx4c-- From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 05 11:16:38 2015 Received: (at 20006) by debbugs.gnu.org; 5 Oct 2015 15:16:38 +0000 Received: from localhost ([127.0.0.1]:55351 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj7VG-0001RC-CK for submit@debbugs.gnu.org; Mon, 05 Oct 2015 11:16:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47544) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zj7VE-0001R1-JD for 20006@debbugs.gnu.org; Mon, 05 Oct 2015 11:16:37 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 0C8B78CF73; Mon, 5 Oct 2015 15:16:36 +0000 (UTC) Received: from [10.3.113.192] (ovpn-113-192.phx2.redhat.com [10.3.113.192]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t95FGY6A020921; Mon, 5 Oct 2015 11:16:35 -0400 Subject: Re: Bash-specific performance by avoiding sed To: Pavel Raiskup , libtool@gnu.org References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <1594368.rU6RqJ2yht@nb.usersys.redhat.com> <18282557.LaRZhCTEKb@nb.usersys.redhat.com> <1541901.1iDc6jCD0b@nb.usersys.redhat.com> From: Eric Blake Openpgp: url=http://people.redhat.com/eblake/eblake.gpg Organization: Red Hat, Inc. Message-ID: <5612944E.2050806@redhat.com> Date: Mon, 5 Oct 2015 09:16:30 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1541901.1iDc6jCD0b@nb.usersys.redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="iDBQmxrUxK8QiVRFHDnQrLN3rQUgXiMVJ" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --iDBQmxrUxK8QiVRFHDnQrLN3rQUgXiMVJ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 10/05/2015 01:47 AM, Pavel Raiskup wrote: >=20 > Hmm, one might-be-a-problem with this (catched by testsuite), when you s/catched/caught/ > We would like to have an output \"*\". I'm not aware of portable way > how to disable wildcard expansion in shell, and autoconf 'Shellology' > section haven't helped me. In particular, the problem is here: It is portable to use 'set +f' to disable wildcard globbing, then 'set -f' to turn them back on. (POSIX requires it). >=20 > x=3D'a"[a-z]*"c' > IFS=3D'"' > for i in $x; do # Here we wan't to disable wildcard expansion s/wan't/want/ --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --iDBQmxrUxK8QiVRFHDnQrLN3rQUgXiMVJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJWEpRSAAoJEKeha0olJ0NqjncH/RzhXltA8WRNSlfberhyfCgg dfXYW2mxbD09ikJVaozvMd4iBGTtb2TMV443MRJf2b/T+fOZAXjMSCag6J4bNmXY gYphr28ed65R47t/NPf3mzVNC/Ru07YpcdBwM7p2f1mQPuP/WLWVV5mhT+6Gx+G1 IHpiFpkQRS1gLuFCP/zZyjr7I7RzN3GhWJeI4ARuiVckPODQevBIy3+9yVibCGZx oG6nFekeCvmivIfhjbacmHVgckL1K8TakaupVoiE/MQXIKfYKnCp941uXn68z2q2 uylDbnbpVmsphmTSh0CxqURPZDkszD08mtss8gNg28rGRNAwyeP41ur6U4Hm7rw= =+gob -----END PGP SIGNATURE----- --iDBQmxrUxK8QiVRFHDnQrLN3rQUgXiMVJ-- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 07 08:28:40 2015 Received: (at 20006) by debbugs.gnu.org; 7 Oct 2015 12:28:40 +0000 Received: from localhost ([127.0.0.1]:57888 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zjnpl-00032j-Ux for submit@debbugs.gnu.org; Wed, 07 Oct 2015 08:28:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54257) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zjnpi-00032Y-4a for 20006@debbugs.gnu.org; Wed, 07 Oct 2015 08:28:36 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id AB54EC0BE076; Wed, 7 Oct 2015 12:28:33 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-157.brq.redhat.com [10.34.4.157]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t97CSU7E012080 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Wed, 7 Oct 2015 08:28:32 -0400 From: Pavel Raiskup To: libtool@gnu.org Subject: Re: Bash-specific performance by avoiding sed Date: Wed, 07 Oct 2015 14:28:30 +0200 Message-ID: <17836452.nsWZIHcxGW@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.2-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <2468314.8kiGoXDBPl@nb.usersys.redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <1541901.1iDc6jCD0b@nb.usersys.redhat.com> <2468314.8kiGoXDBPl@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart2726378.O234mx4xem" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Eric Blake , 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart2726378.O234mx4xem Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Monday 05 of October 2015 15:28:56 Pavel Raiskup wrote: > On Monday 05 of October 2015 09:47:05 Pavel Raiskup wrote: > > On Monday 05 of October 2015 01:25:24 Pavel Raiskup wrote: > > > On Monday 05 of October 2015 00:45:50 Pavel Raiskup wrote: > > > > > should we test the size of the string first ? i've written such raw shell > > > > > string parsing functions before, and once you hit a certain size (like 1k+ > > > > > iirc), forking out to sed is way faster, especially when running in multibyte > > > > > locales (like UTF8) which most people are doing nowadays. > > > > > -mike > > > > > > > > Well, that optimization would require (fast) strlen()-like construct. > > > > Anyway, the vast majority of calls to func_quote () function will have > > > > short ARG, and its complexity is still "just" linear. We could optimize > > > > later if that was a real issue. > > > > > > > > I would like to propose solution based on Eric's one, without using of > > > > '${VAR%.}' and '${VAR#.}' constructs -- sounds like this could be even > > > > more portable while it keeps almost the same speed (if we can use += its > > > > even faster). > > > > > > > > I have yet a another patch trying to minimize option-parser overhead > > > > (that is focused on the POV of Richard, but that needs to be cleaned up a > > > > bit, I'll post hopefully tomorrow). > > > > > > > > Any comment is welcome! > > > > > > Re-attached (fixes for 'make syntax-check' and fixed one comment). > > > > Hmm, one might-be-a-problem with this (catched by testsuite), when you > > have: > > > > $ cat build-aux/test-quoting > > . `echo "$0" |${SED-sed} 's|[^/]*$||'`/funclib.sh > > # source this for "GNU m4" detection methods > > . `echo "$0" |${SED-sed} 's|[^/]*$||'`/extract-trace > > > > func_quote_for_eval "$@" > > echo "$func_quote_for_eval_result" > > > > Then: > > > > $ ./build-aux/test-quoting '"a b"' # fine > > "\"a b\"" > > > > $ ./build-aux/test-quoting '"*tool"' # broken > > ./build-aux/test-quoting '"*tool"' > > \"libtool\" > > > > We would like to have an output \"*\". I'm not aware of portable way > > how to disable wildcard expansion in shell, and autoconf 'Shellology' > > section haven't helped me. In particular, the problem is here: > > > > x='a"[a-z]*"c' > > IFS='"' > > for i in $x; do # Here we wan't to disable wildcard expansion > > echo $i > > done > > > > Any idea other than fallback to $sed_quote_subst in case of '*' or '[' > > exists in ARG? > > Attaching two (yet to be cleaned) patches doing the optimization. Is > anybody able to test/comment on this particular solution? That would be > really appreciated. The cleaned patches are attached. I would like to push those very soon, probably before weekend. If you see any issues worth holding this change, please let me know soon, thanks! FWIW, some numbers (systemd.git build time, right after 'make clean'): The old libtool v2.4.2: $ time make -j5 real 2m3.163s user 3m54.849s sys 3m28.684s The latest released libtool v2.4.6: $ time make -j5 LIBTOOL=/usr/bin/libtool real 8m24.604s user 9m56.977s sys 19m45.620s The patched git libtool: $ time make -j5 LIBTOOL=~/rh/projects/libtool/libtool real 2m34.682s user 6m37.158s sys 2m21.123s .. so it is (2.4.6 vs. 2.4.7~dev, user+sys) 7m23.5s vs 8m58.3s. It's not completely back yet but it's much better than v2.4.6. Pavel --nextPart2726378.O234mx4xem Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From 74940e9306df18deddd11621791973df886e313a Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH 1/2] libtool: mitigate the $sed_quote_subst slowdown When it is reasonably possible, use shell implementation for quoting. References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substitution for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 56 +++++++++++++++++++++++++++++++++++++++++++++++-- build-aux/ltmain.in | 10 +++++---- gl/build-aux/funclib.sh | 56 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 117 insertions(+), 8 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..2649478 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *'*'*|*'['*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1275,7 +1326,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..8032e6b 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *'*'*|*'['*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1044,7 +1095,8 @@ func_quote_for_eval () while test 0 -lt $#; do case $1 in *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; + func_quote "$1" + _G_unquoted_arg=$func_quote_result ;; *) _G_unquoted_arg=$1 ;; esac -- 2.5.0 --nextPart2726378.O234mx4xem Content-Disposition: attachment; filename="0002-libtool-optimizing-options-parser-hooks.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0002-libtool-optimizing-options-parser-hooks.patch" >From 97d39fb30958e86e1fe4d5fc8c953c66183cba20 Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Mon, 5 Oct 2015 13:16:08 +0200 Subject: [PATCH 2/2] libtool: optimizing options-parser hooks Its not necessary to (re)func_quote_for_eval in each function in the hook hierarchy. Usually is enough if the leaf function does func_quote_for_eval and its caller just re-uses the _return variable. This is follow up for the previous commit. * gl/build-aux/options-parser (func_run_hooks): Propagate $EXIT_SUCCESS return code down to caller if *any* hook succeeded. Never re-quote the result -- either the arguments are left untouched, or the options have already been properly quoted by succeeding hooks. (func_parse_options): Quote '$@' and return $EXIT_SUCCESS only if we changed something. (func_validate_options): Likewise. (func_options_prep): Likewise. (func_options_finish): New hook-caller for 'func_options' hooks. (func_options): Propagate return value down to top-level caller, but pay attention we have always set $func_options_result. * build-aux/ltmain.in (libtool_options_prep): Quote '$@' and return $EXIT_SUCCESS only if we changed something. (libtool_parse_options): Likewise. * bootstrap: Sync gl/build-aux/with option-parser. --- bootstrap | 171 +++++++++++++++++++++++++++++++------------- build-aux/ltmain.in | 35 ++++++--- gl/build-aux/options-parser | 171 +++++++++++++++++++++++++++++++------------- 3 files changed, 271 insertions(+), 106 deletions(-) diff --git a/bootstrap b/bootstrap index 2649478..f9aac68 100755 --- a/bootstrap +++ b/bootstrap @@ -1588,7 +1588,7 @@ func_lt_ver () #! /bin/sh # Set a version string for this script. -scriptversion=2014-01-07.03; # UTC +scriptversion=2015-10-07.11; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -1748,6 +1748,8 @@ func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -1756,16 +1758,16 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' - - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=: + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + $_G_rc_run_hooks && func_run_hooks_result=$_G_hook_result } @@ -1775,10 +1777,16 @@ func_run_hooks () ## --------------- ## # In order to add your own option parsing hooks, you must accept the -# full positional parameter list in your hook function, remove any -# options that you action, and then pass back the remaining unprocessed +# full positional parameter list in your hook function, you may remove/edit +# any options that you action, and then pass back the remaining unprocessed # options in '_result', escaped suitably for -# 'eval'. Like this: +# 'eval'. In this case you also must return $EXIT_SUCCESS to let the +# hook's caller know that it should pay attention to +# '_result'. Returning $EXIT_FAILURE signalizes that +# arguments are left untouched by the hook and therefore caller will ignore the +# result variable. +# +# Like this: # # my_options_prep () # { @@ -1788,9 +1796,11 @@ func_run_hooks () # usage_message=$usage_message' # -s, --silent don'\''t print informational messages # ' -# -# func_quote_for_eval ${1+"$@"} -# my_options_prep_result=$func_quote_for_eval_result +# # No change in '$@' (ignored completely by this hook). There is +# # no need to do the equivalent (but slower) action: +# # func_quote_for_eval ${1+"$@"} +# # my_options_prep_result=$func_quote_for_eval_result +# false # } # func_add_hook func_options_prep my_options_prep # @@ -1799,25 +1809,37 @@ func_run_hooks () # { # $debug_cmd # +# args_changed=false +# # # Note that for efficiency, we parse as many options as we can # # recognise in a loop before passing the remainder back to the # # caller on the first unrecognised argument we encounter. # while test $# -gt 0; do # opt=$1; shift # case $opt in -# --silent|-s) opt_silent=: ;; +# --silent|-s) opt_silent=: +# args_changed=: +# ;; # # Separate non-argument short options: # -s*) func_split_short_opt "$_G_opt" # set dummy "$func_split_short_opt_name" \ # "-$func_split_short_opt_arg" ${1+"$@"} # shift +# args_changed=: # ;; -# *) set dummy "$_G_opt" "$*"; shift; break ;; +# *) # Make sure the first unrecognised option "$_G_opt" +# # is added back to "$@", we could need that later +# # if $args_changed is true. +# set dummy "$_G_opt" ${1+"$@"}; shift; break ;; # esac # done # -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# if $args_changed; then +# func_quote_for_eval ${1+"$@"} +# my_silent_option_result=$func_quote_for_eval_result +# fi +# +# $args_changed # } # func_add_hook func_parse_options my_silent_option # @@ -1829,16 +1851,32 @@ func_run_hooks () # $opt_silent && $opt_verbose && func_fatal_help "\ # '--silent' and '--verbose' options are mutually exclusive." # -# func_quote_for_eval ${1+"$@"} -# my_option_validation_result=$func_quote_for_eval_result +# false # } # func_add_hook func_validate_options my_option_validation # -# You'll alse need to manually amend $usage_message to reflect the extra +# You'll also need to manually amend $usage_message to reflect the extra # options you parse. It's preferable to append if you can, so that # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# Finishing the option parse loop (call 'func_options' hooks ATM). +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -1848,17 +1886,28 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + # Save modified positional parameters for caller. As a top-level + # options-parser function we always need to set the 'func_options_result' + # variable (regardless the $_G_rc_options value). + if $_G_rc_options; then + func_options_result=$_G_res_var + else + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -1867,9 +1916,9 @@ func_options () # All initialisations required before starting the option parse loop. # Note that when calling hook functions, we pass through the list of # positional parameters. If a hook function modifies that list, and -# needs to propogate that back to rest of this script, then the complete +# needs to propagate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning $EXIT_SUCCESS (otherwise $EXIT_FAILURE is returned). func_hookable func_options_prep func_options_prep () { @@ -1879,10 +1928,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -1896,18 +1949,20 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -1922,7 +1977,10 @@ func_parse_options () ;; --warnings|--warning|-W) - test $# = 0 && func_missing_arg $_G_opt && break + if test $# = 0 && func_missing_arg $_G_opt; then + _G_rc_parse_options=: + break + fi case " $warning_categories $1" in *" $1 "*) # trailing space prevents matching last $1 above @@ -1975,15 +2033,25 @@ func_parse_options () shift ;; - --) break ;; + --) _G_rc_parse_options=: ; break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"}; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -1996,16 +2064,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 24acefd..b4c6bcd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -358,6 +358,8 @@ libtool_options_prep () nonopt= preserve_args= + _G_rc_lt_options_prep=: + # Shorthand for --mode=foo, only valid as the first argument case $1 in clean|clea|cle|cl) @@ -381,11 +383,18 @@ libtool_options_prep () uninstall|uninstal|uninsta|uninst|unins|unin|uni|un|u) shift; set dummy --mode uninstall ${1+"$@"}; shift ;; + *) + _G_rc_lt_options_prep=false + ;; esac - # Pass back the list of options. - func_quote_for_eval ${1+"$@"} - libtool_options_prep_result=$func_quote_for_eval_result + if $_G_rc_lt_options_prep; then + # Pass back the list of options. + func_quote_for_eval ${1+"$@"} + libtool_options_prep_result=$func_quote_for_eval_result + fi + + $_G_rc_lt_options_prep } func_add_hook func_options_prep libtool_options_prep @@ -397,9 +406,12 @@ libtool_parse_options () { $debug_cmd + _G_rc_lt_parse_options=false + # Perform our own loop to consume as many options as possible in # each iteration. while test $# -gt 0; do + _G_match_lt_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -474,15 +486,22 @@ libtool_parse_options () func_append preserve_args " $_G_opt" ;; - # An option not handled by this hook function: - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + # An option not handled by this hook function: + *) set dummy "$_G_opt" ${1+"$@"} ; shift + _G_match_lt_parse_options=false + break + ;; esac + $_G_match_lt_parse_options && _G_rc_lt_parse_options=: done + if $_G_rc_lt_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + libtool_parse_options_result=$func_quote_for_eval_result + fi - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - libtool_parse_options_result=$func_quote_for_eval_result + $_G_rc_lt_parse_options } func_add_hook func_parse_options libtool_parse_options diff --git a/gl/build-aux/options-parser b/gl/build-aux/options-parser index d651f1d..4c7e9cb 100644 --- a/gl/build-aux/options-parser +++ b/gl/build-aux/options-parser @@ -1,7 +1,7 @@ #! /bin/sh # Set a version string for this script. -scriptversion=2014-01-07.03; # UTC +scriptversion=2015-10-07.11; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -161,6 +161,8 @@ func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -169,16 +171,16 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' - - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=: + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + $_G_rc_run_hooks && func_run_hooks_result=$_G_hook_result } @@ -188,10 +190,16 @@ func_run_hooks () ## --------------- ## # In order to add your own option parsing hooks, you must accept the -# full positional parameter list in your hook function, remove any -# options that you action, and then pass back the remaining unprocessed +# full positional parameter list in your hook function, you may remove/edit +# any options that you action, and then pass back the remaining unprocessed # options in '_result', escaped suitably for -# 'eval'. Like this: +# 'eval'. In this case you also must return $EXIT_SUCCESS to let the +# hook's caller know that it should pay attention to +# '_result'. Returning $EXIT_FAILURE signalizes that +# arguments are left untouched by the hook and therefore caller will ignore the +# result variable. +# +# Like this: # # my_options_prep () # { @@ -201,9 +209,11 @@ func_run_hooks () # usage_message=$usage_message' # -s, --silent don'\''t print informational messages # ' -# -# func_quote_for_eval ${1+"$@"} -# my_options_prep_result=$func_quote_for_eval_result +# # No change in '$@' (ignored completely by this hook). There is +# # no need to do the equivalent (but slower) action: +# # func_quote_for_eval ${1+"$@"} +# # my_options_prep_result=$func_quote_for_eval_result +# false # } # func_add_hook func_options_prep my_options_prep # @@ -212,25 +222,37 @@ func_run_hooks () # { # $debug_cmd # +# args_changed=false +# # # Note that for efficiency, we parse as many options as we can # # recognise in a loop before passing the remainder back to the # # caller on the first unrecognised argument we encounter. # while test $# -gt 0; do # opt=$1; shift # case $opt in -# --silent|-s) opt_silent=: ;; +# --silent|-s) opt_silent=: +# args_changed=: +# ;; # # Separate non-argument short options: # -s*) func_split_short_opt "$_G_opt" # set dummy "$func_split_short_opt_name" \ # "-$func_split_short_opt_arg" ${1+"$@"} # shift +# args_changed=: # ;; -# *) set dummy "$_G_opt" "$*"; shift; break ;; +# *) # Make sure the first unrecognised option "$_G_opt" +# # is added back to "$@", we could need that later +# # if $args_changed is true. +# set dummy "$_G_opt" ${1+"$@"}; shift; break ;; # esac # done # -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# if $args_changed; then +# func_quote_for_eval ${1+"$@"} +# my_silent_option_result=$func_quote_for_eval_result +# fi +# +# $args_changed # } # func_add_hook func_parse_options my_silent_option # @@ -242,16 +264,32 @@ func_run_hooks () # $opt_silent && $opt_verbose && func_fatal_help "\ # '--silent' and '--verbose' options are mutually exclusive." # -# func_quote_for_eval ${1+"$@"} -# my_option_validation_result=$func_quote_for_eval_result +# false # } # func_add_hook func_validate_options my_option_validation # -# You'll alse need to manually amend $usage_message to reflect the extra +# You'll also need to manually amend $usage_message to reflect the extra # options you parse. It's preferable to append if you can, so that # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# Finishing the option parse loop (call 'func_options' hooks ATM). +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -261,17 +299,28 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + # Save modified positional parameters for caller. As a top-level + # options-parser function we always need to set the 'func_options_result' + # variable (regardless the $_G_rc_options value). + if $_G_rc_options; then + func_options_result=$_G_res_var + else + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -280,9 +329,9 @@ func_options () # All initialisations required before starting the option parse loop. # Note that when calling hook functions, we pass through the list of # positional parameters. If a hook function modifies that list, and -# needs to propogate that back to rest of this script, then the complete +# needs to propagate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning $EXIT_SUCCESS (otherwise $EXIT_FAILURE is returned). func_hookable func_options_prep func_options_prep () { @@ -292,10 +341,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -309,18 +362,20 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -335,7 +390,10 @@ func_parse_options () ;; --warnings|--warning|-W) - test $# = 0 && func_missing_arg $_G_opt && break + if test $# = 0 && func_missing_arg $_G_opt; then + _G_rc_parse_options=: + break + fi case " $warning_categories $1" in *" $1 "*) # trailing space prevents matching last $1 above @@ -388,15 +446,25 @@ func_parse_options () shift ;; - --) break ;; + --) _G_rc_parse_options=: ; break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"}; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -409,16 +477,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } -- 2.5.0 --nextPart2726378.O234mx4xem-- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 07 10:03:40 2015 Received: (at 20006) by debbugs.gnu.org; 7 Oct 2015 14:03:40 +0000 Received: from localhost ([127.0.0.1]:59119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZjpJf-0005Um-Fd for submit@debbugs.gnu.org; Wed, 07 Oct 2015 10:03:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48832) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZjpJF-0005UD-Hu for 20006@debbugs.gnu.org; Wed, 07 Oct 2015 10:03:31 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 5AC63AED54; Wed, 7 Oct 2015 14:03:08 +0000 (UTC) Received: from [10.3.113.202] (ovpn-113-202.phx2.redhat.com [10.3.113.202]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t97E37lD004595; Wed, 7 Oct 2015 10:03:07 -0400 Subject: Re: Bash-specific performance by avoiding sed To: Pavel Raiskup , libtool@gnu.org References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <1541901.1iDc6jCD0b@nb.usersys.redhat.com> <2468314.8kiGoXDBPl@nb.usersys.redhat.com> <17836452.nsWZIHcxGW@nb.usersys.redhat.com> From: Eric Blake Openpgp: url=http://people.redhat.com/eblake/eblake.gpg X-Enigmail-Draft-Status: N1110 Organization: Red Hat, Inc. Message-ID: <56152615.60903@redhat.com> Date: Wed, 7 Oct 2015 08:03:01 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <17836452.nsWZIHcxGW@nb.usersys.redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="mSvquapeUP9SdwEWqFL1NFCiFKijgCqAf" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: 20006@debbugs.gnu.org, Richard Purdie , Mike Frysinger X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mSvquapeUP9SdwEWqFL1NFCiFKijgCqAf Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 10/07/2015 06:28 AM, Pavel Raiskup wrote: > .. so it is (2.4.6 vs. 2.4.7~dev, user+sys) 7m23.5s vs 8m58.3s. It's n= ot > completely back yet but it's much better than v2.4.6. Thanks again for working on this. > =20 > +# func_quote ARG > +# -------------- > +# Aesthetically quote one ARG, store the result into $func_quote_resul= t. Note > +# that we keep attention to performance here (so far O(N) complexity a= s long as > +# func_append is O(1)). > +func_quote () > +{ > + $debug_cmd > + > + func_quote_result=3D$1 Common case - nothing needs escaping. Converts 'abc' to 'abc', as well as 'a b' to 'a b'. The caller can still blindly add outer "" for a printed form '"abc"' (unnecessary but harmless ""), or for '"a b"' (necessary in that case). > + > + case $func_quote_result in > + *[\\\`\"\$]*) Something needs escaping before being placed in double quotes. > + case $func_quote_result in > + *'*'*|*'['*) Problem: shouldn't this also include *'?'* to avoid globbing a single character? Could probably be written *[\[\*\?]*) The string itself contains globs, so... > + func_quote_result=3D`$ECHO "$func_quote_result" | $SED "$s= ed_quote_subst"` > + return 0 =2E..rather than worry about set +f, we just use sed in this rare case. Converts 'a*b' into 'a*b', converts 'a*"b' into 'a*\"b'. The caller then supplies outer "", for '"a*b"' or '"a*\"b"' (outer quotes necessary in both cases). > + ;; > + esac > + > + func_quote_old_IFS=3D$IFS > + for _G_char in '\' '`' '"' '$' > + do > + # STATE($1) PREV($2) SEPARATOR($3) > + set start "" "" > + func_quote_result=3Ddummy"$_G_char$func_quote_result$_G_char= "dummy > + IFS=3D$_G_char > + for _G_part in $func_quote_result > + do > + case $1 in > + quote) > + func_append func_quote_result "$3$2" > + set quote "$_G_part" "\\$_G_char" > + ;; > + start) > + set first "" "" > + func_quote_result=3D > + ;; > + first) > + set quote "$_G_part" "" > + ;; > + esac > + done > + IFS=3D$func_quote_old_IFS > + done > + ;; The string does not contain globs, so do four linear passes that escape each problem character. converts 'a"b' into 'a\"b', then caller adds outer quotes to become '"a\"b"'. > + *) ;; > + esac > +} Looks correct. However, is it also worth attempting a shell-specific speedup? After all, func_append uses shell-specific speedups (it is NOT as fast on shells that don't support +=3D). That is, the above function appears to be portable to all shells, but if we detect that a shell supports printf -v %q, can we use that instead for some additional speed? Of course, printf -v %q in bash prefers output that does NOT embed inside "" (it quotes ALL shell metacharacters using backslash), so we'd have to rework the contract of the function. That is, instead of the current function which returns a ready-escaped string but no outer "", we would instead be returning a string that already includes all necessary quoting. Which would mean rewriting all callers :( For example, we'd have to figure out what to do for func_quote_for_eval, which returns two values - func_quote_for_eval_result being fully quoted is easy with printf -v %q, and func_quote_for_eval_unquoted_result which is not. Here's a (lightly-tested) idea of what it would look like, where we'd have to audit every caller to deal with the result already including full quoting: if test yes =3D `(x=3D; printf -v x %q yes; echo $x) 2>/dev/null`; then func_quote() { printf -v func_quote_result %q "$1" } else func_quote() { portable version, except add: func_quote_result=3D"\"$func_quote_result\"" } fi Note that with this variant, the portable version converts 'a *"b' into '"a *\"b"', while the bash version converts it into 'a\ \ \*\"b'. > @@ -8596,8 +8597,8 @@ EOF > relink_command=3D"$var=3D$func_quote_for_eval_result; export $var= ; $relink_command" > fi > done > - relink_command=3D"(cd `pwd`; $relink_command)" > - relink_command=3D`$ECHO "$relink_command" | $SED "$sed_quote_subst"` > + func_quote "(cd `pwd`; $relink_command)" > + relink_command=3D$func_quote_result Unrelated to your patch, but doesn't this fail if pwd is called in an absolute directory containing spaces? --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --mSvquapeUP9SdwEWqFL1NFCiFKijgCqAf Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJWFSYVAAoJEKeha0olJ0NqQ8QIAIYcPLARZ8rcnPQTrrr7c6xl HtOdlMtgrCTT6u5uLUfn3hhT/L4XUZg3xonkyb3PH3q/WCA3p589dvocDqGW9eqQ hESk6XWzB7dZssEG7VBMzzpzNw5Xq8zJWHyqNzEuiEA54Zb7UWoSOyQ/ckLhR0S8 q6WiHov98lRjKK2Ihw/aQLVUjlOmYbfYa5SbbmoM7O8iuJoyN9ypMP+GDrCqB3Tx Wie1tf3Xg5HypHdw3nlFeuuGoek7pudDH6/35g5Ua2lZaV1NaT2dGJ4xQUY/RtbE qSM0HDM4DM93+Wkf9i2NWsRVFd525wkFz7Cj9gocNOu8PIdge10/6W6I5VA9dNE= =VobO -----END PGP SIGNATURE----- --mSvquapeUP9SdwEWqFL1NFCiFKijgCqAf-- From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 09 11:16:11 2015 Received: (at 20006) by debbugs.gnu.org; 9 Oct 2015 15:16:11 +0000 Received: from localhost ([127.0.0.1]:34431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZkZP0-00077P-P5 for submit@debbugs.gnu.org; Fri, 09 Oct 2015 11:16:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39914) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZkZOx-00077F-Jj for 20006@debbugs.gnu.org; Fri, 09 Oct 2015 11:16:09 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id B1F5F92462; Fri, 9 Oct 2015 15:16:06 +0000 (UTC) Received: from nb.usersys.redhat.com (ovpn-200-33.brq.redhat.com [10.40.200.33]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t99FG4MU023319 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Fri, 9 Oct 2015 11:16:05 -0400 From: Pavel Raiskup To: Eric Blake Subject: Re: Bash-specific performance by avoiding sed Date: Fri, 09 Oct 2015 17:16:03 +0200 Message-ID: <2297414.EmTqi36Kih@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.2-300.fc23.x86_64+debug; KDE/4.14.11; x86_64; ; ) In-Reply-To: <56152615.60903@redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <17836452.nsWZIHcxGW@nb.usersys.redhat.com> <56152615.60903@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Mike Frysinger , 20006@debbugs.gnu.org, Richard Purdie , libtool@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On Wednesday 07 of October 2015 08:03:01 Eric Blake wrote: > On 10/07/2015 06:28 AM, Pavel Raiskup wrote: > > > .. so it is (2.4.6 vs. 2.4.7~dev, user+sys) 7m23.5s vs 8m58.3s. It's not > > completely back yet but it's much better than v2.4.6. > > Thanks again for working on this. Thanks for perfect review, Eric. > > + case $func_quote_result in > > + *'*'*|*'['*) > > Problem: shouldn't this also include *'?'* to avoid globbing a single > character? Oh, sorry for this - and thanks, I'll fix that! > Could probably be written *[\[\*\?]*) > > The string itself contains globs, so... Ok. > > + ;; > > + esac > > + > > + func_quote_old_IFS=$IFS > > + for _G_char in '\' '`' '"' '$' > > + do > > + # STATE($1) PREV($2) SEPARATOR($3) > > + set start "" "" > > + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy > > + IFS=$_G_char > > + for _G_part in $func_quote_result > > + do > > + case $1 in > > + quote) > > + func_append func_quote_result "$3$2" > > + set quote "$_G_part" "\\$_G_char" > > + ;; > > + start) > > + set first "" "" > > + func_quote_result= > > + ;; > > + first) > > + set quote "$_G_part" "" > > + ;; > > + esac > > + done > > + IFS=$func_quote_old_IFS > > + done > > + ;; > > The string does not contain globs, so do four linear passes that escape > each problem character. To be honest, in shells without '+=' operator, these are four O(N^2) passes.. But still very acceptable parabola IMO. Firstly I tried to do this without this ugly state machine -- using $@ array to store strings. Very roughly: for _G_char in '\' '`' ... do IFS=$_G_char set dummy for _G_part $func_quote_result do set "$@" "\\$_G_char$_G_part" ... done shift # _now join $@ into one string_ done ... but that ended in quadratic complexity _everywhere_. I was curious why and it was because of the 'set "$@"' command being longer and longer for strings like '"""""""""""""""""""...'. > converts 'a"b' into 'a\"b', then caller adds outer quotes to become > '"a\"b"'. > > > + *) ;; > > + esac > > +} > > Looks correct. However, is it also worth attempting a shell-specific > speedup? After all, func_append uses shell-specific speedups (it is NOT > as fast on shells that don't support +=). Yes. We could do that (at least in future if not in this thread). > That is, the above function appears to be portable to all shells, but if > we detect that a shell supports printf -v %q, can we use that instead > for some additional speed? Of course, printf -v %q in bash prefers > output that does NOT embed inside "" (it quotes ALL shell metacharacters > using backslash), so we'd have to rework the contract of the function. Well, I was thinking about 'printf %q' firstly, but I rejected that idea because I missed the point that bash's printf has '-v' option (thus we can avoid forking!). Thanks for this point. > That is, instead of the current function which returns a ready-escaped > string but no outer "", we would instead be returning a string that > already includes all necessary quoting. Which would mean rewriting all > callers :( For example, we'd have to figure out what to do for > func_quote_for_eval, which returns two values - > func_quote_for_eval_result being fully quoted is easy with printf -v %q, > and func_quote_for_eval_unquoted_result which is not. Yes, rewriting all callers would raise a chance of my mistake. So I'm not really in favour of this dangerous action :) .. it would be just Bash speedup anyway (even zsh does not have 'printf -v'). Also, we would have to be careful where this optimized function would be used -- e.g. 'sed_quote_subst' has been historically used to generate '*.la' files. We should keep that format as is -- so we would have to have two versions of func_quote (one possibly with printf %q, one producing the old output). > Here's a (lightly-tested) idea of what it would look like, where we'd > have to audit every caller to deal with the result already including > full quoting: > > if test yes = `(x=; printf -v x %q yes; echo $x) 2>/dev/null`; then > func_quote() > { > printf -v func_quote_result %q "$1" > } > else > func_quote() > { > portable version, except add: > func_quote_result="\"$func_quote_result\"" > } > fi > > Note that with this variant, the portable version converts 'a *"b' into > '"a *\"b"', while the bash version converts it into 'a\ \ \*\"b'. Thanks! I've done a simple testing too .. and it seems to be equivalent to '$SED $sed_quote_subst' (in usecases like: string -> quote -> eval). Well -- I'm not 100% sure I want to hack this in and risk some issues. On the other hand, I'm able to review the patches if somebody wanted to give it a try (the small bash slowdown from v2.4.2 to v2.4.6 is not good motivation for me). Much more interesting (than fighting this quotation issue) would be an attempt to standardize some shell builtin which would have the same effects as 'func_quote_for_eval' (because returning "$@" array unchanged up to caller function is not an easy task). If that was available, we could easily (conditionally) reuse that... (some shell hackers could implement binary simulating this builtin..., etc.)... ... while I'm on it -- we could avoid a lot of calls to func_quote and func_quote_for_eval if we were able to _return arrays_ (or set global array variables). Because some shells can do this, maybe we could detect this feature and return $_result_array instead of quoted-for-eval $_result? > > @@ -8596,8 +8597,8 @@ EOF > > relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" > > fi > > done > > - relink_command="(cd `pwd`; $relink_command)" > > - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` > > + func_quote "(cd `pwd`; $relink_command)" > > + relink_command=$func_quote_result > > Unrelated to your patch, but doesn't this fail if pwd is called in an > absolute directory containing spaces? Yes, good catch. This should be fixed. Pavel From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 12 13:38:56 2015 Received: (at 20006) by debbugs.gnu.org; 12 Oct 2015 17:38:56 +0000 Received: from localhost ([127.0.0.1]:37467 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zlh3j-0000Vr-TG for submit@debbugs.gnu.org; Mon, 12 Oct 2015 13:38:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50893) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zlh3c-0000Vb-Ni for 20006@debbugs.gnu.org; Mon, 12 Oct 2015 13:38:50 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 3C328C0C1B0D; Mon, 12 Oct 2015 17:38:43 +0000 (UTC) Received: from nb.usersys.redhat.com (ovpn-200-23.brq.redhat.com [10.40.200.23]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t9CHcekb032420 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Mon, 12 Oct 2015 13:38:42 -0400 From: Pavel Raiskup To: Eric Blake Subject: Re: Bash-specific performance by avoiding sed Date: Mon, 12 Oct 2015 19:38:40 +0200 Message-ID: <3697027.6QQg1pv7aR@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.3-300.fc23.x86_64; KDE/4.14.11; x86_64; ; ) In-Reply-To: <2297414.EmTqi36Kih@nb.usersys.redhat.com> References: <450369038.1171999.1425925692399.JavaMail.yahoo@mail.yahoo.com> <56152615.60903@redhat.com> <2297414.EmTqi36Kih@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart3071762.M43Bqc6qy0" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 20006 Cc: Mike Frysinger , 20006@debbugs.gnu.org, Richard Purdie , libtool@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This is a multi-part message in MIME format. --nextPart3071762.M43Bqc6qy0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Friday 09 of October 2015 17:16:03 Pavel Raiskup wrote: > > Here's a (lightly-tested) idea of what it would look like, where we'd > > have to audit every caller to deal with the result already including > > full quoting: > > > > if test yes = `(x=; printf -v x %q yes; echo $x) 2>/dev/null`; then > > func_quote() > > { > > printf -v func_quote_result %q "$1" > > } > > else > > func_quote() > > { > > portable version, except add: > > func_quote_result="\"$func_quote_result\"" > > } > > fi > > > > Note that with this variant, the portable version converts 'a *"b' into > > '"a *\"b"', while the bash version converts it into 'a\ \ \*\"b'. > > Thanks! I've done a simple testing too .. and it seems to be equivalent > to '$SED $sed_quote_subst' (in usecases like: string -> quote -> eval). > > Well -- I'm not 100% sure I want to hack this in and risk some issues. On > the other hand, I'm able to review the patches if somebody wanted to give > it a try (the small bash slowdown from v2.4.2 to v2.4.6 is not good > motivation for me). Hm, I was thinking thinking about this over the weekend and I wanted to do some testing before I would definitely reject that idea. But it sounds like the printf builtin helps lets say significantly to bash and does not hurt others. I changed the function names so the changes to libtool code should be pretty transparent (even though it is rather large change). I'm posting results of the performance test (on systemd package) and attaching 3 "planned to be pushed" patches. So testing '2.4.2' version (fast), git-version after applying 0001 and 0002 (patch-2) and including 0003 (patch-3). The dirname of test is in format 'PACKAGE-SHELL-LTVERSION'. Pavel ----- %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-bash-2.4.2) 0:28.50 56.84 41.42 0:28.38 56.72 41.41 0:28.26 56.44 41.68 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-bash-2.4.6.13-f729) 1:06.13 232.12 23.33 1:06.52 232.28 23.35 1:06.54 232.16 23.52 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-bash-2.4.6.14-7a09) 0:46.52 151.16 24.48 0:46.64 150.80 24.60 0:46.69 151.12 23.85 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-dash-2.4.2) 0:16.34 22.22 28.62 0:17.13 22.57 29.09 0:17.11 22.37 29.24 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-dash-2.4.6.13-f729) 0:23.38 58.80 21.32 0:23.16 58.81 21.16 0:23.21 58.67 21.15 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-dash-2.4.6.14-7a09) 0:21.36 51.20 21.62 0:21.23 50.59 21.76 0:21.25 50.70 21.55 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-ksh-2.4.2) 0:19.65 30.76 34.12 0:21.09 30.68 34.63 0:19.75 30.83 34.29 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-ksh-2.4.6.13-f729) 0:30.75 85.60 25.41 0:30.70 85.51 25.52 0:30.80 85.65 25.41 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-ksh-2.4.6.14-7a09) 0:30.78 84.25 26.14 0:30.63 83.65 26.17 0:30.62 83.85 26.02 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-zsh-2.4.2) 0:27.94 44.47 48.97 0:28.03 44.29 49.31 0:28.60 44.48 48.93 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-zsh-2.4.6.13-f729) 0:47.60 140.30 41.70 0:47.53 139.84 41.89 0:47.56 139.71 42.12 %E %U %S (/home/praiskup/rh/projects/systemd-perf/systemd-zsh-2.4.6.14-7a09) 0:46.21 131.88 41.72 0:46.62 131.31 42.25 0:46.44 131.35 42.03 --nextPart3071762.M43Bqc6qy0 Content-Disposition: attachment; filename="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-libtool-mitigate-the-sed_quote_subst-slowdown.patch" >From 32f0df9835ac15ac17e04be57c368172c3ad1d19 Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 4 Oct 2015 21:55:03 +0200 Subject: [PATCH 1/3] libtool: mitigate the $sed_quote_subst slowdown When it is reasonably possible, use shell implementation for quoting. References: http://lists.gnu.org/archive/html/libtool/2015-03/msg00005.html http://lists.gnu.org/archive/html/libtool/2015-02/msg00000.html https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20006 * gl/build-aux/funclib.sh (func_quote): New function that can be used as substitution for '$SED $sed_quote_subst' call. * build-aux/ltmain.in (func_emit_wrapper): Use func_quote instead of '$SED $sed_quote_subst'. (func_mode_link): Likewise. * NEWS: Document. * bootstrap: Sync with funclib.sh. --- NEWS | 3 +++ bootstrap | 61 +++++++++++++++++++++++++++++++++++++++++++------ build-aux/ltmain.in | 10 ++++---- gl/build-aux/funclib.sh | 61 +++++++++++++++++++++++++++++++++++++++++++------ 4 files changed, 117 insertions(+), 18 deletions(-) diff --git a/NEWS b/NEWS index a3c5b12..7c23d03 100644 --- a/NEWS +++ b/NEWS @@ -17,6 +17,9 @@ NEWS - list of user-visible changes between releases of GNU Libtool - Fix significant slowdown of libtoolize for certain projects (regression introduced in 2.4.3 release) caused by infinite m4 macro recursion. + - Mitigate the slowdown of libtool script (introduced in v2.4.3) caused by + increased number of calls to '$SED $sed_quote_subst' (bug#20006). + * Noteworthy changes in release 2.4.6 (2015-02-15) [stable] ** New features: diff --git a/bootstrap b/bootstrap index c179f51..fe9e9ca 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1257,6 +1257,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *'*'*|*'['*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1273,12 +1324,8 @@ func_quote_for_eval () func_quote_for_eval_unquoted_result= func_quote_for_eval_result= while test 0 -lt $#; do - case $1 in - *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; - *) - _G_unquoted_arg=$1 ;; - esac + func_quote "$1" + _G_unquoted_arg=$func_quote_result if test -n "$func_quote_for_eval_unquoted_result"; then func_append func_quote_for_eval_unquoted_result " $_G_unquoted_arg" else diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 0c40da0..24acefd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -3346,7 +3346,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - qECHO=`$ECHO "$ECHO" | $SED "$sed_quote_subst"` + func_quote "$ECHO" + qECHO=$func_quote_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -8596,8 +8597,8 @@ EOF relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" fi done - relink_command="(cd `pwd`; $relink_command)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "(cd `pwd`; $relink_command)" + relink_command=$func_quote_result fi # Only actually do things if not in dry run mode. @@ -8843,7 +8844,8 @@ EOF done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - relink_command=`$ECHO "$relink_command" | $SED "$sed_quote_subst"` + func_quote "$relink_command" + relink_command=$func_quote_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 39d972e..47d8b95 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-04.22; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -1026,6 +1026,57 @@ func_relative_path () } +# func_quote ARG +# -------------- +# Aesthetically quote one ARG, store the result into $func_quote_result. Note +# that we keep attention to performance here (so far O(N) complexity as long as +# func_append is O(1)). +func_quote () +{ + $debug_cmd + + func_quote_result=$1 + + case $func_quote_result in + *[\\\`\"\$]*) + case $func_quote_result in + *[\[\*\?]*) + func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` + return 0 + ;; + esac + + func_quote_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' + do + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_result + do + case $1 in + quote) + func_append func_quote_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done + IFS=$func_quote_old_IFS + done + ;; + *) ;; + esac +} + + # func_quote_for_eval ARG... # -------------------------- # Aesthetically quote ARGs to be evaled later. @@ -1042,12 +1093,8 @@ func_quote_for_eval () func_quote_for_eval_unquoted_result= func_quote_for_eval_result= while test 0 -lt $#; do - case $1 in - *[\\\`\"\$]*) - _G_unquoted_arg=`printf '%s\n' "$1" |$SED "$sed_quote_subst"` ;; - *) - _G_unquoted_arg=$1 ;; - esac + func_quote "$1" + _G_unquoted_arg=$func_quote_result if test -n "$func_quote_for_eval_unquoted_result"; then func_append func_quote_for_eval_unquoted_result " $_G_unquoted_arg" else -- 2.5.0 --nextPart3071762.M43Bqc6qy0 Content-Disposition: attachment; filename="0002-libtool-optimizing-options-parser-hooks.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0002-libtool-optimizing-options-parser-hooks.patch" >From b5cadd557ef345fb90cd4375f3bc1299d704077c Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Mon, 5 Oct 2015 13:16:08 +0200 Subject: [PATCH 2/3] libtool: optimizing options-parser hooks Its not necessary to (re)func_quote_for_eval in each function in the hook hierarchy. Usually is enough if the leaf function does func_quote_for_eval and its caller just re-uses the _return variable. This is follow up for the previous commit. * gl/build-aux/options-parser (func_run_hooks): Propagate $EXIT_SUCCESS return code down to caller if *any* hook succeeded. Never re-quote the result -- either the arguments are left untouched, or the options have already been properly quoted by succeeding hooks. (func_parse_options): Quote '$@' and return $EXIT_SUCCESS only if we changed something. (func_validate_options): Likewise. (func_options_prep): Likewise. (func_options_finish): New hook-caller for 'func_options' hooks. (func_options): Propagate return value down to top-level caller, but pay attention we have always set $func_options_result. * build-aux/ltmain.in (libtool_options_prep): Quote '$@' and return $EXIT_SUCCESS only if we changed something. (libtool_parse_options): Likewise. * bootstrap: Sync gl/build-aux/with option-parser. --- bootstrap | 171 +++++++++++++++++++++++++++++++------------- build-aux/ltmain.in | 35 ++++++--- gl/build-aux/options-parser | 171 +++++++++++++++++++++++++++++++------------- 3 files changed, 271 insertions(+), 106 deletions(-) diff --git a/bootstrap b/bootstrap index fe9e9ca..4f00096 100755 --- a/bootstrap +++ b/bootstrap @@ -1583,7 +1583,7 @@ func_lt_ver () #! /bin/sh # Set a version string for this script. -scriptversion=2014-01-07.03; # UTC +scriptversion=2015-10-07.11; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -1743,6 +1743,8 @@ func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -1751,16 +1753,16 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' - - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=: + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + $_G_rc_run_hooks && func_run_hooks_result=$_G_hook_result } @@ -1770,10 +1772,16 @@ func_run_hooks () ## --------------- ## # In order to add your own option parsing hooks, you must accept the -# full positional parameter list in your hook function, remove any -# options that you action, and then pass back the remaining unprocessed +# full positional parameter list in your hook function, you may remove/edit +# any options that you action, and then pass back the remaining unprocessed # options in '_result', escaped suitably for -# 'eval'. Like this: +# 'eval'. In this case you also must return $EXIT_SUCCESS to let the +# hook's caller know that it should pay attention to +# '_result'. Returning $EXIT_FAILURE signalizes that +# arguments are left untouched by the hook and therefore caller will ignore the +# result variable. +# +# Like this: # # my_options_prep () # { @@ -1783,9 +1791,11 @@ func_run_hooks () # usage_message=$usage_message' # -s, --silent don'\''t print informational messages # ' -# -# func_quote_for_eval ${1+"$@"} -# my_options_prep_result=$func_quote_for_eval_result +# # No change in '$@' (ignored completely by this hook). There is +# # no need to do the equivalent (but slower) action: +# # func_quote_for_eval ${1+"$@"} +# # my_options_prep_result=$func_quote_for_eval_result +# false # } # func_add_hook func_options_prep my_options_prep # @@ -1794,25 +1804,37 @@ func_run_hooks () # { # $debug_cmd # +# args_changed=false +# # # Note that for efficiency, we parse as many options as we can # # recognise in a loop before passing the remainder back to the # # caller on the first unrecognised argument we encounter. # while test $# -gt 0; do # opt=$1; shift # case $opt in -# --silent|-s) opt_silent=: ;; +# --silent|-s) opt_silent=: +# args_changed=: +# ;; # # Separate non-argument short options: # -s*) func_split_short_opt "$_G_opt" # set dummy "$func_split_short_opt_name" \ # "-$func_split_short_opt_arg" ${1+"$@"} # shift +# args_changed=: # ;; -# *) set dummy "$_G_opt" "$*"; shift; break ;; +# *) # Make sure the first unrecognised option "$_G_opt" +# # is added back to "$@", we could need that later +# # if $args_changed is true. +# set dummy "$_G_opt" ${1+"$@"}; shift; break ;; # esac # done # -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# if $args_changed; then +# func_quote_for_eval ${1+"$@"} +# my_silent_option_result=$func_quote_for_eval_result +# fi +# +# $args_changed # } # func_add_hook func_parse_options my_silent_option # @@ -1824,16 +1846,32 @@ func_run_hooks () # $opt_silent && $opt_verbose && func_fatal_help "\ # '--silent' and '--verbose' options are mutually exclusive." # -# func_quote_for_eval ${1+"$@"} -# my_option_validation_result=$func_quote_for_eval_result +# false # } # func_add_hook func_validate_options my_option_validation # -# You'll alse need to manually amend $usage_message to reflect the extra +# You'll also need to manually amend $usage_message to reflect the extra # options you parse. It's preferable to append if you can, so that # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# Finishing the option parse loop (call 'func_options' hooks ATM). +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -1843,17 +1881,28 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + # Save modified positional parameters for caller. As a top-level + # options-parser function we always need to set the 'func_options_result' + # variable (regardless the $_G_rc_options value). + if $_G_rc_options; then + func_options_result=$_G_res_var + else + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -1862,9 +1911,9 @@ func_options () # All initialisations required before starting the option parse loop. # Note that when calling hook functions, we pass through the list of # positional parameters. If a hook function modifies that list, and -# needs to propogate that back to rest of this script, then the complete +# needs to propagate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning $EXIT_SUCCESS (otherwise $EXIT_FAILURE is returned). func_hookable func_options_prep func_options_prep () { @@ -1874,10 +1923,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -1891,18 +1944,20 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -1917,7 +1972,10 @@ func_parse_options () ;; --warnings|--warning|-W) - test $# = 0 && func_missing_arg $_G_opt && break + if test $# = 0 && func_missing_arg $_G_opt; then + _G_rc_parse_options=: + break + fi case " $warning_categories $1" in *" $1 "*) # trailing space prevents matching last $1 above @@ -1970,15 +2028,25 @@ func_parse_options () shift ;; - --) break ;; + --) _G_rc_parse_options=: ; break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"}; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -1991,16 +2059,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index 24acefd..b4c6bcd 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -358,6 +358,8 @@ libtool_options_prep () nonopt= preserve_args= + _G_rc_lt_options_prep=: + # Shorthand for --mode=foo, only valid as the first argument case $1 in clean|clea|cle|cl) @@ -381,11 +383,18 @@ libtool_options_prep () uninstall|uninstal|uninsta|uninst|unins|unin|uni|un|u) shift; set dummy --mode uninstall ${1+"$@"}; shift ;; + *) + _G_rc_lt_options_prep=false + ;; esac - # Pass back the list of options. - func_quote_for_eval ${1+"$@"} - libtool_options_prep_result=$func_quote_for_eval_result + if $_G_rc_lt_options_prep; then + # Pass back the list of options. + func_quote_for_eval ${1+"$@"} + libtool_options_prep_result=$func_quote_for_eval_result + fi + + $_G_rc_lt_options_prep } func_add_hook func_options_prep libtool_options_prep @@ -397,9 +406,12 @@ libtool_parse_options () { $debug_cmd + _G_rc_lt_parse_options=false + # Perform our own loop to consume as many options as possible in # each iteration. while test $# -gt 0; do + _G_match_lt_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -474,15 +486,22 @@ libtool_parse_options () func_append preserve_args " $_G_opt" ;; - # An option not handled by this hook function: - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + # An option not handled by this hook function: + *) set dummy "$_G_opt" ${1+"$@"} ; shift + _G_match_lt_parse_options=false + break + ;; esac + $_G_match_lt_parse_options && _G_rc_lt_parse_options=: done + if $_G_rc_lt_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + libtool_parse_options_result=$func_quote_for_eval_result + fi - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - libtool_parse_options_result=$func_quote_for_eval_result + $_G_rc_lt_parse_options } func_add_hook func_parse_options libtool_parse_options diff --git a/gl/build-aux/options-parser b/gl/build-aux/options-parser index d651f1d..4c7e9cb 100644 --- a/gl/build-aux/options-parser +++ b/gl/build-aux/options-parser @@ -1,7 +1,7 @@ #! /bin/sh # Set a version string for this script. -scriptversion=2014-01-07.03; # UTC +scriptversion=2015-10-07.11; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -161,6 +161,8 @@ func_run_hooks () { $debug_cmd + _G_rc_run_hooks=false + case " $hookable_fns " in *" $1 "*) ;; *) func_fatal_error "'$1' does not support hook funcions.n" ;; @@ -169,16 +171,16 @@ func_run_hooks () eval _G_hook_fns=\$$1_hooks; shift for _G_hook in $_G_hook_fns; do - eval $_G_hook '"$@"' - - # store returned options list back into positional - # parameters for next 'cmd' execution. - eval _G_hook_result=\$${_G_hook}_result - eval set dummy "$_G_hook_result"; shift + if eval $_G_hook '"$@"'; then + # store returned options list back into positional + # parameters for next 'cmd' execution. + eval _G_hook_result=\$${_G_hook}_result + eval set dummy "$_G_hook_result"; shift + _G_rc_run_hooks=: + fi done - func_quote_for_eval ${1+"$@"} - func_run_hooks_result=$func_quote_for_eval_result + $_G_rc_run_hooks && func_run_hooks_result=$_G_hook_result } @@ -188,10 +190,16 @@ func_run_hooks () ## --------------- ## # In order to add your own option parsing hooks, you must accept the -# full positional parameter list in your hook function, remove any -# options that you action, and then pass back the remaining unprocessed +# full positional parameter list in your hook function, you may remove/edit +# any options that you action, and then pass back the remaining unprocessed # options in '_result', escaped suitably for -# 'eval'. Like this: +# 'eval'. In this case you also must return $EXIT_SUCCESS to let the +# hook's caller know that it should pay attention to +# '_result'. Returning $EXIT_FAILURE signalizes that +# arguments are left untouched by the hook and therefore caller will ignore the +# result variable. +# +# Like this: # # my_options_prep () # { @@ -201,9 +209,11 @@ func_run_hooks () # usage_message=$usage_message' # -s, --silent don'\''t print informational messages # ' -# -# func_quote_for_eval ${1+"$@"} -# my_options_prep_result=$func_quote_for_eval_result +# # No change in '$@' (ignored completely by this hook). There is +# # no need to do the equivalent (but slower) action: +# # func_quote_for_eval ${1+"$@"} +# # my_options_prep_result=$func_quote_for_eval_result +# false # } # func_add_hook func_options_prep my_options_prep # @@ -212,25 +222,37 @@ func_run_hooks () # { # $debug_cmd # +# args_changed=false +# # # Note that for efficiency, we parse as many options as we can # # recognise in a loop before passing the remainder back to the # # caller on the first unrecognised argument we encounter. # while test $# -gt 0; do # opt=$1; shift # case $opt in -# --silent|-s) opt_silent=: ;; +# --silent|-s) opt_silent=: +# args_changed=: +# ;; # # Separate non-argument short options: # -s*) func_split_short_opt "$_G_opt" # set dummy "$func_split_short_opt_name" \ # "-$func_split_short_opt_arg" ${1+"$@"} # shift +# args_changed=: # ;; -# *) set dummy "$_G_opt" "$*"; shift; break ;; +# *) # Make sure the first unrecognised option "$_G_opt" +# # is added back to "$@", we could need that later +# # if $args_changed is true. +# set dummy "$_G_opt" ${1+"$@"}; shift; break ;; # esac # done # -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# if $args_changed; then +# func_quote_for_eval ${1+"$@"} +# my_silent_option_result=$func_quote_for_eval_result +# fi +# +# $args_changed # } # func_add_hook func_parse_options my_silent_option # @@ -242,16 +264,32 @@ func_run_hooks () # $opt_silent && $opt_verbose && func_fatal_help "\ # '--silent' and '--verbose' options are mutually exclusive." # -# func_quote_for_eval ${1+"$@"} -# my_option_validation_result=$func_quote_for_eval_result +# false # } # func_add_hook func_validate_options my_option_validation # -# You'll alse need to manually amend $usage_message to reflect the extra +# You'll also need to manually amend $usage_message to reflect the extra # options you parse. It's preferable to append if you can, so that # multiple option parsing hooks can be added safely. +# func_options_finish [ARG]... +# ---------------------------- +# Finishing the option parse loop (call 'func_options' hooks ATM). +func_options_finish () +{ + $debug_cmd + + _G_func_options_finish_exit=false + if func_run_hooks func_options ${1+"$@"}; then + func_options_finish_result=$func_run_hooks_result + _G_func_options_finish_exit=: + fi + + $_G_func_options_finish_exit +} + + # func_options [ARG]... # --------------------- # All the functions called inside func_options are hookable. See the @@ -261,17 +299,28 @@ func_options () { $debug_cmd - func_options_prep ${1+"$@"} - eval func_parse_options \ - ${func_options_prep_result+"$func_options_prep_result"} - eval func_validate_options \ - ${func_parse_options_result+"$func_parse_options_result"} + _G_rc_options=false - eval func_run_hooks func_options \ - ${func_validate_options_result+"$func_validate_options_result"} + for my_func in options_prep parse_options validate_options options_finish + do + if eval func_$my_func '${1+"$@"}'; then + eval _G_res_var='$'"func_${my_func}_result" + eval set dummy "$_G_res_var" ; shift + _G_rc_options=: + fi + done - # save modified positional parameters for caller - func_options_result=$func_run_hooks_result + # Save modified positional parameters for caller. As a top-level + # options-parser function we always need to set the 'func_options_result' + # variable (regardless the $_G_rc_options value). + if $_G_rc_options; then + func_options_result=$_G_res_var + else + func_quote_for_eval ${1+"$@"} + func_options_result=$func_quote_for_eval_result + fi + + $_G_rc_options } @@ -280,9 +329,9 @@ func_options () # All initialisations required before starting the option parse loop. # Note that when calling hook functions, we pass through the list of # positional parameters. If a hook function modifies that list, and -# needs to propogate that back to rest of this script, then the complete +# needs to propagate that back to rest of this script, then the complete # modified list must be put in 'func_run_hooks_result' before -# returning. +# returning $EXIT_SUCCESS (otherwise $EXIT_FAILURE is returned). func_hookable func_options_prep func_options_prep () { @@ -292,10 +341,14 @@ func_options_prep () opt_verbose=false opt_warning_types= - func_run_hooks func_options_prep ${1+"$@"} + _G_rc_options_prep=false + if func_run_hooks func_options_prep ${1+"$@"}; then + _G_rc_options_prep=: + # save modified positional parameters for caller + func_options_prep_result=$func_run_hooks_result + fi - # save modified positional parameters for caller - func_options_prep_result=$func_run_hooks_result + $_G_rc_options_prep } @@ -309,18 +362,20 @@ func_parse_options () func_parse_options_result= + _G_rc_parse_options=false # this just eases exit handling while test $# -gt 0; do # Defer to hook functions for initial option parsing, so they # get priority in the event of reusing an option name. - func_run_hooks func_parse_options ${1+"$@"} - - # Adjust func_parse_options positional parameters to match - eval set dummy "$func_run_hooks_result"; shift + if func_run_hooks func_parse_options ${1+"$@"}; then + eval set dummy "$func_run_hooks_result"; shift + _G_rc_parse_options=: + fi # Break out of the loop if we already parsed every option. test $# -gt 0 || break + _G_match_parse_options=: _G_opt=$1 shift case $_G_opt in @@ -335,7 +390,10 @@ func_parse_options () ;; --warnings|--warning|-W) - test $# = 0 && func_missing_arg $_G_opt && break + if test $# = 0 && func_missing_arg $_G_opt; then + _G_rc_parse_options=: + break + fi case " $warning_categories $1" in *" $1 "*) # trailing space prevents matching last $1 above @@ -388,15 +446,25 @@ func_parse_options () shift ;; - --) break ;; + --) _G_rc_parse_options=: ; break ;; -*) func_fatal_help "unrecognised option: '$_G_opt'" ;; - *) set dummy "$_G_opt" ${1+"$@"}; shift; break ;; + *) set dummy "$_G_opt" ${1+"$@"}; shift + _G_match_parse_options=false + break + ;; esac + + $_G_match_parse_options && _G_rc_parse_options=: done - # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + + if $_G_rc_parse_options; then + # save modified positional parameters for caller + func_quote_for_eval ${1+"$@"} + func_parse_options_result=$func_quote_for_eval_result + fi + + $_G_rc_parse_options } @@ -409,16 +477,21 @@ func_validate_options () { $debug_cmd + _G_rc_validate_options=false + # Display all warnings if -W was not given. test -n "$opt_warning_types" || opt_warning_types=" $warning_categories" - func_run_hooks func_validate_options ${1+"$@"} + if func_run_hooks func_validate_options ${1+"$@"}; then + # save modified positional parameters for caller + func_validate_options_result=$func_run_hooks_result + _G_rc_validate_options=: + fi # Bail if the options were screwed! $exit_cmd $EXIT_FAILURE - # save modified positional parameters for caller - func_validate_options_result=$func_run_hooks_result + $_G_rc_validate_options } -- 2.5.0 --nextPart3071762.M43Bqc6qy0 Content-Disposition: attachment; filename="0003-funclib-refactor-quoting-methods-a-bit.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0003-funclib-refactor-quoting-methods-a-bit.patch" >From 5762e2c82e53b6dd712f19985c90081684263810 Mon Sep 17 00:00:00 2001 From: Pavel Raiskup Date: Sun, 11 Oct 2015 14:35:15 +0200 Subject: [PATCH 3/3] funclib: refactor quoting methods a bit >From now we have two basic functions to perform string quoting for shell evaluation -- 'func_quote_arg' to quote one argument and 'func_quote' which takes list of arguments to be quoted. New function name-scheme should be more descriptive (previously we called func_quote_for_eval with one argument and also multiple arguments, while we had confusing $func_quote_for_eval_unquoted_result which is redundant for multiple-arguments call). New abstraction allowed us (in an easy way) to implement bash-specific optimization for quoting (using 'printf -v VARNAME %q "$value"', suggested by Eric Blake), this construct may be used on those places where we don't care much about the result aesthetics (its thus not useful for '*.la' generation or for error printing). * gl/build-aux/funclib.sh (func_append_quoted): Use func_quote_arg internally (kept in 'pretty' mode for now). (func_quote): Made to be "main" high-level quoting method taking list of arguments to be quoted into single command. It replaces func_quote_for_{expand,eval}. (func_quote_portable): Implements quoting in shell, falling back to portable sed call (rare cases). (func_quotefast_eval): New internal function using fast bash-specific construct, falling back to func_quote_portable for non-Bash scripts. (func_quote_arg): New function to quote one argument. (func_quote_for_eval): Removed. All callers changed to call func_quote. (func_quote_for_expand): Likewise. * bootstrap: Sync with funclib.sh and options-parser. --- bootstrap | 327 +++++++++++++++++++++++++------------------- build-aux/ltmain.in | 114 +++++++-------- gl/build-aux/bootstrap.in | 22 +-- gl/build-aux/funclib.sh | 279 +++++++++++++++++++++---------------- gl/build-aux/options-parser | 18 +-- gl/doc/bootstrap.texi | 4 +- libtoolize.in | 18 +-- 7 files changed, 440 insertions(+), 342 deletions(-) diff --git a/bootstrap b/bootstrap index 4f00096..1d86ab0 100755 --- a/bootstrap +++ b/bootstrap @@ -230,7 +230,7 @@ vc_ignore= # Source required external libraries: # Set a version string for this script. -scriptversion=2015-10-04.22; # UTC +scriptversion=2015-10-12.13; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -746,16 +746,16 @@ if test yes = "$_G_HAVE_PLUSEQ_OP"; then { $debug_cmd - func_quote_for_eval "$2" - eval "$1+=\\ \$func_quote_for_eval_result" + func_quote_arg pretty "$2" + eval "$1+=\\ \$func_quote_arg_result" }' else func_append_quoted () { $debug_cmd - func_quote_for_eval "$2" - eval "$1=\$$1\\ \$func_quote_for_eval_result" + func_quote_arg pretty "$2" + eval "$1=\$$1\\ \$func_quote_arg_result" } fi @@ -1257,132 +1257,181 @@ func_relative_path () } -# func_quote ARG -# -------------- -# Aesthetically quote one ARG, store the result into $func_quote_result. Note -# that we keep attention to performance here (so far O(N) complexity as long as -# func_append is O(1)). -func_quote () +# func_quote_portable EVAL ARG +# ---------------------------- +# Internal function to portably implement func_quote_arg. Note that we still +# keep attention to performance here so we as much as possible try to avoid +# calling sed binary (so far O(N) complexity as long as func_append is O(1)). +func_quote_portable () { $debug_cmd - func_quote_result=$1 + func_quote_portable_result=$2 - case $func_quote_result in - *[\\\`\"\$]*) - case $func_quote_result in - *'*'*|*'['*) - func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` - return 0 - ;; - esac + # one-time-loop (easy break) + while true + do + if $1; then + func_quote_portable_result=`$ECHO "$2" | $SED \ + -e "$sed_double_quote_subst" -e "$sed_double_backslash"` + break + fi - func_quote_old_IFS=$IFS - for _G_char in '\' '`' '"' '$' - do - # STATE($1) PREV($2) SEPARATOR($3) - set start "" "" - func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy - IFS=$_G_char - for _G_part in $func_quote_result + # Quote for eval. + case $func_quote_portable_result in + *[\\\`\"\$]*) + case $func_quote_portable_result in + *[\[\*\?]*) + func_quote_portable_result=`$ECHO "$func_quote_portable_result" | $SED "$sed_quote_subst"` + break + ;; + esac + + func_quote_portable_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' do - case $1 in - quote) - func_append func_quote_result "$3$2" - set quote "$_G_part" "\\$_G_char" - ;; - start) - set first "" "" - func_quote_result= - ;; - first) - set quote "$_G_part" "" - ;; - esac + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_portable_result=dummy"$_G_char$func_quote_portable_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_portable_result + do + case $1 in + quote) + func_append func_quote_portable_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_portable_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done done - IFS=$func_quote_old_IFS - done - ;; - *) ;; - esac -} - - -# func_quote_for_eval ARG... -# -------------------------- -# Aesthetically quote ARGs to be evaled later. -# This function returns two values: -# i) func_quote_for_eval_result -# double-quoted, suitable for a subsequent eval -# ii) func_quote_for_eval_unquoted_result -# has all characters that are still active within double -# quotes backslashified. -func_quote_for_eval () -{ - $debug_cmd - - func_quote_for_eval_unquoted_result= - func_quote_for_eval_result= - while test 0 -lt $#; do - func_quote "$1" - _G_unquoted_arg=$func_quote_result - if test -n "$func_quote_for_eval_unquoted_result"; then - func_append func_quote_for_eval_unquoted_result " $_G_unquoted_arg" - else - func_append func_quote_for_eval_unquoted_result "$_G_unquoted_arg" - fi - - case $_G_unquoted_arg in - # Double-quote args containing shell metacharacters to delay - # word splitting, command substitution and variable expansion - # for a subsequent eval. - # Many Bourne shells cannot handle close brackets correctly - # in scan sets, so we specify it separately. - *[\[\~\#\^\&\*\(\)\{\}\|\;\<\>\?\'\ \ ]*|*]*|"") - _G_quoted_arg=\"$_G_unquoted_arg\" + IFS=$func_quote_portable_old_IFS ;; - *) - _G_quoted_arg=$_G_unquoted_arg - ;; + *) ;; esac - - if test -n "$func_quote_for_eval_result"; then - func_append func_quote_for_eval_result " $_G_quoted_arg" - else - func_append func_quote_for_eval_result "$_G_quoted_arg" - fi - shift + break done -} - -# func_quote_for_expand ARG -# ------------------------- -# Aesthetically quote ARG to be evaled later; same as above, -# but do not quote variable references. -func_quote_for_expand () -{ - $debug_cmd - - case $1 in - *[\\\`\"]*) - _G_arg=`$ECHO "$1" | $SED \ - -e "$sed_double_quote_subst" -e "$sed_double_backslash"` ;; - *) - _G_arg=$1 ;; - esac - - case $_G_arg in - # Double-quote args containing shell metacharacters to delay - # word splitting and command substitution for a subsequent eval. - # Many Bourne shells cannot handle close brackets correctly + func_quote_portable_unquoted_result=$func_quote_portable_result + case $func_quote_portable_result in + # double-quote args containing shell metacharacters to delay + # word splitting, command substitution and variable expansion + # for a subsequent eval. + # many bourne shells cannot handle close brackets correctly # in scan sets, so we specify it separately. *[\[\~\#\^\&\*\(\)\{\}\|\;\<\>\?\'\ \ ]*|*]*|"") - _G_arg=\"$_G_arg\" + func_quote_portable_result=\"$func_quote_portable_result\" ;; esac +} - func_quote_for_expand_result=$_G_arg + +# func_quotefast_eval ARG +# ----------------------- +# Quote one ARG (internal). This is equivalent to 'func_quote_arg eval ARG', +# but optimized for speed. Result is stored in $func_quotefast_eval. +if test xyes = `(x=; printf -v x %q yes; echo x"$x") 2>/dev/null`; then + func_quotefast_eval () + { + printf -v func_quotefast_eval_result %q "$1" + } +else + func_quotefast_eval () + { + func_quote_portable false "$1" + func_quotefast_eval_result=$func_quote_portable_result + } +fi + + +# func_quote_arg MODEs ARG +# ------------------------ +# Quote one ARG to be evaled later. MODEs argument may contain zero ore more +# specifiers listed below separated by ',' character. This function returns two +# values: +# i) func_quote_arg_result +# double-quoted (when needed), suitable for a subsequent eval +# ii) func_quote_arg_unquoted_result +# has all characters that are still active within double +# quotes backslashified. Available only if 'unquoted' is specified. +# +# Available modes: +# ---------------- +# 'eval' (default) +# - escape shell special characters +# 'expand' +# - the same as 'eval'; but do not quote variable references +# 'pretty' +# - request aesthetic output, i.e. '"a b"' instead of 'a\ b'. This might +# later used in func_quote to get output like: 'echo "a b"' instead of +# 'echo a\ b'. This is slower than default on some shells. +# 'unquoted' +# - produce also $func_quote_arg_unquoted_result which does not contain +# wrapping double-quotes. +# +# Examples for 'func_quote_arg pretty,unquoted string': +# +# string | *_result | *_unquoted_result +# ------------+-----------------------+------------------- +# " | \" | \" +# a b | "a b" | a b +# "a b" | "\"a b\"" | \"a b\" +# * | "*" | * +# z="${x-$y}" | "z=\"\${x-\$y}\"" | z=\"\${x-\$y}\" +# +# Examples for 'func_quote_arg pretty,unquoted,expand string': +# +# string | *_result | *_unquoted_result +# --------------+---------------------+-------------------- +# z="${x-$y}" | "z=\"${x-$y}\"" | z=\"${x-$y}\" +func_quote_arg () +{ + _G_quote_expand=false + case ,$1, in + *,expand,*) + _G_quote_expand=: + ;; + esac + + case ,$1, in + *,pretty,*|*,expand,*|*,unquoted,*) + func_quote_portable $_G_quote_expand "$2" + func_quote_arg_result=$func_quote_portable_result + func_quote_arg_unquoted_result=$func_quote_portable_unquoted_result + ;; + *) + # Faster quote-for-eval for some shells. + func_quotefast_eval "$2" + func_quote_arg_result=$func_quotefast_eval_result + ;; + esac +} + + +# func_quote MODEs ARGs... +# ------------------------ +# Quote all ARGs to be evaled later and join them into single command. See +# func_quote_arg's description for more info. +func_quote () +{ + $debug_cmd + _G_func_quote_mode=$1 ; shift + func_quote_result= + while test 0 -lt $#; do + func_quote_arg "$_G_func_quote_mode" "$1" + if test -n "$func_quote_result"; then + func_append func_quote_result " $func_quote_arg_result" + else + func_append func_quote_result "$func_quote_arg_result" + fi + shift + done } @@ -1428,8 +1477,8 @@ func_show_eval () _G_cmd=$1 _G_fail_exp=${2-':'} - func_quote_for_expand "$_G_cmd" - eval "func_notquiet $func_quote_for_expand_result" + func_quote_arg pretty,expand "$_G_cmd" + eval "func_notquiet $func_quote_arg_result" $opt_dry_run || { eval "$_G_cmd" @@ -1454,8 +1503,8 @@ func_show_eval_locale () _G_fail_exp=${2-':'} $opt_quiet || { - func_quote_for_expand "$_G_cmd" - eval "func_echo $func_quote_for_expand_result" + func_quote_arg expand,pretty "$_G_cmd" + eval "func_echo $func_quote_arg_result" } $opt_dry_run || { @@ -1583,7 +1632,7 @@ func_lt_ver () #! /bin/sh # Set a version string for this script. -scriptversion=2015-10-07.11; # UTC +scriptversion=2015-10-12.13; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -1793,8 +1842,8 @@ func_run_hooks () # ' # # No change in '$@' (ignored completely by this hook). There is # # no need to do the equivalent (but slower) action: -# # func_quote_for_eval ${1+"$@"} -# # my_options_prep_result=$func_quote_for_eval_result +# # func_quote eval ${1+"$@"} +# # my_options_prep_result=$func_quote_result # false # } # func_add_hook func_options_prep my_options_prep @@ -1830,8 +1879,8 @@ func_run_hooks () # done # # if $args_changed; then -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# func_quote eval ${1+"$@"} +# my_silent_option_result=$func_quote_result # fi # # $args_changed @@ -1898,8 +1947,8 @@ func_options () if $_G_rc_options; then func_options_result=$_G_res_var else - func_quote_for_eval ${1+"$@"} - func_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + func_options_result=$func_quote_result fi $_G_rc_options @@ -2042,8 +2091,8 @@ func_parse_options () if $_G_rc_parse_options; then # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + func_parse_options_result=$func_quote_result fi $_G_rc_parse_options @@ -2753,7 +2802,7 @@ test extract-trace = "$progname" && func_main "$@" # End: # Set a version string for *this* script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-12.13; # UTC ## ------------------- ## @@ -2781,8 +2830,8 @@ func_bootstrap () # Save the current positional parameters to prevent them being # corrupted by calls to 'set' in 'func_init'. - func_quote_for_eval ${1+"$@"} - _G_saved_positional_parameters=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + _G_saved_positional_parameters=$func_quote_result # Initialisation. func_init @@ -4821,8 +4870,8 @@ func_show_eval () _G_fail_exp=${2-':'} ${opt_silent-'false'} || { - func_quote_for_eval $_G_cmd - eval func_truncate_cmd $func_quote_for_eval_result + func_quote eval $_G_cmd + eval func_truncate_cmd $func_quote_result func_echo "running: $tc_bold$func_truncate_cmd_result$tc_reset" } @@ -5209,8 +5258,8 @@ bootstrap_options_prep () opt_skip_po=false # Pass back the list of options we consumed. - func_quote_for_eval ${1+"$@"} - bootstrap_options_prep_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_options_prep_result=$func_quote_result } func_add_hook func_options_prep bootstrap_options_prep @@ -5260,8 +5309,8 @@ bootstrap_parse_options () done # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - bootstrap_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_parse_options_result=$func_quote_result } func_add_hook func_parse_options bootstrap_parse_options @@ -5279,8 +5328,8 @@ bootstrap_validate_options () && func_fatal_help "too many arguments" # Pass back the (empty) list of unconsumed options. - func_quote_for_eval ${1+"$@"} - bootstrap_validate_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_validate_options_result=$func_quote_result } func_add_hook func_validate_options bootstrap_validate_options diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index b4c6bcd..1cbe875 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -390,8 +390,8 @@ libtool_options_prep () if $_G_rc_lt_options_prep; then # Pass back the list of options. - func_quote_for_eval ${1+"$@"} - libtool_options_prep_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtool_options_prep_result=$func_quote_result fi $_G_rc_lt_options_prep @@ -497,8 +497,8 @@ libtool_parse_options () if $_G_rc_lt_parse_options; then # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - libtool_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtool_parse_options_result=$func_quote_result fi $_G_rc_lt_parse_options @@ -558,8 +558,8 @@ libtool_validate_options () } # Pass back the unparsed argument list - func_quote_for_eval ${1+"$@"} - libtool_validate_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtool_validate_options_result=$func_quote_result } func_add_hook func_validate_options libtool_validate_options @@ -1525,8 +1525,8 @@ func_mode_compile () esac done - func_quote_for_eval "$libobj" - test "X$libobj" != "X$func_quote_for_eval_result" \ + func_quote_arg pretty "$libobj" + test "X$libobj" != "X$func_quote_arg_result" \ && $ECHO "X$libobj" | $GREP '[]~#^*{};<>?"'"'"' &()|`$[]' \ && func_warning "libobj name '$libobj' may not contain shell special characters." func_dirname_and_basename "$obj" "/" "" @@ -1599,8 +1599,8 @@ compiler." func_to_tool_file "$srcfile" func_convert_file_msys_to_w32 srcfile=$func_to_tool_file_result - func_quote_for_eval "$srcfile" - qsrcfile=$func_quote_for_eval_result + func_quote_arg pretty "$srcfile" + qsrcfile=$func_quote_arg_result # Only build a PIC object if we are building libtool libraries. if test yes = "$build_libtool_libs"; then @@ -2203,8 +2203,8 @@ func_mode_install () case $nonopt in *shtool*) :;; *) false;; esac then # Aesthetically quote it. - func_quote_for_eval "$nonopt" - install_prog="$func_quote_for_eval_result " + func_quote_arg pretty "$nonopt" + install_prog="$func_quote_arg_result " arg=$1 shift else @@ -2214,8 +2214,8 @@ func_mode_install () # The real first argument should be the name of the installation program. # Aesthetically quote it. - func_quote_for_eval "$arg" - func_append install_prog "$func_quote_for_eval_result" + func_quote_arg pretty "$arg" + func_append install_prog "$func_quote_arg_result" install_shared_prog=$install_prog case " $install_prog " in *[\\\ /]cp\ *) install_cp=: ;; @@ -2272,12 +2272,12 @@ func_mode_install () esac # Aesthetically quote the argument. - func_quote_for_eval "$arg" - func_append install_prog " $func_quote_for_eval_result" + func_quote_arg pretty "$arg" + func_append install_prog " $func_quote_arg_result" if test -n "$arg2"; then - func_quote_for_eval "$arg2" + func_quote_arg pretty "$arg2" fi - func_append install_shared_prog " $func_quote_for_eval_result" + func_append install_shared_prog " $func_quote_arg_result" done test -z "$install_prog" && \ @@ -2288,8 +2288,8 @@ func_mode_install () if test -n "$install_override_mode" && $no_mode; then if $install_cp; then :; else - func_quote_for_eval "$install_override_mode" - func_append install_shared_prog " -m $func_quote_for_eval_result" + func_quote_arg pretty "$install_override_mode" + func_append install_shared_prog " -m $func_quote_arg_result" fi fi @@ -2585,8 +2585,8 @@ func_mode_install () relink_command=`$ECHO "$relink_command" | $SED 's%@OUTPUT@%'"$outputname"'%g'` $opt_quiet || { - func_quote_for_expand "$relink_command" - eval "func_echo $func_quote_for_expand_result" + func_quote_arg expand,pretty "$relink_command" + eval "func_echo $func_quote_arg_result" } if eval "$relink_command"; then : else @@ -3365,8 +3365,8 @@ else if test \"\$libtool_execute_magic\" != \"$magic\"; then file=\"\$0\"" - func_quote "$ECHO" - qECHO=$func_quote_result + func_quote_arg pretty "$ECHO" + qECHO=$func_quote_arg_result $ECHO "\ # A function that is used when there is no print builtin or printf. @@ -3376,7 +3376,7 @@ func_fallback_echo () \$1 _LTECHO_EOF' } - ECHO=\"$qECHO\" + ECHO=$qECHO fi # Very basic option parsing. These options are (a) specific to @@ -4719,9 +4719,9 @@ func_mode_link () while test "$#" -gt 0; do arg=$1 shift - func_quote_for_eval "$arg" - qarg=$func_quote_for_eval_unquoted_result - func_append libtool_args " $func_quote_for_eval_result" + func_quote_arg pretty,unquoted "$arg" + qarg=$func_quote_arg_unquoted_result + func_append libtool_args " $func_quote_arg_result" # If the previous option needs an argument, assign it. if test -n "$prev"; then @@ -5319,9 +5319,9 @@ func_mode_link () save_ifs=$IFS; IFS=, for flag in $args; do IFS=$save_ifs - func_quote_for_eval "$flag" - func_append arg " $func_quote_for_eval_result" - func_append compiler_flags " $func_quote_for_eval_result" + func_quote_arg pretty "$flag" + func_append arg " $func_quote_arg_result" + func_append compiler_flags " $func_quote_arg_result" done IFS=$save_ifs func_stripname ' ' '' "$arg" @@ -5335,10 +5335,10 @@ func_mode_link () save_ifs=$IFS; IFS=, for flag in $args; do IFS=$save_ifs - func_quote_for_eval "$flag" - func_append arg " $wl$func_quote_for_eval_result" - func_append compiler_flags " $wl$func_quote_for_eval_result" - func_append linker_flags " $func_quote_for_eval_result" + func_quote_arg pretty "$flag" + func_append arg " $wl$func_quote_arg_result" + func_append compiler_flags " $wl$func_quote_arg_result" + func_append linker_flags " $func_quote_arg_result" done IFS=$save_ifs func_stripname ' ' '' "$arg" @@ -5362,8 +5362,8 @@ func_mode_link () # -msg_* for osf cc -msg_*) - func_quote_for_eval "$arg" - arg=$func_quote_for_eval_result + func_quote_arg pretty "$arg" + arg=$func_quote_arg_result ;; # Flags to be passed through unchanged, with rationale: @@ -5386,8 +5386,8 @@ func_mode_link () -t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \ -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \ -specs=*) - func_quote_for_eval "$arg" - arg=$func_quote_for_eval_result + func_quote_arg pretty "$arg" + arg=$func_quote_arg_result func_append compile_command " $arg" func_append finalize_command " $arg" func_append compiler_flags " $arg" @@ -5408,15 +5408,15 @@ func_mode_link () continue else # Otherwise treat like 'Some other compiler flag' below - func_quote_for_eval "$arg" - arg=$func_quote_for_eval_result + func_quote_arg pretty "$arg" + arg=$func_quote_arg_result fi ;; # Some other compiler flag. -* | +*) - func_quote_for_eval "$arg" - arg=$func_quote_for_eval_result + func_quote_arg pretty "$arg" + arg=$func_quote_arg_result ;; *.$objext) @@ -5536,8 +5536,8 @@ func_mode_link () *) # Unknown arguments in both finalize_command and compile_command need # to be aesthetically quoted because they are evaled later. - func_quote_for_eval "$arg" - arg=$func_quote_for_eval_result + func_quote_arg pretty "$arg" + arg=$func_quote_arg_result ;; esac # arg @@ -8043,8 +8043,8 @@ EOF for cmd in $concat_cmds; do IFS=$save_ifs $opt_quiet || { - func_quote_for_expand "$cmd" - eval "func_echo $func_quote_for_expand_result" + func_quote_arg expand,pretty "$cmd" + eval "func_echo $func_quote_arg_result" } $opt_dry_run || eval "$cmd" || { lt_exit=$? @@ -8137,8 +8137,8 @@ EOF eval cmd=\"$cmd\" IFS=$save_ifs $opt_quiet || { - func_quote_for_expand "$cmd" - eval "func_echo $func_quote_for_expand_result" + func_quote_arg expand,pretty "$cmd" + eval "func_echo $func_quote_arg_result" } $opt_dry_run || eval "$cmd" || { lt_exit=$? @@ -8612,12 +8612,12 @@ EOF elif eval var_value=\$$var; test -z "$var_value"; then relink_command="$var=; export $var; $relink_command" else - func_quote_for_eval "$var_value" - relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" + func_quote_arg pretty "$var_value" + relink_command="$var=$func_quote_arg_result; export $var; $relink_command" fi done - func_quote "(cd `pwd`; $relink_command)" - relink_command=$func_quote_result + func_quote_arg pretty,unquoted "(cd `pwd`; $relink_command)" + relink_command=$func_quote_arg_unquoted_result fi # Only actually do things if not in dry run mode. @@ -8857,14 +8857,14 @@ EOF elif eval var_value=\$$var; test -z "$var_value"; then relink_command="$var=; export $var; $relink_command" else - func_quote_for_eval "$var_value" - relink_command="$var=$func_quote_for_eval_result; export $var; $relink_command" + func_quote_arg pretty,unquoted "$var_value" + relink_command="$var=$func_quote_arg_unquoted_result; export $var; $relink_command" fi done # Quote the link command for shipping. relink_command="(cd `pwd`; $SHELL \"$progpath\" $preserve_args --mode=relink $libtool_args @inst_prefix_dir@)" - func_quote "$relink_command" - relink_command=$func_quote_result + func_quote_arg pretty,unquoted "$relink_command" + relink_command=$func_quote_arg_unquoted_result if test yes = "$hardcode_automatic"; then relink_command= fi diff --git a/gl/build-aux/bootstrap.in b/gl/build-aux/bootstrap.in index 6e686da..6da21d9 100755 --- a/gl/build-aux/bootstrap.in +++ b/gl/build-aux/bootstrap.in @@ -232,7 +232,7 @@ vc_ignore= . `echo "$0" |${SED-sed} 's|[^/]*$||'`"extract-trace" # Set a version string for *this* script. -scriptversion=2015-01-20.17; # UTC +scriptversion=2015-10-12.13; # UTC ## ------------------- ## @@ -260,8 +260,8 @@ func_bootstrap () # Save the current positional parameters to prevent them being # corrupted by calls to 'set' in 'func_init'. - func_quote_for_eval ${1+"$@"} - _G_saved_positional_parameters=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + _G_saved_positional_parameters=$func_quote_result # Initialisation. func_init @@ -2300,8 +2300,8 @@ func_show_eval () _G_fail_exp=${2-':'} ${opt_silent-'false'} || { - func_quote_for_eval $_G_cmd - eval func_truncate_cmd $func_quote_for_eval_result + func_quote eval $_G_cmd + eval func_truncate_cmd $func_quote_result func_echo "running: $tc_bold$func_truncate_cmd_result$tc_reset" } @@ -2688,8 +2688,8 @@ bootstrap_options_prep () opt_skip_po=false # Pass back the list of options we consumed. - func_quote_for_eval ${1+"$@"} - bootstrap_options_prep_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_options_prep_result=$func_quote_result } func_add_hook func_options_prep bootstrap_options_prep @@ -2739,8 +2739,8 @@ bootstrap_parse_options () done # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - bootstrap_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_parse_options_result=$func_quote_result } func_add_hook func_parse_options bootstrap_parse_options @@ -2758,8 +2758,8 @@ bootstrap_validate_options () && func_fatal_help "too many arguments" # Pass back the (empty) list of unconsumed options. - func_quote_for_eval ${1+"$@"} - bootstrap_validate_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + bootstrap_validate_options_result=$func_quote_result } func_add_hook func_validate_options bootstrap_validate_options diff --git a/gl/build-aux/funclib.sh b/gl/build-aux/funclib.sh index 47d8b95..2c22db1 100644 --- a/gl/build-aux/funclib.sh +++ b/gl/build-aux/funclib.sh @@ -1,5 +1,5 @@ # Set a version string for this script. -scriptversion=2015-10-04.22; # UTC +scriptversion=2015-10-12.13; # UTC # General shell script boiler plate, and helper functions. # Written by Gary V. Vaughan, 2004 @@ -515,16 +515,16 @@ if test yes = "$_G_HAVE_PLUSEQ_OP"; then { $debug_cmd - func_quote_for_eval "$2" - eval "$1+=\\ \$func_quote_for_eval_result" + func_quote_arg pretty "$2" + eval "$1+=\\ \$func_quote_arg_result" }' else func_append_quoted () { $debug_cmd - func_quote_for_eval "$2" - eval "$1=\$$1\\ \$func_quote_for_eval_result" + func_quote_arg pretty "$2" + eval "$1=\$$1\\ \$func_quote_arg_result" } fi @@ -1026,135 +1026,184 @@ func_relative_path () } -# func_quote ARG -# -------------- -# Aesthetically quote one ARG, store the result into $func_quote_result. Note -# that we keep attention to performance here (so far O(N) complexity as long as -# func_append is O(1)). -func_quote () +# func_quote_portable EVAL ARG +# ---------------------------- +# Internal function to portably implement func_quote_arg. Note that we still +# keep attention to performance here so we as much as possible try to avoid +# calling sed binary (so far O(N) complexity as long as func_append is O(1)). +func_quote_portable () { $debug_cmd - func_quote_result=$1 + func_quote_portable_result=$2 - case $func_quote_result in - *[\\\`\"\$]*) - case $func_quote_result in - *[\[\*\?]*) - func_quote_result=`$ECHO "$func_quote_result" | $SED "$sed_quote_subst"` - return 0 - ;; - esac + # one-time-loop (easy break) + while true + do + if $1; then + func_quote_portable_result=`$ECHO "$2" | $SED \ + -e "$sed_double_quote_subst" -e "$sed_double_backslash"` + break + fi - func_quote_old_IFS=$IFS - for _G_char in '\' '`' '"' '$' - do - # STATE($1) PREV($2) SEPARATOR($3) - set start "" "" - func_quote_result=dummy"$_G_char$func_quote_result$_G_char"dummy - IFS=$_G_char - for _G_part in $func_quote_result + # Quote for eval. + case $func_quote_portable_result in + *[\\\`\"\$]*) + case $func_quote_portable_result in + *[\[\*\?]*) + func_quote_portable_result=`$ECHO "$func_quote_portable_result" | $SED "$sed_quote_subst"` + break + ;; + esac + + func_quote_portable_old_IFS=$IFS + for _G_char in '\' '`' '"' '$' do - case $1 in - quote) - func_append func_quote_result "$3$2" - set quote "$_G_part" "\\$_G_char" - ;; - start) - set first "" "" - func_quote_result= - ;; - first) - set quote "$_G_part" "" - ;; - esac + # STATE($1) PREV($2) SEPARATOR($3) + set start "" "" + func_quote_portable_result=dummy"$_G_char$func_quote_portable_result$_G_char"dummy + IFS=$_G_char + for _G_part in $func_quote_portable_result + do + case $1 in + quote) + func_append func_quote_portable_result "$3$2" + set quote "$_G_part" "\\$_G_char" + ;; + start) + set first "" "" + func_quote_portable_result= + ;; + first) + set quote "$_G_part" "" + ;; + esac + done done - IFS=$func_quote_old_IFS - done + IFS=$func_quote_portable_old_IFS + ;; + *) ;; + esac + break + done + + func_quote_portable_unquoted_result=$func_quote_portable_result + case $func_quote_portable_result in + # double-quote args containing shell metacharacters to delay + # word splitting, command substitution and variable expansion + # for a subsequent eval. + # many bourne shells cannot handle close brackets correctly + # in scan sets, so we specify it separately. + *[\[\~\#\^\&\*\(\)\{\}\|\;\<\>\?\'\ \ ]*|*]*|"") + func_quote_portable_result=\"$func_quote_portable_result\" ;; - *) ;; esac } -# func_quote_for_eval ARG... -# -------------------------- -# Aesthetically quote ARGs to be evaled later. -# This function returns two values: -# i) func_quote_for_eval_result -# double-quoted, suitable for a subsequent eval -# ii) func_quote_for_eval_unquoted_result +# func_quotefast_eval ARG +# ----------------------- +# Quote one ARG (internal). This is equivalent to 'func_quote_arg eval ARG', +# but optimized for speed. Result is stored in $func_quotefast_eval. +if test xyes = `(x=; printf -v x %q yes; echo x"$x") 2>/dev/null`; then + func_quotefast_eval () + { + printf -v func_quotefast_eval_result %q "$1" + } +else + func_quotefast_eval () + { + func_quote_portable false "$1" + func_quotefast_eval_result=$func_quote_portable_result + } +fi + + +# func_quote_arg MODEs ARG +# ------------------------ +# Quote one ARG to be evaled later. MODEs argument may contain zero ore more +# specifiers listed below separated by ',' character. This function returns two +# values: +# i) func_quote_arg_result +# double-quoted (when needed), suitable for a subsequent eval +# ii) func_quote_arg_unquoted_result # has all characters that are still active within double -# quotes backslashified. -func_quote_for_eval () +# quotes backslashified. Available only if 'unquoted' is specified. +# +# Available modes: +# ---------------- +# 'eval' (default) +# - escape shell special characters +# 'expand' +# - the same as 'eval'; but do not quote variable references +# 'pretty' +# - request aesthetic output, i.e. '"a b"' instead of 'a\ b'. This might +# later used in func_quote to get output like: 'echo "a b"' instead of +# 'echo a\ b'. This is slower than default on some shells. +# 'unquoted' +# - produce also $func_quote_arg_unquoted_result which does not contain +# wrapping double-quotes. +# +# Examples for 'func_quote_arg pretty,unquoted string': +# +# string | *_result | *_unquoted_result +# ------------+-----------------------+------------------- +# " | \" | \" +# a b | "a b" | a b +# "a b" | "\"a b\"" | \"a b\" +# * | "*" | * +# z="${x-$y}" | "z=\"\${x-\$y}\"" | z=\"\${x-\$y}\" +# +# Examples for 'func_quote_arg pretty,unquoted,expand string': +# +# string | *_result | *_unquoted_result +# --------------+---------------------+-------------------- +# z="${x-$y}" | "z=\"${x-$y}\"" | z=\"${x-$y}\" +func_quote_arg () +{ + _G_quote_expand=false + case ,$1, in + *,expand,*) + _G_quote_expand=: + ;; + esac + + case ,$1, in + *,pretty,*|*,expand,*|*,unquoted,*) + func_quote_portable $_G_quote_expand "$2" + func_quote_arg_result=$func_quote_portable_result + func_quote_arg_unquoted_result=$func_quote_portable_unquoted_result + ;; + *) + # Faster quote-for-eval for some shells. + func_quotefast_eval "$2" + func_quote_arg_result=$func_quotefast_eval_result + ;; + esac +} + + +# func_quote MODEs ARGs... +# ------------------------ +# Quote all ARGs to be evaled later and join them into single command. See +# func_quote_arg's description for more info. +func_quote () { $debug_cmd - - func_quote_for_eval_unquoted_result= - func_quote_for_eval_result= + _G_func_quote_mode=$1 ; shift + func_quote_result= while test 0 -lt $#; do - func_quote "$1" - _G_unquoted_arg=$func_quote_result - if test -n "$func_quote_for_eval_unquoted_result"; then - func_append func_quote_for_eval_unquoted_result " $_G_unquoted_arg" + func_quote_arg "$_G_func_quote_mode" "$1" + if test -n "$func_quote_result"; then + func_append func_quote_result " $func_quote_arg_result" else - func_append func_quote_for_eval_unquoted_result "$_G_unquoted_arg" - fi - - case $_G_unquoted_arg in - # Double-quote args containing shell metacharacters to delay - # word splitting, command substitution and variable expansion - # for a subsequent eval. - # Many Bourne shells cannot handle close brackets correctly - # in scan sets, so we specify it separately. - *[\[\~\#\^\&\*\(\)\{\}\|\;\<\>\?\'\ \ ]*|*]*|"") - _G_quoted_arg=\"$_G_unquoted_arg\" - ;; - *) - _G_quoted_arg=$_G_unquoted_arg - ;; - esac - - if test -n "$func_quote_for_eval_result"; then - func_append func_quote_for_eval_result " $_G_quoted_arg" - else - func_append func_quote_for_eval_result "$_G_quoted_arg" + func_append func_quote_result "$func_quote_arg_result" fi shift done } -# func_quote_for_expand ARG -# ------------------------- -# Aesthetically quote ARG to be evaled later; same as above, -# but do not quote variable references. -func_quote_for_expand () -{ - $debug_cmd - - case $1 in - *[\\\`\"]*) - _G_arg=`$ECHO "$1" | $SED \ - -e "$sed_double_quote_subst" -e "$sed_double_backslash"` ;; - *) - _G_arg=$1 ;; - esac - - case $_G_arg in - # Double-quote args containing shell metacharacters to delay - # word splitting and command substitution for a subsequent eval. - # Many Bourne shells cannot handle close brackets correctly - # in scan sets, so we specify it separately. - *[\[\~\#\^\&\*\(\)\{\}\|\;\<\>\?\'\ \ ]*|*]*|"") - _G_arg=\"$_G_arg\" - ;; - esac - - func_quote_for_expand_result=$_G_arg -} - - # func_stripname PREFIX SUFFIX NAME # --------------------------------- # strip PREFIX and SUFFIX from NAME, and store in func_stripname_result. @@ -1197,8 +1246,8 @@ func_show_eval () _G_cmd=$1 _G_fail_exp=${2-':'} - func_quote_for_expand "$_G_cmd" - eval "func_notquiet $func_quote_for_expand_result" + func_quote_arg pretty,expand "$_G_cmd" + eval "func_notquiet $func_quote_arg_result" $opt_dry_run || { eval "$_G_cmd" @@ -1223,8 +1272,8 @@ func_show_eval_locale () _G_fail_exp=${2-':'} $opt_quiet || { - func_quote_for_expand "$_G_cmd" - eval "func_echo $func_quote_for_expand_result" + func_quote_arg expand,pretty "$_G_cmd" + eval "func_echo $func_quote_arg_result" } $opt_dry_run || { diff --git a/gl/build-aux/options-parser b/gl/build-aux/options-parser index 4c7e9cb..c2bf9b4 100644 --- a/gl/build-aux/options-parser +++ b/gl/build-aux/options-parser @@ -1,7 +1,7 @@ #! /bin/sh # Set a version string for this script. -scriptversion=2015-10-07.11; # UTC +scriptversion=2015-10-12.13; # UTC # A portable, pluggable option parser for Bourne shell. # Written by Gary V. Vaughan, 2010 @@ -211,8 +211,8 @@ func_run_hooks () # ' # # No change in '$@' (ignored completely by this hook). There is # # no need to do the equivalent (but slower) action: -# # func_quote_for_eval ${1+"$@"} -# # my_options_prep_result=$func_quote_for_eval_result +# # func_quote eval ${1+"$@"} +# # my_options_prep_result=$func_quote_result # false # } # func_add_hook func_options_prep my_options_prep @@ -248,8 +248,8 @@ func_run_hooks () # done # # if $args_changed; then -# func_quote_for_eval ${1+"$@"} -# my_silent_option_result=$func_quote_for_eval_result +# func_quote eval ${1+"$@"} +# my_silent_option_result=$func_quote_result # fi # # $args_changed @@ -316,8 +316,8 @@ func_options () if $_G_rc_options; then func_options_result=$_G_res_var else - func_quote_for_eval ${1+"$@"} - func_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + func_options_result=$func_quote_result fi $_G_rc_options @@ -460,8 +460,8 @@ func_parse_options () if $_G_rc_parse_options; then # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - func_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + func_parse_options_result=$func_quote_result fi $_G_rc_parse_options diff --git a/gl/doc/bootstrap.texi b/gl/doc/bootstrap.texi index 2f7b382..b28bd71 100755 --- a/gl/doc/bootstrap.texi +++ b/gl/doc/bootstrap.texi @@ -390,8 +390,8 @@ my_silent_option () esac # return modified option list - func_quote_for_eval $@{1+"$@@"@} - func_run_hooks_result=$func_quote_for_eval_result + func_quote eval $@{1+"$@@"@} + func_run_hooks_result=$func_quote_result @} func_add_hook func_parse_options my_silent_option @end smallexample diff --git a/libtoolize.in b/libtoolize.in index 798bd0a..3fe61ce 100644 --- a/libtoolize.in +++ b/libtoolize.in @@ -151,11 +151,11 @@ libtoolize_environment_options () # Pass back the updated list of options. if test -n "$envopts"; then - func_quote_for_eval "$envopts" ${1+"$@"} + func_quote eval "$envopts" ${1+"$@"} else - func_quote_for_eval ${1+"$@"} + func_quote eval ${1+"$@"} fi - libtoolize_environment_options_result=$func_quote_for_eval_result + libtoolize_environment_options_result=$func_quote_result } func_add_hook func_options_prep libtoolize_environment_options @@ -181,8 +181,8 @@ libtoolize_options_prep () ltdl_mode= # Pass back the list of options. - func_quote_for_eval ${1+"$@"} - libtoolize_options_prep_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtoolize_options_prep_result=$func_quote_result } func_add_hook func_options_prep libtoolize_options_prep @@ -252,8 +252,8 @@ libtoolize_parse_options () done # save modified positional parameters for caller - func_quote_for_eval ${1+"$@"} - libtoolize_parse_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtoolize_parse_options_result=$func_quote_result } func_add_hook func_parse_options libtoolize_parse_options @@ -286,8 +286,8 @@ libtoolize_validate_options () func_fatal_help "unknown additional arguments: '${1+$@}'" # Pass back the empty argument list - func_quote_for_eval ${1+"$@"} - libtoolize_validate_options_result=$func_quote_for_eval_result + func_quote eval ${1+"$@"} + libtoolize_validate_options_result=$func_quote_result } func_add_hook func_validate_options libtoolize_validate_options -- 2.5.0 --nextPart3071762.M43Bqc6qy0-- From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 04 01:59:34 2015 Received: (at submit) by debbugs.gnu.org; 4 Nov 2015 06:59:34 +0000 Received: from localhost ([127.0.0.1]:52133 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zts2g-00039p-B2 for submit@debbugs.gnu.org; Wed, 04 Nov 2015 01:59:34 -0500 Received: from eggs.gnu.org ([208.118.235.92]:60941) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zts2N-000394-DR for submit@debbugs.gnu.org; Wed, 04 Nov 2015 01:59:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zts2M-00076m-A8 for submit@debbugs.gnu.org; Wed, 04 Nov 2015 01:59:14 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:35153) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zts2M-00076i-6q for submit@debbugs.gnu.org; Wed, 04 Nov 2015 01:59:14 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42704) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zts2L-0007IS-FK for bug-libtool@gnu.org; Wed, 04 Nov 2015 01:59:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zts2I-00076I-8w for bug-libtool@gnu.org; Wed, 04 Nov 2015 01:59:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40541) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zts2I-00076A-3y for bug-libtool@gnu.org; Wed, 04 Nov 2015 01:59:10 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id D94A4C0D61AF; Wed, 4 Nov 2015 06:59:08 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-219.brq.redhat.com [10.34.4.219]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tA46x7H6007145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 4 Nov 2015 01:59:08 -0500 From: Pavel Raiskup To: bug-libtool@gnu.org, 20006@debbugs.gnu.org Subject: Re: bug#20006: Get rid of excessive sed forks Date: Wed, 04 Nov 2015 07:58:21 +0100 Message-ID: <1478904.RdK7LIIg8U@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.3-300.fc23.x86_64; KDE/4.14.13; x86_64; ; ) In-Reply-To: <54F81935.2000904@redhat.com> References: <54F813FD.7080409@redhat.com> <54F81935.2000904@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: control@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) close 20006 libtool-2.4.6.19-aabc thanks I installed two test-cases worth running a bit before the future releases; so any pre-release reports would be really appreciated. $ make check TESTSUITEFLAGS='-k none' TESTS='test-funclib-quote.sh test-option-parser.sh' Pavel From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 04 02:15:10 2015 Received: (at control) by debbugs.gnu.org; 4 Nov 2015 07:15:10 +0000 Received: from localhost ([127.0.0.1]:52162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZtsHm-0003fH-0T for submit@debbugs.gnu.org; Wed, 04 Nov 2015 02:15:10 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60214) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZtsHj-0003cb-Im for control@debbugs.gnu.org; Wed, 04 Nov 2015 02:15:07 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 4A37D8E3FF for ; Wed, 4 Nov 2015 07:15:07 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-219.brq.redhat.com [10.34.4.219]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tA47F5Io011139 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 4 Nov 2015 02:15:06 -0500 From: Pavel Raiskup To: GNU bug tracker automated control server Subject: Re: Processed (with 2 errors): Closing "Get rid of excessive sed forks" Date: Wed, 04 Nov 2015 08:15:05 +0100 Message-ID: <6667711.75vtntvtTj@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.3-300.fc23.x86_64; KDE/4.14.13; x86_64; ; ) In-Reply-To: References: <13521076.Cu9tilhY16@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) close 20005 2.4.6.19-f187 thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 04 02:17:28 2015 Received: (at control) by debbugs.gnu.org; 4 Nov 2015 07:17:28 +0000 Received: from localhost ([127.0.0.1]:52169 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZtsK0-0005DU-Ia for submit@debbugs.gnu.org; Wed, 04 Nov 2015 02:17:28 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50358) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZtsJz-0005DM-8O for control@debbugs.gnu.org; Wed, 04 Nov 2015 02:17:27 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 016A1802 for ; Wed, 4 Nov 2015 07:17:26 +0000 (UTC) Received: from nb.usersys.redhat.com (unused-4-219.brq.redhat.com [10.34.4.219]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tA47HPBN012199 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 4 Nov 2015 02:17:26 -0500 From: Pavel Raiskup To: GNU bug tracker automated control server Subject: Re: Processed (with 2 errors): Closing "Get rid of excessive sed forks" Date: Wed, 04 Nov 2015 08:17:24 +0100 Message-ID: <4452396.Z6HtCbmVI7@nb.usersys.redhat.com> User-Agent: KMail/4.14.10 (Linux/4.2.3-300.fc23.x86_64; KDE/4.14.13; x86_64; ; ) In-Reply-To: References: <13521076.Cu9tilhY16@nb.usersys.redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) close 20005 2.4.6.19-aabc thanks From unknown Thu Aug 14 17:27:20 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 02 Dec 2015 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator