GNU bug report logs -
#16843
Error caused due to regression in git
Previous Next
Reported by: Darshit Shah <darnir <at> gmail.com>
Date: Sat, 22 Feb 2014 16:07:01 UTC
Severity: normal
Tags: notabug
Done: Eric Blake <eblake <at> redhat.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16843 in the body.
You can then email your comments to 16843 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Sat, 22 Feb 2014 16:07:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Darshit Shah <darnir <at> gmail.com>:
New bug report received and forwarded. Copy sent to
bug-automake <at> gnu.org.
(Sat, 22 Feb 2014 16:07:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
automake --version: 1.14.1
autoconf --version: 2.69
The error occured when attempting to run the test suite GNU Wget's git
source.
`make` works correctly, but `make check` happens to fail with the following
output:
if test -d ./.git \
&& git --version >/dev/null 2>&1; then \
cd . && \
git submodule --quiet foreach \
test '"$(git rev-parse "$sha1")"' \
= '"$(git merge-base origin "$sha1")"' \
|| { echo 'maint.mk: found non-public submodule commit' >&2; \
exit 1; }; \
else \
: ; \
fi
Stopping at 'gnulib'; script returned non-zero status.
maint.mk: found non-public submodule commit
maint.mk:1394: recipe for target 'public-submodule-commit' failed
make: *** [public-submodule-commit] Error 1
In the repository, gnulib is used as a submodule. Running:
$ git submodule foreach echo $name, $path, $sha1
returns an empty string which implies that git is unable to see the
submodules correctly.
I bisected the git source to find the commit causing the problems. The
following is the commit message:
commit 1c4fb136dbad762c9c4350ee79c3474ae8037587
Author: Anders Kaseorg <andersk <at> MIT.EDU>
Date: Fri Sep 27 06:23:55 2013 -0400
submodule foreach: skip eval for more than one argument
'eval "$@"' creates an extra layer of shell interpretation, which is
probably not expected by a user who passes multiple arguments to git
submodule foreach:
$ git grep "'"
[searches for single quotes]
$ git submodule foreach git grep "'"
Entering '[submodule]'
/usr/lib/git-core/git-submodule: 1: eval: Syntax error: Unterminated
quoted string
Stopping at '[submodule]'; script returned non-zero status.
To fix this, if the user passes more than one argument, execute "$@"
directly instead of passing it to eval.
Examples:
* Typical usage when adding an extra level of quoting is to pass a
single argument representing the entire command to be passed to the
shell. This doesn't change that.
* One can imagine someone feeding untrusted input as an argument:
git submodule foreach git grep "$variable"
That currently results in a nonobvious shell code injection
vulnerability. Executing the command named by the arguments
directly, as in this patch, fixes it.
Signed-off-by: Anders Kaseorg <andersk <at> mit.edu>
Acked-by: Johan Herland <johan <at> herland.net>
Signed-off-by: Jonathan Nieder <jrnieder <at> gmail.com>
Hence, this seems to me as a feature, not a bug in Git. Maybe, automake
needs to fix its scripts accordingly? Because the issue with `make check`
was introduced through this specific commit.
Do let me know if I can provide any more help.
Also, I am not sunscribed to this mailing list, hence, please ensure to
leave my email in the CC list.
--
Thanking You,
Darshit Shah
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 14:05:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 02/24/2014 07:00 AM, Eric Blake wrote:
>
> Based on the test that is failing, can you please show the output of:
>
> git submodule --quiet foreach echo 'a b' '"c d"'
>
> On my version (1.8.5.3), I get 'a b c d' (that is, echo saw three
> arguments rather than 2, where only my 'c d' argument survived because
> it was double-quoted). I'm guessing your version preserves extra
> quotes, because it is not being passed through a layer of shell
> interpretation?
What about:
git submodule --quiet foreach 'echo a b "c d" "$sha1"'
That is, does presenting the entire command as a single argument to git
still manage to interpolate the shell variable and preserve the double
space between c and d while doing word splitting between a and b?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 14:08:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
thanks
[adding gnulib]
On 02/22/2014 02:01 AM, Darshit Shah wrote:
> automake --version: 1.14.1
> autoconf --version: 2.69
>
> The error occured when attempting to run the test suite GNU Wget's git
> source.
> `make` works correctly, but `make check` happens to fail with the following
> output:
>
> if test -d ./.git \
> && git --version >/dev/null 2>&1; then \
> cd . && \
> git submodule --quiet foreach \
> test '"$(git rev-parse "$sha1")"' \
> = '"$(git merge-base origin "$sha1")"' \
> || { echo 'maint.mk: found non-public submodule commit' >&2; \
> exit 1; }; \
> else \
> : ; \
> fi
> Stopping at 'gnulib'; script returned non-zero status.
> maint.mk: found non-public submodule commit
> maint.mk:1394: recipe for target 'public-submodule-commit' failed
> make: *** [public-submodule-commit] Error 1
This rule comes from upstream gnulib, so it will break on more than just
automake as more people start upgrading to newer git.
>
>
> In the repository, gnulib is used as a submodule. Running:
> $ git submodule foreach echo $name, $path, $sha1
> returns an empty string which implies that git is unable to see the
> submodules correctly.
>
> I bisected the git source to find the commit causing the problems. The
> following is the commit message:
>
> commit 1c4fb136dbad762c9c4350ee79c3474ae8037587
>
> To fix this, if the user passes more than one argument, execute "$@"
> directly instead of passing it to eval.
>
>
> Hence, this seems to me as a feature, not a bug in Git. Maybe, automake
> needs to fix its scripts accordingly? Because the issue with `make check`
> was introduced through this specific commit.
Yes, we need to update the gnulib rule to behave nicely on both old and
new versions of git.
Based on the test that is failing, can you please show the output of:
git submodule --quiet foreach echo 'a b' '"c d"'
On my version (1.8.5.3), I get 'a b c d' (that is, echo saw three
arguments rather than 2, where only my 'c d' argument survived because
it was double-quoted). I'm guessing your version preserves extra
quotes, because it is not being passed through a layer of shell
interpretation?
Have you reported this regression to the git folks?
>
> Do let me know if I can provide any more help.
> Also, I am not sunscribed to this mailing list, hence, please ensure to
> leave my email in the CC list.
Yes, reply-to-all is list policy (both for automake and for gnulib).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 19:39:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, Feb 24, 2014 at 3:04 PM, Eric Blake <eblake <at> redhat.com> wrote:
> On 02/24/2014 07:00 AM, Eric Blake wrote:
>
> >
> > Based on the test that is failing, can you please show the output of:
> >
> > git submodule --quiet foreach echo 'a b' '"c d"'
> >
> > On my version (1.8.5.3), I get 'a b c d' (that is, echo saw three
> > arguments rather than 2, where only my 'c d' argument survived because
> > it was double-quoted). I'm guessing your version preserves extra
> > quotes, because it is not being passed through a layer of shell
> > interpretation?
>
> What about:
>
> git submodule --quiet foreach 'echo a b "c d" "$sha1"'
>
> That is, does presenting the entire command as a single argument to git
> still manage to interpolate the shell variable and preserve the double
> space between c and d while doing word splitting between a and b?
>
The first command:
git submodule --quiet foreach echo 'a b' '"c d"'
gives the output: a b "c d"
Where there is a double space between a and b while c and d still retain
their quotes.
The second command:
git submodule --quiet foreach 'echo a b "c d" "$sha1"'
Gives the output:
a b c d 0ac90c5a98030c998f3e1db3a0d7f19d4630b6b6
So, it does seem to do word splitting between a and b, while retaining the
double space between c and d.
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
>
--
Thanking You,
Darshit Shah
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 19:54:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 02/24/2014 12:37 PM, Darshit Shah wrote:
>>> Based on the test that is failing, can you please show the output of:
>>>
>>> git submodule --quiet foreach echo 'a b' '"c d"'
>> What about:
>>
>> git submodule --quiet foreach 'echo a b "c d" "$sha1"'
>>
>
> The first command:
> git submodule --quiet foreach echo 'a b' '"c d"'
> gives the output: a b "c d"
Good, I was right - newer git is doing one less round of shell
interpolation, so the second layer of quotes that we were adding for
older git now show up as literal arguments in newer git.
> Where there is a double space between a and b while c and d still retain
> their quotes.
>
> The second command:
> git submodule --quiet foreach 'echo a b "c d" "$sha1"'
>
> Gives the output:
> a b c d 0ac90c5a98030c998f3e1db3a0d7f19d4630b6b6
Yay - you confirmed that the solution that works for both versions of
git is to ensure that we provide exactly one argument to the foreach
command, where that argument gets passed through the shell with
identical results. I'll patch maint.mk shortly.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 20:06:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 02/24/2014 12:53 PM, Eric Blake wrote:
> Yay - you confirmed that the solution that works for both versions of
> git is to ensure that we provide exactly one argument to the foreach
> command, where that argument gets passed through the shell with
> identical results. I'll patch maint.mk shortly.
Actually, it turns out that the problem has already been fixed upstream:
commit 7a0c729aacc3716a665ce3b0c8cb34843bdecc75
Author: Jim Meyering <meyering <at> fb.com>
Date: Wed Jan 1 20:25:44 2014 -0800
maint: fix public-submodule-commit to work with newer git
* top/maint.mk (public-submodule-commit): Remove excess quoting.
We were over-quoting the test arguments, and somewhere prior to
version 1.8.5.2.229, git stopped removing those excess quotes,
which made the test fail, since the unexpanded strings would
always differ; using GIT_TRACE=1 confirmed that the git merge-base
command wasn't even being run.
So all that remains now is for the package you were complaining about to
upgrade to a newer version of gnulib that plays nicely with git 1.9 and
newer.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-automake <at> gnu.org:
bug#16843; Package
automake.
(Mon, 24 Feb 2014 20:11:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 16843 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, Feb 24, 2014 at 9:05 PM, Eric Blake <eblake <at> redhat.com> wrote:
> On 02/24/2014 12:53 PM, Eric Blake wrote:
> > Yay - you confirmed that the solution that works for both versions of
> > git is to ensure that we provide exactly one argument to the foreach
> > command, where that argument gets passed through the shell with
> > identical results. I'll patch maint.mk shortly.
>
> Actually, it turns out that the problem has already been fixed upstream:
>
> commit 7a0c729aacc3716a665ce3b0c8cb34843bdecc75
> Author: Jim Meyering <meyering <at> fb.com>
> Date: Wed Jan 1 20:25:44 2014 -0800
>
> maint: fix public-submodule-commit to work with newer git
>
> * top/maint.mk (public-submodule-commit): Remove excess quoting.
> We were over-quoting the test arguments, and somewhere prior to
> version 1.8.5.2.229, git stopped removing those excess quotes,
> which made the test fail, since the unexpanded strings would
> always differ; using GIT_TRACE=1 confirmed that the git merge-base
> command wasn't even being run.
>
> So all that remains now is for the package you were complaining about to
> upgrade to a newer version of gnulib that plays nicely with git 1.9 and
> newer.
>
> Thanks Eric!
I'll notify them of the fact.
--
Thanking You,
Darshit Shah
[Message part 2 (text/html, inline)]
Added tag(s) notabug.
Request was from
Eric Blake <eblake <at> redhat.com>
to
control <at> debbugs.gnu.org.
(Mon, 24 Feb 2014 20:12:02 GMT)
Full text and
rfc822 format available.
Reply sent
to
Eric Blake <eblake <at> redhat.com>:
You have taken responsibility.
(Mon, 24 Feb 2014 20:12:03 GMT)
Full text and
rfc822 format available.
Notification sent
to
Darshit Shah <darnir <at> gmail.com>:
bug acknowledged by developer.
(Mon, 24 Feb 2014 20:12:04 GMT)
Full text and
rfc822 format available.
Message #30 received at 16843-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
tag 16843 notabug
thanks
On 02/22/2014 02:01 AM, Darshit Shah wrote:
> automake --version: 1.14.1
> autoconf --version: 2.69
>
> The error occured when attempting to run the test suite GNU Wget's git
> source.
Then this bug should be filed against wget, not automake. I'm closing
this bug in the automake tracker, as the fix is for wget to upgrade to a
newer version of gnulib, and automake has no bearing on it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Tue, 25 Mar 2014 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 11 years and 86 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.