From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 21 20:36:43 2013 Received: (at submit) by debbugs.gnu.org; 22 Mar 2013 00:36:43 +0000 Received: from localhost ([127.0.0.1]:39577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIpyN-0003QR-1k for submit@debbugs.gnu.org; Thu, 21 Mar 2013 20:36:43 -0400 Received: from eggs.gnu.org ([208.118.235.92]:41876) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIiVU-0000FS-4O for submit@debbugs.gnu.org; Thu, 21 Mar 2013 12:38:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UIiTd-0001ai-Kq for submit@debbugs.gnu.org; Thu, 21 Mar 2013 12:36:32 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:48630) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIiTd-0001ae-IN for submit@debbugs.gnu.org; Thu, 21 Mar 2013 12:36:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:53220) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIiTa-0000Cc-RS for bug-coreutils@gnu.org; Thu, 21 Mar 2013 12:36:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UIiTY-0001ZQ-0i for bug-coreutils@gnu.org; Thu, 21 Mar 2013 12:36:26 -0400 Received: from mail-we0-x232.google.com ([2a00:1450:400c:c03::232]:62247) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIiTX-0001Z4-Ib for bug-coreutils@gnu.org; Thu, 21 Mar 2013 12:36:23 -0400 Received: by mail-we0-f178.google.com with SMTP id o45so2637798wer.9 for ; Thu, 21 Mar 2013 09:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mythmardyl.org; s=google; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=lPW+ruB1zreWNX8MorriESjN5UWPj3SxOr7vdojjIH0=; b=FYsAZJLQRT8KeJqQszALF4I2Y4oOXAA2+EVq2sUeYY8ezT+HnwB7NQyCu1aduTEH+Z 0waxq0Z66tzh2Czo9PDbtwLwfUEgWo1eEFuzFDIgtkNEaIIym3Ga6xo9hkf0NQRFwqE/ mxwnVM4Kkw2qYSxjPlvR2Qejs+FfhvXMDzpvI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=lPW+ruB1zreWNX8MorriESjN5UWPj3SxOr7vdojjIH0=; b=ot50Ul7fay4W+o4/uGFMtZVRS2Z/Ne6pVQEK42mWPgyuR7C7CI1+Mu6zN6oIEVl2K8 ppSCpFUuR1V0Ztb/kkbIK3p1TTy/r/CF+vKHTCEL5wpcr3WDvHgkwlp658CpNr/V8sp8 6/SYQ5jmfxDNoKdn2FDrG1Yk6T7ztDcovRFbd0pHtgux65LJ1RkoARZF9+Ty9Pz25mKo 4mNtHfj/a/xaJfNFuV6x61hxfEv0Jawa8A/eZLVL4wiSKloh5av3Cwdu4ExekhVYCfo6 ulRDtXgj8yiDlR0gSHioLh0cknhOWrdlTd69g3HncNbhMVJ/MdAQ0QM4O8n0CtgqFv2x u9XA== MIME-Version: 1.0 X-Received: by 10.194.237.129 with SMTP id vc1mr18496094wjc.20.1363883782048; Thu, 21 Mar 2013 09:36:22 -0700 (PDT) Received: by 10.180.89.143 with HTTP; Thu, 21 Mar 2013 09:36:21 -0700 (PDT) Date: Thu, 21 Mar 2013 18:36:21 +0200 Message-ID: Subject: dirname/basename unexpected results when run by xargs -I From: Juho-Pekka Kuitunen To: bug-coreutils@gnu.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQksKqFDR5w903km0c+4WLoQbzaGhFP/cY6gUirqAsx6uyGtltPG+F9QVYbBdY3eShf6i62j X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 208.118.235.17 X-Spam-Score: -4.3 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Thu, 21 Mar 2013 20:36:31 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.2 (------) Reproduce example; $ echo "testdir/testfile" | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') Expected output; testdir/testfile, dir: dirname testdir/testfile = testdir Actual output; testdir/testfile, dir: dirname testdir/testfile = . This behavior seems to be limited to the xargs -I replace-str usage pattern, any other way I can think of running dirname works fine. Basename has similarly wonky behavior, it prints the full string instead of doing any stripping. Not sure if there's a bug involved (or if it's on the findutils or coreutils side). Could also just be something silly I'm overlooking. I've tested this with findutils 4.4.2 and coreutils 8.5 & 8.13 with identical results. -- Juho-Pekka Kuitunen From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 21 21:07:56 2013 Received: (at 14023) by debbugs.gnu.org; 22 Mar 2013 01:07:56 +0000 Received: from localhost ([127.0.0.1]:39640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIqSV-0004Et-23 for submit@debbugs.gnu.org; Thu, 21 Mar 2013 21:07:55 -0400 Received: from joseki.proulx.com ([216.17.153.58]:54972) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIqSN-0004Ej-UH for 14023@debbugs.gnu.org; Thu, 21 Mar 2013 21:07:49 -0400 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 78369211D5; Thu, 21 Mar 2013 19:05:49 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 3DEEF2DCC4; Thu, 21 Mar 2013 19:05:49 -0600 (MDT) Date: Thu, 21 Mar 2013 19:05:49 -0600 From: Bob Proulx To: Juho-Pekka Kuitunen Subject: Re: bug#14023: dirname/basename unexpected results when run by xargs -I Message-ID: <20130322010549.GA23675@hysteria.proulx.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: 14023 Cc: 14023@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.4 (----) Juho-Pekka Kuitunen wrote: > Reproduce example; > $ echo "testdir/testfile" | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') Thank you for the report and the very nice test case. It made debugging this problem so very much simpler. > Expected output; > testdir/testfile, dir: dirname testdir/testfile = testdir > > Actual output; > testdir/testfile, dir: dirname testdir/testfile = . Ah... But you have missed a critical point! :-) The $(...) is expanded by the current shell and not by the xargs. Use echo to see what you are asking. $ echo xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') xargs -I {} echo {}, dir: dirname {} = . > This behavior seems to be limited to the xargs -I replace-str usage > pattern, any other way I can think of running dirname works fine. The problem is the $(...) which is running the dirname during the earlier shell command line parsing pass and passing the result off to the xargs command. > Basename has similarly wonky behavior, it prints the full string > instead of doing any stripping. Also using $(...)? :-) > Not sure if there's a bug involved (or if it's on the findutils or > coreutils side). Could also just be something silly I'm overlooking. > I've tested this with findutils 4.4.2 and coreutils 8.5 & 8.13 with > identical results. So... Not a bug. Agreed? (I am having a good chuckle. I hope you will too.) We will close the bug then. Bob From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 21 21:16:42 2013 Received: (at control) by debbugs.gnu.org; 22 Mar 2013 01:16:42 +0000 Received: from localhost ([127.0.0.1]:39644 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIqaz-0004RZ-Qg for submit@debbugs.gnu.org; Thu, 21 Mar 2013 21:16:40 -0400 Received: from joseki.proulx.com ([216.17.153.58]:55010) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIqav-0004RP-4y for control@debbugs.gnu.org; Thu, 21 Mar 2013 21:16:35 -0400 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 2355E211DF for ; Thu, 21 Mar 2013 19:14:38 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id DCBCB2DCC4; Thu, 21 Mar 2013 19:14:37 -0600 (MDT) Date: Thu, 21 Mar 2013 19:14:37 -0600 From: Bob Proulx To: control@debbugs.gnu.org Subject: tag moreinfo Message-ID: <20130322011437.GA28331@hysteria.proulx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.5 (--) tag 14023 + moreinfo thanks Message already sent to the poster. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 21 22:54:23 2013 Received: (at 14023) by debbugs.gnu.org; 22 Mar 2013 02:54:24 +0000 Received: from localhost ([127.0.0.1]:39711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIs7b-0007i9-H0 for submit@debbugs.gnu.org; Thu, 21 Mar 2013 22:54:23 -0400 Received: from mail-wg0-f54.google.com ([74.125.82.54]:43147) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIs7Y-0007i1-EF for 14023@debbugs.gnu.org; Thu, 21 Mar 2013 22:54:21 -0400 Received: by mail-wg0-f54.google.com with SMTP id fm10so2874275wgb.9 for <14023@debbugs.gnu.org>; Thu, 21 Mar 2013 19:52:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mythmardyl.org; s=google; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=guzSfCvK9Jla5G5HXs/zPGdgyUqxmuojX6B9y4rR990=; b=Zz/JiK3x391nvh1mrHGJZG4PvsF57FucM4jwWW/UDp/D0qu17/atx3btBlnGpRb8Oc 5yI4J6Y7y9OhTF7cyGffjBpCvjszPLgrYrwHvH5xjHiEg6549hu/vc+g60fySd+9WPQm tYYVYsvZiKn3bUKSutqSPDLr6yEYtLsfoz9QE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=guzSfCvK9Jla5G5HXs/zPGdgyUqxmuojX6B9y4rR990=; b=oUfx/5ck0xfdavn87E9SWl448qsNhdtP42413JJDjJV705Wic5pi+Nmw2qAn8j8jOt G+SCQJ0UWSIVH/Gy6qiz4CjPkRLXCgYSslNdAkAAZPrXA/pAXJ1gmzNcdGwqf48Zb7Er hSU0FQbpP5yVeQzTHQhpEtwVrjG8kcaxX4x5OrdMCR1DaLQyDb68oF1Glyaps2dc+0NP JPvUtLEtnc2JFqgf4XGYh9fZSEvNyQYt/RCm+f7L/HAkkj0hOSq+Zgsbhyh5lnvVHcTo 9mnRF8snr+WCJYiWoY8ypJ4EzsikF4GtBMMuF2GW9rPS5H4ZsMkWTR8TCyflW9nazsnv 0Y6w== MIME-Version: 1.0 X-Received: by 10.194.71.241 with SMTP id y17mr183159wju.28.1363920745756; Thu, 21 Mar 2013 19:52:25 -0700 (PDT) Received: by 10.180.89.143 with HTTP; Thu, 21 Mar 2013 19:52:25 -0700 (PDT) In-Reply-To: <20130322010549.GA23675@hysteria.proulx.com> References: <20130322010549.GA23675@hysteria.proulx.com> Date: Fri, 22 Mar 2013 04:52:25 +0200 Message-ID: Subject: Re: bug#14023: dirname/basename unexpected results when run by xargs -I From: Juho-Pekka Kuitunen To: Bob Proulx Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQks+h9sGAUgpI12l8Mu+34kezalH3P7X+XI6wPq/lQW0IB49xJ7CQOZ7640OBCw2ZnPscKt X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 14023 Cc: 14023@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On Fri, Mar 22, 2013 at 3:05 AM, Bob Proulx wrote: > Juho-Pekka Kuitunen wrote: >> Reproduce example; >> $ echo "testdir/testfile" | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') > > Thank you for the report and the very nice test case. It made > debugging this problem so very much simpler. > >> Expected output; >> testdir/testfile, dir: dirname testdir/testfile = testdir >> >> Actual output; >> testdir/testfile, dir: dirname testdir/testfile = . > > Ah... But you have missed a critical point! :-) The $(...) is > expanded by the current shell and not by the xargs. Use echo to see > what you are asking. > > $ echo xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') > xargs -I {} echo {}, dir: dirname {} = . > >> This behavior seems to be limited to the xargs -I replace-str usage >> pattern, any other way I can think of running dirname works fine. > > The problem is the $(...) which is running the dirname during the > earlier shell command line parsing pass and passing the result off to > the xargs command. So if I understand correctly, subshell replacements gets executed before xargs fills in the replacements? I tried to rule out this wonkyness with the "$(echo '{}')" bit and it seemed to work in the correct order. The echo should produce an empty string if the subshell was evaluated too soon? > >> Basename has similarly wonky behavior, it prints the full string >> instead of doing any stripping. > > Also using $(...)? :-) > >> Not sure if there's a bug involved (or if it's on the findutils or >> coreutils side). Could also just be something silly I'm overlooking. >> I've tested this with findutils 4.4.2 and coreutils 8.5 & 8.13 with >> identical results. > > So... Not a bug. Agreed? (I am having a good chuckle. I hope you > will too.) We will close the bug then. > Very much possibly not a bug but I'm not 100% convinced yet. I've been chuckling at this all evening, love a good brain teaser even if it turns out to be something I overlooked instead of a bug. :-) > Bob -- Juho-Pekka Kuitunen From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 21 23:13:13 2013 Received: (at 14023) by debbugs.gnu.org; 22 Mar 2013 03:13:13 +0000 Received: from localhost ([127.0.0.1]:39729 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIsPp-000892-6t for submit@debbugs.gnu.org; Thu, 21 Mar 2013 23:13:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:5372) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UIsPk-00088r-GX for 14023@debbugs.gnu.org; Thu, 21 Mar 2013 23:13:11 -0400 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id r2M3BDob020439 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 21 Mar 2013 23:11:13 -0400 Received: from [10.3.113.168] (ovpn-113-168.phx2.redhat.com [10.3.113.168]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id r2M3BBCK007315; Thu, 21 Mar 2013 23:11:12 -0400 Message-ID: <514BCBCD.6040108@redhat.com> Date: Thu, 21 Mar 2013 21:11:09 -0600 From: Eric Blake Organization: Red Hat, Inc. User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: Juho-Pekka Kuitunen Subject: Re: bug#14023: dirname/basename unexpected results when run by xargs -I References: <20130322010549.GA23675@hysteria.proulx.com> In-Reply-To: X-Enigmail-Version: 1.5.1 OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="----enig2OHLVITOILRVAPUIESCBB" X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Spam-Score: -8.0 (--------) X-Debbugs-Envelope-To: 14023 Cc: 14023@debbugs.gnu.org, Bob Proulx X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -9.4 (---------) This is an OpenPGP/MIME signed message (RFC 4880 and 3156) ------enig2OHLVITOILRVAPUIESCBB Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 03/21/2013 08:52 PM, Juho-Pekka Kuitunen wrote: > On Fri, Mar 22, 2013 at 3:05 AM, Bob Proulx wrote: >> Juho-Pekka Kuitunen wrote: >>> Reproduce example; >>> $ echo "testdir/testfile" | xargs -I '{}' echo '{}', dir: $(echo dirn= ame '{}') =3D $(dirname '{}') >> The problem is the $(...) which is running the dirname during the >> earlier shell command line parsing pass and passing the result off to >> the xargs command. >=20 > So if I understand correctly, subshell replacements gets executed > before xargs fills in the replacements? I tried to rule out this > wonkyness with the "$(echo '{}')" bit and it seemed to work in the > correct order. The echo should produce an empty string if the subshell > was evaluated too soon? If you want to see when the shell is evaluating $(), use 'set -vx': $ set -vx $ echo testdir/testfile | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') =3D $(dirname '{}') echo testdir/testfile | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') =3D $(dirname '{}') + echo testdir/testfile echo dirname '{}') dirname '{}') echo dirname '{}') echo dirname '{}' ++ echo dirname '{}' dirname '{}') dirname '{}' ++ dirname '{}' + xargs -I '{}' echo '{},' dir: dirname '{}' =3D . testdir/testfile, dir: dirname testdir/testfile =3D . $ set - set - + set - What you seem to want to do is invoke a shell instance on each file name; that would be done as follows: $ echo testdir/testfile | xargs -I {} sh -c \ 'echo "$1", dir: $(echo dirname "$1") =3D $(dirname "$1")' sh {} testdir/testfile, dir: dirname testdir/testfile =3D testdir Or again with shell tracing (note there are two levels of shells - the shell that spawns xargs, and the shell that xargs spawns, so I'm posting two different traces): $ set -vx $ echo testdir/testfile | xargs -I {} sh -c 'echo "$1", dir: $(echo dirname "$1") =3D $(dirname "$1")' sh {} echo testdir/testfile | xargs -I {} sh -c 'echo "$1", dir: $(echo dirname "$1") =3D $(dirname "$1")' sh {} + xargs -I '{}' sh -c 'echo "$1", dir: $(echo dirname "$1") =3D $(dirname= "$1")' sh '{}' + echo testdir/testfile testdir/testfile, dir: dirname testdir/testfile =3D testdir $ set - set - + set - $ echo testdir/testfile | xargs -I {} sh -cvx 'echo "$1", dir: $(echo dirname "$1") =3D $(dirname "$1")' sh {} echo "$1", dir: $(echo dirname "$1") =3D $(dirname "$1") echo dirname "$1") echo dirname "$1" ++ echo dirname testdir/testfile dirname "$1") dirname "$1" ++ dirname testdir/testfile + echo testdir/testfile, dir: dirname testdir/testfile =3D testdir testdir/testfile, dir: dirname testdir/testfile =3D testdir By the way, your question is mostly related to shell, and a bit with xargs, and practically nothing to do with dirname. Your confusion on WHEN $() is expanded would apply no matter what executable you plug in instead of dirname. But since neither sh nor xargs belongs to coreutils, you might get better answers by asking on a general forum on shell programming subtleties. >=20 > Very much possibly not a bug but I'm not 100% convinced yet. I've been > chuckling at this all evening, love a good brain teaser even if it > turns out to be something I overlooked instead of a bug. :-) Hopefully shell tracing has managed to convince you. --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org ------enig2OHLVITOILRVAPUIESCBB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJRS8vOAAoJEKeha0olJ0Nq72wH/3SF1VSSUqNRcxvD+6hs/gng qxS9nxnIyW2hteABZbyqCljGrRHHh4yOpEkOVXMp4m/6+IW9Y4deOyC89NlZy5eA dz89V46Wm1o8a7Yh96+yr+jyC9seKedjsGvzWusD6MjSBPza5vqHqUZgjysem9WF MI7MsgZHk0QF+mXH8SPWjQ53/4QNlXA4MqaY880Mf4+rJRQxar3RI6YvHS6PqwFX pNWoCc/uSSmrBHDRPt7EPQxpTMLjp/TvHed4BwspsxwQ+Sm9VdSAvFUnZebCIeg4 9gUvaQ53ZQw0ryE/A/93gCpsdZp0wDy5q2VhrTv3yOTf7MPr7dgFpDu6VnD/g+I= =QtQC -----END PGP SIGNATURE----- ------enig2OHLVITOILRVAPUIESCBB-- From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 23 07:05:20 2013 Received: (at 14023) by debbugs.gnu.org; 23 Mar 2013 11:05:20 +0000 Received: from localhost ([127.0.0.1]:41660 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJMGE-0006cj-NR for submit@debbugs.gnu.org; Sat, 23 Mar 2013 07:05:20 -0400 Received: from mail-we0-f171.google.com ([74.125.82.171]:64667) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJMG9-0006cZ-Ui for 14023@debbugs.gnu.org; Sat, 23 Mar 2013 07:05:17 -0400 Received: by mail-we0-f171.google.com with SMTP id d46so1976645wer.2 for <14023@debbugs.gnu.org>; Sat, 23 Mar 2013 04:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mythmardyl.org; s=google; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=6RQU9UvnBWupIeDizjNAIrnx1MpxrICkZ6LZYC9/QP8=; b=KnvOcJgk10q9vmmA+FuA4j2Rd6luo/YsIWC+cCYCzjJIRWqG8Hq8gDBjnzU8WUCa5A 7/p7VnIOzunrka9gZzmlHrnsI1CPZIiiyfCQ6FKMQUoytytFaVzyaTM4nJpoi4vxNF6u sNFv/eTbQHFslyA9K4JIb/Co+my0hNxn0KmPk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=6RQU9UvnBWupIeDizjNAIrnx1MpxrICkZ6LZYC9/QP8=; b=dgLtJcFY6vCBBlet+LRZ9DNfBEScoCdSEKczledFQhskR1IW6P2DUOBOpgCHY0iebb k2mc4dldN/VUQF8xYShbEw5jvbewhXhOVh0K59QAyulor1/Shzniaf5v7ni5zYHgpnkC jMglYEzMKQvOmXvrNVhkReLgsbU2nXS+v1ziWrVaHIkPGWsUrebwv+XJOG9PP4xHuAi7 Astvf+ThVF28byUZ6dsj2UpPcCxTjp+Ylra5yffqa/RoaoNdsRYoQ5qXUe6M5J5J8yBR QGgMhuUrv3cV+kU5gAQWGz/tGHvMHBczgSMds4UKW5KmVKyUhkJtrdcCXu8TCJQzi56z tWBg== MIME-Version: 1.0 X-Received: by 10.194.237.129 with SMTP id vc1mr8210113wjc.20.1364036591640; Sat, 23 Mar 2013 04:03:11 -0700 (PDT) Received: by 10.180.89.143 with HTTP; Sat, 23 Mar 2013 04:03:11 -0700 (PDT) In-Reply-To: <514BCBCD.6040108@redhat.com> References: <20130322010549.GA23675@hysteria.proulx.com> <514BCBCD.6040108@redhat.com> Date: Sat, 23 Mar 2013 13:03:11 +0200 Message-ID: Subject: Re: bug#14023: dirname/basename unexpected results when run by xargs -I From: Juho-Pekka Kuitunen To: Eric Blake Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQlx93ynvlX/l+TVqSOvw5XBhGxOwmo9yOgz7zmitFqY4lxLB5ZFO33zHrrb9j9Vldt97+9Y X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 14023 Cc: 14023@debbugs.gnu.org, Bob Proulx X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On Fri, Mar 22, 2013 at 5:11 AM, Eric Blake wrote: > On 03/21/2013 08:52 PM, Juho-Pekka Kuitunen wrote: >> On Fri, Mar 22, 2013 at 3:05 AM, Bob Proulx wrote: >>> Juho-Pekka Kuitunen wrote: >>>> Reproduce example; >>>> $ echo "testdir/testfile" | xargs -I '{}' echo '{}', dir: $(echo dirname '{}') = $(dirname '{}') > >>> The problem is the $(...) which is running the dirname during the >>> earlier shell command line parsing pass and passing the result off to >>> the xargs command. >> >> So if I understand correctly, subshell replacements gets executed >> before xargs fills in the replacements? I tried to rule out this >> wonkyness with the "$(echo '{}')" bit and it seemed to work in the >> correct order. The echo should produce an empty string if the subshell >> was evaluated too soon? > > If you want to see when the shell is evaluating $(), use 'set -vx': > > $ set -vx > $ echo testdir/testfile | xargs -I '{}' echo '{}', dir: $(echo dirname > '{}') = $(dirname '{}') > echo testdir/testfile | xargs -I '{}' echo '{}', dir: $(echo dirname > '{}') = $(dirname '{}') > + echo testdir/testfile > echo dirname '{}') > dirname '{}') > echo dirname '{}') > echo dirname '{}' > ++ echo dirname '{}' > dirname '{}') > dirname '{}' > ++ dirname '{}' > + xargs -I '{}' echo '{},' dir: dirname '{}' = . > testdir/testfile, dir: dirname testdir/testfile = . > $ set - > set - > + set - > > What you seem to want to do is invoke a shell instance on each file > name; that would be done as follows: > > $ echo testdir/testfile | xargs -I {} sh -c \ > 'echo "$1", dir: $(echo dirname "$1") = $(dirname "$1")' sh {} > testdir/testfile, dir: dirname testdir/testfile = testdir > > Or again with shell tracing (note there are two levels of shells - the > shell that spawns xargs, and the shell that xargs spawns, so I'm posting > two different traces): > > $ set -vx > $ echo testdir/testfile | xargs -I {} sh -c 'echo "$1", dir: $(echo > dirname "$1") = $(dirname "$1")' sh {} > echo testdir/testfile | xargs -I {} sh -c 'echo "$1", dir: $(echo > dirname "$1") = $(dirname "$1")' sh {} > + xargs -I '{}' sh -c 'echo "$1", dir: $(echo dirname "$1") = $(dirname > "$1")' sh '{}' > + echo testdir/testfile > testdir/testfile, dir: dirname testdir/testfile = testdir > $ set - > set - > + set - > $ echo testdir/testfile | xargs -I {} sh -cvx 'echo "$1", dir: $(echo > dirname "$1") = $(dirname "$1")' sh {} > echo "$1", dir: $(echo dirname "$1") = $(dirname "$1") > echo dirname "$1") > echo dirname "$1" > ++ echo dirname testdir/testfile > dirname "$1") > dirname "$1" > ++ dirname testdir/testfile > + echo testdir/testfile, dir: dirname testdir/testfile = testdir > testdir/testfile, dir: dirname testdir/testfile = testdir > > By the way, your question is mostly related to shell, and a bit with > xargs, and practically nothing to do with dirname. Your confusion on > WHEN $() is expanded would apply no matter what executable you plug in > instead of dirname. But since neither sh nor xargs belongs to > coreutils, you might get better answers by asking on a general forum on > shell programming subtleties. > >> >> Very much possibly not a bug but I'm not 100% convinced yet. I've been >> chuckling at this all evening, love a good brain teaser even if it >> turns out to be something I overlooked instead of a bug. :-) > > Hopefully shell tracing has managed to convince you. > Yup, that did the trick. Looks like I didn't understand the way xargs works nearly well enough. Thanks for the lesson, my bad! > -- > Eric Blake eblake redhat com +1-919-301-3266 > Libvirt virtualization library http://libvirt.org > -- Juho-Pekka Kuitunen From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 23 11:04:48 2013 Received: (at control) by debbugs.gnu.org; 23 Mar 2013 15:04:48 +0000 Received: from localhost ([127.0.0.1]:42340 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJPzy-0004lu-V6 for submit@debbugs.gnu.org; Sat, 23 Mar 2013 11:04:48 -0400 Received: from joseki.proulx.com ([216.17.153.58]:38557) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJPzv-0004ll-8S for control@debbugs.gnu.org; Sat, 23 Mar 2013 11:04:44 -0400 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 176C0211D5 for ; Sat, 23 Mar 2013 09:02:40 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 095C72DCC4; Sat, 23 Mar 2013 09:02:39 -0600 (MDT) Date: Sat, 23 Mar 2013 09:02:39 -0600 From: Bob Proulx To: control@debbugs.gnu.org Subject: close 14023 Message-ID: <20130323150239.GA31682@hysteria.proulx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.5 (--) close 14023 thanks Message with explanation being sent. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 23 11:13:11 2013 Received: (at 14023) by debbugs.gnu.org; 23 Mar 2013 15:13:11 +0000 Received: from localhost ([127.0.0.1]:42353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJQ87-0004yG-4w for submit@debbugs.gnu.org; Sat, 23 Mar 2013 11:13:11 -0400 Received: from joseki.proulx.com ([216.17.153.58]:38598) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UJQ85-0004y9-CU for 14023@debbugs.gnu.org; Sat, 23 Mar 2013 11:13:10 -0400 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 617F1211D5; Sat, 23 Mar 2013 09:11:06 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 261A12DCC4; Sat, 23 Mar 2013 09:11:06 -0600 (MDT) Date: Sat, 23 Mar 2013 09:11:06 -0600 From: Bob Proulx To: Juho-Pekka Kuitunen Subject: Re: bug#14023: dirname/basename unexpected results when run by xargs -I Message-ID: <20130323151106.GA31625@hysteria.proulx.com> References: <20130322010549.GA23675@hysteria.proulx.com> <514BCBCD.6040108@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: 14023 Cc: 14023@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.4 (----) Juho-Pekka Kuitunen wrote: > Eric Blake wrote: > > By the way, your question is mostly related to shell, and a bit with > > xargs, and practically nothing to do with dirname. Your confusion on > > WHEN $() is expanded would apply no matter what executable you plug in > > instead of dirname. But since neither sh nor xargs belongs to > > coreutils, you might get better answers by asking on a general forum on > > shell programming subtleties. > > > >> Very much possibly not a bug but I'm not 100% convinced yet. I've been > >> chuckling at this all evening, love a good brain teaser even if it > >> turns out to be something I overlooked instead of a bug. :-) > > > > Hopefully shell tracing has managed to convince you. > > Yup, that did the trick. Looks like I didn't understand the way xargs > works nearly well enough. Thanks for the lesson, my bad! The place to look for the explanation is in the shell documentation. In the bash manual this is in the "EXPANSION" section. In that section it walks through a number of expansions. One of those is "Command Substitution". This is done by the shell uniformly for all commands before the command is invoked so it and the other expansions are a good thing to understand. The shell effectively modifies the command line you typed into a dynamically created set of arguments for the command being invoked. I have closed the bug ticket to finish our accounting. But feel free to continue adding information to this bug log. It continues to be active and available. Bob From unknown Sat Jun 21 10:44:44 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 21 Apr 2013 11:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator