From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 07 22:45:01 2011 Received: (at submit) by debbugs.gnu.org; 8 Feb 2011 03:45:01 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PmeVg-0000c8-RV for submit@debbugs.gnu.org; Mon, 07 Feb 2011 22:45:01 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PmeVf-0000by-IQ for submit@debbugs.gnu.org; Mon, 07 Feb 2011 22:45:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pmee2-00065m-Vj for submit@debbugs.gnu.org; Mon, 07 Feb 2011 22:53:40 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:58987) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pmee2-00065g-Te for submit@debbugs.gnu.org; Mon, 07 Feb 2011 22:53:38 -0500 Received: from [140.186.70.92] (port=35697 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pmee2-0000Wr-3G for bug-coreutils@gnu.org; Mon, 07 Feb 2011 22:53:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pmee0-000651-PK for bug-coreutils@gnu.org; Mon, 07 Feb 2011 22:53:37 -0500 Received: from smtp.gentoo.org ([140.211.166.183]:51222) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pmee0-00064d-H9 for bug-coreutils@gnu.org; Mon, 07 Feb 2011 22:53:36 -0500 Received: from vapier.localnet (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 1A0721B400C for ; Tue, 8 Feb 2011 03:53:35 +0000 (UTC) From: Mike Frysinger Organization: wh0rd.org To: bug-coreutils@gnu.org Subject: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) Date: Mon, 7 Feb 2011 22:53:26 -0500 User-Agent: KMail/1.13.5 (Linux/2.6.37; KDE/4.5.5; x86_64; ; ) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2540096.OVA0AVK6Ax"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201102072253.34476.vapier@gentoo.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.211.166.183 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 199.232.76.165 X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) --nextPart2540096.OVA0AVK6Ax Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable after upgrading from coreutils 8.9 to 8.10, the sparse handling in cp is=20 silently breaking on btrfs filesystems with the compressed option enabled. = =20 using --sparse=3Dnever works fine, but "auto" or "always" tend to fail. th= e=20 cp/fiemap-2 test catches the issue nicely. to reproduce (i'm using linux-2.6.37): file=3Dbtrfs.img mntp=3D/mnt/tmp/ dd if=3D/dev/zero of=3D$file bs=3D1M count=3D0 seek=3D1024 mkfs.btrfs $file=20 mount -t btrfs -o compress $file $mntp cd $mntp tar xf ~/coreutils-8.10.tar.xz cd coreutils-8.10/tests =2E/cp/fiemap-2 and this last test shows: =46ilesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop0 btrfs 1048576 20420 377028 6% /mnt/tmp 0+0 records in 0+0 records out 0 bytes (0 B) copied, 2.7598e-05 s, 0.0 kB/s k k2 differ: byte 1, line 1 255+0 records in 255+0 records out 261120 bytes (261 kB) copied, 0.00310926 s, 84.0 MB/s =2Dmike --nextPart2540096.OVA0AVK6Ax Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAABAgAGBQJNUL4+AAoJEEFjO5/oN/WBffQQAM3Xq0fkVWP7pnl+RKq7MLHs JibB3e0j2XAZCm7o6GXT/0dssBZ9+5Ce4S7R4lAuBkskHESRys2PCKlM/iwMYY1p +UiKTkA/O9LjCIeHt5ztw6gJDVg1tHDjnqzg5dLFDSBacPz+S1Y4nZPfqNFn5kPO WzYIeJ3DME4NAWu0mettVqU0A2JDURi2sGsRHqvyGgVioR2p0brgx5n8i8onXKPh 5vgEqDDqMLMRwnI4FvZs+Exx0XVI/aBeYCjHTJAv64T435wasLG+HmGs17xX/bRV q2VVB87+Mph5G7/ZOJ3JtytnpR8hcD4t4I3unCezYoXhcRfv98g2tzrmSVsmU86R Keb1Atv8qZbv/4PKe5Y7rEwHStW9FdfpFS23EEpjaZPb8bnjtVly93rh8zxhEnYI 4YWNOBe/qhtcSbdubALt9x+bdnYMtmfz8Uj4pHYDr0iJxzdJblLpatFUSgsobSRk 3rT5wj5pcklfZ2GqzMzkGPqjCEedW4d3eqdYfobZEa8GbT7sPqhe90bRDzolaTpl Fkjtzjf8LDXsIm+CxSVPpe2yp1IMQvJs3hk0YoJhFQRbtPOhI9Jym7RUEhvumnOx m5rQP6mXYaWkNGuwXteAwGYmvcw3pXP9x/OYzcU+daNu2VZ4T8Uw+6GOSJTBpnSI 68QFQrTpYA9Z6QWVW2tu =9VVi -----END PGP SIGNATURE----- --nextPart2540096.OVA0AVK6Ax-- From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 09 05:51:19 2011 Received: (at 8001) by debbugs.gnu.org; 9 Feb 2011 10:51:19 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Pn7dn-0002kB-A3 for submit@debbugs.gnu.org; Wed, 09 Feb 2011 05:51:19 -0500 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1Pn7dk-0002jr-Ok for 8001@debbugs.gnu.org; Wed, 09 Feb 2011 05:51:17 -0500 Received: (qmail 99178 invoked from network); 9 Feb 2011 10:59:59 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 9 Feb 2011 10:59:59 -0000 Message-ID: <4D52730A.2080008@draigBrady.com> Date: Wed, 09 Feb 2011 10:57:14 +0000 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Mike Frysinger Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) References: <201102072253.34476.vapier@gentoo.org> In-Reply-To: <201102072253.34476.vapier@gentoo.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8001 Cc: BTRFS MAILING LIST , 8001@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 08/02/11 03:53, Mike Frysinger wrote: > after upgrading from coreutils 8.9 to 8.10, the sparse handling in cp is > silently breaking on btrfs filesystems with the compressed option enabled. > using --sparse=never works fine, but "auto" or "always" tend to fail. the > cp/fiemap-2 test catches the issue nicely. > > to reproduce (i'm using linux-2.6.37): > file=btrfs.img > mntp=/mnt/tmp/ > dd if=/dev/zero of=$file bs=1M count=0 seek=1024 > mkfs.btrfs $file > mount -t btrfs -o compress $file $mntp > cd $mntp > tar xf ~/coreutils-8.10.tar.xz > cd coreutils-8.10/tests > ./cp/fiemap-2 > > and this last test shows: > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/loop0 btrfs 1048576 20420 377028 6% /mnt/tmp > 0+0 records in > 0+0 records out > 0 bytes (0 B) copied, 2.7598e-05 s, 0.0 kB/s > k k2 differ: byte 1, line 1 Eek. That doesn't trigger here (2.6.35.10-72.fc14.i686) because I guess this kernel doesn't honor the compress attribute: dd if=/dev/zero of=test.size count=1000 # du -B512 test.size 1000 test.size But on a general note, we may read more (or possibly less) than is stored in the extent. So how to detect that? I suppose one could use lseek() to get the current position and see if it's ext_start + ext_length, otherwise adjust accordingly. That would add a little overhead though. I also notice the FIEMAP_EXTENT_DATA_ENCRYPTED and FIEMAP_EXTENT_ENCODED flags, which could mean we only need to handle these extents specially. Does `filefrag -v` show those for you? I see nothing here. I don't suppose there is any facility to read the raw data to also avoid decompressing and compressing again (sendfile, mmap?). cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 15 06:24:57 2011 Received: (at 8001) by debbugs.gnu.org; 15 Feb 2011 11:24:57 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PpJ1d-0007N6-Ae for submit@debbugs.gnu.org; Tue, 15 Feb 2011 06:24:57 -0500 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1PpJ1a-0007Mu-Hv for 8001@debbugs.gnu.org; Tue, 15 Feb 2011 06:24:55 -0500 Received: (qmail 91056 invoked from network); 15 Feb 2011 11:33:53 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 15 Feb 2011 11:33:53 -0000 Message-ID: <4D5A63DE.3040600@draigBrady.com> Date: Tue, 15 Feb 2011 11:30:38 +0000 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Marti Raudsepp Subject: Re: btrfs: compression breaks cp and cross-FS mv, FS_IOC_FIEMAP bug? References: <20110213155730.GK19533@dhcp231-156.rdu.redhat.com> <20110213161329.GL19533@dhcp231-156.rdu.redhat.com> <1297695521-sup-9460@think> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8001 Cc: btrfs hackers , 8001@debbugs.gnu.org, Josef Bacik , Chris Mason , Kernel hackers X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 14/02/11 17:58, Marti Raudsepp wrote: > On Mon, Feb 14, 2011 at 17:01, Chris Mason wrote: >> Or, it could just be delalloc ;) > > I suspect delalloc. After creating the file, filefrag reports "1 > extent found", but for some reason it doesn't actually print out > details of the extent. That's a bug in `filefrag -v` that I noticed independently yesterday. Without -v it will correctly report 0 extents. I've already suggested a patch to fix upstream. > After a "sync" call, the extent appears and "cp" starts working as expected: About that sync. I've noticed on ext4 loop back at least (and I suspect BTRFS is the same) that specifying FIEMAP_FLAG_SYNC (which cp does) is ineffective. I worked around this for cp tests by explicitly syncing with: dd if=/dev/null of=foo conv=notrunc,fdatasync > % rm -f foo bar > % echo foo > foo > % sync > % filefrag -v foo > Filesystem type is: 9123683e > File size of foo is 4 (1 block, blocksize 4096) > ext logical physical expected length flags > 0 0 0 4096 not_aligned,inline,eof > foo: 1 extent found > % cp foo bar > % hexdump bar > 0000000 6f66 0a6f > 0000004 OK that's fine for normal files. cp (from coreutils >= 8.10) may still do the wrong thing as it currently ignores FIEMAP_EXTENT_DATA_ENCRYPTED and FIEMAP_EXTENT_ENCODED as I've already reported: http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg08356.html I'd appreciate some `filefrag -v` output from a large compressed file. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 15 08:38:48 2011 Received: (at 8001) by debbugs.gnu.org; 15 Feb 2011 13:38:48 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PpL78-0002gD-U6 for submit@debbugs.gnu.org; Tue, 15 Feb 2011 08:38:47 -0500 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PpKjx-00028p-Dj for 8001@debbugs.gnu.org; Tue, 15 Feb 2011 08:14:50 -0500 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p1FDNmea029065 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 15 Feb 2011 08:23:48 -0500 Received: from dhcp231-156.rdu.redhat.com (dhcp231-156.rdu.redhat.com [10.11.231.156]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p1FDNlf2002359; Tue, 15 Feb 2011 08:23:47 -0500 Date: Tue, 15 Feb 2011 08:18:50 -0500 From: Josef Bacik To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: btrfs: compression breaks cp and cross-FS mv, FS_IOC_FIEMAP bug? Message-ID: <20110215131849.GC28010@dhcp231-156.rdu.redhat.com> References: <20110213155730.GK19533@dhcp231-156.rdu.redhat.com> <20110213161329.GL19533@dhcp231-156.rdu.redhat.com> <1297695521-sup-9460@think> <4D5A63DE.3040600@draigBrady.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <4D5A63DE.3040600@draigBrady.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id p1FDNmea029065 X-Spam-Score: -10.6 (----------) X-Debbugs-Envelope-To: 8001 X-Mailman-Approved-At: Tue, 15 Feb 2011 08:38:45 -0500 Cc: 8001@debbugs.gnu.org, Kernel hackers , Chris Mason , Josef Bacik , Marti Raudsepp , btrfs hackers X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.6 (----------) On Tue, Feb 15, 2011 at 11:30:38AM +0000, P=E1draig Brady wrote: > On 14/02/11 17:58, Marti Raudsepp wrote: > > On Mon, Feb 14, 2011 at 17:01, Chris Mason w= rote: > >> Or, it could just be delalloc ;) > >=20 > > I suspect delalloc. After creating the file, filefrag reports "1 > > extent found", but for some reason it doesn't actually print out > > details of the extent. >=20 > That's a bug in `filefrag -v` that I noticed independently yesterday. > Without -v it will correctly report 0 extents. > I've already suggested a patch to fix upstream. >=20 > > After a "sync" call, the extent appears and "cp" starts working as ex= pected: >=20 > About that sync. > I've noticed on ext4 loop back at least (and I suspect BTRFS is the sam= e) > that specifying FIEMAP_FLAG_SYNC (which cp does) is ineffective. > I worked around this for cp tests by explicitly syncing with: > dd if=3D/dev/null of=3Dfoo conv=3Dnotrunc,fdatasync > Well thats not good, thats all take care of in the generic code before it= gets to the fs, I'll take a look at that when I try and fix delalloc fiemap fo= r btrfs. Thanks, Josef=20 From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 19 13:28:51 2011 Received: (at 8001) by debbugs.gnu.org; 19 Feb 2011 18:28:52 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqrY3-0003Xm-Gm for submit@debbugs.gnu.org; Sat, 19 Feb 2011 13:28:51 -0500 Received: from smtp.gentoo.org ([140.211.166.183]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqrY1-0003Xa-PH for 8001@debbugs.gnu.org; Sat, 19 Feb 2011 13:28:50 -0500 Received: from vapier.localnet (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 94EC11B409A; Sat, 19 Feb 2011 18:28:43 +0000 (UTC) From: Mike Frysinger Organization: wh0rd.org To: =?iso-8859-1?q?P=E1draig_Brady?= Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) Date: Sat, 19 Feb 2011 13:28:40 -0500 User-Agent: KMail/1.13.5 (Linux/2.6.37; KDE/4.5.5; x86_64; ; ) References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> In-Reply-To: <4D52730A.2080008@draigBrady.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2628434.WugHTYgDEu"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201102191328.42067.vapier@gentoo.org> X-Spam-Score: -6.3 (------) X-Debbugs-Envelope-To: 8001 Cc: BTRFS MAILING LIST , 8001@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.3 (------) --nextPart2628434.WugHTYgDEu Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable based on other threads (which i havent been following too closely), did we= =20 settle on this being a btrfs bug ? =2Dmike --nextPart2628434.WugHTYgDEu Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAABAgAGBQJNYAvaAAoJEEFjO5/oN/WBmrYP/3C+SQT+j5Mp0VFrMMnsOVJ3 d3MAVuB4fzbbp3sl3eW6+ZRECTymAR+iXhwQ4Y5UQK53PFOshK43XSz22QCE3Njg Oxoc/w2J6SO9eszhPufLS6NqGTA4EZp0pjgbAW/VGpiHUDsi9IPwhciSsSM/0cAq IFlIiAm9plU8vS8vARXCA0a0H/YFASOnGvtPAOyprxiOqKgiFJtTKKeMxDGfshWt mFnzW94PlCNz2m6po8SzOm6h7OmTlfJzWhQV8QoAX2IHO3xv4cZ98x/neC1ZLzXn yZDHf2mOMji4iBV2F29qtIPR00iuRchGnK9xSRXzCAMIEYsex2jIUL/dQOJKZNZk pXWMSsUn2+/0HvjzHWEdxiq+AqP56zOrBuOrEDbroDsXjo4amtWRacIAniQNsajD o6jFNoEa2KcTkN1ClUqMd30SAw9YEWj0R1PFl7b3JI/awbi++IWed92GtAlkiKTW SCBjP8mtvYyhy1RnDMwEpafFzFAqWQFa74OvImHPjsKqrCZZ/StcktfRpxn8gvJh SR6gemOfvcRN+4Um7dZTKaUbUTqCiQk9d1hFiN7WF7nBBgRw9+yrZY6ER1s7t7bR LWqTJFok/qwMtZ+9G0fxTv6Cgk5svd1WD24Trj3yvHzsE/2vFJ41ujR/TeShkD7Q DFtJGNs3VIlOJWEmWqqf =vVtt -----END PGP SIGNATURE----- --nextPart2628434.WugHTYgDEu-- From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 19 18:26:00 2011 Received: (at 8001) by debbugs.gnu.org; 19 Feb 2011 23:26:00 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqwBb-00039h-QP for submit@debbugs.gnu.org; Sat, 19 Feb 2011 18:26:00 -0500 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1PqwBZ-00039U-HV for 8001@debbugs.gnu.org; Sat, 19 Feb 2011 18:25:58 -0500 Received: (qmail 49341 invoked from network); 19 Feb 2011 23:25:51 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 19 Feb 2011 23:25:51 -0000 Message-ID: <4D6050A7.5090407@draigBrady.com> Date: Sat, 19 Feb 2011 23:22:15 +0000 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Mike Frysinger Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> <201102191328.42067.vapier@gentoo.org> In-Reply-To: <201102191328.42067.vapier@gentoo.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8001 Cc: BTRFS MAILING LIST , 8001@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 19/02/11 18:28, Mike Frysinger wrote: > based on other threads (which i havent been following too closely), did we > settle on this being a btrfs bug ? > -mike Nope, cp 8.10 is not absolved yet. It may be btrfs not honoring FIEMAP_FLAG_SYNC, and/or it may be cp needing to handle FIEMAP_EXTENT_ENCODED specially. It would help if you ran `sync` before the copy, to exclude that as a possible issue. Also `filefrag -v` output for the file on the compressed BTRFS file system would be helpful. thanks, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 19 19:43:36 2011 Received: (at 8001) by debbugs.gnu.org; 20 Feb 2011 00:43:36 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqxOh-0004qU-MU for submit@debbugs.gnu.org; Sat, 19 Feb 2011 19:43:35 -0500 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1PqxOc-0004qC-86 for 8001@debbugs.gnu.org; Sat, 19 Feb 2011 19:43:34 -0500 Received: (qmail 59850 invoked from network); 20 Feb 2011 00:43:24 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 20 Feb 2011 00:43:24 -0000 Message-ID: <4D6062D3.3090905@draigBrady.com> Date: Sun, 20 Feb 2011 00:39:47 +0000 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Mike Frysinger Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> <201102191328.42067.vapier@gentoo.org> <4D6050A7.5090407@draigBrady.com> In-Reply-To: <4D6050A7.5090407@draigBrady.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8001 Cc: BTRFS MAILING LIST , 8001@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 19/02/11 23:22, Pádraig Brady wrote: > On 19/02/11 18:28, Mike Frysinger wrote: >> based on other threads (which i havent been following too closely), did we >> settle on this being a btrfs bug ? >> -mike > > Nope, cp 8.10 is not absolved yet. > It may be btrfs not honoring FIEMAP_FLAG_SYNC, > and/or it may be cp needing to handle FIEMAP_EXTENT_ENCODED > specially. > > It would help if you ran `sync` before the copy, > to exclude that as a possible issue. > > Also `filefrag -v` output for the file on > the compressed BTRFS file system would be helpful. Hmm I just tried the latest Fedora 15 live image and was tripped up a bit by https://bugzilla.redhat.com/show_bug.c Anyway it uses 2.6.38 rc4 and "compress" doesn't seem to be implemented for BTRFS there? Writing a file of zeros showed a normal disk usage, and filefrag -v didn't report any flags for the extents. In any case, cp/fiemap-2 passed on each of about 100 runs, even without introducing and syncs. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 19 19:53:47 2011 Received: (at 8001) by debbugs.gnu.org; 20 Feb 2011 00:53:47 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqxYY-00054a-7d for submit@debbugs.gnu.org; Sat, 19 Feb 2011 19:53:46 -0500 Received: from mail-fx0-f44.google.com ([209.85.161.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PqxYV-00054O-Ru for 8001@debbugs.gnu.org; Sat, 19 Feb 2011 19:53:44 -0500 Received: by fxm15 with SMTP id 15so1386766fxm.3 for <8001@debbugs.gnu.org>; Sat, 19 Feb 2011 16:53:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=K+LCfXwL0NnheFfAT7kfZ7+qz6mD2Kml7OZ+PvdjzsQ=; b=MSsfiJXHN0ysoP3B1daoaYKGuJdnoM3PcZyttfJLeNm6raEeOlwmeanfbn9dF7u7gv EdhpiYVGL5P1bSj2opluoZynzX2zOVHPAPnt+xqTolkTul1IGwvbYAJEuWdbYPJ7UTAb cUm/UwOKC05O4asguytKlwoXGlmT0MBPg8uz8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=AhFWB5FIwsvoT5QlTlxsuWi47wu5uJYzyMp67yNZwn4EY5tCcw4Ne3gi1qDUBqnnAG M3RxduNZXgQTuApy0SLKA8QGApqtJN63nb0pZdWyUpP96MdGvQQPXwiwN83EVqk+fi9e KQ5yNZogk7vSv4qSoMIJskf+x2KelHua9UsBs= Received: by 10.223.53.68 with SMTP id l4mr3056602fag.44.1298163218100; Sat, 19 Feb 2011 16:53:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.115.140 with HTTP; Sat, 19 Feb 2011 16:53:18 -0800 (PST) In-Reply-To: <4D6050A7.5090407@draigBrady.com> References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> <201102191328.42067.vapier@gentoo.org> <4D6050A7.5090407@draigBrady.com> From: Mike Frysinger Date: Sat, 19 Feb 2011 19:53:18 -0500 X-Google-Sender-Auth: niyE_IrRLb58YeJ_YU0RPgVQAPE Message-ID: Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) To: =?ISO-8859-1?Q?P=E1draig_Brady?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.6 (---) X-Debbugs-Envelope-To: 8001 Cc: BTRFS MAILING LIST , 8001@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.6 (---) 2011/2/19 P=E1draig Brady: > On 19/02/11 18:28, Mike Frysinger wrote: >> based on other threads (which i havent been following too closely), did = we >> settle on this being a btrfs bug ? > > Nope, cp 8.10 is not absolved yet. > It may be btrfs not honoring FIEMAP_FLAG_SYNC, > and/or it may be cp needing to handle FIEMAP_EXTENT_ENCODED > specially. > > It would help if you ran `sync` before the copy, > to exclude that as a possible issue. makes no difference after applying this patch: --- coreutils-8.10/tests/cp/fiemap-2 +++ coreutils-8.10/tests/cp/fiemap-2 @@ -43,12 +43,19 @@ printf x > k || framework_failure_ dd bs=3D1k seek=3D1 of=3Dk count=3D255 < /dev/zero || framework_failure_ +filefrag -v k +sync +filefrag -v k + # cp should detect the all-zero blocks and convert some of them to holes. # How many it detects/converts currently depends on io_blksize. # Currently, on my F14/ext4 desktop, this K starts off with size 256KiB, # (note that the K in the preceding test starts off with size 4KiB). # cp from coreutils-8.9 with --sparse=3Dalways reduces the size to 32KiB. cp --sparse=3Dalways k k2 || fail=3D1 + +sync + test $(stat -c %b k2) -lt $(stat -c %b k) || fail=3D1 Exit $fail > Also `filefrag -v` output for the file on > the compressed BTRFS file system would be helpful. the output from the aforementioned patch: ... + dd bs=3D1k seek=3D1 of=3Dk count=3D255 255+0 records in 255+0 records out 261120 bytes (261 kB) copied, 0.00313737 s, 83.2 MB/s + filefrag -v k Filesystem type is: 9123683e File size of k is 262144 (64 blocks, blocksize 4096) ext logical physical expected length flags k: 1 extent found + sync + filefrag -v k Filesystem type is: 9123683e File size of k is 262144 (64 blocks, blocksize 4096) ext logical physical expected length flags 0 0 47917 32 1 32 47918 47948 32 eof k: 2 extents found ... -mike From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 22 22:35:55 2011 Received: (at 8001) by debbugs.gnu.org; 23 Feb 2011 03:35:56 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Ps5W7-0006oE-IC for submit@debbugs.gnu.org; Tue, 22 Feb 2011 22:35:55 -0500 Received: from rcsinet10.oracle.com ([148.87.113.121]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Ps5Sr-0006jg-Ab for 8001@debbugs.gnu.org; Tue, 22 Feb 2011 22:32:33 -0500 Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by rcsinet10.oracle.com (Switch-3.4.2/Switch-3.4.2) with ESMTP id p1N3WPtu008481 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 23 Feb 2011 03:32:27 GMT Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id p1N3WOjG004343; Wed, 23 Feb 2011 03:32:24 GMT Received: from abhmt015.oracle.com by acsmt354.oracle.com with ESMTP id 1077992321298431881; Tue, 22 Feb 2011 19:31:21 -0800 Received: from localhost (/66.66.148.185) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 22 Feb 2011 19:31:21 -0800 Content-Type: text/plain; charset=UTF-8 From: Chris Mason To: Mike Frysinger Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) In-reply-to: References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> <201102191328.42067.vapier@gentoo.org> <4D6050A7.5090407@draigBrady.com> Date: Tue, 22 Feb 2011 22:31:19 -0500 Message-Id: <1298431849-sup-5365@think> User-Agent: Sup/git X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4D647FC9.0071:SCFMA4539814,ss=1,fgs=0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by rcsinet10.oracle.com id p1N3WPtu008481 X-Spam-Score: -6.6 (------) X-Debbugs-Envelope-To: 8001 X-Mailman-Approved-At: Tue, 22 Feb 2011 22:35:54 -0500 Cc: =?utf-8?q?P=C3=A1draig_Brady?= , BTRFS MAILING LIST , 8001 <8001@debbugs.gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.6 (------) Excerpts from Mike Frysinger's message of 2011-02-19 19:53:18 -0500: > 2011/2/19 P=C3=A1draig Brady: > > On 19/02/11 18:28, Mike Frysinger wrote: > >> based on other threads (which i havent been following too closely), = did we > >> settle on this being a btrfs bug ? > > > > Nope, cp 8.10 is not absolved yet. > > It may be btrfs not honoring FIEMAP_FLAG_SYNC, > > and/or it may be cp needing to handle FIEMAP_EXTENT_ENCODED > > specially. > > > > It would help if you ran `sync` before the copy, > > to exclude that as a possible issue. >=20 > makes no difference after applying this patch: I've got a fix for this on the btrfs side in testing. It is definitely a btrfs bug. -chris From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 23 04:53:57 2011 Received: (at 8001-done) by debbugs.gnu.org; 23 Feb 2011 09:53:57 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PsBPx-00009F-3q for submit@debbugs.gnu.org; Wed, 23 Feb 2011 04:53:57 -0500 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1PsBPu-000091-As for 8001-done@debbugs.gnu.org; Wed, 23 Feb 2011 04:53:55 -0500 Received: (qmail 50754 invoked from network); 23 Feb 2011 09:53:48 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 23 Feb 2011 09:53:48 -0000 Message-ID: <4D64D843.9080903@draigBrady.com> Date: Wed, 23 Feb 2011 09:49:55 +0000 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Chris Mason , 8001-done@debbugs.gnu.org Subject: Re: bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2) References: <201102072253.34476.vapier@gentoo.org> <4D52730A.2080008@draigBrady.com> <201102191328.42067.vapier@gentoo.org> <4D6050A7.5090407@draigBrady.com> <1298431849-sup-5365@think> In-Reply-To: <1298431849-sup-5365@think> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8001-done Cc: Mike Frysinger , BTRFS MAILING LIST X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 23/02/11 03:31, Chris Mason wrote: > I've got a fix for this on the btrfs side in testing. It is definitely > a btrfs bug. [resend] Cool. I'm closing this ticket, so. Interesting I didn't notice this with kernel-2.6.38-0.rc4.git0.2.fc15.i686 (with a 1G loop back BTRFS on ext4) I'd appreciate a link to the fix at some stage. If we know what's going on exactly we might be able to know how widespread the issue is and whether we could work around it in `cp`. cheers, Pádraig. From unknown Mon Jun 16 23:34:33 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 23 Mar 2011 11:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator