From unknown Sun Jun 22 22:42:53 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#8411 <8411@debbugs.gnu.org> To: bug#8411 <8411@debbugs.gnu.org> Subject: Status: due to missing sync even on 2.6.39, cp fails to copy an odd file Reply-To: bug#8411 <8411@debbugs.gnu.org> Date: Mon, 23 Jun 2025 05:42:53 +0000 retitle 8411 due to missing sync even on 2.6.39, cp fails to copy an odd fi= le reassign 8411 coreutils submitter 8411 Jim Meyering severity 8411 normal tag 8411 fixed thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 07:16:57 2011 Received: (at submit) by debbugs.gnu.org; 2 Apr 2011 11:16:57 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q5yp6-000408-Mc for submit@debbugs.gnu.org; Sat, 02 Apr 2011 07:16:57 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q5yp3-0003zt-R5 for submit@debbugs.gnu.org; Sat, 02 Apr 2011 07:16:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Q5yox-0003bQ-6a for submit@debbugs.gnu.org; Sat, 02 Apr 2011 07:16:48 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:38123) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Q5yox-0003bH-35 for submit@debbugs.gnu.org; Sat, 02 Apr 2011 07:16:47 -0400 Received: from [140.186.70.92] (port=60323 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Q5you-0006Zk-Ga for bug-coreutils@gnu.org; Sat, 02 Apr 2011 07:16:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Q5yos-0003Yt-VD for bug-coreutils@gnu.org; Sat, 02 Apr 2011 07:16:44 -0400 Received: from mx.meyering.net ([82.230.74.64]:59073) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Q5yos-0003XL-MD for bug-coreutils@gnu.org; Sat, 02 Apr 2011 07:16:42 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 3265B602C8; Sat, 2 Apr 2011 13:16:39 +0200 (CEST) From: Jim Meyering To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: due to missing sync even on 2.6.39, cp fails to copy an odd file Date: Sat, 02 Apr 2011 13:16:39 +0200 Message-ID: <87fwq0gay0.fsf@rho.meyering.net> Lines: 65 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 199.232.76.165 X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: submit Cc: bug-coreutils@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) Hi P=E1draig, As of this change, copy: with fiemap copy, only sync when needed http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=3Df69876e55 fiemap copy with extents beyond EOF can fail on ext4 even with Fedora 15 (2.6.38) and rawhide's 2.6.39 kernel. Here we construct an odd file. First, preallocate 10MB of space, and then write 5KiB of random data into the beginning of that: $ fallocate -l 10000000 -n k $ dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of= =3Dk However, when we try to copy "k", we get a file, "k2" of the expected size, but with only NUL bytes for contents: $ strace -o log ./cp k k2 $ cmp -l k k2|head -2 1 60 0 2 164 0 $ uname -r 2.6.39-0.rc0.git11.0.fc16.x86_64 The strace showed no reads were required, so to make that test succeed, we would have to sync k's metadata to disk before invoking cp. This change (to make cp require FIEMAP sync for kernels before 2.6.40) solves the problem, but I hope there's a better way... diff --git a/src/extent-scan.c b/src/extent-scan.c index c0a5de6..1d9d06b 100644 --- a/src/extent-scan.c +++ b/src/extent-scan.c @@ -50,7 +50,7 @@ extent_need_sync (void) unsigned long val; if (xstrtoul (name.release + 4, NULL, 10, &val, NULL) =3D=3D LO= NGINT_OK) { - if (val < 38) + if (val < 40) need_sync =3D 1; } } -------------------------------- The above is part of P=E1draig's new fiemap-empty test, which fails on both F15 and rawhide: FAIL: cp/fiemap-empty (exit: 1) ... + fallocate -l 10000000 -n unwritten.withdata + dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of=3Du= nwritten.withdata 10+0 records in 10+0 records out 5120 bytes (5.1 kB) copied, 0.00248136 s, 2.1 MB/s + cp unwritten.withdata cp.test ++ stat -c %s unwritten.withdata ++ stat -c %s cp.test + test 5120 =3D 5120 + cmp unwritten.withdata cp.test unwritten.withdata cp.test differ: char 1, line 1 + fail=3D1 From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 09:30:06 2011 Received: (at 8411) by debbugs.gnu.org; 2 Apr 2011 13:30:06 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q60ty-0007kZ-0Q for submit@debbugs.gnu.org; Sat, 02 Apr 2011 09:30:06 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1Q60tu-0007jn-Nb for 8411@debbugs.gnu.org; Sat, 02 Apr 2011 09:30:04 -0400 Received: (qmail 40442 invoked from network); 2 Apr 2011 13:29:56 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 2 Apr 2011 13:29:56 -0000 Message-ID: <4D9724AC.2070009@draigBrady.com> Date: Sat, 02 Apr 2011 14:29:16 +0100 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: 8411@debbugs.gnu.org Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file References: <87fwq0gay0.fsf@rho.meyering.net> In-Reply-To: <87fwq0gay0.fsf@rho.meyering.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8411 Cc: linux-ext4@vger.kernel.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 02/04/11 12:16, Jim Meyering wrote: > Hi Pádraig, > > As of this change, > > copy: with fiemap copy, only sync when needed > http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=f69876e55 > > fiemap copy with extents beyond EOF can fail on ext4 even with > Fedora 15 (2.6.38) and rawhide's 2.6.39 kernel. > > Here we construct an odd file. First, preallocate 10MB of space, > and then write 5KiB of random data into the beginning of that: > > $ fallocate -l 10000000 -n k > $ dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k > > However, when we try to copy "k", we get a file, "k2" of the > expected size, but with only NUL bytes for contents: So the extent info is not updated until sync(), which means cp will consider the "unwritten" extent as NUL data and not bother to read it :( I guess this is a corner case that was missed in the fixups for ext4 (and btrfs?) in 2.6.38 for this? I've copied ext4 devs for clarification. I.E. if you do this: fallocate -l 10000000 -n k dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k filefrag -v k Do you get all extents still unwritten. I do on my 2.6.35.11-83.fc14.i686 kernel, but I expected that. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 09:50:23 2011 Received: (at 8411) by debbugs.gnu.org; 2 Apr 2011 13:50:23 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q61Db-0008BO-8T for submit@debbugs.gnu.org; Sat, 02 Apr 2011 09:50:23 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q61DT-0008B3-Ta for 8411@debbugs.gnu.org; Sat, 02 Apr 2011 09:50:21 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 189C8602C8; Sat, 2 Apr 2011 15:50:09 +0200 (CEST) From: Jim Meyering To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file In-Reply-To: <4D9724AC.2070009@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?= =?iso-8859-1?Q?g?= Brady"'s message of "Sat, 02 Apr 2011 14:29:16 +0100") References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> Date: Sat, 02 Apr 2011 15:50:08 +0200 Message-ID: <8739m0g3u7.fsf@rho.meyering.net> Lines: 82 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) P=E1draig Brady wrote: > On 02/04/11 12:16, Jim Meyering wrote: >> Hi P=E1draig, >> As of this change, >> >> copy: with fiemap copy, only sync when needed >> http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=3Df69876e55 >> >> fiemap copy with extents beyond EOF can fail on ext4 even with >> Fedora 15 (2.6.38) and rawhide's 2.6.39 kernel. >> >> Here we construct an odd file. First, preallocate 10MB of space, >> and then write 5KiB of random data into the beginning of that: >> >> $ fallocate -l 10000000 -n k >> $ dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock o= f=3Dk >> >> However, when we try to copy "k", we get a file, "k2" of the >> expected size, but with only NUL bytes for contents: > > So the extent info is not updated until sync(), > which means cp will consider the "unwritten" extent > as NUL data and not bother to read it :( > > I guess this is a corner case that was missed > in the fixups for ext4 (and btrfs?) in 2.6.38 for this? > I've copied ext4 devs for clarification. > I.E. if you do this: > > fallocate -l 10000000 -n k > dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of=3Dk > filefrag -v k > > Do you get all extents still unwritten. > I do on my 2.6.35.11-83.fc14.i686 kernel, but I expected that. Right. The new extent remains unwritten until after a sync with ext4 on both Fedora 15 and rawhide kernels. Here's F15: $ fallocate -l 10000000 -n k $ dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of= =3Dk ... $ filefrag -v k Filesystem type is: ef53 File size of k is 5120 (2 blocks, blocksize 4096) ext logical physical expected length flags 0 0 6498304 2442 unwritten,eof k: 1 extent found $ sync $ filefrag -v k Filesystem type is: ef53 File size of k is 5120 (2 blocks, blocksize 4096) ext logical physical expected length flags 0 0 6498304 2 eof 1 2 6498306 2440 unwritten,eof k: 1 extent found Same problem with rawhide, but slightly different layout: $ fallocate -l 10000000 -n k $ dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of= =3Dk ... $ filefrag -v k Filesystem type is: ef53 File size of k is 5120 (2 blocks, blocksize 4096) ext logical physical expected length flags 0 0 2054144 2048 unwritten,eof 1 2048 2060288 2056191 394 unwritten,eof k: 2 extents found $ filefrag -v k [Exit 1] $ sync $ filefrag -v k Filesystem type is: ef53 File size of k is 5120 (2 blocks, blocksize 4096) ext logical physical expected length flags 0 0 2054144 2 eof 1 2 2054146 2046 unwritten,eof 2 2048 2060288 2056191 394 unwritten,eof k: 2 extents found From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 14:08:46 2011 Received: (at 8411) by debbugs.gnu.org; 2 Apr 2011 18:08:46 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q65Fd-0005PU-Qe for submit@debbugs.gnu.org; Sat, 02 Apr 2011 14:08:46 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q65FZ-0005PG-Vl for 8411@debbugs.gnu.org; Sat, 02 Apr 2011 14:08:44 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 675EA6040D; Sat, 2 Apr 2011 20:08:34 +0200 (CEST) From: Jim Meyering To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file In-Reply-To: <8739m0g3u7.fsf@rho.meyering.net> (Jim Meyering's message of "Sat, 02 Apr 2011 15:50:08 +0200") References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> Date: Sat, 02 Apr 2011 20:08:34 +0200 Message-ID: <87d3l4edb1.fsf@rho.meyering.net> Lines: 81 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) Jim Meyering wrote: > P=E1draig Brady wrote: >> On 02/04/11 12:16, Jim Meyering wrote: >>> Hi P=E1draig, >>> As of this change, >>> >>> copy: with fiemap copy, only sync when needed >>> http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=3Df69876e= 55 >>> >>> fiemap copy with extents beyond EOF can fail on ext4 even with >>> Fedora 15 (2.6.38) and rawhide's 2.6.39 kernel. >>> >>> Here we construct an odd file. First, preallocate 10MB of space, >>> and then write 5KiB of random data into the beginning of that: >>> >>> $ fallocate -l 10000000 -n k >>> $ dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock = of=3Dk >>> >>> However, when we try to copy "k", we get a file, "k2" of the >>> expected size, but with only NUL bytes for contents: >> >> So the extent info is not updated until sync(), >> which means cp will consider the "unwritten" extent >> as NUL data and not bother to read it :( >> >> I guess this is a corner case that was missed >> in the fixups for ext4 (and btrfs?) in 2.6.38 for this? >> I've copied ext4 devs for clarification. >> I.E. if you do this: >> >> fallocate -l 10000000 -n k >> dd count=3D10 if=3D/dev/urandom conv=3Dnotrunc iflag=3Dfullblock of=3Dk >> filefrag -v k >> >> Do you get all extents still unwritten. >> I do on my 2.6.35.11-83.fc14.i686 kernel, but I expected that. > > Right. The new extent remains unwritten until after a sync > with ext4 on both Fedora 15 and rawhide kernels. Here's the quick band-aid I'll push tomorrow. I'll welcome a better fix, but this, at least, lets "make check" pass once again. >From 0a6d128d0d17c1604245f1caafe6af73584a0bb8 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Sat, 2 Apr 2011 19:59:30 +0200 Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 kerne= ls * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38 and 2.6.39. Without this, part of the cp/fiemap-empty test would fail both on F15-to-be and rawhide. For discussion and details, see: http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 --- src/extent-scan.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/extent-scan.c b/src/extent-scan.c index 4ddf46e..752de85 100644 --- a/src/extent-scan.c +++ b/src/extent-scan.c @@ -31,7 +31,7 @@ # include "fiemap.h" #endif -/* Work around Linux kernel issues on BTRFS and EXT4 before 2.6.38. +/* Work around Linux kernel issues on BTRFS and EXT4 before 2.6.40. FIXME: remove in 2013, or whenever we're pretty confident that the offending, unpatched kernels are no longer in use. */ static bool @@ -50,7 +50,7 @@ extent_need_sync (void) unsigned long val; if (xstrtoul (name.release + 4, NULL, 10, &val, NULL) =3D=3D LO= NGINT_OK) { - if (val < 38) + if (val < 40) need_sync =3D 1; } } -- 1.7.4.2.662.gcbd0 From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 15:00:32 2011 Received: (at 8411) by debbugs.gnu.org; 2 Apr 2011 19:00:32 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q663k-0006Y0-12 for submit@debbugs.gnu.org; Sat, 02 Apr 2011 15:00:32 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1Q663h-0006Xj-DN for 8411@debbugs.gnu.org; Sat, 02 Apr 2011 15:00:30 -0400 Received: (qmail 81094 invoked from network); 2 Apr 2011 19:00:22 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 2 Apr 2011 19:00:22 -0000 Message-ID: <4D97721E.2060701@draigBrady.com> Date: Sat, 02 Apr 2011 19:59:42 +0100 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: 8411@debbugs.gnu.org Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> In-Reply-To: <87d3l4edb1.fsf@rho.meyering.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8411 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 02/04/11 19:08, Jim Meyering wrote: > Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 kernels That's the best workaround for the moment, as I can't think of anything else that could be done in user space. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 02 21:16:35 2011 Received: (at 8411) by debbugs.gnu.org; 3 Apr 2011 01:16:35 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6Bve-0007dh-6E for submit@debbugs.gnu.org; Sat, 02 Apr 2011 21:16:35 -0400 Received: from li9-11.members.linode.com ([67.18.176.11] helo=test.thunk.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q69nl-0003JU-Sb for 8411@debbugs.gnu.org; Sat, 02 Apr 2011 19:00:22 -0400 Received: from root (helo=tytso-glaptop) by test.thunk.org with local-esmtp (Exim 4.69) (envelope-from ) id 1Q69nb-0008EC-DK; Sat, 02 Apr 2011 23:00:07 +0000 Received: from tytso by tytso-glaptop with local (Exim 4.71) (envelope-from ) id 1Q69nZ-0004S5-D9; Sat, 02 Apr 2011 19:00:05 -0400 Date: Sat, 2 Apr 2011 19:00:05 -0400 From: Ted Ts'o To: Jim Meyering Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file Message-ID: <20110402230005.GL21075@thunk.org> References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87d3l4edb1.fsf@rho.meyering.net> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: tytso@thunk.org X-SA-Exim-Scanned: No (on test.thunk.org); SAEximRunCond expanded to false X-Spam-Score: -2.6 (--) X-Debbugs-Envelope-To: 8411 X-Mailman-Approved-At: Sat, 02 Apr 2011 21:16:33 -0400 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org, =?iso-8859-1?Q?P=E1draig?= Brady X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On Sat, Apr 02, 2011 at 08:08:34PM +0200, Jim Meyering wrote: > From 0a6d128d0d17c1604245f1caafe6af73584a0bb8 Mon Sep 17 00:00:00 2001 > From: Jim Meyering > Date: Sat, 2 Apr 2011 19:59:30 +0200 > Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 kernels > > * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38 > and 2.6.39. Without this, part of the cp/fiemap-empty test would fail > both on F15-to-be and rawhide. For discussion and details, see: > http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 FYI, the following fix has been merged into mainline, which should fix the problem for 2.6.39 once it is finally released, at least for ext4. It was merged right before Linus released 2.6.39-rc1. I'm assuming that Rawhide released a pre-2.6.39-rc1 kernel in the middle of the merge window. Some distro's will informally, but incorrectly, refer to such a release as "2.6.39-rc0". I prefer the more technically correct 2.6.38-git18 (which is the first git tag in the Linux git repo which contained the patch below, as of March 25, 2011). Unfortunately, RPM doesn't understand that 2.6.38-rc1 sorts before 2.6.38, while 2.6.38-git17 sorts *after* 2.6.38. (Hence the incorrect, but convenient, use of 2.6.39-rc0.) - Ted commit 6d9c85eb700bd3ac59e63bb9de463dea1aca084c Author: Yongqiang Yang Date: Sun Feb 27 17:25:47 2011 -0500 ext4: make FIEMAP and delayed allocation play well together Fix the FIEMAP ioctl so that it returns all of the page ranges which are still subject to delayed allocation. We were missing some cases if the file was sparse. Reported by Chris Mason : >We've had reports on btrfs that cp is giving us files full of zeros >instead of actually copying them. It was tracked down to a bug with >the btrfs fiemap implementation where it was returning holes for >delalloc ranges. > >Newer versions of cp are trusting fiemap to tell it where the holes >are, which does seem like a pretty neat trick. > >I decided to give xfs and ext4 a shot with a few tests cases too, xfs >passed with all the ones btrfs was getting wrong, and ext4 got the basic >delalloc case right. >$ mkfs.ext4 /dev/xxx >$ mount /dev/xxx /mnt >$ dd if=/dev/zero of=/mnt/foo bs=1M count=1 >$ fiemap-test foo >ext: 0 logical: [ 0.. 255] phys: 0.. 255 >flags: 0x007 tot: 256 > >Horray! But once we throw a hole in, things go bad: >$ mkfs.ext4 /dev/xxx >$ mount /dev/xxx /mnt >$ dd if=/dev/zero of=/mnt/foo bs=1M count=1 seek=1 >$ fiemap-test foo >< no output > > >We've got a delalloc extent after the hole and ext4 fiemap didn't find >it. If I run sync to kick the delalloc out: >$sync >$ fiemap-test foo >ext: 0 logical: [ 256.. 511] phys: 34048.. 34303 >flags: 0x001 tot: 256 > >fiemap-test is sitting in my /usr/local/bin, and I have no idea how it >got there. It's full of pretty comments so I know it isn't mine, but >you can grab it here: > >http://oss.oracle.com/~mason/fiemap-test.c > >xfsqa has a fiemap program too. After Fix, test results are as follows: ext: 0 logical: [ 256.. 511] phys: 0.. 255 flags: 0x007 tot: 256 ext: 0 logical: [ 256.. 511] phys: 33280.. 33535 flags: 0x001 tot: 256 $ mkfs.ext4 /dev/xxx $ mount /dev/xxx /mnt $ dd if=/dev/zero of=/mnt/foo bs=1M count=1 seek=1 $ sync $ dd if=/dev/zero of=/mnt/foo bs=1M count=1 seek=3 $ dd if=/dev/zero of=/mnt/foo bs=1M count=1 seek=5 $ fiemap-test foo ext: 0 logical: [ 256.. 511] phys: 33280.. 33535 flags: 0x000 tot: 256 ext: 1 logical: [ 768.. 1023] phys: 0.. 255 flags: 0x006 tot: 256 ext: 2 logical: [ 1280.. 1535] phys: 0.. 255 flags: 0x007 tot: 256 Tested-by: Eric Sandeen Reviewed-by: Andreas Dilger Signed-off-by: Yongqiang Yang Signed-off-by: "Theodore Ts'o" diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index d16f6b5..9ea1bc6 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -3775,6 +3775,7 @@ int ext4_convert_unwritten_extents(struct inode *inode, loff_t offset, } return ret > 0 ? ret2 : ret; } + /* * Callback function called for each extent to gather FIEMAP information. */ @@ -3782,38 +3783,162 @@ static int ext4_ext_fiemap_cb(struct inode *inode, struct ext4_ext_path *path, struct ext4_ext_cache *newex, struct ext4_extent *ex, void *data) { - struct fiemap_extent_info *fieinfo = data; - unsigned char blksize_bits = inode->i_sb->s_blocksize_bits; __u64 logical; __u64 physical; __u64 length; + loff_t size; __u32 flags = 0; - int error; + int ret = 0; + struct fiemap_extent_info *fieinfo = data; + unsigned char blksize_bits; - logical = (__u64)newex->ec_block << blksize_bits; + blksize_bits = inode->i_sb->s_blocksize_bits; + logical = (__u64)newex->ec_block << blksize_bits; if (newex->ec_start == 0) { - pgoff_t offset; - struct page *page; + /* + * No extent in extent-tree contains block @newex->ec_start, + * then the block may stay in 1)a hole or 2)delayed-extent. + * + * Holes or delayed-extents are processed as follows. + * 1. lookup dirty pages with specified range in pagecache. + * If no page is got, then there is no delayed-extent and + * return with EXT_CONTINUE. + * 2. find the 1st mapped buffer, + * 3. check if the mapped buffer is both in the request range + * and a delayed buffer. If not, there is no delayed-extent, + * then return. + * 4. a delayed-extent is found, the extent will be collected. + */ + ext4_lblk_t end = 0; + pgoff_t last_offset; + pgoff_t offset; + pgoff_t index; + struct page **pages = NULL; struct buffer_head *bh = NULL; + struct buffer_head *head = NULL; + unsigned int nr_pages = PAGE_SIZE / sizeof(struct page *); + + pages = kmalloc(PAGE_SIZE, GFP_KERNEL); + if (pages == NULL) + return -ENOMEM; offset = logical >> PAGE_SHIFT; - page = find_get_page(inode->i_mapping, offset); - if (!page || !page_has_buffers(page)) - return EXT_CONTINUE; +repeat: + last_offset = offset; + head = NULL; + ret = find_get_pages_tag(inode->i_mapping, &offset, + PAGECACHE_TAG_DIRTY, nr_pages, pages); + + if (!(flags & FIEMAP_EXTENT_DELALLOC)) { + /* First time, try to find a mapped buffer. */ + if (ret == 0) { +out: + for (index = 0; index < ret; index++) + page_cache_release(pages[index]); + /* just a hole. */ + kfree(pages); + return EXT_CONTINUE; + } - bh = page_buffers(page); + /* Try to find the 1st mapped buffer. */ + end = ((__u64)pages[0]->index << PAGE_SHIFT) >> + blksize_bits; + if (!page_has_buffers(pages[0])) + goto out; + head = page_buffers(pages[0]); + if (!head) + goto out; - if (!bh) - return EXT_CONTINUE; + bh = head; + do { + if (buffer_mapped(bh)) { + /* get the 1st mapped buffer. */ + if (end > newex->ec_block + + newex->ec_len) + /* The buffer is out of + * the request range. + */ + goto out; + goto found_mapped_buffer; + } + bh = bh->b_this_page; + end++; + } while (bh != head); - if (buffer_delay(bh)) { - flags |= FIEMAP_EXTENT_DELALLOC; - page_cache_release(page); + /* No mapped buffer found. */ + goto out; } else { - page_cache_release(page); - return EXT_CONTINUE; + /*Find contiguous delayed buffers. */ + if (ret > 0 && pages[0]->index == last_offset) + head = page_buffers(pages[0]); + bh = head; + } + +found_mapped_buffer: + if (bh != NULL && buffer_delay(bh)) { + /* 1st or contiguous delayed buffer found. */ + if (!(flags & FIEMAP_EXTENT_DELALLOC)) { + /* + * 1st delayed buffer found, record + * the start of extent. + */ + flags |= FIEMAP_EXTENT_DELALLOC; + newex->ec_block = end; + logical = (__u64)end << blksize_bits; + } + /* Find contiguous delayed buffers. */ + do { + if (!buffer_delay(bh)) + goto found_delayed_extent; + bh = bh->b_this_page; + end++; + } while (bh != head); + + for (index = 1; index < ret; index++) { + if (!page_has_buffers(pages[index])) { + bh = NULL; + break; + } + head = page_buffers(pages[index]); + if (!head) { + bh = NULL; + break; + } + if (pages[index]->index != + pages[0]->index + index) { + /* Blocks are not contiguous. */ + bh = NULL; + break; + } + bh = head; + do { + if (!buffer_delay(bh)) + /* Delayed-extent ends. */ + goto found_delayed_extent; + bh = bh->b_this_page; + end++; + } while (bh != head); + } + } else if (!(flags & FIEMAP_EXTENT_DELALLOC)) + /* a hole found. */ + goto out; + +found_delayed_extent: + newex->ec_len = min(end - newex->ec_block, + (ext4_lblk_t)EXT_INIT_MAX_LEN); + if (ret == nr_pages && bh != NULL && + newex->ec_len < EXT_INIT_MAX_LEN && + buffer_delay(bh)) { + /* Have not collected an extent and continue. */ + for (index = 0; index < ret; index++) + page_cache_release(pages[index]); + goto repeat; } + + for (index = 0; index < ret; index++) + page_cache_release(pages[index]); + kfree(pages); } physical = (__u64)newex->ec_start << blksize_bits; @@ -3822,32 +3947,16 @@ static int ext4_ext_fiemap_cb(struct inode *inode, struct ext4_ext_path *path, if (ex && ext4_ext_is_uninitialized(ex)) flags |= FIEMAP_EXTENT_UNWRITTEN; - /* - * If this extent reaches EXT_MAX_BLOCK, it must be last. - * - * Or if ext4_ext_next_allocated_block is EXT_MAX_BLOCK, - * this also indicates no more allocated blocks. - * - * XXX this might miss a single-block extent at EXT_MAX_BLOCK - */ - if (ext4_ext_next_allocated_block(path) == EXT_MAX_BLOCK || - newex->ec_block + newex->ec_len - 1 == EXT_MAX_BLOCK) { - loff_t size = i_size_read(inode); - loff_t bs = EXT4_BLOCK_SIZE(inode->i_sb); - + size = i_size_read(inode); + if (logical + length >= size) flags |= FIEMAP_EXTENT_LAST; - if ((flags & FIEMAP_EXTENT_DELALLOC) && - logical+length > size) - length = (size - logical + bs - 1) & ~(bs-1); - } - error = fiemap_fill_next_extent(fieinfo, logical, physical, + ret = fiemap_fill_next_extent(fieinfo, logical, physical, length, flags); - if (error < 0) - return error; - if (error == 1) + if (ret < 0) + return ret; + if (ret == 1) return EXT_BREAK; - return EXT_CONTINUE; } From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 03 06:12:58 2011 Received: (at 8411) by debbugs.gnu.org; 3 Apr 2011 10:12:58 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6KIj-0003hN-Q9 for submit@debbugs.gnu.org; Sun, 03 Apr 2011 06:12:58 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1Q6KIh-0003hA-PM for 8411@debbugs.gnu.org; Sun, 03 Apr 2011 06:12:56 -0400 Received: (qmail 89050 invoked from network); 3 Apr 2011 10:12:49 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 3 Apr 2011 10:12:49 -0000 Message-ID: <4D9847F5.8030804@draigBrady.com> Date: Sun, 03 Apr 2011 11:12:05 +0100 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Ted Ts'o Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> <20110402230005.GL21075@thunk.org> In-Reply-To: <20110402230005.GL21075@thunk.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org, Jim Meyering X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 03/04/11 00:00, Ted Ts'o wrote: > On Sat, Apr 02, 2011 at 08:08:34PM +0200, Jim Meyering wrote: >> From 0a6d128d0d17c1604245f1caafe6af73584a0bb8 Mon Sep 17 00:00:00 2001 >> From: Jim Meyering >> Date: Sat, 2 Apr 2011 19:59:30 +0200 >> Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 kernels >> >> * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38 >> and 2.6.39. Without this, part of the cp/fiemap-empty test would fail >> both on F15-to-be and rawhide. For discussion and details, see: >> http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 > > FYI, the following fix has been merged into mainline, which should fix > the problem for 2.6.39 once it is finally released, at least for ext4. > It was merged right before Linus released 2.6.39-rc1. I'm assuming > that Rawhide released a pre-2.6.39-rc1 kernel in the middle of the > merge window. So this fix is not in 2.6.38? It was committed before 2.6.38-rc6 was released, and I would have thought it appropriate for 2.6.38 :( Anyway I guess that we now have to assume that there can be 2.6.38 kernels in the wild with this issue, even if the stable branch does get the fix soon. As for 2.6.39, I guess we can assume it's OK, and ignore the rawhide aberration for a while. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 03 06:15:28 2011 Received: (at 8411) by debbugs.gnu.org; 3 Apr 2011 10:15:28 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6KLA-0003lG-3A for submit@debbugs.gnu.org; Sun, 03 Apr 2011 06:15:28 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6KL7-0003l2-BC for 8411@debbugs.gnu.org; Sun, 03 Apr 2011 06:15:26 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 0F68D601ED; Sun, 3 Apr 2011 12:15:18 +0200 (CEST) From: Jim Meyering To: Ted Ts'o Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file In-Reply-To: <20110402230005.GL21075@thunk.org> (Ted Ts'o's message of "Sat, 2 Apr 2011 19:00:05 -0400") References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> <20110402230005.GL21075@thunk.org> Date: Sun, 03 Apr 2011 12:15:17 +0200 Message-ID: <87wrjbd4ju.fsf@rho.meyering.net> Lines: 34 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org, =?iso-8859-1?Q?P=E1draig?= Brady X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) Ted Ts'o wrote: > On Sat, Apr 02, 2011 at 08:08:34PM +0200, Jim Meyering wrote: >> From 0a6d128d0d17c1604245f1caafe6af73584a0bb8 Mon Sep 17 00:00:00 2001 >> From: Jim Meyering >> Date: Sat, 2 Apr 2011 19:59:30 +0200 >> Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 kernels >> >> * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38 >> and 2.6.39. Without this, part of the cp/fiemap-empty test would fail >> both on F15-to-be and rawhide. For discussion and details, see: >> http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 > > FYI, the following fix has been merged into mainline, which should fix > the problem for 2.6.39 once it is finally released, at least for ext4. > It was merged right before Linus released 2.6.39-rc1. I'm assuming > that Rawhide released a pre-2.6.39-rc1 kernel in the middle of the > merge window. > > Some distro's will informally, but incorrectly, refer to such a > release as "2.6.39-rc0". I prefer the more technically correct > 2.6.38-git18 (which is the first git tag in the Linux git repo which > contained the patch below, as of March 25, 2011). Unfortunately, RPM > doesn't understand that 2.6.38-rc1 sorts before 2.6.38, while > 2.6.38-git17 sorts *after* 2.6.38. (Hence the incorrect, but > convenient, use of 2.6.39-rc0.) Right. Rawhide's 2.6.39-0.rc0.git11.0.fc16.x86_64 kernel is from March 22. Good. That means we needn't condemn 2.6.39. This sort of uname-based kernel check is precisely why we should minimize use of -rcN named kernels, but if it affects only rawhide (and that only briefly), I won't complain too loudly. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 03 06:27:06 2011 Received: (at 8411) by debbugs.gnu.org; 3 Apr 2011 10:27:06 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6KWQ-00044P-0f for submit@debbugs.gnu.org; Sun, 03 Apr 2011 06:27:06 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6KWM-00043t-5v for 8411@debbugs.gnu.org; Sun, 03 Apr 2011 06:27:03 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 1FA23601D7; Sun, 3 Apr 2011 12:26:56 +0200 (CEST) From: Jim Meyering To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file In-Reply-To: <4D9847F5.8030804@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?= =?iso-8859-1?Q?g?= Brady"'s message of "Sun, 03 Apr 2011 11:12:05 +0100") References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> <20110402230005.GL21075@thunk.org> <4D9847F5.8030804@draigBrady.com> Date: Sun, 03 Apr 2011 12:26:55 +0200 Message-ID: <87r59jd40g.fsf@rho.meyering.net> Lines: 77 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org, Ted Ts'o X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) P=E1draig Brady wrote: > On 03/04/11 00:00, Ted Ts'o wrote: >> On Sat, Apr 02, 2011 at 08:08:34PM +0200, Jim Meyering wrote: >>> From 0a6d128d0d17c1604245f1caafe6af73584a0bb8 Mon Sep 17 00:00:00 2001 >>> From: Jim Meyering >>> Date: Sat, 2 Apr 2011 19:59:30 +0200 >>> Subject: [PATCH] copy: require fiemap sync also for 2.6.38 and 2.6.39 k= ernels >>> >>> * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38 >>> and 2.6.39. Without this, part of the cp/fiemap-empty test would fail >>> both on F15-to-be and rawhide. For discussion and details, see: >>> http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 >> >> FYI, the following fix has been merged into mainline, which should fix >> the problem for 2.6.39 once it is finally released, at least for ext4. >> It was merged right before Linus released 2.6.39-rc1. I'm assuming >> that Rawhide released a pre-2.6.39-rc1 kernel in the middle of the >> merge window. > > So this fix is not in 2.6.38? > It was committed before 2.6.38-rc6 was released, > and I would have thought it appropriate for 2.6.38 :( > Anyway I guess that we now have to assume that there > can be 2.6.38 kernels in the wild with this issue, > even if the stable branch does get the fix soon. Yes, this is unfortunate. But it's just an optimization, so probably not a big deal for anyone. I haven't measured the performance difference. Have you? If it's a problem, I suppose once we've seen that most major distros have patched their 2.6.38, we can turn off the sync also for 2.6.38 kernels. > As for 2.6.39, I guess we can assume it's OK, > and ignore the rawhide aberration for a while. Yes. I've adjusted and pushed this: >From 1c3654cb1fb0d8f3c422c766028d0783a40f4a42 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Sat, 2 Apr 2011 19:59:30 +0200 Subject: [PATCH] copy: require fiemap sync also for 2.6.38 kernels * src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38. Without this, part of the cp/fiemap-empty test would fail both on F15-to-be (2.6.38.1-6.fc15.x86_64) and rawhide. For details, see http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190 --- src/extent-scan.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/extent-scan.c b/src/extent-scan.c index c0a5de6..d84746c 100644 --- a/src/extent-scan.c +++ b/src/extent-scan.c @@ -31,7 +31,7 @@ # include "fiemap.h" #endif -/* Work around Linux kernel issues on BTRFS and EXT4 before 2.6.38. +/* Work around Linux kernel issues on BTRFS and EXT4 before 2.6.39. FIXME: remove in 2013, or whenever we're pretty confident that the offending, unpatched kernels are no longer in use. */ static bool @@ -50,7 +50,7 @@ extent_need_sync (void) unsigned long val; if (xstrtoul (name.release + 4, NULL, 10, &val, NULL) =3D=3D LO= NGINT_OK) { - if (val < 38) + if (val < 39) need_sync =3D 1; } } -- 1.7.4.2.662.gcbd0 From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 03 06:46:26 2011 Received: (at 8411) by debbugs.gnu.org; 3 Apr 2011 10:46:26 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6Kp8-0004Vb-H8 for submit@debbugs.gnu.org; Sun, 03 Apr 2011 06:46:26 -0400 Received: from dmz-mailsec-scanner-1.mit.edu ([18.9.25.12]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Q6Kp5-0004VO-AT for 8411@debbugs.gnu.org; Sun, 03 Apr 2011 06:46:24 -0400 X-AuditID: 1209190c-b7b7aae0000047c7-77-4d984ffebfe6 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP id F6.CE.18375.EFF489D4; Sun, 3 Apr 2011 06:46:22 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id p33AkFgv030161; Sun, 3 Apr 2011 06:46:15 -0400 Received: from [10.0.42.108] (c-98-216-98-217.hsd1.ma.comcast.net [98.216.98.217]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p33AkD69019021 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sun, 3 Apr 2011 06:46:14 -0400 (EDT) Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Theodore Tso In-Reply-To: <4D9847F5.8030804@draigBrady.com> Date: Sun, 3 Apr 2011 06:46:09 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <4F38B379-26DC-4E8D-AD8C-2C0E5D3D4E86@mit.edu> References: <87fwq0gay0.fsf@rho.meyering.net> <4D9724AC.2070009@draigBrady.com> <8739m0g3u7.fsf@rho.meyering.net> <87d3l4edb1.fsf@rho.meyering.net> <20110402230005.GL21075@thunk.org> <4D9847F5.8030804@draigBrady.com> To: =?iso-8859-1?Q?P=E1draig_Brady?= X-Mailer: Apple Mail (2.1084) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprKKsWRmVeSWpSXmKPExsUixG6nrvvPf4avwbrXuhaz1nSyWFzpP8Zi MXPeHTaLdXNNHVg8Lk7ax+xxa2o7u8eeuZuZPD5vkgtgieKySUnNySxLLdK3S+DKWLjmKXvB GtaKX6tuszUw9rF0MXJySAiYSBxr6WGFsMUkLtxbzwZiCwnsY5SY8yC5i5ELyF7PKPH57RsW COcUk8TW3XeYQKqEBcIl3qz9zw5i8woYSizd1A5mMwvoSOzcegdsEpuAksSdT/vBtnEK6EnM XbMZrJdFQEXi8KQ2Roj6OIklO16zQNjaEssWvmaGmGkl8fDVRXaIi24wSrz6ZgNiiwiYS/ye uwNoPgfQ1bISTcsyJjAKzkJyxSwkV8xCMnUBI/MqRtmU3Crd3MTMnOLUZN3i5MS8vNQiXUO9 3MwSvdSU0k2MoBDnlOTZwfjmoNIhRgEORiUe3kV2032FWBPLiitzDzFKcjApifJaAyNEiC8p P6UyI7E4I76oNCe1+BCjBAezkghvFgdQjjclsbIqtSgfJiXNwaIkzjtDUt1XSCA9sSQ1OzW1 ILUIJivDwaEkwesCMlSwKDU9tSItM6cEIc3EwQkynAdo+FQ/kOHFBYm5xZnpEPlTjLocv29P 3McoxJKXn5cqJc7rBzJIAKQoozQPbg4sNb1iFAd6S5jXBqSKB5jW4Ca9AlrCBLTE234ayJKS RISUVANjKPfrdRfst7QJXXuUcjv18LobAYc3v+RPzc3mVQtNnVhgskntlJa9Z+YbEZMmecPK pe1fTLf2OKZtqZ2x31ldkfnOVy6JyUwRUe2RuWuN1r7LajzkpXL6/xJx31jxotjqwrdzgjqe z0xofW64tkbjXmp42MyFeh0LZBqPBW7KEPnprzOtXFqJpTgj0VCLuag4EQBbgUTWKAMAAA== X-Spam-Score: -2.6 (--) X-Debbugs-Envelope-To: 8411 Cc: 8411@debbugs.gnu.org, linux-ext4@vger.kernel.org, Jim Meyering X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On Apr 3, 2011, at 6:12 AM, P=E1draig Brady wrote: > So this fix is not in 2.6.38? > It was committed before 2.6.38-rc6 was released, > and I would have thought it appropriate for 2.6.38 :( > Anyway I guess that we now have to assume that there > can be 2.6.38 kernels in the wild with this issue, > even if the stable branch does get the fix soon. Yeah, the patch was complicated enough that I was concerned about = pushing it that late in the 2.6.38 development cycle. When I was told = that this feature was in a pre-release shellutils, I made the call not = to try to rush things.... We can get it into a 2.6.38 stable release = fairly quickly. -- Ted From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 15 09:29:21 2018 Received: (at control) by debbugs.gnu.org; 15 Oct 2018 13:29:21 +0000 Received: from localhost ([127.0.0.1]:49901 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gC2vx-000810-Ci for submit@debbugs.gnu.org; Mon, 15 Oct 2018 09:29:21 -0400 Received: from mail-pg1-f170.google.com ([209.85.215.170]:39366) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gC2vv-00080n-Lm for control@debbugs.gnu.org; Mon, 15 Oct 2018 09:29:19 -0400 Received: by mail-pg1-f170.google.com with SMTP id r9-v6so9173470pgv.6 for ; Mon, 15 Oct 2018 06:29:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:message-id:date:user-agent:mime-version:content-language :content-transfer-encoding; bh=ERMZcaPPvl6soqkEEL7KQa21ZvlQzs6QOlHEU55/gxw=; b=K+RUVMTZ7h3cw2YC6WzQmCQGSy9wE/4SBmrn5wZzyXJrVr+13Cjpx4AzGmYoGeedEq tsMAcG40KD7yf7ozbNbZBi7wmYvEQEFWKqOqxAvJwGOl4SJBP1XWfrQqJWGYdIGaTKn9 QS2xKPmjm4GnSG3OOylvl/zSnva0IE0TL9dYNvkrggSREPhxka7wOX3HewiwpQ8NsQnw ZUTP0n8h8aiAfASQyZEewWXXtKmIV4zcyIlb3sGztkU+HTKknv4tnY9H0MMLuaVaU5o+ H3CSCsiZklgBL9Ac4jmQ5ybwU9nwc0ntt9A4vCYt5fK1a3TAMf5ZyNcYH7f/XvB+m/nY JPhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=ERMZcaPPvl6soqkEEL7KQa21ZvlQzs6QOlHEU55/gxw=; b=K280B9D2nn2EsmD6pHqOzEK2rLxaIfBDjEOA5Ng2IMk5hYsm6/0nOQTDIR/foE6R7E APqXuwqDn8Tc7c5uWMHlNM+vN/itShP7QJZORw8IGeqWnjVujWJauDUlOmXnOPe8V1Xp NRwdBftrnvOuWMsbfeslxq3fRKXhuiLUuLxrJzY18keEMAf0sc5y1HPum9RKRuZw8HTC KMwuIHX2rqzn6pO8uhf9jwKWGJXXfODJj6X/77xsBW037IWb97E6tWE4SOoI3ghtgs5h eD9mcl1pncOczO1vCKTWPSM+l2decWnpufRh65jjzGBWeYyknaA6+zvX4pg5h9WOmKh9 75oA== X-Gm-Message-State: ABuFfojCLYh/sjBDkd5Ygi2WZ0cV2LU8Ss7in1Cxv04izCCJOKEcT3NE 7cTobABI3AiQlwaLlp9nrOmn3Bai X-Google-Smtp-Source: ACcGV62+dwUg/XQyncwWernuFSG5hoasFwbso9sqTqboyy1BM5FpIbQbjF0nODpG8UsoEAfIGWHFrw== X-Received: by 2002:a63:a012:: with SMTP id r18-v6mr16371478pge.282.1539610153116; Mon, 15 Oct 2018 06:29:13 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id o24-v6sm20963382pfa.90.2018.10.15.06.29.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 15 Oct 2018 06:29:11 -0700 (PDT) To: control@debbugs.gnu.org From: Assaf Gordon Message-ID: Date: Mon, 15 Oct 2018 07:29:10 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: severity 8103 wishlist tags 8103 easy close 8271 tags 8411 fixed close 8411 [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [209.85.215.170 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.215.170 listed in wl.mailspike.net] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (assafgordon[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) severity 8103 wishlist tags 8103 easy close 8271 tags 8411 fixed close 8411 tags 8527 fixed close 8527 From unknown Sun Jun 22 22:42:53 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 13 Nov 2018 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator