Subject: Re: bug#20570: Test dd/sparse.sh fail with btrfs

From: Pádraig Brady <[email-masked]>
Date: Thu, 14 May 2015 11:48:30 +0100

On 14/05/15 03:53, Sébastien Luttringer wrote:
> Hello,
> 
> I experience one test suite failure when I tried to rebuild the
> coreutils v8.23 package. My system is an archlinux host with a btrfs
> filesystem.
> 
> The test doesn't fail when I run it in a tmpfs. I'm pretty sure it used
> to works over btrfs in july 2014[1] when I built the previous package
> for archlinux.
> 
> The failing section is the following:
> 
>   # Ensure NUL blocks smaller than the block size are not made sparse.
>   # Here, with a 2MiB block size, dd's conv=sparse must *not*
> introduce a hole. 
>   dd if=file.in of=file.out bs=2M conv=sparse
>   test 2500 -lt $(kb_alloc file.out) || fail=1
> 
> You can see the full shell output here: http://pastebin.com/XADb1BKj
> 
> Regards,
> 
> [1] https://www.archlinux.org/packages/core/x86_64/coreutils/

It seems the allocation is async.
I also see this intermittently with BTRFS on 4.0.0-0.rc5.git4.1.fc22.x86_64
I've been unable to reproduce with the attached
which syncs the file before checking the allocation.

thanks!
Pádraig.
From 77e092c85ed9ae2221042323cf87f4fc8c847e75 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email-masked]>
Date: Thu, 14 May 2015 11:43:54 +0100
Subject: [PATCH] tests: fix async allocation race on BTRFS

* tests/dd/sparse.sh: Sync files before checking allocations,
which may be done asynchronously on NFS and BTRFS at least.
Also mark this test as very expensive on remote file systems.
* tests/tail-2/inotify-rotate-resources.sh: Use the more
standard is_local_dir_() to check remoteness.
* tests/cp/fiemap-empty.sh: Comment on the sync issue
for this currerntly unused test.
Fixes http://bugs.gnu.org/20570
---
 tests/cp/fiemap-empty.sh                 | 1 +
 tests/dd/sparse.sh                       | 9 ++++++++-
 tests/tail-2/inotify-rotate-resources.sh | 2 +-
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tests/cp/fiemap-empty.sh b/tests/cp/fiemap-empty.sh
index a5eaac5..b3b2cd7 100755
--- a/tests/cp/fiemap-empty.sh
+++ b/tests/cp/fiemap-empty.sh
@@ -22,6 +22,7 @@ print_ver_ cp
 # FIXME: enable any part of this test that is still relevant,
 # or, if none are relevant (now that cp does not handle unwritten
 # extents), just remove the test altogether.
+# Note also if checking allocations may need to sync first on BTRFS at least
 skip_ 'disabled for now'
 
 touch fiemap_chk
diff --git a/tests/dd/sparse.sh b/tests/dd/sparse.sh
index 4fdabad..fde5b32 100755
--- a/tests/dd/sparse.sh
+++ b/tests/dd/sparse.sh
@@ -17,6 +17,7 @@
 
 . "${srcdir=.}/tests/init.sh"; path_prepend_ ./src
 print_ver_ dd
+is_local_dir_ . || very_expensive_
 require_sparse_support_
 
 # Ensure basic sparse generation works
@@ -50,6 +51,9 @@ dd if=/dev/zero    of=file.in bs=1M count=1 seek=1 conv=notrunc || fail=1
 
 kb_alloc() { du -k "$1"|cut -f1; }
 
+# sync out data for async allocators like NFS/BTRFS
+# sync file.in || fail=1
+
 # If our just-created input file appears to be too small,
 # skip the remaining tests.  On at least Solaris 10 with NFS,
 # file.in is reported to occupy <= 1KiB for about 50 seconds
@@ -58,7 +62,10 @@ if test $(kb_alloc file.in) -gt 3000; then
 
   # Ensure NUL blocks smaller than the block size are not made sparse.
   # Here, with a 2MiB block size, dd's conv=sparse must *not* introduce a hole.
-  dd if=file.in of=file.out bs=2M conv=sparse
+  dd if=file.in of=file.out bs=2M conv=sparse || fail=1
+
+  # Intermittently BTRFS returns 0 allocation for file.out unless synced
+  sync file.out || fail=1
   test 2500 -lt $(kb_alloc file.out) || fail=1
 
   # Note we recreate a sparse file first to avoid
diff --git a/tests/tail-2/inotify-rotate-resources.sh b/tests/tail-2/inotify-rotate-resources.sh
index c13959e..d6629a8 100755
--- a/tests/tail-2/inotify-rotate-resources.sh
+++ b/tests/tail-2/inotify-rotate-resources.sh
@@ -25,7 +25,7 @@ grep '^#define HAVE_INOTIFY 1' "$CONFIG_HEADER" >/dev/null \
 require_strace_ 'inotify_add_watch,inotify_rm_watch'
 
 # Quickly skip on remote file systems
-df --local . >/dev/null 2>&1 ||
+is_local_dir . >/dev/null 2>&1 ||
   skip_ 'inotify not used on remote file system'
 
 check_tail_output()
-- 
2.3.4