Package: coreutils;
Reported by: David Schleimer <dschleimer <at> fb.com>
Date: Tue, 3 Jun 2014 15:35:02 UTC
Severity: normal
To reply to this bug, email your comments to 17676 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-coreutils <at> gnu.org:bug#17676; Package coreutils.
(Tue, 03 Jun 2014 15:35:02 GMT) Full text and rfc822 format available.David Schleimer <dschleimer <at> fb.com>:bug-coreutils <at> gnu.org.
(Tue, 03 Jun 2014 15:35:03 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: David Schleimer <dschleimer <at> fb.com> To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org> Subject: btrfs subvolumes and bind-mounts make df report incorrect and/or extra results Date: Tue, 3 Jun 2014 15:21:15 +0000
[Message part 1 (text/plain, inline)]
When you have btrfs mount where there is a both a subvolume of that mount, and the source of a bind-mount under that mount, df will report confusing results. It will show results for the bind-mount instead of the main btrfs mount when you pass the subvoulme on the command line. It may (depending on the system) show both the bind-mount and main mount in bare df results.
I've attached a script with the minimal repro instructions. It needs to be run as root since it (temporarily) mounts filesystems under your tmpdir.
I tested on two machines:
Fedora 20 running kernel 3.14.4-200.fc20.x86_64, against both the system df 8.21 and df 8.22 built from source.
Centos 6.4 running a 3.10.39 variant, with significant backports, notably including most btrfs changes from mainline. Testing against the system df 8.4, and df 8.22 again built from source.
On the fedora machine, /etc/mtab is a symlink to /proc/self/mounts. We see output that refers to the correct block device in all cases. However, we see a report for the bind-mount destination instead of the main btrfs mount when asking specifically for the subvolume path, and see results for both the main btrfs filesystem and the bind-mount when we run a bare df.
Transcript of Fedora output:
[root <at> caitsidhe dschleimer]# ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.00963354 s, 1.1 GB/s SMALL VOLUME: forcing mixed metadata/data groups
WARNING! - Btrfs v3.12 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using
Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block
nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume'
Begin unexpected output
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 10240 36 6112 1% /tmp/tmp.biUryZcwD9/binddest
Expected report for the main btrfs mount, not the bind-mount
/dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.biUryZcwD9/filesystem
/dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.biUryZcwD9/binddest
Expected only one btrfs filesystem
/home/dschleimer
[root <at> caitsidhe dschleimer]# PATH=/home/dschleimer/Downloads/coreutils-8.22/src/:$PATH ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.012752 s, 822 MB/s SMALL VOLUME: forcing mixed metadata/data groups
WARNING! - Btrfs v3.12 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using
Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block
nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume'
Begin unexpected output
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/binddest
Expected report for the main btrfs mount, not the bind-mount
/dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/filesystem
/dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/binddest
Expected only one btrfs filesystem
/home/dschleimer
On Centos, /etc/mtab is a plain file. When asking specifically for the subvolume, we see a report for the bind-mount dest which lists the bind-mount source as the filesystem. When we run a bare df, we see only one record with the file backing the loopback device as the filesystem, but the bind-mount as the mount-point.
Centos transcript:
[08:03:09 Tue Jun 03 2014] root <at> devbig100.prn2 /home/dschleimer dschleimer 282 $ ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.0101568 s, 1.0 GB/s
WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using
SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block
nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
Begin unexpected output
Filesystem 1K-blocks Used Available Use% Mounted on
/tmp/tmp.V4FNWcBXke/filesystem/subvolume/bindsource
10240 36 6112 1% /tmp/tmp.V4FNWcBXke/binddest
Expected report for the main btrfs mount, not the bind-mount
df: `/mnt/gvfs': Function not implemented /tmp/tmp.V4FNWcBXke/block
btrfs 10240 36 6112 1% /tmp/tmp.V4FNWcBXke/filesystem
Expected only one btrfs filesystem
/home/dschleimer
[08:09:39 Tue Jun 03 2014] root <at> devbig100.prn2 /home/dschleimer dschleimer 282 $ PATH=/data/users/dschleimer/coreutils-8.22/src/:$PATH ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.0116229 s, 902 MB/s
WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using
SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block
nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
Begin unexpected output
Filesystem 1K-blocks Used Available Use% Mounted on
/tmp/tmp.o7w6NcN178/filesystem/subvolume/bindsource 10240 36 6112 1% /tmp/tmp.o7w6NcN178/binddest
Expected report for the main btrfs mount, not the bind-mount
/tmp/tmp.o7w6NcN178/block btrfs 10240 36 6112 1% /tmp/tmp.o7w6NcN178/filesystem
Expected only one btrfs filesystem
/home/dschleimer
[repro_instructions.sh (application/octet-stream, attachment)]
bug-coreutils <at> gnu.org:bug#17676; Package coreutils.
(Wed, 04 Jun 2014 01:53:02 GMT) Full text and rfc822 format available.Message #8 received at 17676 <at> debbugs.gnu.org (full text, mbox):
From: Pádraig Brady <P <at> draigBrady.com> To: David Schleimer <dschleimer <at> fb.com> Cc: 17676 <at> debbugs.gnu.org Subject: Re: bug#17676: btrfs subvolumes and bind-mounts make df report incorrect and/or extra results Date: Wed, 04 Jun 2014 02:51:43 +0100
On 06/03/2014 04:21 PM, David Schleimer wrote: > When you have btrfs mount where there is a both a subvolume of that mount, and the source of a bind-mount under that mount, df will report confusing results. It will show results for the bind-mount instead of the main btrfs mount when you pass the subvoulme on the command line. It may (depending on the system) show both the bind-mount and main mount in bare df results. > > I've attached a script with the minimal repro instructions. It needs to be run as root since it (temporarily) mounts filesystems under your tmpdir. > > I tested on two machines: > Fedora 20 running kernel 3.14.4-200.fc20.x86_64, against both the system df 8.21 and df 8.22 built from source. > > Centos 6.4 running a 3.10.39 variant, with significant backports, notably including most btrfs changes from mainline. Testing against the system df 8.4, and df 8.22 again built from source. > > On the fedora machine, /etc/mtab is a symlink to /proc/self/mounts. We see output that refers to the correct block device in all cases. However, we see a report for the bind-mount destination instead of the main btrfs mount when asking specifically for the subvolume path, and see results for both the main btrfs filesystem and the bind-mount when we run a bare df. > > Transcript of Fedora output: > > [root <at> caitsidhe dschleimer]# ./repro_instructions.sh > 1024+0 records in > 1024+0 records out > 10485760 bytes (10 MB) copied, 0.00963354 s, 1.1 GB/s SMALL VOLUME: forcing mixed metadata/data groups > > WARNING! - Btrfs v3.12 IS EXPERIMENTAL > WARNING! - see http://btrfs.wiki.kernel.org before using > > Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block > nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume' > > > > Begin unexpected output > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/loop0 10240 36 6112 1% /tmp/tmp.biUryZcwD9/binddest > Expected report for the main btrfs mount, not the bind-mount > /dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.biUryZcwD9/filesystem > /dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.biUryZcwD9/binddest > Expected only one btrfs filesystem > /home/dschleimer > [root <at> caitsidhe dschleimer]# PATH=/home/dschleimer/Downloads/coreutils-8.22/src/:$PATH ./repro_instructions.sh > 1024+0 records in > 1024+0 records out > 10485760 bytes (10 MB) copied, 0.012752 s, 822 MB/s SMALL VOLUME: forcing mixed metadata/data groups > > WARNING! - Btrfs v3.12 IS EXPERIMENTAL > WARNING! - see http://btrfs.wiki.kernel.org before using > > Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block > nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume' > > > > Begin unexpected output > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/loop0 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/binddest > Expected report for the main btrfs mount, not the bind-mount > /dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/filesystem > /dev/loop0 btrfs 10240 36 6112 1% /tmp/tmp.TEBR2D9Vta/binddest > Expected only one btrfs filesystem > /home/dschleimer > > On Centos, /etc/mtab is a plain file. When asking specifically for the subvolume, we see a report for the bind-mount dest which lists the bind-mount source as the filesystem. When we run a bare df, we see only one record with the file backing the loopback device as the filesystem, but the bind-mount as the mount-point. > > Centos transcript: > > [08:03:09 Tue Jun 03 2014] root <at> devbig100.prn2 /home/dschleimer dschleimer 282 $ ./repro_instructions.sh > 1024+0 records in > 1024+0 records out > 10485760 bytes (10 MB) copied, 0.0101568 s, 1.0 GB/s > > WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using > > SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block > nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume' > > > > Begin unexpected output > Filesystem 1K-blocks Used Available Use% Mounted on > /tmp/tmp.V4FNWcBXke/filesystem/subvolume/bindsource > 10240 36 6112 1% /tmp/tmp.V4FNWcBXke/binddest > Expected report for the main btrfs mount, not the bind-mount > df: `/mnt/gvfs': Function not implemented /tmp/tmp.V4FNWcBXke/block > btrfs 10240 36 6112 1% /tmp/tmp.V4FNWcBXke/filesystem > Expected only one btrfs filesystem > /home/dschleimer > > [08:09:39 Tue Jun 03 2014] root <at> devbig100.prn2 /home/dschleimer dschleimer 282 $ PATH=/data/users/dschleimer/coreutils-8.22/src/:$PATH ./repro_instructions.sh > 1024+0 records in > 1024+0 records out > 10485760 bytes (10 MB) copied, 0.0116229 s, 902 MB/s > > WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using > > SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block > nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume' > > > > Begin unexpected output > Filesystem 1K-blocks Used Available Use% Mounted on > /tmp/tmp.o7w6NcN178/filesystem/subvolume/bindsource 10240 36 6112 1% /tmp/tmp.o7w6NcN178/binddest > Expected report for the main btrfs mount, not the bind-mount > /tmp/tmp.o7w6NcN178/block btrfs 10240 36 6112 1% /tmp/tmp.o7w6NcN178/filesystem > Expected only one btrfs filesystem > /home/dschleimer So the crux of the confusion is that the subvolume has a different device ID. I.E. a separate virtual file system within the main file system, that shares storage but are otherwise treated like separate file systems for hard links etc. Here are the device IDs for your test setup: # find -printf "%D %p\n" 34 . 63 ./binddest 62 ./filesystem 63 ./filesystem/subvolume 63 ./filesystem/subvolume/directory 63 ./filesystem/subvolume/bindsource 34 ./block When you expose that virtual file system to df with a bind mount it will be identified separately as you've shown. BTW it's not specific to bind mounts, as you can see the same thing by mounting the subvolume separately like: mkdir submntdest mount -o subvolid=256 -o loop block submntdest It could be argued that since there are explicit mounts to this subvolume that one (the shortest) of these is the most appropriate to display? If you don't mount separately, then df notices that ./filesystem/subvolume is a "separate mount" due to the different device id, and will display that as the mount dir and "-" as the device which isn't ideal either. It's more problematic that the btrfs file system is listed multiple times, especially with --total which will be incorrect. We might be able to avoid this problem by taking the first entry for duplicate device names, when those device names exist. However I'm worried that there are cases where separate file systems with separate storage (distinguished by mount options) have the same device listed. Maybe we could suppress these duplicates by scanning /proc/self/mountinfo and discarding entries that didn't have '/' in the fourth column, though that's awkward and I'm not sure how general it is either. thanks, Pádraig.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.