GNU bug report logs - #78623
cp --preserve=xattr copies attr as well as xattr which breaks cross-FS copy

Previous Next

To reply to this bug, email your comments to 78623 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: cp --preserve=xattr copies attr as well as xattr which breaks
 cross-FS copy
Date: Wed, 28 May 2025 15:14:31 +0000

Dear list,

We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.

I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.

Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).

Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?

Thanks
Gaël



#!/bin/bash

workspace="/tmp/cp_attr"

echo "WORKDIR: ${workspace}"
echo
echo 'SETTING UP WORKSPACE'
mkdir -p ${workspace}/{xfs,btrfs}
truncate -s 512M ${workspace}/xfs.img
truncate -s 512M ${workspace}/btrfs.img

mkfs.xfs -q ${workspace}/xfs.img
mkfs.btrfs -q ${workspace}/btrfs.img

echo 'BECOMING ROOT'
sudo sh -c "
    cd ${workspace}
    echo '  - mounting filesystems'
    mount -o loop xfs.img xfs
    mount -o loop btrfs.img btrfs
    echo '  - creating files in BTRFS'
    touch btrfs/{foo,bar,baz}

    echo '  - original attr state:'
    lsattr btrfs/

    echo '  - original xattr state:'
    getfattr btrfs/*

    echo

    echo 'CHANGING ATTRIBUTES'
    echo '  - unchanged: btrfs/foo'
    echo '  - set xattr: btrfs/bar | setfattr -n user.foobar -v BAR'
    setfattr -n user.foobar -v BAR btrfs/bar

    echo '  - set  attr: btrfs/baz | chattr +c btrfs/baz'
    chattr +c btrfs/baz

    echo '  - current attr state:'
    lsattr btrfs/

    echo '  - current xattr state:'
    getfattr btrfs/*

    echo

    echo 'COPIES'
    # Works:
    echo '  - copying btrfs/foo -> xfs/foo'
    cp --preserve=xattr btrfs/foo xfs/ && echo 'Success!' || echo 'Failed!'
    
    # Works:
    echo '  - copying btrfs/bar -> xfs/bar'
    cp --preserve=xattr btrfs/bar xfs/ && echo 'Success!' || echo 'Failed!'

    # PROBLEM IS RIGHT HERE:
    echo '  - copying btrfs/baz -> xfs/baz'
    cp --preserve=xattr btrfs/baz xfs/ && echo 'Success!' || echo 'Failed!'


    umount btrfs
    umount xfs
"

rm ${workspace}/{xfs,btrfs}.img
rmdir ${workspace}/{xfs,btrfs}

Message #8 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Gael Donval <gael.donval <at> manchester.ac.uk>, 78623 <at> debbugs.gnu.org
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Thu, 29 May 2025 10:10:48 +0100

On 28/05/2025 16:14, Gael Donval wrote:
> Dear list,
> 
> We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
> 
> I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
> 
> Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
> 
> Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?

I've not looked in detail at your case,
but it's worth noting that /etc/xattr.conf
gives the facility to skip copying certain xattrs.
Does that suffice to handle your issue?

thanks,
Pádraig

Message #11 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Thu, 29 May 2025 09:56:07 +0000

On Thu, 2025-05-29 at 10:10 +0100, Pádraig Brady wrote:
> On 28/05/2025 16:14, Gael Donval wrote:
> > Dear list,
> > 
> > We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
> > 
> > I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
> > 
> > Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
> > 
> > Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?
> 
> I've not looked in detail at your case,
> but it's worth noting that /etc/xattr.conf
> gives the facility to skip copying certain xattrs.
> Does that suffice to handle your issue?

As far as I can tell, xattr.conf cannot be used to ignore normal file
attributes (as opposed to extended attributes), can it? We need a way
to copy extended attributes without copying normal file attributes.

Thanks,
Gaël

> 
> thanks,
> Pádraig

Message #14 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Gael Donval <gael.donval <at> manchester.ac.uk>,
 "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Thu, 29 May 2025 12:21:35 +0100

On 29/05/2025 10:56, Gael Donval wrote:
> On Thu, 2025-05-29 at 10:10 +0100, Pádraig Brady wrote:
>> On 28/05/2025 16:14, Gael Donval wrote:
>>> Dear list,
>>>
>>> We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
>>>
>>> I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
>>>
>>> Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
>>>
>>> Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?
>>
>> I've not looked in detail at your case,
>> but it's worth noting that /etc/xattr.conf
>> gives the facility to skip copying certain xattrs.
>> Does that suffice to handle your issue?
> 
> As far as I can tell, xattr.conf cannot be used to ignore normal file
> attributes (as opposed to extended attributes), can it? We need a way
> to copy extended attributes without copying normal file attributes.

Well there is no code in coreutils to copy "chattr" attributes.
It was discussed previously, but we decided not to support those
due to the incompat issues you're hitting.

That version of cp are you using?

strace may be instructive as to that is reading/writing these chattrs
(this is done with ioctl FS_IOC_GETFLAGS IIRC).

cheers,
Pádraig

Message #17 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Thu, 29 May 2025 11:53:18 +0000

> On 29/05/2025 10:56, Gael Donval wrote:
> > On Thu, 2025-05-29 at 10:10 +0100, Pádraig Brady wrote:
> > > On 28/05/2025 16:14, Gael Donval wrote:
> > > > Dear list,
> > > > 
> > > > We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
> > > > 
> > > > I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
> > > > 
> > > > Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
> > > > 
> > > > Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?
> > > 
> > > I've not looked in detail at your case,
> > > but it's worth noting that /etc/xattr.conf
> > > gives the facility to skip copying certain xattrs.
> > > Does that suffice to handle your issue?
> > 
> > As far as I can tell, xattr.conf cannot be used to ignore normal file
> > attributes (as opposed to extended attributes), can it? We need a way
> > to copy extended attributes without copying normal file attributes.
> 
> Well there is no code in coreutils to copy "chattr" attributes.
> It was discussed previously, but we decided not to support those
> due to the incompat issues you're hitting.

It does copy them though. If it's not the intent of `preserve=xattr`,
I'm afraid it's a bug. My reproducer makes it obvious that it tries to
copy them. If you change `mkfs.xfs` to `mkfs.btrfs`, it will all
succeed and you'll see chattr attributes are copied. If your remove `--
preserver=xattr`, you'll see chattr attributes are not copied anymore. 
> 
> That version of cp are you using?

cp (GNU coreutils) 9.7

> 
> strace may be instructive as to that is reading/writing these chattrs
> (this is done with ioctl FS_IOC_GETFLAGS IIRC).

I might be wrong but it looks like everything xattr-related is deferred
to libattr. 
I'll investigate that tomorrow!

Gaël

> 
> cheers,
> Pádraig

Message #20 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Fri, 30 May 2025 09:02:00 +0000

On Thu, 2025-05-29 at 12:53 +0100, Gaël Donval wrote:
> > On 29/05/2025 10:56, Gael Donval wrote:
> > > On Thu, 2025-05-29 at 10:10 +0100, Pádraig Brady wrote:
> > > > On 28/05/2025 16:14, Gael Donval wrote:
> > > > > Dear list,
> > > > > 
> > > > > We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
> > > > > 
> > > > > I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
> > > > > 
> > > > > Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
> > > > > 
> > > > > Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?
> > > > 
> > > > I've not looked in detail at your case,
> > > > but it's worth noting that /etc/xattr.conf
> > > > gives the facility to skip copying certain xattrs.
> > > > Does that suffice to handle your issue?
> > > 
> > > As far as I can tell, xattr.conf cannot be used to ignore normal file
> > > attributes (as opposed to extended attributes), can it? We need a way
> > > to copy extended attributes without copying normal file attributes.
> > 
> > Well there is no code in coreutils to copy "chattr" attributes.
> > It was discussed previously, but we decided not to support those
> > due to the incompat issues you're hitting.
> 
> It does copy them though. If it's not the intent of `preserve=xattr`,
> I'm afraid it's a bug. My reproducer makes it obvious that it tries to
> copy them. If you change `mkfs.xfs` to `mkfs.btrfs`, it will all
> succeed and you'll see chattr attributes are copied. If your remove `--
> preserver=xattr`, you'll see chattr attributes are not copied anymore. 
> > 
> > That version of cp are you using?
> 
> cp (GNU coreutils) 9.7
> 
> > 
> > strace may be instructive as to that is reading/writing these chattrs
> > (this is done with ioctl FS_IOC_GETFLAGS IIRC).
> 
> I might be wrong but it looks like everything xattr-related is deferred
> to libattr. 
> I'll investigate that tomorrow!

Ok, I've done that and you were right. I guess BTRFS implements its
file attributes as extended attributes. The relevant part of the strace
is this:

openat(AT_FDCWD, "btrfs/baz", O_RDONLY) = 4
openat(3, "baz", O_WRONLY|O_CREAT|O_EXCL, 0644) = 5
flistxattr(4, NULL, 0)                  = 18
flistxattr(4, "btrfs.compression\0", 18) = 18
openat(AT_FDCWD, "/etc/xattr.conf", O_RDONLY) = 6
read(6, "# /etc/xattr.conf\n#\n# Format:\n# "..., 4096) = 622
fgetxattr(4, "btrfs.compression", NULL, 0) = 4
fgetxattr(4, "btrfs.compression", "zlib", 4) = 4
fsetxattr(5, "btrfs.compression", "zlib", 4, 0) = -1 EOPNOTSUPP (Operation not supported)

and updating /etc/xattr.conf to ignore it does work.

I can't see any way to tweak this behaviour at command-line level yet
the behaviour can be triggered by normal users: would it make sense to
add a flag to filter xattr out at CLI level?

Cheers,
Gaël


> 
> Gaël
> 
> > 
> > cheers,
> > Pádraig

Message #23 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Fri, 30 May 2025 09:24:06 +0000

On Fri, 2025-05-30 at 10:02 +0100, Gaël Donval wrote:
> On Thu, 2025-05-29 at 12:53 +0100, Gaël Donval wrote:
> > > On 29/05/2025 10:56, Gael Donval wrote:
> > > > On Thu, 2025-05-29 at 10:10 +0100, Pádraig Brady wrote:
> > > > > On 28/05/2025 16:14, Gael Donval wrote:
> > > > > > Dear list,
> > > > > > 
> > > > > > We've unearth an odd behaviour in cp: `cp --preserve=xattr` tries to copy both attributes of the chattr kind and extended attributes of the getfattr kind with apparently no way to disable either one of them (it's all or nothing). This is problematic in tools like mkosi where non-filesystem-specific xattributes need to be preserved whilst FS-specific attributes must be discarded for cross-filesystem support.
> > > > > > 
> > > > > > I have added a MWE at the end of this email after my signature: it creates two raw partitions as files (one as XFS, one as BTRFS), mounts them in local folders and creates 3 files in the BTRFS partitions later altered before copy.
> > > > > > 
> > > > > > Referring to the script, we think there should be an option to copy files foo (no-attr), bar (setfattr) and baz (chattr), keeping the setfattr's xattr and discarding chattr's attr. Looking at the code, it seems like cp eventually defers the actual attribute copying libattr, which seems to handle both, but separately (which is what we want).
> > > > > > 
> > > > > > Would it make sense to add a separate `attr` preserve value for the chattr case and keep `xattr` for getfattr case?
> > > > > 
> > > > > I've not looked in detail at your case,
> > > > > but it's worth noting that /etc/xattr.conf
> > > > > gives the facility to skip copying certain xattrs.
> > > > > Does that suffice to handle your issue?
> > > > 
> > > > As far as I can tell, xattr.conf cannot be used to ignore normal file
> > > > attributes (as opposed to extended attributes), can it? We need a way
> > > > to copy extended attributes without copying normal file attributes.
> > > 
> > > Well there is no code in coreutils to copy "chattr" attributes.
> > > It was discussed previously, but we decided not to support those
> > > due to the incompat issues you're hitting.
> > 
> > It does copy them though. If it's not the intent of `preserve=xattr`,
> > I'm afraid it's a bug. My reproducer makes it obvious that it tries to
> > copy them. If you change `mkfs.xfs` to `mkfs.btrfs`, it will all
> > succeed and you'll see chattr attributes are copied. If your remove `--
> > preserver=xattr`, you'll see chattr attributes are not copied anymore. 
> > > 
> > > That version of cp are you using?
> > 
> > cp (GNU coreutils) 9.7
> > 
> > > 
> > > strace may be instructive as to that is reading/writing these chattrs
> > > (this is done with ioctl FS_IOC_GETFLAGS IIRC).
> > 
> > I might be wrong but it looks like everything xattr-related is deferred
> > to libattr. 
> > I'll investigate that tomorrow!
> 
> Ok, I've done that and you were right. I guess BTRFS implements its
> file attributes as extended attributes. The relevant part of the strace
> is this:
> 
> openat(AT_FDCWD, "btrfs/baz", O_RDONLY) = 4
> openat(3, "baz", O_WRONLY|O_CREAT|O_EXCL, 0644) = 5
> flistxattr(4, NULL, 0)                  = 18
> flistxattr(4, "btrfs.compression\0", 18) = 18
> openat(AT_FDCWD, "/etc/xattr.conf", O_RDONLY) = 6
> read(6, "# /etc/xattr.conf\n#\n# Format:\n# "..., 4096) = 622
> fgetxattr(4, "btrfs.compression", NULL, 0) = 4
> fgetxattr(4, "btrfs.compression", "zlib", 4) = 4
> fsetxattr(5, "btrfs.compression", "zlib", 4, 0) = -1 EOPNOTSUPP (Operation not supported)
> 
> and updating /etc/xattr.conf to ignore it does work.
> 
> I can't see any way to tweak this behaviour at command-line level yet
> the behaviour can be triggered by normal users: would it make sense to
> add a flag to filter xattr out at CLI level?

Or perhaps introduce a `preserve=xattr-safe` alternative that ignores
all but security, system, trusted and user namespaces? It won't compose
well with `all` but it's not introducing a new flag and works better
semantically.

Gaël 
> 
> Cheers,
> Gaël
> 
> 
> > 
> > Gaël
> > 
> > > 
> > > cheers,
> > > Pádraig

Message #26 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Gael Donval <gael.donval <at> manchester.ac.uk>,
 "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Fri, 30 May 2025 10:37:47 +0100

On 30/05/2025 10:02, Gael Donval wrote:
> Ok, I've done that and you were right. I guess BTRFS implements its
> file attributes as extended attributes. The relevant part of the strace
> is this:
> 
> openat(AT_FDCWD, "btrfs/baz", O_RDONLY) = 4
> openat(3, "baz", O_WRONLY|O_CREAT|O_EXCL, 0644) = 5
> flistxattr(4, NULL, 0)                  = 18
> flistxattr(4, "btrfs.compression\0", 18) = 18
> openat(AT_FDCWD, "/etc/xattr.conf", O_RDONLY) = 6
> read(6, "# /etc/xattr.conf\n#\n# Format:\n# "..., 4096) = 622
> fgetxattr(4, "btrfs.compression", NULL, 0) = 4
> fgetxattr(4, "btrfs.compression", "zlib", 4) = 4
> fsetxattr(5, "btrfs.compression", "zlib", 4, 0) = -1 EOPNOTSUPP (Operation not supported)
> 
> and updating /etc/xattr.conf to ignore it does work.
> 
> I can't see any way to tweak this behaviour at command-line level yet
> the behaviour can be triggered by normal users: would it make sense to
> add a flag to filter xattr out at CLI level?

It would get unwieldy as CLI options I think,
and you'd usually want a static list of FS specific options.

There are additional complications with a static list though,
as you would probably want to attempt to copy xattrs within
or between file systems of the same type, so unconditionally
avoiding them in a static list may not be appropriate.

Note `cp -a` will attempt to copy all xattrs
but ignore "operation not supported" errors.
So we only have this issue with --preserve=xattr which diagnoses any issues.
Perhaps we would benefit from a --preserve=supported-xattr option?

BTW there is a further complication with the "btrfs.compression" xattr
as its behavior is dependent on whether you write that xattr
before or after you write the data:
https://lists.gnu.org/archive/html/coreutils/2020-05/msg00013.html
(Though note we reflink by default now, since the above discussion
which alleviates the issue somewhat).

cheers,
Padraig

Message #29 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: Gael Donval <gael.donval <at> manchester.ac.uk>, 78623 <at> debbugs.gnu.org
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Fri, 30 May 2025 10:16:08 -0700

On 2025-05-30 02:37, Pádraig Brady wrote:
> we only have this issue with --preserve=xattr which diagnoses any 
> issues.
> Perhaps we would benefit from a --preserve=supported-xattr option?

If we go that route, it might be a bit better if the new option-arg 
began with 'xattr' rather than ended with 'xattr' so that it's easier to 
find in the doc. Perhaps something like --preserve='xattr-try'?

I'm not quite seeing the motivation, though. Why are scripts using 
--preserve=xattr rather than the much-simpler '-a'? That is, why 
preserve xattr but not other metadata?

Message #32 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Cc: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Fri, 30 May 2025 19:42:14 +0000

On Fri, 2025-05-30 at 10:16 -0700, Paul Eggert wrote:
> On 2025-05-30 02:37, Pádraig Brady wrote:

>> Note `cp -a` will attempt to copy all xattrs
>> but ignore "operation not supported" errors.

I wasn't aware of this. `cp -a --no-preserve=...` would solve the issue
but I'm pretty positive the tool is coded the way it is specifically to
catch errors. 

Mkosi is made to craft OS images: it should error out if important
xattrs are left out, just like it should error out if `--preserve=mode`
failed: you don't want to figure out your booted system is entirely
UID=0 with perm 777, do you? :) 

> > we only have this issue with --preserve=xattr which diagnoses any 
> > issues.
> > Perhaps we would benefit from a --preserve=supported-xattr option?
> 
> If we go that route, it might be a bit better if the new option-arg 
> began with 'xattr' rather than ended with 'xattr' so that it's easier to 
> find in the doc. Perhaps something like --preserve='xattr-try'?

We are looking for the diagnostics unfortunately.

At this stage, I have no doubt this issue is not a bug. But in terms of
user experience, sorting this out without having to alter a system-wide
static admin-only file would be awesome! (especially when your user is
unprivileged)

> 
> I'm not quite seeing the motivation, though. Why are scripts using 
> --preserve=xattr rather than the much-simpler '-a'? That is, why 
> preserve xattr but not other metadata?

I simplified for the MWE, mkosi actually preserves more than that:

https://github.com/systemd/mkosi/blob/main/mkosi/tree.py#L111-L117

Ideally, we would come up with a `xattr-portable` set of rules to cover
system, user, trusted and security namespaces (and skip the well-known
exceptions to them) but if xattrs are too messy for that, providing a
filtering option that normal users can use without too much gymnastics
would be a great improvement.

Best,
Gaël

Message #35 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Gael Donval <gael.donval <at> manchester.ac.uk>, 78623 <at> debbugs.gnu.org
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Wed, 4 Jun 2025 11:18:43 +0100

On 30/05/2025 18:16, Paul Eggert wrote:
> On 2025-05-30 02:37, Pádraig Brady wrote:
>> we only have this issue with --preserve=xattr which diagnoses any
>> issues.
>> Perhaps we would benefit from a --preserve=supported-xattr option?
> 
> If we go that route, it might be a bit better if the new option-arg
> began with 'xattr' rather than ended with 'xattr' so that it's easier to
> find in the doc. Perhaps something like --preserve='xattr-try'?
> 
> I'm not quite seeing the motivation, though. Why are scripts using
> --preserve=xattr rather than the much-simpler '-a'? That is, why
> preserve xattr but not other metadata?

Thinking a bit more about this I think I'll add it, because:

1. ENOTSUP is common with xattrs

2. `--preserve=all --no-preserve=mode,ownership,timestamps,links,context`
is too verbose, and not future proof if we ever add another --preserve option.

3. It's more descriptive to explicitly say that -a implies --preserve=xattr-supported

I would also like to get feedback though on why -a doesn't suffice.
Though I suppose by that argument you could ask why we need the
various --preserve options at all.

cheers,
Pádraig

Message #38 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Cc: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Wed, 4 Jun 2025 11:55:36 +0000

> On 30/05/2025 18:16, Paul Eggert wrote:
> > On 2025-05-30 02:37, Pádraig Brady wrote:
> > > we only have this issue with --preserve=xattr which diagnoses any
> > > issues.
> > > Perhaps we would benefit from a --preserve=supported-xattr option?
> > 
> > If we go that route, it might be a bit better if the new option-arg
> > began with 'xattr' rather than ended with 'xattr' so that it's easier to
> > find in the doc. Perhaps something like --preserve='xattr-try'?
> > 
> > I'm not quite seeing the motivation, though. Why are scripts using
> > --preserve=xattr rather than the much-simpler '-a'? That is, why
> > preserve xattr but not other metadata?
> 
> Thinking a bit more about this I think I'll add it, because:
> 
> 1. ENOTSUP is common with xattrs
> 
> 2. `--preserve=all --no-preserve=mode,ownership,timestamps,links,context`
> is too verbose, and not future proof if we ever add another --preserve option.
> 
> 3. It's more descriptive to explicitly say that -a implies --preserve=xattr-supported
> 
> I would also like to get feedback though on why -a doesn't suffice.
> Though I suppose by that argument you could ask why we need the
> various --preserve options at all.

I'm not sure whether you have received my previous message (#32 on the
bug tracker). Just in case: we use cp to copy full OS trees and need
the diagnostic to determine when intermediaries break xattrs.

`cp -a` offers that flexibility (good) but also silently drops
everything if any intermediary does not support xattrs (very bad). The
latter is relatively common on distributed FS but could also happen
depending on vendor/admin choices.


The current approach of setting up xattr copy rules in /etc/xattr.conf
is unwieldy: 

1. it does not work for non-root users;
2. and does not allow for any call-specific adjustment.

Setting up a container so that a normal user can setup a scoped local
/etc/xattr.conf tailored to their use, to make a specific tree copy
checking for the existence of a minimal set of supported xattrs is a
bit convoluted. 

Any alternative suggestion would be good.

Thanks,
Gaël

> 
> cheers,
> Pádraig

Message #41 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Gael Donval <gael.donval <at> manchester.ac.uk>,
 "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>
Cc: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Wed, 4 Jun 2025 14:31:44 +0100

On 04/06/2025 12:55, Gael Donval wrote:
>> On 30/05/2025 18:16, Paul Eggert wrote:
>>> On 2025-05-30 02:37, Pádraig Brady wrote:
>>>> we only have this issue with --preserve=xattr which diagnoses any
>>>> issues.
>>>> Perhaps we would benefit from a --preserve=supported-xattr option?
>>>
>>> If we go that route, it might be a bit better if the new option-arg
>>> began with 'xattr' rather than ended with 'xattr' so that it's easier to
>>> find in the doc. Perhaps something like --preserve='xattr-try'?
>>>
>>> I'm not quite seeing the motivation, though. Why are scripts using
>>> --preserve=xattr rather than the much-simpler '-a'? That is, why
>>> preserve xattr but not other metadata?
>>
>> Thinking a bit more about this I think I'll add it, because:
>>
>> 1. ENOTSUP is common with xattrs
>>
>> 2. `--preserve=all --no-preserve=mode,ownership,timestamps,links,context`
>> is too verbose, and not future proof if we ever add another --preserve option.
>>
>> 3. It's more descriptive to explicitly say that -a implies --preserve=xattr-supported
>>
>> I would also like to get feedback though on why -a doesn't suffice.
>> Though I suppose by that argument you could ask why we need the
>> various --preserve options at all.
> 
> I'm not sure whether you have received my previous message (#32 on the
> bug tracker). Just in case: we use cp to copy full OS trees and need
> the diagnostic to determine when intermediaries break xattrs.

Oh right, I missed that sorry.
I'll hold off so on --preserve=xattr-supported.

> `cp -a` offers that flexibility (good) but also silently drops
> everything if any intermediary does not support xattrs (very bad). The
> latter is relatively common on distributed FS but could also happen
> depending on vendor/admin choices.
> 
> 
> The current approach of setting up xattr copy rules in /etc/xattr.conf
> is unwieldy:
> 
> 1. it does not work for non-root users;
> 2. and does not allow for any call-specific adjustment.
> 
> Setting up a container so that a normal user can setup a scoped local
> /etc/xattr.conf tailored to their use, to make a specific tree copy
> checking for the existence of a minimal set of supported xattrs is a
> bit convoluted.
> 
> Any alternative suggestion would be good.
So you're suggesting supporting something like --xattrs-include=... and --xattrs-exclude.
Looking now, I see that tar supports those exact options.
Also rsync support xattr filtering though its --filter rules.
This does give more control, though also less general control
as one would need to config the xattrs for each particular system / call.
Also given other archiving tools support this already,
I'm not sure it's warranted in cp.

cheers,
Pádraig

Message #44 received at 78623 <at> debbugs.gnu.org (full text, mbox):

From: Gael Donval <gael.donval <at> manchester.ac.uk>
To: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "P <at> draigBrady.com"
 <P <at> draigBrady.com>
Cc: "78623 <at> debbugs.gnu.org" <78623 <at> debbugs.gnu.org>
Subject: Re: bug#78623: cp --preserve=xattr copies attr as well as xattr which
 breaks cross-FS copy
Date: Thu, 5 Jun 2025 10:05:08 +0000

> > `cp -a` offers that flexibility (good) but also silently drops
> > everything if any intermediary does not support xattrs (very bad). The
> > latter is relatively common on distributed FS but could also happen
> > depending on vendor/admin choices.
> > 
> > 
> > The current approach of setting up xattr copy rules in /etc/xattr.conf
> > is unwieldy:
> > 
> > 1. it does not work for non-root users;
> > 2. and does not allow for any call-specific adjustment.
> > 
> > Setting up a container so that a normal user can setup a scoped local
> > /etc/xattr.conf tailored to their use, to make a specific tree copy
> > checking for the existence of a minimal set of supported xattrs is a
> > bit convoluted.
> > 
> > Any alternative suggestion would be good.
> So you're suggesting supporting something like --xattrs-include=... and --xattrs-exclude.
> Looking now, I see that tar supports those exact options.
> Also rsync support xattr filtering though its --filter rules.
> This does give more control, though also less general control
> as one would need to config the xattrs for each particular system / call.
> Also given other archiving tools support this already,
> I'm not sure it's warranted in cp.

The only problem with rsync is that it doesn't support reflinks.
Otherwise it would be perfect (there is a series of patch that exist
since 2021 but no one ever merged them in main).

Gaël


> 
> cheers,
> Pádraig

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.