Subject: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS

From: George Valkov <[email-masked]>
Date: Sun, 12 Feb 2023 02:38:39 +0200

> On 2023-02-11, at 11:53 PM, Paul Eggert <[email-masked]> wrote:
> 
> On 2023-02-10 17:21, George Valkov wrote:
> 
>> remember to trace all code paths errors.
> 
> Yes, I've been doing that. However, I don't use macOS and the two of us are debugging remotely where I write the code and you debug it but neither of us know how macOS works. Of course it would be much more efficient if we weren't operating with these obstacles, but here we are.

Yes, it’s true. It takes a lot of energy from both of us. Especially the time to document observations in hope to be helpful. There is a saying that two people working together is more than each on their own.
Let’s work together! I have a good experience with mutiplatform development: macOS, Linux, BSD, Windows. For the most part it’s the same as Linux with minor differences, so pretty easy to support according to my experience. Build environment and debugging tools are all set. That will keep you comfortable and productive. Just drop me a private mail and we can set a conference call. Then test for standard compliance according to manpage.

> Because the macOS behavior is poorly documented and we lack the source, if we can't figure this out fairly quickly coreutils should simply stop using fclonefileat because it's unreliable for users and a time sink for us. I'm hoping, though, we can get something working with another round or two.

Sad but true. This might help:
https://github.com/apple/darwin-xnu/blob/main/bsd/sys/clonefile.h
The kernel should be open source. We might be able to find the implementation there.
I hope we can keep fclonefileat.

>> With CLONE_ACL, I get 22 Invalid argument.
> 
> OK, this means that although Apple has documented CLONE_ACL and it's in your include files, it doesn't work in your version of macOS, because either it's not supported by the kernel, or by the filesystem code, or by the particular filesystem instance, or for some other reason. EINVAL hints that it's the kernel but that's by no means certain.
> 
> One possible way to defend against this macOS misbehavior is for cp to try to use CLONE_ACL, and if that fails with errno == EINVAL try again without CLONE_ACL. I attempted to implement this in the attached patch; please give it a try.

Good approach. Tour patch confirms it works. copy.c says it has been introduced in 12.6, I’m using 12.6.3. That suggests it is available in headers, but not available in the kernel yet. I will check if it is implemented in the recovery environment for macOS 13.

I added some code for tracing. The check works as intended.

int s = fclonefileat (source_desc, dst_dirfd, dst_relname, fc_flags);
int a = s ? errno : 0;
if (s != 0 && (fc_flags & CLONE_ACL) != 0 && errno == EINVAL)
  {
    printf("s %2i  errno %2i %s  fc_flags %x !!\n", s, a, strerror(a), fc_flags);
    fc_flags &= ~CLONE_ACL;
    s = fclonefileat (source_desc, dst_dirfd, dst_relname, fc_flags);
    a = s ? errno : 0;
  }
  printf("s %2i  errno %2i %s  fc_flags %x\n", s, a, strerror(a), fc_flags);

./coreutils-clone/src/cp A B
s  0  errno  0 Success  fc_flags 2

./coreutils-clone/src/cp -p A C
s -1  errno 22 Invalid argument  fc_flags 4 !!
s  0  errno  0 Success  fc_flags 0

Extended attributes are preserved in both copies: note the @ flag.
-rw-------@    1 g     staff       415 Feb 11 02:10 A
-rw-------@    1 g     staff       415 Feb 12 01:30 B
-rw-------@    1 g     staff       415 Feb 11 02:10 C

Good luck, Paul!

Georgi Valkov
httpstorm.com
nano RTOS