Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS

From: George Valkov <[email-masked]>
Date: Mon, 20 Feb 2023 12:21:27 +0200

> On 2023-02-19, at 8:08 AM, Paul Eggert <[email-masked]> wrote:
> 
> George, given what you've written I suppose we should give up the idea of copying sparse files efficiently on macOS (and on FreeBSD 13.0-RELEASE, as it has a similar bug with SEEK_HOLE and SEEK_DATA), in cases where fclonefileat does not work.

I agree, Paul. I just ran some benchmarks: the time it takes to call msync unconditionally hurts the performance: 502 us if the memory view is already synced seems fine, however 28177-36585 us whenever a sync is required is more than the time it takes to performing a full-copy 7042-21528 us.

> Please try the attached Gnulib patch; it uses your idea of disabling SEEK_HOLE and SEEK_DATA on macOS, except on steroids because it disables these two macros everywhere, for all Gnulib-using applications, and it also disables them on FreeBSD < 14. For coreutils you need only this patch's change to lib/unistd.in.h. If this works for you I suppose we can install it into Gnulib and propagate that into coreutils etc.

What is the correct way to apply your patch? I tried
git apply 0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch 
error: patch failed: ChangeLog:1
error: ChangeLog: patch does not apply
error: doc/posix-functions/lseek.texi: No such file or directory
error: lib/unistd.in.h: wrong type

Then I made the changes to lib/unistd.in.h manually, and ran the tests:
* before: corrupted
* after: fixed

Regarding FreeBSD, I have a FreeBSD 13.1-RELEASE VM running on Hyper-V, so I compiled and ran my samples: m.c, d.c: the copy was correct, however d.c reports that there are no segments to skip, which means that the file created by m.c using mmap is not sparse even though it should contain one blank segment and be sparse. Next I installed a fresh copy of FreeBSD 13.0-RELEASE and observed the same behaviour. Both installations use ZFS and the tests were ran as root.
If you know what conditions are required to reproduce the issue on FreeBSD, let me know and I will test it for you. Otherwise I am not sure if we need to make any changes to FreeBSD, since I did not observe anything wrong there.
Is it ok if I continue testing on FreeBSD 13.1, because the 13.0 I just installed lacks any configuration and is quite uncomfortable to use. It also rejects my password over SSH, so I’m forced to use the console. I can test pretty much anything that runs on Hyper-V.

> Also, you reported the bug against macOS 12.6. Can you check whether the same bug occurs on macOS 13? If it's still there I suppose the attached patch will need to be updated since it guesses the bug is fixed in macOS 13.

Yes it does. I just ran my samples under macOS Internet recovery, which reports as Ventura with kernel compiled on the 30th of January 2023. See the attached log:

> I'll cc this email to bug-gnulib to give them a heads-up. (Gnulib readers can see <https://bugs.gnu.org/61386> for context.)

Thanks, I’ll check it.

> Really, Apple should fix this serious data corruption bug in APFS and macOS.
> <0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch>

I agree, I opened a new ticket with Apple on 2023-02-16. Both via e-mail, and https://security.apple.com/reports/OE1928608366012

This was at the same time I reported about mmap and msync here. I got this replay on the following day:
> Thanks for the additional information. We are reviewing it and will follow up if we have any questions or need additional details.

Feel free to write them at [email-masked] and include this line
OE0928602070811 - please include this ID in replies to this thread.

Introduce your self as part of GNU/coreutils and let them know it has been 18 months since the issue was reported privately to them, so we’ve gone public now. CC me <same as gmail> at abv dot bg.

Georgi Valkov
httpstorm.com
nano RTOS