Subject: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS

From: George Valkov <[email-masked]>
Date: Tue, 14 Feb 2023 09:04:02 +0200

> On 2023-02-14, at 8:12 AM, Paul Eggert <[email-masked]> wrote:
> 
> On 2023-02-13 09:16, George Valkov wrote:
>> 1. We apply my original patch to disable sparse-copy on macOS. Otherwise since fclonefileat is not used whenever we overwrite a file, corruption will still occur.
> 
> I'm not entirely sold on this patch, because I still don't understand what's happening. The original bug report in <https://github.com/openwrt/openwrt/pull/11960> basically says only "it doesn't work", and I'd like to know more.
> 
> Part of the reason I'm hesitating is that there are multiple ways of interpreting what SEEK_HOLE and SEEK_DATA mean, and perhaps it's just that Apple has come up with yet another way to interpret it, which we need to know about.
> 
> Another reason to hesitate is that GNU coreutils is not the only core program that uses SEEK_HOLE and SEEK_DATA. If this really is a generic Apple problem we need to know which Apple releases have it, so that we can program around it at the Gnulib level instead of mucking about with each individual program.
> 
> Yet another reason is that perhaps the bug was introduced by this pair of changes introduced to Gnulib in November 2021 to try to address <https://bugs.gnu.org/51857>:
> 
> https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=4db8db34112b86ddf8bac48f16b5acff732b5fa9
> https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=1a268176fbb184e393c98575e61fe692264c7d91
> 
> These Gnulib patches are attached. If you revert these, does the problem go away?
> 
> And yet another reason is that perhaps the bug was introduced by this pair of Coreutils changes made in October 2021 to try to address <https://bugs.gnu.org/51433>:
> 
> https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=a9e31457bf6a63072b54a9324cdf59a09441a21f
> https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=1753012b8d8cdd0817155f4756f5bf9bfe347816
> 
> These Coreutils patches are also attached. Does reverting them help?
> 
> All things considered, I'd like a copy of the dtruss output for unmodified coreutils 9.1 cp on the failing case. This is really helpful for debugging this sort of thing. It's what helped us debug Bug#51857.<0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch><0002-lseek-port-around-macOS-SEEK_HOLE-glitch.patch><0001-cp-defend-better-against-FreeBSD-9.1-zfs-bug.patch><0002-cp-revert-unnecessary-FreeBSD-workaround.patch>

Hi Paul, I can check later today. One thing I notices is that the sparse-copy corruption has been present on 2021-10-06. (October 6th). As I recall it was not present on coreutils-8, it was introduced as soon as coreutils-9 came, precisely ever since sparse-copy was enabled. I will have to check if I can find any old logs or investigate again. But I don’t think reverting these patches will make any difference, since the issue was present before them.

Can we bring someone from the gcc team here? They know best how the build system for gcc works, and might be able to create a simple PoC to craft a sparse file that gets corrupted on copy. Currently I need to rebuild gcc, which takes 2-3 minutes and we have no clue how it is crafted.

Any idea how to make Apple aware? Last time I contacted the [email-masked] team on 2021-10-06, the replayed two days later that they are investigating, but did not bother fixing the underlaying issue. Perhaps if you contact them as a member of GNU/coreutils, they might be more open to work? If you do, refer them to Follow-up: 782620861.

I’ll come back to test your requests, at a late point today.

Georgi Valkov
httpstorm.com
nano RTOS