GNU bug report logs - #79267
cp --sparse=auto heuristic fails on a squashfs mounted drive.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Jeremy Allison <jallison <at> ciq.com>
Subject: bug#79267: closed (Re: bug#79267: cp --sparse=auto heuristic
 fails on a squashfs mounted drive.)
Date: Thu, 21 Aug 2025 23:25:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#79267: cp --sparse=auto heuristic fails on a squashfs mounted drive.

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 79267 <at> debbugs.gnu.org.

-- 
79267: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=79267
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jeremy Allison <jallison <at> ciq.com>
Cc: 79267-done <at> debbugs.gnu.org
Subject: Re: bug#79267: cp --sparse=auto heuristic fails on a squashfs mounted
 drive.
Date: Thu, 21 Aug 2025 17:24:16 -0600

Thanks for checking; closing the bug report.

[Message part 3 (message/rfc822, inline)]

From: Jeremy Allison <jallison <at> ciq.com>
To: bug-coreutils <at> gnu.org, Jeremy Allison <jallison <at> ciq.com>, 
 Howard Van Der Wal <hvanderwal <at> ciq.com>
Subject: cp --sparse=auto heuristic fails on a squashfs mounted drive.
Date: Mon, 18 Aug 2025 14:25:32 -0700

It turns out that: lseek(3, 0, SEEK_HOLE) returns end-of-file for a
sparse file copied from a Linux squashfs mounted drive. This breaks
the --sparse=auto heuristic that detects a sparse file.

I have a fix for you to consider.

To reproduce:

First, create a squashfs drive containing a file output_file.bin.

mkdir squashfs-root
cd squashfs-root

Then run the following script mkhole.sh:

--------------------------------------------------------------
#!/bin/bash
OUTPUT="output_file.bin"

# Remove file if it exists
rm -f "$OUTPUT"

# Write 4KB of 'A'
dd if=<(yes A | tr -d '\n' | head -c 4096) of="$OUTPUT" bs=4096 count=1

# Create a 4k*100 hole followed by 4KB of zeros
dd if=/dev/zero of="$OUTPUT" bs=4096 count=1 seek=101

# Write another 4KB of 'A' after the hole (overwriting the 4k of zeros)
dd if=<(yes A | tr -d '\n' | head -c 4096) of="$OUTPUT" bs=4096 count=1 seek=101
--------------------------------------------------------------

Now create the mysquashfs.img file to mount:

cd ..
mksquashfs squashfs-root mysquashfs.img
sudo mount -o loop  mysquashfs.img /mnt

Check that /mnt/output_file.bin is sparse:

ls -lh /mnt/output_file.bin
du -sh /mnt/output_file.bin

(the second value should be less).

Now use a newly built cp command from coreutils to copy this file to a
local filesystem.

mkdir ~/tmp
cd ~/tmp
~/src/coreutils/src/cp --reflink=never /mnt/output_file.bin nonsparse

Even though --sparse=auto and the file is sparse, it is not detected
as such. This can be confirmed by
running:

strace ~/src/coreutils/src/cp --reflink=never /mnt/output_file.bin nonsparse

and you will see:

lseek(3, 0, SEEK_DATA)                  = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
lseek(3, 0, SEEK_HOLE)                  = 417792

The following diff fixes this for me, and still passes "make check".

diff --git a/src/copy.c b/src/copy.c
index 77f0c561e..91136cd7c 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -592,7 +592,7 @@ lseek_copy (int src_fd, int dest_fd, char **abuf,
size_t buf_size,
           else if (sparse_mode != SPARSE_NEVER)
             {
               if (! create_hole (dest_fd, dst_name,
-                                 sparse_mode == SPARSE_ALWAYS,
+                                 sparse_mode != SPARSE_NEVER,
                                  ext_hole_size))
                 return false;
             }
@@ -621,7 +621,7 @@ lseek_copy (int src_fd, int dest_fd, char **abuf,
size_t buf_size,
       if ( ! sparse_copy (src_fd, dest_fd, abuf, buf_size,
                           true, allow_reflink, src_name, dst_name,
                           ext_len,
-                          sparse_mode == SPARSE_ALWAYS ? hole_size : nullptr,
+                          sparse_mode != SPARSE_NEVER ? hole_size : nullptr,
                           &n_read))
         return false;

@@ -1576,7 +1576,7 @@ copy_reg (char const *src_name, char const *dst_name,
              :
 #endif
                sparse_copy (source_desc, dest_desc, &buf, buf_size,
-                            x->sparse_mode == SPARSE_ALWAYS,
+                            x->sparse_mode != SPARSE_NEVER,
                             x->reflink_mode != REFLINK_NEVER,
                             src_name, dst_name, UINTMAX_MAX,
                             make_holes ? &hole_size : nullptr, &n_read)))

Thanks !

Jeremy Allison,
CIQ Inc.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.