From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: Korn Andras Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 19 Nov 2022 09:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: 59382@debbugs.gnu.org X-Debbugs-Original-To: bug-coreutils@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.166884992512806 (code B ref -1); Sat, 19 Nov 2022 09:26:03 +0000 Received: (at submit) by debbugs.gnu.org; 19 Nov 2022 09:25:25 +0000 Received: from localhost ([127.0.0.1]:39012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owK6F-0003KO-Gw for submit@debbugs.gnu.org; Sat, 19 Nov 2022 04:25:24 -0500 Received: from lists.gnu.org ([209.51.188.17]:57608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owJ0A-0001bG-Nd for submit@debbugs.gnu.org; Sat, 19 Nov 2022 03:15:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owJ09-00019M-5t for bug-coreutils@gnu.org; Sat, 19 Nov 2022 03:15:02 -0500 Received: from elan.rulez.org ([81.0.124.36] helo=arcadia.elan.rulez.org) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1owJ06-0007Du-HV for bug-coreutils@gnu.org; Sat, 19 Nov 2022 03:15:00 -0500 Received: (qmail 12172 invoked from network); 19 Nov 2022 09:14:50 +0100 Received: from courier.intra.guy (192.168.0.10) by arcadia.intra.guy with ESMTP; 19 Nov 2022 09:14:50 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=elan.rulez.org; s=sel1; t=1668845690; bh=kFV9BDo9xzTP0dJxfW5wxrQqWbkv9I46POvqRqg1CO8=; h=Date:From:To:Subject:From; b=k8dBmL1wKmE/vahUPMicdvUYNW4un2SURh0nUTQD3p7/hyxxUasPUAb/1lG2zs2L9 kXhGob90o+AE8+eRmJ6Cqxcgii/EhLxQOQ4nWOvtUPXrLFeOBpH+sq24MRGo3sGnOz goHMRDUIxX2jqGUJJBOX6q8BEgzf/33jxws8wioM= Received: from hellgate.av.hu (hellgate.intra.guy [192.168.0.4]) by courier.intra.guy (Postfix) with ESMTP id 9CFA380A for ; Sat, 19 Nov 2022 09:14:50 +0100 (CET) Received: (qmail 27540 invoked by uid 1000); 19 Nov 2022 09:14:50 +0100 Date: Sat, 19 Nov 2022 09:14:50 +0100 From: Korn Andras Message-ID: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=81.0.124.36; envelope-from=korn-bug-coreutils=gnu.org@elan.rulez.org; helo=arcadia.elan.rulez.org X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Mailman-Approved-At: Sat, 19 Nov 2022 04:25:13 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi, on zfs, newfstatat() can return an st_blksize that is approximately equal to the file size in bytes (if the file fit into a single zfs record). The block size for filesystems can also be quite large (currently, up to 16M). The code at https://github.com/coreutils/coreutils/blob/master/src/copy.c#L1343 will try to compute the least common multiple of the input and output block sizes, then allocate a buffer of that size using mmap(). With such unusual block sizes, the least common multiple can be very large, causing the mmap() to return ENOMEM and cp to abort with "memory exhausted". Example: openat(AT_FDCWD, "tmp", O_RDONLY|O_PATH|O_DIRECTORY) = 3 newfstatat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_SYMLINK_NOFOLLOW) = 0 newfstatat(3, "configure", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_mtime_nsec=809758585, st_ctime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_ctime_nsec=809758585}, 0) = 0 openat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", O_RDONLY|O_NOFOLLOW) = 4 newfstatat(4, "", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_EMPTY_PATH) = 0 openat(3, "configure", O_WRONLY|O_TRUNC) = 5 ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = -1 EXDEV (Invalid cross-device link) newfstatat(5, "", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_mtime_nsec=326056957, st_ctime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_ctime_nsec=326056957}, AT_EMPTY_PATH) = 0 fadvise64(4, 0, 0, POSIX_FADV_SEQUENTIAL) = 0 copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 EXDEV (Invalid cross-device link) mmap(NULL, 21688754176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) brk(0x55a7fa4b0000) = 0x55a2ed8ab000 mmap(NULL, 21689794560, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) write(2>, "cp: ", 4) = 4 write(2>, "memory exhausted", 16) = 16 write(2>, "\n", 1) = 1 lseek(0>, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) close(0>) = 0 close(1>) = 0 close(2>) = 0 exit_group(1) = ? I think cp(1) shouldn't insist on using the lcm if it's very large; additionally, it should never try to allocate a buffer that is (much) larger than the source file. Perhaps you could also try to use successively smaller buffers if allocating a large one fails? András -- Write your complaints in this box: ---------> [] From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: "=?UTF-8?Q?P=C3=A1draig?= Brady" Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 19 Nov 2022 17:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Korn Andras , 59382@debbugs.gnu.org Received: via spool by 59382-submit@debbugs.gnu.org id=B59382.166887872313250 (code B ref 59382); Sat, 19 Nov 2022 17:26:01 +0000 Received: (at 59382) by debbugs.gnu.org; 19 Nov 2022 17:25:23 +0000 Received: from localhost ([127.0.0.1]:41047 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owRak-0003Rd-79 for submit@debbugs.gnu.org; Sat, 19 Nov 2022 12:25:23 -0500 Received: from mail-wr1-f42.google.com ([209.85.221.42]:36584) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owRag-0003RM-EE for 59382@debbugs.gnu.org; Sat, 19 Nov 2022 12:25:20 -0500 Received: by mail-wr1-f42.google.com with SMTP id j15so13267067wrq.3 for <59382@debbugs.gnu.org>; Sat, 19 Nov 2022 09:25:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-language:references:to:subject:user-agent :mime-version:date:message-id:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=xswd7HM4BLaQ+CQtpz1iOCt+Vi4hbJhcqHoZFf19SB8=; b=dWOaSqqjILGoVhHMW/5Y1vFsYPrIV0n3F3gYtPbXVpA0UYbFbCyJkzu3/XvOibsWa9 7jHHZ4LgiwXlRril2m+qvCcPWLPl73kudsi/wajhT7arVYvhcuRw4KtTnw5WxYCCQBLa fc+ckkes3JZixyspUfxi3IeXzDn3ugNTDnOeVFK7xMWHpfWQBTVDD79LYWXlWnh/paQj G5OAL8g0XAKvveocuQ78J61gmI4NpHAHFs5OyyzUNHvewi1XIdAR8StFeK21+SMsJc6Y RphH3dDcEetAH4mnp2M10OOszJk3oO9OMFtJ2Ku+uv155FNJs9JU7jaxsFGe6GczXbx+ +3fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-language:references:to:subject:user-agent :mime-version:date:message-id:from:sender:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xswd7HM4BLaQ+CQtpz1iOCt+Vi4hbJhcqHoZFf19SB8=; b=TlQHidSrOIS4CITd1ZjibVbDmeKIOSCBbrlYGGlpS+fKqG1SS/6B2ZRc5ujufp6wsT Pv64bEP+PHGtSeCLfsW4cAane+JZDG4V5PSS48hNP4w4PduGwJwCETGMsB+bz55xPuIX VpY+qTuR+oCVUVeA4T5HlL6Hc+z3OIX1dIru7aQBetcSn8waBx6pEdsmIyA7zzZhvdaJ k8MVILU0/ZEFgL9TQTjtMtFm1ry2GOciXMe+FS0OLQUqdbYH7f6JnQiWvg5YcVnH6+S3 HNPBJDjlLrgvP4XZ84j9W6BZPGtNKHhsyNgs2MGEDX51cCqbHB2XlrSJu31lwxNtncU9 8auA== X-Gm-Message-State: ANoB5pmy24IC/GsGN1TOMdCD4FQ0301ST+3uZGtNyveZxepFmXGw92KP P0aVVPXC9qynIF6iXvQOekU= X-Google-Smtp-Source: AA0mqf6kcnQ+ODBNJbpJgNpO1XYgQyEwnfnQUcEzeZdtXKt8BdTqsiz05/qbRR6B1oMsUgGGa3E4xA== X-Received: by 2002:adf:fb46:0:b0:22e:2d49:7d3d with SMTP id c6-20020adffb46000000b0022e2d497d3dmr6771739wrs.505.1668878712501; Sat, 19 Nov 2022 09:25:12 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id b14-20020a5d4d8e000000b0022cbf4cda62sm7949359wru.27.2022.11.19.09.25.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Nov 2022 09:25:11 -0800 (PST) From: "=?UTF-8?Q?P=C3=A1draig?= Brady" X-Google-Original-From: =?UTF-8?Q?P=C3=A1draig?= Brady Content-Type: multipart/mixed; boundary="------------j8gv8KGowdnb4qJDBVQ85Sgm" Message-ID: <36c2c912-2424-da14-31af-439d540607d4@lincor.com> Date: Sat, 19 Nov 2022 17:25:10 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.0 References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> Content-Language: en-US In-Reply-To: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) This is a multi-part message in MIME format. --------------j8gv8KGowdnb4qJDBVQ85Sgm Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 19/11/2022 08:14, Korn Andras wrote: > Hi, > > on zfs, newfstatat() can return an st_blksize that is approximately equal to the file size in bytes (if the file fit into a single zfs record). > > The block size for filesystems can also be quite large (currently, up to 16M). > > The code at https://github.com/coreutils/coreutils/blob/master/src/copy.c#L1343 will try to compute the least common multiple of the input and output block sizes, then allocate a buffer of that size using mmap(). With such unusual block sizes, the least common multiple can be very large, causing the mmap() to return ENOMEM and cp to abort with "memory exhausted". > > Example: > > openat(AT_FDCWD, "tmp", O_RDONLY|O_PATH|O_DIRECTORY) = 3 > newfstatat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_SYMLINK_NOFOLLOW) = 0 > newfstatat(3, "configure", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_mtime_nsec=809758585, st_ctime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_ctime_nsec=809758585}, 0) = 0 > openat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", O_RDONLY|O_NOFOLLOW) = 4 > newfstatat(4, "", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_EMPTY_PATH) = 0 > openat(3, "configure", O_WRONLY|O_TRUNC) = 5 > ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = -1 EXDEV (Invalid cross-device link) > newfstatat(5, "", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_mtime_nsec=326056957, st_ctime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_ctime_nsec=326056957}, AT_EMPTY_PATH) = 0 > fadvise64(4, 0, 0, POSIX_FADV_SEQUENTIAL) = 0 > copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 EXDEV (Invalid cross-device link) > mmap(NULL, 21688754176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) > brk(0x55a7fa4b0000) = 0x55a2ed8ab000 > mmap(NULL, 21689794560, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) > openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) > write(2>, "cp: ", 4) = 4 > write(2>, "memory exhausted", 16) = 16 > write(2>, "\n", 1) = 1 > lseek(0>, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) > close(0>) = 0 > close(1>) = 0 > close(2>) = 0 > exit_group(1) = ? > > I think cp(1) shouldn't insist on using the lcm if it's very large; additionally, it should never try to allocate a buffer that is (much) larger than the source file. > > Perhaps you could also try to use successively smaller buffers if allocating a large one fails? Thank you for the detailed report. This buffer_lcm code has been present for a long time, but I agree in the presence of disparate st_blksize values it can result in inappropriate buffer sizes. In your case: src st_blksize = 2647552 (0x286600) (st_size = 2647046) dst st_blksize = 4194304 (0x400000) (st_size = 0) Which implies a lowest common multiple of 21688745984 Once the st_blksize is based on st_size we can get very large values like this. In the zfs case it seems to set st_blksize to st_size rounded to next multiple of 512. The proposed patch attached removes the use of buffer_lcm() and just picks the largest st_blksize, which would be 4MiB in your case. It also limits the max buffer size to 32MiB in the edge case where st_blksize returns a larger value that this. cheers, Pádraig --------------j8gv8KGowdnb4qJDBVQ85Sgm Content-Type: text/x-patch; charset=UTF-8; name="copy-buffer-limit.patch" Content-Disposition: attachment; filename="copy-buffer-limit.patch" Content-Transfer-Encoding: base64 RnJvbSA0OGIwM2ViODg4YTExNzhmOGIzMWI3YWIxNzM1MWNjYWVlMmQ1MzNjIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UD1DMz1BMWRyYWlnPTIwQnJhZHk/ PSA8UEBkcmFpZ0JyYWR5LmNvbT4KRGF0ZTogU2F0LCAxOSBOb3YgMjAyMiAxNzowOToxNyAr MDAwMApTdWJqZWN0OiBbUEFUQ0hdIGNvcHk6IGNvbnN0cmFpbiBidWZmZXIgc2l6ZSB0byBh IHJlYXNvbmFibGUgdmFsdWUKClJlbW92ZSB1c2Ugb2YgYnVmZmVyX2xjbSgpIHRvIGRldGVy bWluZSBhIGJ1ZmZlcl9zaXplIHRvIHVzZSwKYW5kIGluc3RlYWQganVzdCBwaWNrIHRoZSBs YXJnZXN0IGlvX2Jsa3NpemUoKQpmb3IgdGhlIHNyYyBhbmQgZHN0IGZpbGUsIHdoaWNoIGlu IHR1cm4gaXMgY29uc3RyYWluZWQKdG8gYSByZWFzb25hYmxlIHZhbHVlIG9mIDMyTWlCIGN1 cnJlbnRseS4KCiogYm9vdHN0cmFwLmNvbmY6IFJlbW92ZSBidWZmZXItbGNtIG1vZHVsZS4K KiBnbC9saWIvYnVmZmVyLWxjbS5jOiBMaWtld2lzZS4KKiBnbC9saWIvYnVmZmVyLWxjbS5o OiBMaWtld2lzZS4KKiBnbC9tb2R1bGVzL2J1ZmZlci1sY206IExpa2V3aXNlLgoqIHNyYy9j b3B5LmM6IFBpY2sgYSBidWZmZXIgdmFsdWUgYXMgcGVyIHRoZSBhYm92ZSBkZXNjcmlwdGlv bi4KKiBzcmMvaW9ibGtzaXplLmg6IENvbnN0cmFpbiBpb19ibGtzaXplKCkgdG8gMzJNaUIs CiogTkVXUzogTWVudGlvbiB0aGUgZml4Lgp0byBhdm9pZCBhbnkgc3Vic2VxdWVudCBtZW1v cnkgb3IgdHlwZSBvdmVyZmxvdyBpc3N1ZXMuCkZpeGVzIGh0dHBzOi8vYnVncy5nbnUub3Jn LzU5MzgyCi0tLQogYm9vdHN0cmFwLmNvbmYgICAgICAgIHwgIDEgLQogZ2wvbGliL2J1ZmZl ci1sY20uYyAgIHwgNTkgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQogZ2wvbGliL2J1ZmZlci1sY20uaCAgIHwgIDIgLS0KIGdsL21vZHVsZXMvYnVmZmVy LWxjbSB8IDIzIC0tLS0tLS0tLS0tLS0tLS0tCiBzcmMvY29weS5jICAgICAgICAgICAgfCAx OCArKysrKy0tLS0tLS0tCiBzcmMvaW9ibGtzaXplLmggICAgICAgfCAxMCArKysrKy0tLQog NiBmaWxlcyBjaGFuZ2VkLCAxNCBpbnNlcnRpb25zKCspLCA5OSBkZWxldGlvbnMoLSkKIGRl bGV0ZSBtb2RlIDEwMDY0NCBnbC9saWIvYnVmZmVyLWxjbS5jCiBkZWxldGUgbW9kZSAxMDA2 NDQgZ2wvbGliL2J1ZmZlci1sY20uaAogZGVsZXRlIG1vZGUgMTAwNjQ0IGdsL21vZHVsZXMv YnVmZmVyLWxjbQoKZGlmZiAtLWdpdCBhL2Jvb3RzdHJhcC5jb25mIGIvYm9vdHN0cmFwLmNv bmYKaW5kZXggOGUyNTdhMjU0Li5mMzdhYzRlNDcgMTAwNjQ0Ci0tLSBhL2Jvb3RzdHJhcC5j b25mCisrKyBiL2Jvb3RzdHJhcC5jb25mCkBAIC00Miw3ICs0Miw2IEBAIGdudWxpYl9tb2R1 bGVzPSIKICAgYmFzZTMyCiAgIGJhc2U2NAogICBidG93YwotICBidWZmZXItbGNtCiAgIGJ5 dGVzd2FwCiAgIGMtc3RyY2FzZQogICBjbC1zdHJ0b2QKZGlmZiAtLWdpdCBhL2dsL2xpYi9i dWZmZXItbGNtLmMgYi9nbC9saWIvYnVmZmVyLWxjbS5jCmRlbGV0ZWQgZmlsZSBtb2RlIDEw MDY0NAppbmRleCAzOTJkY2I3NTMuLjAwMDAwMDAwMAotLS0gYS9nbC9saWIvYnVmZmVyLWxj bS5jCisrKyAvZGV2L251bGwKQEAgLTEsNTkgKzAsMCBAQAotLyogYnVmZmVyLWxjbS5jIC0g Y29tcHV0ZSBhIGdvb2QgYnVmZmVyIHNpemUgZm9yIGRlYWxpbmcgd2l0aCB0d28gZmlsZXMK LQotICAgQ29weXJpZ2h0IChDKSAyMDAyLTIwMjIgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9u LCBJbmMuCi0KLSAgIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOiB5b3UgY2FuIHJl ZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9kaWZ5Ci0gICBpdCB1bmRlciB0aGUgdGVybXMgb2Yg dGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIGFzIHB1Ymxpc2hlZCBieQotICAgdGhl IEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbiwgZWl0aGVyIHZlcnNpb24gMyBvZiB0aGUgTGlj ZW5zZSwgb3IKLSAgIChhdCB5b3VyIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uCi0KLSAg IFRoaXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0aGUgaG9wZSB0aGF0IGl0IHdpbGwg YmUgdXNlZnVsLAotICAgYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3aXRob3V0IGV2ZW4g dGhlIGltcGxpZWQgd2FycmFudHkgb2YKLSAgIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNT IEZPUiBBIFBBUlRJQ1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUKLSAgIEdOVSBHZW5lcmFsIFB1 YmxpYyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuCi0KLSAgIFlvdSBzaG91bGQgaGF2ZSBy ZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlCi0gICBh bG9uZyB3aXRoIHRoaXMgcHJvZ3JhbS4gIElmIG5vdCwgc2VlIDxodHRwczovL3d3dy5nbnUu b3JnL2xpY2Vuc2VzLz4uICAqLwotCi0vKiBXcml0dGVuIGJ5IFBhdWwgRWdnZXJ0LiAgKi8K LQotI2luY2x1ZGUgPGNvbmZpZy5oPgotI2luY2x1ZGUgImJ1ZmZlci1sY20uaCIKLQotLyog UmV0dXJuIGEgYnVmZmVyIHNpemUgc3VpdGFibGUgZm9yIGRvaW5nIEkvTyB3aXRoIGZpbGVz IHdob3NlIGJsb2NrCi0gICBzaXplcyBhcmUgQSBhbmQgQi4gIEhvd2V2ZXIsIG5ldmVyIHJl dHVybiBhIHZhbHVlIGdyZWF0ZXIgdGhhbgotICAgTENNX01BWC4gICovCi0KLXNpemVfdAot YnVmZmVyX2xjbSAoc2l6ZV90IGEsIHNpemVfdCBiLCBzaXplX3QgbGNtX21heCkKLXsKLSAg c2l6ZV90IHNpemU7Ci0KLSAgLyogVXNlIHJlYXNvbmFibGUgdmFsdWVzIGlmIGJ1ZmZlciBz aXplcyBhcmUgemVyby4gICovCi0gIGlmICghYSkKLSAgICBzaXplID0gYiA/IGIgOiA4ICog MTAyNDsKLSAgZWxzZQotICAgIHsKLSAgICAgIGlmIChiKQotICAgICAgICB7Ci0gICAgICAg ICAgLyogUmV0dXJuIGxjbSAoQSwgQikgaWYgaXQgaXMgaW4gcmFuZ2U7IG90aGVyd2lzZSwg ZmFsbCBiYWNrCi0gICAgICAgICAgICAgb24gQS4gICovCi0KLSAgICAgICAgICBzaXplX3Qg bGNtLCBtLCBuLCBxLCByOwotCi0gICAgICAgICAgLyogTiA9IGdjZCAoQSwgQikuICAqLwot ICAgICAgICAgIGZvciAobSA9IGEsIG4gPSBiOyAgKHIgPSBtICUgbikgIT0gMDsgIG0gPSBu LCBuID0gcikKLSAgICAgICAgICAgIGNvbnRpbnVlOwotCi0gICAgICAgICAgLyogTENNID0g bGNtIChBLCBCKSwgaWYgaW4gcmFuZ2UuICAqLwotICAgICAgICAgIHEgPSBhIC8gbjsKLSAg ICAgICAgICBsY20gPSBxICogYjsKLSAgICAgICAgICBpZiAobGNtIDw9IGxjbV9tYXggJiYg bGNtIC8gYiA9PSBxKQotICAgICAgICAgICAgcmV0dXJuIGxjbTsKLSAgICAgICAgfQotCi0g ICAgICBzaXplID0gYTsKLSAgICB9Ci0KLSAgcmV0dXJuIHNpemUgPD0gbGNtX21heCA/IHNp emUgOiBsY21fbWF4OwotfQpkaWZmIC0tZ2l0IGEvZ2wvbGliL2J1ZmZlci1sY20uaCBiL2ds L2xpYi9idWZmZXItbGNtLmgKZGVsZXRlZCBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDQ1NGM2 NWJlOC4uMDAwMDAwMDAwCi0tLSBhL2dsL2xpYi9idWZmZXItbGNtLmgKKysrIC9kZXYvbnVs bApAQCAtMSwyICswLDAgQEAKLSNpbmNsdWRlIDxzdGRkZWYuaD4KLXNpemVfdCBidWZmZXJf bGNtIChzaXplX3QsIHNpemVfdCwgc2l6ZV90KSBfR0xfQVRUUklCVVRFX0NPTlNUOwpkaWZm IC0tZ2l0IGEvZ2wvbW9kdWxlcy9idWZmZXItbGNtIGIvZ2wvbW9kdWxlcy9idWZmZXItbGNt CmRlbGV0ZWQgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAxZDg2ZjcxNWUuLjAwMDAwMDAwMAot LS0gYS9nbC9tb2R1bGVzL2J1ZmZlci1sY20KKysrIC9kZXYvbnVsbApAQCAtMSwyMyArMCww IEBACi1EZXNjcmlwdGlvbjoKLUNvbXB1dGUgYSBnb29kIGJ1ZmZlciBzaXplIGZvciBkZWFs aW5nIHdpdGggdHdvIGZpbGVzLgotCi1GaWxlczoKLWxpYi9idWZmZXItbGNtLmMKLWxpYi9i dWZmZXItbGNtLmgKLQotRGVwZW5kcy1vbjoKLXN0ZGRlZgotCi1jb25maWd1cmUuYWM6Ci0K LU1ha2VmaWxlLmFtOgotbGliX1NPVVJDRVMgKz0gYnVmZmVyLWxjbS5jIGJ1ZmZlci1sY20u aAotCi1JbmNsdWRlOgotImJ1ZmZlci1sY20uaCIKLQotTGljZW5zZToKLUdQTAotCi1NYWlu dGFpbmVyOgotUGF1bCBFZ2dlcnQKZGlmZiAtLWdpdCBhL3NyYy9jb3B5LmMgYi9zcmMvY29w eS5jCmluZGV4IGU0NjUyNzFlZi4uNWNjYWM5OWQ1IDEwMDY0NAotLS0gYS9zcmMvY29weS5j CisrKyBiL3NyYy9jb3B5LmMKQEAgLTM0LDcgKzM0LDYgQEAKICNpbmNsdWRlICJhY2wuaCIK ICNpbmNsdWRlICJhbGlnbmFsbG9jLmgiCiAjaW5jbHVkZSAiYmFja3VwZmlsZS5oIgotI2lu Y2x1ZGUgImJ1ZmZlci1sY20uaCIKICNpbmNsdWRlICJjYW5vbmljYWxpemUuaCIKICNpbmNs dWRlICJjb3B5LmgiCiAjaW5jbHVkZSAiY3AtaGFzaC5oIgpAQCAtMTM0MiwyMiArMTM0MSwx OSBAQCBjb3B5X3JlZyAoY2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9u YW1lLAogICAgICAgICB7CiAgICAgICAgICAgLyogQ29tcHV0ZSB0aGUgbGVhc3QgY29tbW9u IG11bHRpcGxlIG9mIHRoZSBpbnB1dCBhbmQgb3V0cHV0CiAgICAgICAgICAgICAgYnVmZmVy IHNpemVzLCBhZGp1c3RpbmcgZm9yIG91dGxhbmRpc2ggdmFsdWVzLiAgKi8KLSAgICAgICAg ICBzaXplX3QgYmxjbV9tYXggPSBNSU4gKFNJWkVfTUFYLCBTU0laRV9NQVgpOwotICAgICAg ICAgIHNpemVfdCBibGNtID0gYnVmZmVyX2xjbSAoaW9fYmxrc2l6ZSAoc3JjX29wZW5fc2Ip LCBidWZfc2l6ZSwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJsY21f bWF4KTsKKyAgICAgICAgICBzaXplX3QgYm1heCA9IE1BWCAoaW9fYmxrc2l6ZSAoc3JjX29w ZW5fc2IpLCBidWZfc2l6ZSk7CiAKICAgICAgICAgICAvKiBEbyBub3QgYm90aGVyIHdpdGgg YSBidWZmZXIgbGFyZ2VyIHRoYW4gdGhlIGlucHV0IGZpbGUsIHBsdXMgb25lCiAgICAgICAg ICAgICAgYnl0ZSB0byBtYWtlIHN1cmUgdGhlIGZpbGUgaGFzIG5vdCBncm93biB3aGlsZSBy ZWFkaW5nIGl0LiAgKi8KICAgICAgICAgICBpZiAoU19JU1JFRyAoc3JjX29wZW5fc2Iuc3Rf bW9kZSkgJiYgc3JjX29wZW5fc2Iuc3Rfc2l6ZSA8IGJ1Zl9zaXplKQogICAgICAgICAgICAg YnVmX3NpemUgPSBzcmNfb3Blbl9zYi5zdF9zaXplICsgMTsKIAotICAgICAgICAgIC8qIEhv d2V2ZXIsIHN0aWNrIHdpdGggYSBibG9jayBzaXplIHRoYXQgaXMgYSBwb3NpdGl2ZSBtdWx0 aXBsZSBvZgotICAgICAgICAgICAgIGJsY20sIG92ZXJyaWRpbmcgdGhlIGFib3ZlIGFkanVz dG1lbnRzLiAgV2F0Y2ggb3V0IGZvcgotICAgICAgICAgICAgIG92ZXJmbG93LiAgKi8KLSAg ICAgICAgICBidWZfc2l6ZSArPSBibGNtIC0gMTsKLSAgICAgICAgICBidWZfc2l6ZSAtPSBi dWZfc2l6ZSAlIGJsY207Ci0gICAgICAgICAgaWYgKGJ1Zl9zaXplID09IDAgfHwgYmxjbV9t YXggPCBidWZfc2l6ZSkKLSAgICAgICAgICAgIGJ1Zl9zaXplID0gYmxjbTsKKyAgICAgICAg ICAvKiBIb3dldmVyLCBzdGljayB3aXRoIGEgYnVmZmVyIHNpemUgdGhhdCBpcyBhIHBvc2l0 aXZlIG11bHRpcGxlIG9mCisgICAgICAgICAgICAgYm1heCwgb3ZlcnJpZGluZyB0aGUgYWJv dmUgYWRqdXN0bWVudHMuICBXYXRjaCBvdXQgZm9yIG92ZXJmbG93LiAqLworICAgICAgICAg IGJ1Zl9zaXplICs9IGJtYXggLSAxOworICAgICAgICAgIGJ1Zl9zaXplIC09IGJ1Zl9zaXpl ICUgYm1heDsKKyAgICAgICAgICBpZiAoYnVmX3NpemUgPT0gMCB8fCBJT19CVUZTSVpFX01B WCA8IGJ1Zl9zaXplKQorICAgICAgICAgICAgYnVmX3NpemUgPSBibWF4OwogICAgICAgICB9 CiAKICAgICAgIG9mZl90IG5fcmVhZDsKZGlmZiAtLWdpdCBhL3NyYy9pb2Jsa3NpemUuaCBi L3NyYy9pb2Jsa3NpemUuaAppbmRleCA4YmQxOGJhMDUuLmI4ZTllYmYzMiAxMDA2NDQKLS0t IGEvc3JjL2lvYmxrc2l6ZS5oCisrKyBiL3NyYy9pb2Jsa3NpemUuaApAQCAtMTcsNiArMTcs NyBAQAogLyogSW5jbHVkZSB0aGlzIGZpbGUgX2FmdGVyXyBzeXN0ZW0gaGVhZGVycyBpZiBw b3NzaWJsZS4gICovCiAKIC8qIHN5cy9zdGF0LmggYW5kIG1pbm1heC5oIHdpbGwgYWxyZWFk eSBoYXZlIGJlZW4gaW5jbHVkZWQgYnkgc3lzdGVtLmguICovCisjaW5jbHVkZSA8YXNzZXJ0 Lmg+CiAjaW5jbHVkZSAiaWR4LmgiCiAjaW5jbHVkZSAic3RhdC1zaXplLmgiCiAKQEAgLTcy LDExICs3MywxNCBAQAogICAgYW5kIGRlZmF1bHQgdG8gaW9fYmxrc2l6ZSgpIGlmIG5vdC4K ICAqLwogZW51bSB7IElPX0JVRlNJWkUgPSAxMjggKiAxMDI0IH07CisKKy8qIFNldCBhIG1h eCBjb25zdHJhaW50IHRvIGF2b2lkIGV4Y2Vzc2l2ZSBtZW0gdXNhZ2Ugb3IgdHlwZSBvdmVy Zmxvdy4gICovCitlbnVtIHsgSU9fQlVGU0laRV9NQVggPSAxMjggKiBJT19CVUZTSVpFIH07 CitzdGF0aWNfYXNzZXJ0IChJT19CVUZTSVpFX01BWCA8PSBNSU4gKElEWF9NQVgsIFNJWkVf TUFYKSAvIDIgKyAxKTsKKwogc3RhdGljIGlubGluZSBpZHhfdAogaW9fYmxrc2l6ZSAoc3Ry dWN0IHN0YXQgc2IpCiB7Ci0gIC8qIERvbuKAmXQgZ28gYWJvdmUgdGhlIGxhcmdlc3QgcG93 ZXIgb2YgdHdvIHRoYXQgZml0cyBpbiBpZHhfdCBhbmQgc2l6ZV90LAotICAgICBhcyB0aGF0 IGlzIGFza2luZyBmb3IgdHJvdWJsZS4gICovCi0gIHJldHVybiBNSU4gKE1JTiAoSURYX01B WCwgU0laRV9NQVgpIC8gMiArIDEsCisgIHJldHVybiBNSU4gKElPX0JVRlNJWkVfTUFYLAog ICAgICAgICAgICAgICBNQVggKElPX0JVRlNJWkUsIFNUX0JMS1NJWkUgKHNiKSkpOwogfQot LSAKMi4yNi4yCgo= --------------j8gv8KGowdnb4qJDBVQ85Sgm-- From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sun, 20 Nov 2022 03:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: =?UTF-8?Q?P=C3=A1draig?= Brady , Korn Andras , 59382@debbugs.gnu.org Received: via spool by 59382-submit@debbugs.gnu.org id=B59382.166891621815993 (code B ref 59382); Sun, 20 Nov 2022 03:51:02 +0000 Received: (at 59382) by debbugs.gnu.org; 20 Nov 2022 03:50:18 +0000 Received: from localhost ([127.0.0.1]:41956 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owbLV-00049t-Im for submit@debbugs.gnu.org; Sat, 19 Nov 2022 22:50:18 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:47432) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owbLT-00049Z-2W for 59382@debbugs.gnu.org; Sat, 19 Nov 2022 22:50:16 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5FD1C160056; Sat, 19 Nov 2022 19:50:08 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id P6upPWqU1vRt; Sat, 19 Nov 2022 19:50:07 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2EC79160060; Sat, 19 Nov 2022 19:50:07 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 2EC79160060 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1668916207; bh=khnmlz4OmE/V1eEzPFijSYIXrPCHmfLggT+xy1Fbchk=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; b=UkctzkdFsyYDrmI/MlxaqbhqRjLTv/nEubCAfDxuv1lFAFgCXvRxn/74SAgavnzf5 1x5M890gez+inmsf/qD6Q9m1NgQ1z8/32aZMcBmydiG/kcKu8PTqhCLPvovo7QSgSw F8cVtjnhmkEBv0KOp2upf0dhjbiHR3LnbyFTdj8Q= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tdlWKhDw9pTC; Sat, 19 Nov 2022 19:50:07 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id C30E5160056; Sat, 19 Nov 2022 19:50:06 -0800 (PST) Content-Type: multipart/mixed; boundary="------------o42tKUYPmeHWLQ9Pzyti6ZZd" Message-ID: <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> Date: Sat, 19 Nov 2022 19:50:06 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Content-Language: en-US References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <36c2c912-2424-da14-31af-439d540607d4@lincor.com> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) This is a multi-part message in MIME format. --------------o42tKUYPmeHWLQ9Pzyti6ZZd Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit >> The block size for filesystems can also be quite large (currently, up >> to 16M). It seems ZFS tries to "help" apps by reporting misinformation (namely a smaller block size than actually preferred) when the file is small. This is unfortunate, since it messes up cp and similar programs that need to juggle multiple block sizes. Plus, it messes up any program that assumes st_blksize is constant for the life of a file descriptor, which "cp" does assume elsewhere. GNU cp doesn't need ZFS's "help", as it's already smart enough to not over-allocate a buffer when the input file is small but its blocksize is large. Instead, this "help" from ZFS causes GNU cp to over-allocate because it naively trusts the blocksize ZFS that reports. > The proposed patch attached removes the use of buffer_lcm() > and just picks the largest st_blksize, which would be 4MiB in your case. > It also limits the max buffer size to 32MiB in the edge case > where st_blksize returns a larger value that this. I suppose this could break cp if st_blksize is not a power of 2, and if the file is not a regular file, and reads must be a multiple of the block size. POSIX allows such things though I expect nowadays it'd be limited to weird devices. Although we inadvertently removed support for weird devices in 2009 by commit 55efc5f3ee485b3e31a91c331f07c89aeccc4e89, and nobody seems to care (because people use dd or whatever to deal with weird devices), I think it'd be better to limit the fix to regular files. And while we're at it we might as well resurrect support for weird devices. > +#include No need for this, as static_assert works without in C23, and Gnulib's assert-h module support this. > +/* Set a max constraint to avoid excessive mem usage or type overflow. */ > +enum { IO_BUFSIZE_MAX = 128 * IO_BUFSIZE }; > +static_assert (IO_BUFSIZE_MAX <= MIN (IDX_MAX, SIZE_MAX) / 2 + 1); I'm leery of putting in a maximum as low as 16 MiB. Although that's OK now (it matches OpenZFS's current maximum), cp in the future will surely deal with bigger block sizes. Instead, how about if we stick with GNU's "no arbitrary limits" policy and work around the ZFS bug instead? Something like the attached patch, perhaps? --------------o42tKUYPmeHWLQ9Pzyti6ZZd Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-work-around-ZFS-misinformation.patch" Content-Disposition: attachment; filename="0001-cp-work-around-ZFS-misinformation.patch" Content-Transfer-Encoding: base64 RnJvbSA1NTFmM2Y1NTE4MDY2OWFiMGJmZDZjNWQ5ZTNlMGYzOGNiMDM1MTcyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDE5IE5vdiAyMDIyIDE5OjA0OjM2IC0wODAwClN1YmplY3Q6IFtQQVRD SF0gY3A6IHdvcmsgYXJvdW5kIFpGUyBtaXNpbmZvcm1hdGlvbgoKUHJvYmxlbSByZXBvcnRl ZCBieSBLb3JuIEFuZHJhcyAoQnVnIzU5MzgyKS4KKiBib290c3RyYXAuY29uZiAoZ251bGli X21vZHVsZXMpOiBBZGQgY291bnQtbGVhZGluZy16ZXJvcywKd2hpY2ggd2FzIGFscmVhZHkg YW4gaW5kaXJlY3QgZGVwZW5kZW5jeSwgc2luY2UgaW9ibGtzaXplLmgKbm93IHVzZXMgaXQg ZGlyZWN0bHkuCiogc3JjL2lvYmxrc2l6ZS5oOiBJbmNsdWRlIGNvdW50LWxlYWRpbmctemVy b3MuaC4KKGlvX2Jsa3NpemUpOiBUcmVhdCBpbXBvc3NpYmxlIGJsb2Nrc2l6ZXMgYXMgSU9f QlVGU0laRS4KV2hlbiBncm93aW5nIGEgYmxvY2tzaXplIHRvIElPX0JVRlNJWkUsIGtlZXAg aXQgYSBtdWx0aXBsZSBvZiB0aGUKc3RhdGVkIGJsb2Nrc2l6ZS4gIFdvcmsgYXJvdW5kIHRo ZSBaRlMgcGVyZm9ybWFuY2UgYnVnLgotLS0KIE5FV1MgICAgICAgICAgICB8ICAzICsrKwog Ym9vdHN0cmFwLmNvbmYgIHwgIDEgKwogc3JjL2lvYmxrc2l6ZS5oIHwgMjggKysrKysrKysr KysrKysrKysrKysrKysrKysrLQogMyBmaWxlcyBjaGFuZ2VkLCAzMSBpbnNlcnRpb25zKCsp LCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvTkVXUyBiL05FV1MKaW5kZXggYjZiNTIw MWU3Li45MjgyMzUyYzggMTAwNjQ0Ci0tLSBhL05FV1MKKysrIGIvTkVXUwpAQCAtMTMsNiAr MTMsOSBAQCBHTlUgY29yZXV0aWxzIE5FV1MgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAtKi0gb3V0bGluZSAtKi0KICAgJ2NwIC1yeCAvIC9tbnQnIG5vIGxvbmdlciBj b21wbGFpbnMgImNhbm5vdCBjcmVhdGUgZGlyZWN0b3J5IC9tbnQvIi4KICAgW2J1ZyBpbnRy b2R1Y2VkIGluIGNvcmV1dGlscy05LjFdCiAKKyAgY3AsIG12LCBhbmQgaW5zdGFsbCBubyBs b25nZXIgdXNlIG92ZXJseSBsYXJnZSBJL08gYnVmZmVycyB3aGVuIFpGUworICBtaXNpbmZv cm1zIHRoZW0gYWJvdXQgSU8gYmxvY2sgc2l6ZXMuCisKICAgJ212IC0tYmFja3VwPXNpbXBs ZSBmIGQvJyBubyBsb25nZXIgbWlzdGFrZW5seSBiYWNrcyB1cCBkL2YgdG8gZn4uCiAgIFti dWcgaW50cm9kdWNlZCBpbiBjb3JldXRpbHMtOS4xXQogCmRpZmYgLS1naXQgYS9ib290c3Ry YXAuY29uZiBiL2Jvb3RzdHJhcC5jb25mCmluZGV4IDhlMjU3YTI1NC4uZjg3MTUwNjhlIDEw MDY0NAotLS0gYS9ib290c3RyYXAuY29uZgorKysgYi9ib290c3RyYXAuY29uZgpAQCAtNTks NiArNTksNyBAQCBnbnVsaWJfbW9kdWxlcz0iCiAgIGNvbmZpZy1oCiAgIGNvbmZpZ21ha2UK ICAgY29weS1maWxlLXJhbmdlCisgIGNvdW50LWxlYWRpbmctemVyb3MKICAgY3J5cHRvL21k NQogICBjcnlwdG8vc2hhMQogICBjcnlwdG8vc2hhMjU2CmRpZmYgLS1naXQgYS9zcmMvaW9i bGtzaXplLmggYi9zcmMvaW9ibGtzaXplLmgKaW5kZXggOGJkMThiYTA1Li5hYTM2N2FhNGUg MTAwNjQ0Ci0tLSBhL3NyYy9pb2Jsa3NpemUuaAorKysgYi9zcmMvaW9ibGtzaXplLmgKQEAg LTE4LDYgKzE4LDcgQEAKIAogLyogc3lzL3N0YXQuaCBhbmQgbWlubWF4Lmggd2lsbCBhbHJl YWR5IGhhdmUgYmVlbiBpbmNsdWRlZCBieSBzeXN0ZW0uaC4gKi8KICNpbmNsdWRlICJpZHgu aCIKKyNpbmNsdWRlICJjb3VudC1sZWFkaW5nLXplcm9zLmgiCiAjaW5jbHVkZSAic3RhdC1z aXplLmgiCiAKIApAQCAtNzUsOCArNzYsMzMgQEAgZW51bSB7IElPX0JVRlNJWkUgPSAxMjgg KiAxMDI0IH07CiBzdGF0aWMgaW5saW5lIGlkeF90CiBpb19ibGtzaXplIChzdHJ1Y3Qgc3Rh dCBzYikKIHsKKyAgLyogVHJlYXQgaW1wb3NzaWJsZSBibG9ja3NpemVzIGFzIGlmIHRoZXkg d2VyZSBJT19CVUZTSVpFLiAgKi8KKyAgaWR4X3QgYmxvY2tzaXplID0gU1RfQkxLU0laRSAo c2IpIDw9IDAgPyBJT19CVUZTSVpFIDogU1RfQkxLU0laRSAoc2IpOworCisgIC8qIFVzZSBh IGJsb2Nrc2l6ZSBvZiBhdCBsZWFzdCBJT19CVUZTSVpFIGJ5dGVzLCBrZWVwaW5nIGl0IGEK KyAgICAgbXVsdGlwbGUgb2YgdGhlIG9yaWdpbmFsIGJsb2Nrc2l6ZS4gICovCisgIGJsb2Nr c2l6ZSArPSAoSU9fQlVGU0laRSAtIDEpIC0gKElPX0JVRlNJWkUgLSAxKSAlIGJsb2Nrc2l6 ZTsKKworICAvKiBGb3IgcmVndWxhciBmaWxlcyB3ZSBjYW4gaWdub3JlIHRoZSBibG9ja3Np emUgaWYgd2UgdGhpbmsgd2Uga25vdyBiZXR0ZXIuCisgICAgIFpGUyBzb21ldGltZXMgdW5k ZXJzdGF0ZXMgdGhlIGJsb2Nrc2l6ZSwgYmVjYXVzZSBpdCB0aGlua3MKKyAgICAgYXBwcyBz dHVwaWRseSBhbGxvY2F0ZSBhIGJsb2NrIHRoYXQgbGFyZ2UgZXZlbiBmb3Igc21hbGwgZmls ZXMuCisgICAgIFRoaXMgbWlzaW5mb3JtYXRpb24gY2FuIGNhdXNlIGNvcmV1dGlscyB0byB1 c2Ugd3Jvbmctc2l6ZWQgYmxvY2tzLgorICAgICBXb3JrIGFyb3VuZCBzb21lIG9mIHRoZSBw ZXJmb3JtYW5jZSBidWcgYnkgc3Vic3RpdHV0aW5nIHRoZSBuZXh0CisgICAgIHBvd2VyIG9m IHR3byB3aGVuIHRoZSByZXBvcnRlZCBibG9ja3NpemUgaXMgbm90IGEgcG93ZXIgb2YgdHdv LiAgKi8KKyAgaWYgKFNfSVNSRUcgKHNiLnN0X21vZGUpCisgICAgICAmJiBibG9ja3NpemUg JiAoYmxvY2tzaXplIC0gMSkpCisgICAgeworICAgICAgaW50IGxlYWRpbmdfemVyb3MgPSBj b3VudF9sZWFkaW5nX3plcm9zX2xsIChibG9ja3NpemUpOworICAgICAgaWYgKElEWF9NQVgg PCBVTExPTkdfTUFYIHx8IGxlYWRpbmdfemVyb3MpCisgICAgICAgIHsKKyAgICAgICAgICB1 bnNpZ25lZCBsb25nIGxvbmcgcG93ZXIgPSAxdWxsIDw8IChVTExPTkdfV0lEVEggLSBsZWFk aW5nX3plcm9zKTsKKyAgICAgICAgICBpZiAocG93ZXIgPD0gSURYX01BWCkKKyAgICAgICAg ICAgIGJsb2Nrc2l6ZSA9IHBvd2VyOworICAgICAgICB9CisgICAgfQorCiAgIC8qIERvbuKA mXQgZ28gYWJvdmUgdGhlIGxhcmdlc3QgcG93ZXIgb2YgdHdvIHRoYXQgZml0cyBpbiBpZHhf dCBhbmQgc2l6ZV90LAogICAgICBhcyB0aGF0IGlzIGFza2luZyBmb3IgdHJvdWJsZS4gICov CiAgIHJldHVybiBNSU4gKE1JTiAoSURYX01BWCwgU0laRV9NQVgpIC8gMiArIDEsCi0gICAg ICAgICAgICAgIE1BWCAoSU9fQlVGU0laRSwgU1RfQkxLU0laRSAoc2IpKSk7CisgICAgICAg ICAgICAgIGJsb2Nrc2l6ZSk7CiB9Ci0tIAoyLjM3LjIKCg== --------------o42tKUYPmeHWLQ9Pzyti6ZZd-- From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: Korn Andras Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sun, 20 Nov 2022 11:42:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Paul Eggert Cc: 59382@debbugs.gnu.org, =?UTF-8?Q?P=C3=A1draig?= Brady Received: via spool by 59382-submit@debbugs.gnu.org id=B59382.166894451330393 (code B ref 59382); Sun, 20 Nov 2022 11:42:03 +0000 Received: (at 59382) by debbugs.gnu.org; 20 Nov 2022 11:41:53 +0000 Received: from localhost ([127.0.0.1]:42513 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owihr-0007u8-Hd for submit@debbugs.gnu.org; Sun, 20 Nov 2022 06:41:53 -0500 Received: from elan.rulez.org ([81.0.124.36]:57406 helo=arcadia.elan.rulez.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owe3E-0000FA-S5 for 59382@debbugs.gnu.org; Sun, 20 Nov 2022 01:43:38 -0500 Received: (qmail 31351 invoked from network); 20 Nov 2022 07:43:19 +0100 Received: from courier.intra.guy (192.168.0.10) by arcadia.intra.guy with ESMTP; 20 Nov 2022 07:43:19 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=elan.rulez.org; s=sel1; t=1668926599; bh=LhRErbnJ4Hwb2WN8lv3KtVxCz+IDrJm8F/p599pSxc0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=QnYC2GfDVkKLCoE4GkOAkXJGG0RBMxhQRM0GKD1olRoi6aiMzwIlbMoYmlPb6tW9K hnhfaVEuDzE61j7GryZeqyImWbGVm/t61UJTx5DuWBMSymVnTDMHhc3I/gozcAnlFd VLVENaGwktWHz9FeveufGUHJ9spMmOAR2+2r90YM= Received: from hellgate.av.hu (hellgate.intra.guy [192.168.0.4]) by courier.intra.guy (Postfix) with ESMTP id B437417E for <59382@debbugs.gnu.org>; Sun, 20 Nov 2022 07:43:19 +0100 (CET) Received: (qmail 26882 invoked by uid 1000); 20 Nov 2022 07:43:19 +0100 Date: Sun, 20 Nov 2022 07:43:19 +0100 From: Korn Andras Message-ID: <20221120064319.2w7wm6qdvqlkmesh@hellgate> References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> X-Spam-Score: 0.0 (/) X-Mailman-Approved-At: Sun, 20 Nov 2022 06:41:50 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Nov 19, 2022 at 07:50:06PM -0800, Paul Eggert wrote: > > > The block size for filesystems can also be quite large (currently, > > > up to 16M). > > It seems ZFS tries to "help" apps by reporting misinformation (namely a > smaller block size than actually preferred) when the file is small. This is Just a nit: this isn't actually misinformation. ZFS uses variable "block" sizes (it calls these blocks "records"). There is a configurable per-filesystem maximum, which records of new writes will not exceed (but may not reach); but existing files may use larger record sizes than what is currently configured for the fs. The currently-configured recordsize of the filesystem and the recordsize a particular file was written with are not necessarily related. Depending on the write pattern and whether the recordsize of the fs was changed during the lifetime of the file, the same file can contain records of different sizes. Reductio ad absurdum: the "optimal" blocksize for reading may in fact depend on the position within the file (and only apply to the next read). If a file fits into a single record, then, IIUC, it is actually optimal to read it in a single operation; this is the case even if the currently configured recordsize is smaller than what the allowed maximum was when the file was written. If the file is highly fragmented and chunks of it are stored on different physical media (this can easily happen if the zfs pool was expanded with a new "vdev" during the lifetime of the file), it will in fact be fastest to issue reads for several chunks in parallel, with read speed possibly scaling almost linearly with the number of parallel requests. (Not that I'm proposing cp(1) should try to figure this out, presumably in a zfs-specific way, and actually do it.) Since the file may contain records of various sizes that bear no relation to the current per-fs recordsize setting, it's not immediately obvious (at least to me) what st_blocksize zfs should report that can't be construed as misinformation. If you have strong opinions on the matter, you may want to explain them in the pertinent OpenZFS issue: https://github.com/openzfs/zfs/issues/14195. András -- I was once thrown out of a mental hospital for depressing the other patients. From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sun, 20 Nov 2022 17:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Paul Eggert , Korn Andras , 59382@debbugs.gnu.org Received: via spool by 59382-submit@debbugs.gnu.org id=B59382.16689637717501 (code B ref 59382); Sun, 20 Nov 2022 17:03:01 +0000 Received: (at 59382) by debbugs.gnu.org; 20 Nov 2022 17:02:51 +0000 Received: from localhost ([127.0.0.1]:44416 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owniU-0001wu-MU for submit@debbugs.gnu.org; Sun, 20 Nov 2022 12:02:51 -0500 Received: from mail-wr1-f41.google.com ([209.85.221.41]:40950) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owniR-0001wL-HF for 59382@debbugs.gnu.org; Sun, 20 Nov 2022 12:02:48 -0500 Received: by mail-wr1-f41.google.com with SMTP id x5so12586396wrt.7 for <59382@debbugs.gnu.org>; Sun, 20 Nov 2022 09:02:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=UVd/FbjseBxcDG7IeeQadVX95x3MNyIURtBStk0JIZ0=; b=RmjgffamI8bwINu9uQu/EmIdp6H9i9lI8TmUAjCDAvmpXwjLCV6fX3Xh3wGn2otZ8B U+gt90B67aij05Bf7Pe3Gsw6zULnfh2va4MG29DbdMy62W4Y7Dc+Sjeiii1KA5eO/lMF AEDy0WgORhRHhFP5+Ppm1C5qo7ttEAl8WYJF6fwDxe0ILOOe61qw1mmqtxMDcSV0upst YiNy8Y4zGFh+RQs9hSDK55+wHL2nuWvL55Y1pLBC5cauRbiBAY6BtpoelYMJzlzwUdX/ /3DSSk3rp3BXgFxstOE0aFFEk9n+O2TZg10xIRaN2zqF67uHgufAIrVKldgvlC0U2LLf a2lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UVd/FbjseBxcDG7IeeQadVX95x3MNyIURtBStk0JIZ0=; b=TpTSU07MScX+9Wf7YqaHZW5Wzg4PLaJt0YkBFFEhirgB6WunCUTqqsmyuvuXQFqY3I AtZN8KmKJsDpt25s9ARXQJ8DIJAGAoJItqlvHqs9NZ5VifR33LmaUY9TgF8Jh/81w6hd Ha34tARVKw7ZqUEDiz2fJtGryce2oz/t9NCpy8DlmXBHHYxeXpCeNK1Rbm9yvS9ZDXLV ZoP92YF+kLhM+USiX/qn9dUsSzd4WQjqemL00hPmocwFCtQrk5Nv5kjQL/bGeqxanYHd Z1Tn5A6XjH+ZPeNkmgEFpzCZNs+Uk1sShAPc+suXfgOQ85kjSucSs+zgUJF/c2SDMBVJ EfJA== X-Gm-Message-State: ANoB5pmPvUkkmWciY6y06IArvWz/VRCiXHAs1m+HrQ6epSnzY2uZZVHN yxWQ9y6MhZaCEfFnijV6+WQubk6drWc= X-Google-Smtp-Source: AA0mqf62w268vKyoo3QN9pNMsRQOUBnzt6poGg4p5OpMeDCZdlGFfibf9qMo33gZ5j/KssHP0GVZUg== X-Received: by 2002:adf:ffd2:0:b0:236:59ab:cf33 with SMTP id x18-20020adfffd2000000b0023659abcf33mr9014554wrs.568.1668963761798; Sun, 20 Nov 2022 09:02:41 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id o5-20020a05600c510500b003b4ff30e566sm23660539wms.3.2022.11.20.09.02.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 20 Nov 2022 09:02:41 -0800 (PST) Message-ID: <4c557408-1e7f-ae07-abb4-f461b3a259ff@draigBrady.com> Date: Sun, 20 Nov 2022 17:02:40 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.0 Content-Language: en-US References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> From: =?UTF-8?Q?P=C3=A1draig?= Brady In-Reply-To: <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) On 20/11/2022 03:50, Paul Eggert wrote: >>> The block size for filesystems can also be quite large (currently, up >>> to 16M). > > It seems ZFS tries to "help" apps by reporting misinformation (namely a > smaller block size than actually preferred) when the file is small. This > is unfortunate, since it messes up cp and similar programs that need to > juggle multiple block sizes. Plus, it messes up any program that assumes > st_blksize is constant for the life of a file descriptor, which "cp" > does assume elsewhere. > > GNU cp doesn't need ZFS's "help", as it's already smart enough to not > over-allocate a buffer when the input file is small but its blocksize is > large. Instead, this "help" from ZFS causes GNU cp to over-allocate > because it naively trusts the blocksize ZFS that reports. > > >> The proposed patch attached removes the use of buffer_lcm() >> and just picks the largest st_blksize, which would be 4MiB in your case. >> It also limits the max buffer size to 32MiB in the edge case >> where st_blksize returns a larger value that this. > > I suppose this could break cp if st_blksize is not a power of 2, and if > the file is not a regular file, and reads must be a multiple of the > block size. POSIX allows such things though I expect nowadays it'd be > limited to weird devices. > > Although we inadvertently removed support for weird devices in 2009 by > commit 55efc5f3ee485b3e31a91c331f07c89aeccc4e89, and nobody seems to > care (because people use dd or whatever to deal with weird devices), I > think it'd be better to limit the fix to regular files. And while we're > at it we might as well resurrect support for weird devices. I wouldn't worry about weird devices TBH, for the reasons you state, but it doesn't cost too much to support so fair enough. I would definitely add a comment to the code in this regard though as I certainly wasn't aware of that issue when I added commit 55efc5f3. >> +#include > > No need for this, as static_assert works without in C23, and > Gnulib's assert-h module support this. cool >> +/* Set a max constraint to avoid excessive mem usage or type overflow. */ >> +enum { IO_BUFSIZE_MAX = 128 * IO_BUFSIZE }; >> +static_assert (IO_BUFSIZE_MAX <= MIN (IDX_MAX, SIZE_MAX) / 2 + 1); > > I'm leery of putting in a maximum as low as 16 MiB. Although that's OK > now (it matches OpenZFS's current maximum), cp in the future will surely > deal with bigger block sizes. Instead, how about if we stick with GNU's > "no arbitrary limits" policy and work around the ZFS bug instead? Well I wouldn't think of this as a functional limit, more of a defensive programming technique with possible perf benefits. Note as per the table in ioblksize.h, performance is seen to max out at around 2^17 on most systems, and degrades on some systems beyond that. So cp would be faster while being less memory efficient. But yes if we're relying on these multiples for "weird devices" then fair enough, that would be a functional limit, so we need to consider such large buffers. Also to state it explicitly, for regular files, your change to clamp to a power of two will effectively get buffer_lcm to pick the max of the two io_blksize() values. I would change the commit summary from: cp: work around ZFS misinformation to: copy: fix possible over allocation for regular files because: - This also applies to cross device mv etc. - ZFS is giving more info TBH, just a bit naïvely - I do see this is a coreutils bug (as well) Here is a NEWS entry as well: cp, mv, and install avoid allocating too much memory, and possibly triggering "memory exhausted" failures, on file systems like ZFS, which can return varied file system I/O block size values for files. [bug introduced in coreutils-6.0] thanks! Pádraig From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Sun, 20 Nov 2022 17:30:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Korn Andras Cc: 59382@debbugs.gnu.org, =?UTF-8?Q?P=C3=A1draig?= Brady Received: via spool by 59382-submit@debbugs.gnu.org id=B59382.166896538410191 (code B ref 59382); Sun, 20 Nov 2022 17:30:04 +0000 Received: (at 59382) by debbugs.gnu.org; 20 Nov 2022 17:29:44 +0000 Received: from localhost ([127.0.0.1]:44481 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owo8V-0002eI-K7 for submit@debbugs.gnu.org; Sun, 20 Nov 2022 12:29:44 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:36838) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owo8T-0002dy-So for 59382@debbugs.gnu.org; Sun, 20 Nov 2022 12:29:42 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5153B160056; Sun, 20 Nov 2022 09:29:35 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Sbs5lxtXEF1E; Sun, 20 Nov 2022 09:29:34 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 8B667160069; Sun, 20 Nov 2022 09:29:34 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 8B667160069 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1668965374; bh=0TZ93x1ySklqWqFtvr3WVjtoOY6JZyYlrcZM+ruOaEE=; h=Message-ID:Date:MIME-Version:To:From:Subject:Content-Type: Content-Transfer-Encoding; b=TpnNk4o+6Ht5uGbd3vUfh81NoyNn63jiWZVMXwwfZvGc7e69SqTB/+N5GLs2nn+lK Y4V38bZVx78zVUZ3K421eIhPBfwjv7A05N8VCrRBa+a9d/K7LgVa50RmukafEwWaSZ xLpH9U7Bgxn0tztpVq/MCIV5tSXOiCh6rkqFK/sQ= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hz0GiImllI2b; Sun, 20 Nov 2022 09:29:34 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 404C5160056; Sun, 20 Nov 2022 09:29:34 -0800 (PST) Message-ID: <14c7af14-7296-f97a-5276-548eb4a9625a@cs.ucla.edu> Date: Sun, 20 Nov 2022 09:29:33 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Content-Language: en-US References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> <20221120064319.2w7wm6qdvqlkmesh@hellgate> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <20221120064319.2w7wm6qdvqlkmesh@hellgate> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 2022-11-19 22:43, Korn Andras wrote: > the same file can contain records of different > sizes. Reductio ad absurdum: the "optimal" blocksize for reading may in fact > depend on the position within the file (and only apply to the next read). This sort of problem exists on traditional devices as well. A tape drive can have records of different sizes. For these devices, the best approach is to allocate a buffer of the maximum blocksize the drive supports. For the file you describe the situation is different, since ZFS will straddle small blocks during I/O. Although there's no single "best" I would guess that it'd typically be better to report the blocksize currently in use for creating new blocks (which would be a power of two for ZFS), as that will map better to how programs like cp deal with blocksizes. This may not be perfect but it'd be better than what ZFS does now, at least for the instances of 'cp' that are already out there. From unknown Mon Aug 11 19:05:22 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Korn Andras Subject: bug#59382: closed (Re: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual) Message-ID: References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> X-Gnu-PR-Message: they-closed 59382 X-Gnu-PR-Package: coreutils Reply-To: 59382@debbugs.gnu.org Date: Mon, 02 Jan 2023 23:05:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1672700702-26921-1" This is a multi-part message in MIME format... ------------=_1672700702-26921-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #59382: cp(1) tries to allocate too much memory if filesystem blocksizes ar= e unusual which was filed against the coreutils package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 59382@debbugs.gnu.org. --=20 59382: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D59382 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1672700702-26921-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 59382-done) by debbugs.gnu.org; 2 Jan 2023 23:04:04 +0000 Received: from localhost ([127.0.0.1]:44309 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCTqe-0006yx-3o for submit@debbugs.gnu.org; Mon, 02 Jan 2023 18:04:04 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:53950) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCTqZ-0006yQ-MV for 59382-done@debbugs.gnu.org; Mon, 02 Jan 2023 18:04:03 -0500 Received: by mail-wm1-f46.google.com with SMTP id ja17so21466662wmb.3 for <59382-done@debbugs.gnu.org>; Mon, 02 Jan 2023 15:03:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:sender:from:to:cc:subject:date :message-id:reply-to; bh=uCp2h8yUpQhoSOMraxBZVSao82dJf9L8h3FoFY33B3w=; b=T4tXNaBx9WYVRKsRg7bqcU3VmiACukoWpuvV5I+LLONHtrHS+mpH53xBC7MWO0mWis 3rQYRNajr+tpSs6fgiIcnz1ZqeoslQxtHvas8qi7ugKsKe2hEmYaNGJnTC6NtYdwukw1 Uj3Y/8jrQ4JkxeET7/UDO1agLmIos0T/A2g/0UtchmZrHe89zqfhHnKldAJDI7kLyzQD R7t8YBW+wpGweLZqPy7TxQ8ox1tSZEXpKd6LyLE0OSYophgSeDqCVDmo5L0uly8iL7Uy tzowPko0ML+8hmF8mgmRpDqxGKwM6V8QIrDeBses/FR5pj0Qed3dxLVLdVnsCRzEFemB gOBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uCp2h8yUpQhoSOMraxBZVSao82dJf9L8h3FoFY33B3w=; b=3g+/ighWjwMEZQ8yOTsAmDxw4t3nvdb1WE5EIsCrOH4kUPAI5npCc12l7Ffq5iEdRJ BqV1GbUrs6bkkMhk6vO8GN6pnUZk2B2XLtgfOctpHLOUBXXFYjAucAJM6EO028DTcI9N 6xU2FS+ZofarpqI1/RtTjo1uW+ceuKI5iWbB4/6gybqTTLMClI3v09DSCNinQ6mQ9b6E Amn0C6sp8Vp02i+eVyYKoVz36+PR5mXkcU6v5t6oh7Mt0t2lZbck6b12AsNa2LTxLgog AJT5IuqPAmWz4r7ZN0js9yKq0h55a4nKiPWg34TxlSjdfGpqEXlvuA8Sd31L5sMCDhjj qskA== X-Gm-Message-State: AFqh2kru/za34c5gjWPQNc8SoXA+q4EzgmtJFRL9cCBJw5iNh8uJ3+HU REXfF2G+Z4Ufi/YLdHkFixU= X-Google-Smtp-Source: AMrXdXuzfkfxYmD/bSapf24tzWTdVR0WXwYB2M7ptlFj4t9NK4oTxu4jiMGMBf8uXI+dWLjud5zzsQ== X-Received: by 2002:a05:600c:8a9:b0:3cf:6e85:eda7 with SMTP id l41-20020a05600c08a900b003cf6e85eda7mr30279889wmp.14.1672700632614; Mon, 02 Jan 2023 15:03:52 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id j6-20020a05600c1c0600b003b4ff30e566sm70848744wms.3.2023.01.02.15.03.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 02 Jan 2023 15:03:51 -0800 (PST) Content-Type: multipart/mixed; boundary="------------xHF2dJEN8ozijI4ov87GDLFW" Message-ID: Date: Mon, 2 Jan 2023 23:03:50 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.0 Subject: Re: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Content-Language: en-US To: Paul Eggert , Korn Andras , 59382-done@debbugs.gnu.org References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> From: =?UTF-8?Q?P=c3=a1draig_Brady?= In-Reply-To: <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 59382-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------xHF2dJEN8ozijI4ov87GDLFW Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 20/11/2022 03:50, Paul Eggert wrote: > Although we inadvertently removed support for weird devices in 2009 by > commit 55efc5f3ee485b3e31a91c331f07c89aeccc4e89, and nobody seems to > care (because people use dd or whatever to deal with weird devices), I > think it'd be better to limit the fix to regular files. And while we're > at it we might as well resurrect support for weird devices. Paul I'm happy with your patch for this. I've rebased and tweaked the summary and NEWS slightly in the attached. OK for me to apply? Preemptively marking this as done. cheers, Pádraig --------------xHF2dJEN8ozijI4ov87GDLFW Content-Type: text/x-patch; charset=UTF-8; name="copy-fix-large-alloc.patch" Content-Disposition: attachment; filename="copy-fix-large-alloc.patch" Content-Transfer-Encoding: base64 RnJvbSA1NzM5NzllMjFhZjhmOWVhN2Y4MzY2MGU5NDk5MDJiMzMzMGNlMjVlIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDE5IE5vdiAyMDIyIDE5OjA0OjM2IC0wODAwClN1YmplY3Q6IFtQQVRD SF0gY29weTogZml4IHBvc3NpYmxlIG92ZXIgYWxsb2NhdGlvbiBmb3IgcmVndWxhciBmaWxl cwoKKiBib290c3RyYXAuY29uZiAoZ251bGliX21vZHVsZXMpOiBBZGQgY291bnQtbGVhZGlu Zy16ZXJvcywKd2hpY2ggd2FzIGFscmVhZHkgYW4gaW5kaXJlY3QgZGVwZW5kZW5jeSwgc2lu Y2UgaW9ibGtzaXplLmgKbm93IHVzZXMgaXQgZGlyZWN0bHkuCiogc3JjL2lvYmxrc2l6ZS5o OiBJbmNsdWRlIGNvdW50LWxlYWRpbmctemVyb3MuaC4KKGlvX2Jsa3NpemUpOiBUcmVhdCBp bXBvc3NpYmxlIGJsb2Nrc2l6ZXMgYXMgSU9fQlVGU0laRS4KV2hlbiBncm93aW5nIGEgYmxv Y2tzaXplIHRvIElPX0JVRlNJWkUsIGtlZXAgaXQgYSBtdWx0aXBsZSBvZiB0aGUKc3RhdGVk IGJsb2Nrc2l6ZS4gIFdvcmsgYXJvdW5kIHRoZSBaRlMgcGVyZm9ybWFuY2UgYnVnLgoqIE5F V1M6IE1lbnRpb24gdGhlIGJ1ZyBmaXguClByb2JsZW0gcmVwb3J0ZWQgYnkgS29ybiBBbmRy YXMgYXQgaHR0cHM6Ly9idWdzLmdudS5vcmcvNTkzODIKLS0tCiBORVdTICAgICAgICAgICAg fCAgNSArKysrKwogYm9vdHN0cmFwLmNvbmYgIHwgIDEgKwogc3JjL2lvYmxrc2l6ZS5oIHwg MjggKysrKysrKysrKysrKysrKysrKysrKysrKysrLQogMyBmaWxlcyBjaGFuZ2VkLCAzMyBp bnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvTkVXUyBiL05FV1MK aW5kZXggMGZiZmU3YjA5Li40MGViNjY5NjggMTAwNjQ0Ci0tLSBhL05FV1MKKysrIGIvTkVX UwpAQCAtMjAsNiArMjAsMTEgQEAgR05VIGNvcmV1dGlscyBORVdTICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUgLSotCiAgIHdoaWNoIG1heSBoYXZl IHJlc3VsdGVkIGluIGRhdGEgY29ycnVwdGlvbi4KICAgW2J1ZyBpbnRyb2R1Y2VkIGluIGNv cmV1dGlscy03LjUgYW5kIGVuYWJsZWQgYnkgZGVmYXVsdCBpbiBjb3JldXRpbHMtOS4wXQog CisgIGNwLCBtdiwgYW5kIGluc3RhbGwgYXZvaWQgYWxsb2NhdGluZyB0b28gbXVjaCBtZW1v cnksIGFuZCBwb3NzaWJseQorICB0cmlnZ2VyaW5nICJtZW1vcnkgZXhoYXVzdGVkIiBmYWls dXJlcywgb24gZmlsZSBzeXN0ZW1zIGxpa2UgWkZTLAorICB3aGljaCBjYW4gcmV0dXJuIHZh cmllZCBmaWxlIHN5c3RlbSBJL08gYmxvY2sgc2l6ZSB2YWx1ZXMgZm9yIGZpbGVzLgorICBb YnVnIGludHJvZHVjZWQgaW4gY29yZXV0aWxzLTYuMF0KKwogICAnbXYgLS1iYWNrdXA9c2lt cGxlIGYgZC8nIG5vIGxvbmdlciBtaXN0YWtlbmx5IGJhY2tzIHVwIGQvZiB0byBmfi4KICAg W2J1ZyBpbnRyb2R1Y2VkIGluIGNvcmV1dGlscy05LjFdCiAKZGlmZiAtLWdpdCBhL2Jvb3Rz dHJhcC5jb25mIGIvYm9vdHN0cmFwLmNvbmYKaW5kZXggMjQ3NzM0NjczLi5mMDRjOTA2MjQg MTAwNjQ0Ci0tLSBhL2Jvb3RzdHJhcC5jb25mCisrKyBiL2Jvb3RzdHJhcC5jb25mCkBAIC01 OSw2ICs1OSw3IEBAIGdudWxpYl9tb2R1bGVzPSIKICAgY29uZmlnLWgKICAgY29uZmlnbWFr ZQogICBjb3B5LWZpbGUtcmFuZ2UKKyAgY291bnQtbGVhZGluZy16ZXJvcwogICBjcnlwdG8v bWQ1CiAgIGNyeXB0by9zaGExCiAgIGNyeXB0by9zaGEyNTYKZGlmZiAtLWdpdCBhL3NyYy9p b2Jsa3NpemUuaCBiL3NyYy9pb2Jsa3NpemUuaAppbmRleCA3YTU2YzFhNTEuLjgwZDc5MzFi YSAxMDA2NDQKLS0tIGEvc3JjL2lvYmxrc2l6ZS5oCisrKyBiL3NyYy9pb2Jsa3NpemUuaApA QCAtMTgsNiArMTgsNyBAQAogCiAvKiBzeXMvc3RhdC5oIGFuZCBtaW5tYXguaCB3aWxsIGFs cmVhZHkgaGF2ZSBiZWVuIGluY2x1ZGVkIGJ5IHN5c3RlbS5oLiAqLwogI2luY2x1ZGUgImlk eC5oIgorI2luY2x1ZGUgImNvdW50LWxlYWRpbmctemVyb3MuaCIKICNpbmNsdWRlICJzdGF0 LXNpemUuaCIKIAogCkBAIC03NSw4ICs3NiwzMyBAQCBlbnVtIHsgSU9fQlVGU0laRSA9IDEy OCAqIDEwMjQgfTsKIHN0YXRpYyBpbmxpbmUgaWR4X3QKIGlvX2Jsa3NpemUgKHN0cnVjdCBz dGF0IHNiKQogeworICAvKiBUcmVhdCBpbXBvc3NpYmxlIGJsb2Nrc2l6ZXMgYXMgaWYgdGhl eSB3ZXJlIElPX0JVRlNJWkUuICAqLworICBpZHhfdCBibG9ja3NpemUgPSBTVF9CTEtTSVpF IChzYikgPD0gMCA/IElPX0JVRlNJWkUgOiBTVF9CTEtTSVpFIChzYik7CisKKyAgLyogVXNl IGEgYmxvY2tzaXplIG9mIGF0IGxlYXN0IElPX0JVRlNJWkUgYnl0ZXMsIGtlZXBpbmcgaXQg YQorICAgICBtdWx0aXBsZSBvZiB0aGUgb3JpZ2luYWwgYmxvY2tzaXplLiAgKi8KKyAgYmxv Y2tzaXplICs9IChJT19CVUZTSVpFIC0gMSkgLSAoSU9fQlVGU0laRSAtIDEpICUgYmxvY2tz aXplOworCisgIC8qIEZvciByZWd1bGFyIGZpbGVzIHdlIGNhbiBpZ25vcmUgdGhlIGJsb2Nr c2l6ZSBpZiB3ZSB0aGluayB3ZSBrbm93IGJldHRlci4KKyAgICAgWkZTIHNvbWV0aW1lcyB1 bmRlcnN0YXRlcyB0aGUgYmxvY2tzaXplLCBiZWNhdXNlIGl0IHRoaW5rcworICAgICBhcHBz IHN0dXBpZGx5IGFsbG9jYXRlIGEgYmxvY2sgdGhhdCBsYXJnZSBldmVuIGZvciBzbWFsbCBm aWxlcy4KKyAgICAgVGhpcyBtaXNpbmZvcm1hdGlvbiBjYW4gY2F1c2UgY29yZXV0aWxzIHRv IHVzZSB3cm9uZy1zaXplZCBibG9ja3MuCisgICAgIFdvcmsgYXJvdW5kIHNvbWUgb2YgdGhl IHBlcmZvcm1hbmNlIGJ1ZyBieSBzdWJzdGl0dXRpbmcgdGhlIG5leHQKKyAgICAgcG93ZXIg b2YgdHdvIHdoZW4gdGhlIHJlcG9ydGVkIGJsb2Nrc2l6ZSBpcyBub3QgYSBwb3dlciBvZiB0 d28uICAqLworICBpZiAoU19JU1JFRyAoc2Iuc3RfbW9kZSkKKyAgICAgICYmIGJsb2Nrc2l6 ZSAmIChibG9ja3NpemUgLSAxKSkKKyAgICB7CisgICAgICBpbnQgbGVhZGluZ196ZXJvcyA9 IGNvdW50X2xlYWRpbmdfemVyb3NfbGwgKGJsb2Nrc2l6ZSk7CisgICAgICBpZiAoSURYX01B WCA8IFVMTE9OR19NQVggfHwgbGVhZGluZ196ZXJvcykKKyAgICAgICAgeworICAgICAgICAg IHVuc2lnbmVkIGxvbmcgbG9uZyBwb3dlciA9IDF1bGwgPDwgKFVMTE9OR19XSURUSCAtIGxl YWRpbmdfemVyb3MpOworICAgICAgICAgIGlmIChwb3dlciA8PSBJRFhfTUFYKQorICAgICAg ICAgICAgYmxvY2tzaXplID0gcG93ZXI7CisgICAgICAgIH0KKyAgICB9CisKICAgLyogRG9u 4oCZdCBnbyBhYm92ZSB0aGUgbGFyZ2VzdCBwb3dlciBvZiB0d28gdGhhdCBmaXRzIGluIGlk eF90IGFuZCBzaXplX3QsCiAgICAgIGFzIHRoYXQgaXMgYXNraW5nIGZvciB0cm91YmxlLiAg Ki8KICAgcmV0dXJuIE1JTiAoTUlOIChJRFhfTUFYLCBTSVpFX01BWCkgLyAyICsgMSwKLSAg ICAgICAgICAgICAgTUFYIChJT19CVUZTSVpFLCBTVF9CTEtTSVpFIChzYikpKTsKKyAgICAg ICAgICAgICAgYmxvY2tzaXplKTsKIH0KLS0gCjIuMjYuMgoK --------------xHF2dJEN8ozijI4ov87GDLFW-- ------------=_1672700702-26921-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 19 Nov 2022 09:25:25 +0000 Received: from localhost ([127.0.0.1]:39012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owK6F-0003KO-Gw for submit@debbugs.gnu.org; Sat, 19 Nov 2022 04:25:24 -0500 Received: from lists.gnu.org ([209.51.188.17]:57608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owJ0A-0001bG-Nd for submit@debbugs.gnu.org; Sat, 19 Nov 2022 03:15:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owJ09-00019M-5t for bug-coreutils@gnu.org; Sat, 19 Nov 2022 03:15:02 -0500 Received: from elan.rulez.org ([81.0.124.36] helo=arcadia.elan.rulez.org) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1owJ06-0007Du-HV for bug-coreutils@gnu.org; Sat, 19 Nov 2022 03:15:00 -0500 Received: (qmail 12172 invoked from network); 19 Nov 2022 09:14:50 +0100 Received: from courier.intra.guy (192.168.0.10) by arcadia.intra.guy with ESMTP; 19 Nov 2022 09:14:50 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=elan.rulez.org; s=sel1; t=1668845690; bh=kFV9BDo9xzTP0dJxfW5wxrQqWbkv9I46POvqRqg1CO8=; h=Date:From:To:Subject:From; b=k8dBmL1wKmE/vahUPMicdvUYNW4un2SURh0nUTQD3p7/hyxxUasPUAb/1lG2zs2L9 kXhGob90o+AE8+eRmJ6Cqxcgii/EhLxQOQ4nWOvtUPXrLFeOBpH+sq24MRGo3sGnOz goHMRDUIxX2jqGUJJBOX6q8BEgzf/33jxws8wioM= Received: from hellgate.av.hu (hellgate.intra.guy [192.168.0.4]) by courier.intra.guy (Postfix) with ESMTP id 9CFA380A for ; Sat, 19 Nov 2022 09:14:50 +0100 (CET) Received: (qmail 27540 invoked by uid 1000); 19 Nov 2022 09:14:50 +0100 Date: Sat, 19 Nov 2022 09:14:50 +0100 From: Korn Andras To: bug-coreutils@gnu.org Subject: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Message-ID: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=81.0.124.36; envelope-from=korn-bug-coreutils=gnu.org@elan.rulez.org; helo=arcadia.elan.rulez.org X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Sat, 19 Nov 2022 04:25:13 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi, on zfs, newfstatat() can return an st_blksize that is approximately equal to the file size in bytes (if the file fit into a single zfs record). The block size for filesystems can also be quite large (currently, up to 16M). The code at https://github.com/coreutils/coreutils/blob/master/src/copy.c#L1343 will try to compute the least common multiple of the input and output block sizes, then allocate a buffer of that size using mmap(). With such unusual block sizes, the least common multiple can be very large, causing the mmap() to return ENOMEM and cp to abort with "memory exhausted". Example: openat(AT_FDCWD, "tmp", O_RDONLY|O_PATH|O_DIRECTORY) = 3 newfstatat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_SYMLINK_NOFOLLOW) = 0 newfstatat(3, "configure", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_mtime_nsec=809758585, st_ctime=1668788255 /* 2022-11-18T17:17:35.809758585+0100 */, st_ctime_nsec=809758585}, 0) = 0 openat(AT_FDCWD, "usr/src/zfs-2.1.6/configure", O_RDONLY|O_NOFOLLOW) = 4 newfstatat(4, "", {st_dev=makedev(0, 0x30), st_ino=29267, st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=2647552, st_blocks=5177, st_size=2647046, st_atime=1668786119 /* 2022-11-18T16:41:59.760703544+0100 */, st_atime_nsec=760703544, st_mtime=1667705386 /* 2022-11-06T04:29:46+0100 */, st_mtime_nsec=0, st_ctime=1668785062 /* 2022-11-18T16:24:22.866737598+0100 */, st_ctime_nsec=866737598}, AT_EMPTY_PATH) = 0 openat(3, "configure", O_WRONLY|O_TRUNC) = 5 ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = -1 EXDEV (Invalid cross-device link) newfstatat(5, "", {st_dev=makedev(0, 0x32), st_ino=3838, st_mode=S_IFREG|0700, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=1, st_size=0, st_atime=1668788205 /* 2022-11-18T17:16:45.416328293+0100 */, st_atime_nsec=416328293, st_mtime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_mtime_nsec=326056957, st_ctime=1668788266 /* 2022-11-18T17:17:46.326056957+0100 */, st_ctime_nsec=326056957}, AT_EMPTY_PATH) = 0 fadvise64(4, 0, 0, POSIX_FADV_SEQUENTIAL) = 0 copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 EXDEV (Invalid cross-device link) mmap(NULL, 21688754176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) brk(0x55a7fa4b0000) = 0x55a2ed8ab000 mmap(NULL, 21689794560, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) write(2>, "cp: ", 4) = 4 write(2>, "memory exhausted", 16) = 16 write(2>, "\n", 1) = 1 lseek(0>, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) close(0>) = 0 close(1>) = 0 close(2>) = 0 exit_group(1) = ? I think cp(1) shouldn't insist on using the lcm if it's very large; additionally, it should never try to allocate a buffer that is (much) larger than the source file. Perhaps you could also try to use successively smaller buffers if allocating a large one fails? András -- Write your complaints in this box: ---------> [] ------------=_1672700702-26921-1-- From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 02 Jan 2023 23:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: =?UTF-8?Q?P=C3=A1draig?= Brady , Korn Andras , 59382-done@debbugs.gnu.org Received: via spool by 59382-done@debbugs.gnu.org id=D59382.167270150228148 (code D ref 59382); Mon, 02 Jan 2023 23:19:02 +0000 Received: (at 59382-done) by debbugs.gnu.org; 2 Jan 2023 23:18:22 +0000 Received: from localhost ([127.0.0.1]:44318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCU4T-0007Jw-PY for submit@debbugs.gnu.org; Mon, 02 Jan 2023 18:18:21 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:44058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCU4O-0007Jb-NA for 59382-done@debbugs.gnu.org; Mon, 02 Jan 2023 18:18:20 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 40419160044; Mon, 2 Jan 2023 15:18:10 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id xxxxNNO40Xhh; Mon, 2 Jan 2023 15:18:09 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 827DC160045; Mon, 2 Jan 2023 15:18:09 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 827DC160045 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1672701489; bh=1vEAM9UchoKmmxs9Ah6OY2CxJnGxr0pXMZ+Yq+hH0wM=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=FaE1sqnFJTr2gFydr6G9ojoySYEZVulOUXe0A51QJH4M7dnFRwE608aRyKuwjGyRz qIA5dt451H7SVu/ZI35mxLtthJ0PVB6jr2EsE7ncvoZNeo1sHw2emQgIiaqDq7RoK2 bFUkvzN3xJ44nuXqNXvww6nD/8H5msScQHdI3cdY= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id rwIbJcGMSVMu; Mon, 2 Jan 2023 15:18:09 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 58D45160044; Mon, 2 Jan 2023 15:18:09 -0800 (PST) Message-ID: <85fb1df1-fcf8-69ac-c1d0-6a727c330251@cs.ucla.edu> Date: Mon, 2 Jan 2023 15:18:09 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Content-Language: en-US References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.4 (----) On 2023-01-02 15:03, P=C3=A1draig Brady wrote: > On 20/11/2022 03:50, Paul Eggert wrote: >> Although we inadvertently removed support for weird devices in 2009 by >> commit 55efc5f3ee485b3e31a91c331f07c89aeccc4e89, and nobody seems to >> care (because people use dd or whatever to deal with weird devices), I >> think it'd be better to limit the fix to regular files. And while we'r= e >> at it we might as well resurrect support for weird devices. >=20 > Paul I'm happy with your patch for this. > I've rebased and tweaked the summary and NEWS slightly in the attached. >=20 > OK for me to apply? Sure, that looks good, thanks. From unknown Mon Aug 11 19:05:22 2025 X-Loop: help-debbugs@gnu.org Subject: bug#59382: cp(1) tries to allocate too much memory if filesystem blocksizes are unusual Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 02 Jan 2023 23:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59382 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Paul Eggert , Korn Andras , 59382-done@debbugs.gnu.org Received: via spool by 59382-done@debbugs.gnu.org id=D59382.167270185128695 (code D ref 59382); Mon, 02 Jan 2023 23:25:02 +0000 Received: (at 59382-done) by debbugs.gnu.org; 2 Jan 2023 23:24:11 +0000 Received: from localhost ([127.0.0.1]:44322 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCUA7-0007Sl-H1 for submit@debbugs.gnu.org; Mon, 02 Jan 2023 18:24:11 -0500 Received: from mail-wm1-f50.google.com ([209.85.128.50]:50698) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pCUA4-0007SV-RF for 59382-done@debbugs.gnu.org; Mon, 02 Jan 2023 18:24:09 -0500 Received: by mail-wm1-f50.google.com with SMTP id m3so12599067wmq.0 for <59382-done@debbugs.gnu.org>; Mon, 02 Jan 2023 15:24:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:sender:from:to:cc:subject:date :message-id:reply-to; bh=4dW6KpIs5of+Z9S5GX5w9X/7cfN4MhriCVBEMMTZKls=; b=XjeepjZERx+OlxCxHKZooMNB4pc5nycRdXK2LuupGv9IgxhK9mXZc5xl9e+bl8q7jp ZdNJD10ubGNmWZ2bA8CIo8tkDUbHWJyoZW2X1Tg7KJq2QE9D10RjTeVOMtEVhDqlWxZ8 mBfMiNvX+S0c2aompckOlEN9gk57lP0SOuW+SdSjKqZiCr7cwhXOecYGR5060vgnFcRx j3cQZCaGhVFfhFbvmeB0rnd20FCv6yeXkbvQzPYyMHt89ZRmy9QwHR5dCMzscllLBlqt THs7ICg/ADgdTuQKMUPIqfel+g3XmYRRW/OqkEXJ3M6q6VJR2mn9n5XuN82XMa0rUA2K +avg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:to:content-language:subject:user-agent :mime-version:date:message-id:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4dW6KpIs5of+Z9S5GX5w9X/7cfN4MhriCVBEMMTZKls=; b=04gzft2rOv50fRouYMzPNxGKlLVLlGFhOS1Wz0vtxxIt1aHWx8Wpj86uRbWUIjc7aG NNhwaOqFZ9lxUtquyoWm3T+4z/Vywe14m0noLuNNh4nkyAOlaAsR7pwUvN0VPz51i3VE 5ryQ45+F9REhH8PMA6aqfZZah9M60r3XsiCUUtZfReVBYIeX+0YxSVL5Lz40k5ZOmvuT I6GjPI4/Xfwa2mHubV2/z7Sz6xqFG36Yi9WabJKGUa7bRB2aFRbm++x4g0KaNBgBYdgn d0lbENW7KEZcyp9nxJVumaUsuXMX8vmhQvtNqwhxZSnZSCqDnTxivJsuE0+0RNvF4w1O GTQQ== X-Gm-Message-State: AFqh2kog1ms0smn4dV6KSfzy3qLKfPLKLBOHgCVHR6Q8bELGVwiY/kDr aAnHV+3NdA9OgfdV0cpKtI0= X-Google-Smtp-Source: AMrXdXvoLO2LHl/LrDYRq0eOm0pRWh/uKFxx8rUVUJoqF/bev5HRzmw3E4fRXKv7nweay+SijNnGvw== X-Received: by 2002:a05:600c:4e51:b0:3cf:7b8b:6521 with SMTP id e17-20020a05600c4e5100b003cf7b8b6521mr29331191wmq.32.1672701842936; Mon, 02 Jan 2023 15:24:02 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id p19-20020a1c5453000000b003d2157627a8sm43741572wmi.47.2023.01.02.15.24.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 02 Jan 2023 15:24:02 -0800 (PST) Content-Type: multipart/mixed; boundary="------------1BdOS1mPlUqxzUPcOJYRrJUz" Message-ID: Date: Mon, 2 Jan 2023 23:24:01 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.0 Content-Language: en-US References: <20221119081450.bzw7egapqhfh7lqn@hellgate.intra.guy> <36c2c912-2424-da14-31af-439d540607d4@lincor.com> <0f46c8ed-e3fb-bf9a-69e1-904632e39ad3@cs.ucla.edu> <85fb1df1-fcf8-69ac-c1d0-6a727c330251@cs.ucla.edu> From: =?UTF-8?Q?P=C3=A1draig?= Brady In-Reply-To: <85fb1df1-fcf8-69ac-c1d0-6a727c330251@cs.ucla.edu> X-Spam-Score: -0.9 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------1BdOS1mPlUqxzUPcOJYRrJUz Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 02/01/2023 23:18, Paul Eggert wrote: > On 2023-01-02 15:03, Pádraig Brady wrote: >> On 20/11/2022 03:50, Paul Eggert wrote: >>> Although we inadvertently removed support for weird devices in 2009 by >>> commit 55efc5f3ee485b3e31a91c331f07c89aeccc4e89, and nobody seems to >>> care (because people use dd or whatever to deal with weird devices), I >>> think it'd be better to limit the fix to regular files. And while we're >>> at it we might as well resurrect support for weird devices. >> >> Paul I'm happy with your patch for this. >> I've rebased and tweaked the summary and NEWS slightly in the attached. >> >> OK for me to apply? > > Sure, that looks good, thanks. I forgot about documenting the reinstated I/O block size multiple constraint, which I'll do with the attached. cheers, Pádraig --------------1BdOS1mPlUqxzUPcOJYRrJUz Content-Type: text/x-patch; charset=UTF-8; name="copy-doc-io-size.patch" Content-Disposition: attachment; filename="copy-doc-io-size.patch" Content-Transfer-Encoding: base64 RnJvbSBlMDFjMDZmYjljMTgyMTUxZDRiYmJlOTA4NzNmNzliNWIyMjk1MjQ1IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UD1DMz1BMWRyYWlnPTIwQnJhZHk/ PSA8UEBkcmFpZ0JyYWR5LmNvbT4KRGF0ZTogTW9uLCAyIEphbiAyMDIzIDIzOjE2OjA3ICsw MDAwClN1YmplY3Q6IFtQQVRDSF0gZG9jOiBjb3B5OiBtZW50aW9uIHRoZSByZWluc3RhdGVk IEkvTyBzaXplIGNvbnN0cmFpbnRzCgoqIE5FV1M6IE1lbnRpb24gdGhlIGNoYW5nZSBpbiBi ZWhhdmlvciByZSBibG9jayBzaXplIG11bHRpcGxlcwp0byBzdXBwb3J0IHVudXN1YWwgZGV2 aWNlcyB3aXRoIHRoaXMgY29uc3RyYWludC4KKiBzcmMvY29weS5jIChjb3B5X3JlZyk6IExp a2V3aXNlLgotLS0KIE5FV1MgICAgICAgfCA0ICsrKysKIHNyYy9jb3B5LmMgfCA0ICsrKy0K IDIgZmlsZXMgY2hhbmdlZCwgNyBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZm IC0tZ2l0IGEvTkVXUyBiL05FV1MKaW5kZXggN2E2NTU5NzRhLi5kY2UwOWU5YmEgMTAwNjQ0 Ci0tLSBhL05FV1MKKysrIGIvTkVXUwpAQCAtNTgsNiArNTgsMTAgQEAgR05VIGNvcmV1dGls cyBORVdTICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUg LSotCiAgICdjcCAtLXJlZmxpbms9YWx3YXlzIEEgQicgbm8gbG9uZ2VyIGxlYXZlcyBiZWhp bmQgYSBuZXdseSBjcmVhdGVkCiAgIGVtcHR5IGZpbGUgQiBtZXJlbHkgYmVjYXVzZSBjb3B5 LW9uLXdyaXRlIGNsb25lcyBhcmUgbm90IHN1cHBvcnRlZC4KIAorICBjcCwgbXYsIGFuZCBp bnN0YWxsIGFnYWluIHJlYWQgaW4gbXVsdGlwbGVzIG9mIHRoZSByZXBvcnRlZCBibG9jayBz aXplLAorICB0byBzdXBwb3J0IHVudXN1YWwgZGV2aWNlcyB0aGF0IG1heSBoYXZlIHRoaXMg Y29uc3RyYWludC4KKyAgW2JlaGF2aW9yIGluYWR2ZXJ0ZW50bHkgY2hhbmdlZCBpbiBjb3Jl dXRpbHMtNy4yXQorCiAgICdscyAtdicgYW5kICdzb3J0IC1WJyBnbyBiYWNrIHRvIHNvcnRp bmcgIi4wIiBiZWZvcmUgIi5BIiwKICAgcmV2ZXJ0aW5nIHRvIHRoZSBiZWhhdmlvciBpbiBj b3JldXRpbHMtOS4wIGFuZCBlYXJsaWVyLgogICBUaGlzIGJlaGF2aW9yIGlzIG5vdyBkb2N1 bWVudGVkLgpkaWZmIC0tZ2l0IGEvc3JjL2NvcHkuYyBiL3NyYy9jb3B5LmMKaW5kZXggMTA1 OGIwOGVjLi4yNTE3NjkzMDAgMTAwNjQ0Ci0tLSBhL3NyYy9jb3B5LmMKKysrIGIvc3JjL2Nv cHkuYwpAQCAtMTM4Nyw3ICsxMzg3LDkgQEAgY29weV9yZWcgKGNoYXIgY29uc3QgKnNyY19u YW1lLCBjaGFyIGNvbnN0ICpkc3RfbmFtZSwKICAgICAgIGlmICghIG1ha2VfaG9sZXMpCiAg ICAgICAgIHsKICAgICAgICAgICAvKiBDb21wdXRlIHRoZSBsZWFzdCBjb21tb24gbXVsdGlw bGUgb2YgdGhlIGlucHV0IGFuZCBvdXRwdXQKLSAgICAgICAgICAgICBidWZmZXIgc2l6ZXMs IGFkanVzdGluZyBmb3Igb3V0bGFuZGlzaCB2YWx1ZXMuICAqLworICAgICAgICAgICAgIGJ1 ZmZlciBzaXplcywgYWRqdXN0aW5nIGZvciBvdXRsYW5kaXNoIHZhbHVlcy4KKyAgICAgICAg ICAgICBOb3RlIHdlIHJlYWQgaW4gbXVsdGlwbGVzIG9mIHRoZSByZXBvcnRlZCBibG9jayBz aXplCisgICAgICAgICAgICAgdG8gc3VwcG9ydCAodW51c3VhbCkgZGV2aWNlcyB0aGF0IGhh dmUgdGhpcyBjb25zdHJhaW50LiAgKi8KICAgICAgICAgICBzaXplX3QgYmxjbV9tYXggPSBN SU4gKFNJWkVfTUFYLCBTU0laRV9NQVgpOwogICAgICAgICAgIHNpemVfdCBibGNtID0gYnVm ZmVyX2xjbSAoaW9fYmxrc2l6ZSAoc3JjX29wZW5fc2IpLCBidWZfc2l6ZSwKICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJsY21fbWF4KTsKLS0gCjIuMjYuMgoK --------------1BdOS1mPlUqxzUPcOJYRrJUz--