From unknown Thu Jun 19 14:15:46 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#42034 <42034@debbugs.gnu.org> To: bug#42034 <42034@debbugs.gnu.org> Subject: Status: option to truncate at end or should that be default? Reply-To: bug#42034 <42034@debbugs.gnu.org> Date: Thu, 19 Jun 2025 21:15:46 +0000 retitle 42034 option to truncate at end or should that be default? reassign 42034 coreutils submitter 42034 L A Walsh severity 42034 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 24 15:57:00 2020 Received: (at submit) by debbugs.gnu.org; 24 Jun 2020 19:57:00 +0000 Received: from localhost ([127.0.0.1]:39169 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joBVz-00047m-Py for submit@debbugs.gnu.org; Wed, 24 Jun 2020 15:57:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:36620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joBVx-00047e-Gl for submit@debbugs.gnu.org; Wed, 24 Jun 2020 15:56:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48104) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1joBVx-0001Z2-8C for bug-coreutils@gnu.org; Wed, 24 Jun 2020 15:56:57 -0400 Received: from mail-io1-f42.google.com ([209.85.166.42]:45238) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1joBVv-0006jr-6g for bug-coreutils@gnu.org; Wed, 24 Jun 2020 15:56:56 -0400 Received: by mail-io1-f42.google.com with SMTP id y5so3515200iob.12 for ; Wed, 24 Jun 2020 12:56:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=rpdDWF5zRoCOdwnHcP5A7Nz63BumzIqw8Ny6ptLqfrE=; b=cIqV6Vc3xOBfcUPfDAOPp3iirq/iAkK7plXmczhIYCiCutud7YKuCGmm774Pvy+fAe bmiDj3shSbL0gNPp5NP2SBqCPwSJwGI75fWBzRJM5Q3WQnuuYQezfJ/Vf4E782Ieg4gw f7ZxK9YwH2de3/44CIWIQ/A9Q23No2QTQXdty7/o1+NIKlKBP4yQFqpIN+Aj8pUL/Yoi PiMHrEhCVL9KeNXFRCU42NeXWRS7Xks94MKBujiw9yw9kvVAVIjCDGHBpjBzBbU0L1Uz aPG7PMKvOIP5Vzr68M3jiqiKa2Dy/P4+XP8+dgrTrSQw4Z/zq4DUt1yHVUc3m+aFLFea dYIg== X-Gm-Message-State: AOAM533FMqCbOeM2Ia473eTXR5bvIfDOB904h/GJGZunIihN7uxipWI7 XtuUxxs38nAcWSDgaCiiOhpQWdutVAWE5Li5NuvucOfW X-Google-Smtp-Source: ABdhPJy/6n623QCZ5qbohlFoWrMca8Q8UEAlTkyIap8mq+CRZl+MjoiXLTK9SHtutEfeKWfMk1n9bm1mkpRPma2caiE= X-Received: by 2002:a6b:e20b:: with SMTP id z11mr22713950ioc.2.1593028612978; Wed, 24 Jun 2020 12:56:52 -0700 (PDT) MIME-Version: 1.0 From: L A Walsh Date: Wed, 24 Jun 2020 12:56:41 -0700 Message-ID: Subject: option to truncate at end or should that be default? To: bug-coreutils@gnu.org Content-Type: multipart/alternative; boundary="00000000000098134805a8d9e0b4" Received-SPF: softfail client-ip=209.85.166.42; envelope-from=gsuite@tlinx.org; helo=mail-io1-f42.google.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 15:56:53 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=_AUTOLEARN X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --00000000000098134805a8d9e0b4 Content-Type: text/plain; charset="UTF-8" I allocated a large file of contiguous space (~3.6T), the size of a disk image I was going to copy into it with 'dd'. I have the disk image 'overwrite' the existing file, in place using "conv=nocreat,notrunc" (among other switches) and that works with the final file still using max-sized 8GB extents. I realize that I _do_ want it to truncate the file to the actual size when done. The 'notrunc' switch doesn't work for this purpose as its meaning is overloaded (it really specifies multiple behaviors) both the non-truncation effect, as well as a directive to preserve any blocks not written during that specific invocation of 'dd'. A possible 3rd behavior arises from a vague definition of a block. Do they mean to preserve the data in the block, or do they mean to preserve the position of the block on disk? It seems they mean to preserve data, but whether or not that also preserves a place on disk isn't specified. There really needs to be something to specify that writes occur "in-place" such that no "_RE_-allocation" of blocks occurs (except to extend the file, if needed) A second option would be to truncate the file to the last position written. Maybe a oflag=overwrite, and a 'ftrunc' for 'final trunc' to the position of the final byte written? --00000000000098134805a8d9e0b4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I allocated a large file of contiguous space (~3.6T), th= e size of a disk image I was going to copy into it with 'dd'.=C2=A0= I have the disk image 'overwrite' the existing file, in place usin= g "conv=3Dnocreat,notrunc" (among other switches) and that works = with the final file still using max-sized 8GB extents.

I rea= lize that I _do_ want it to truncate the file to the actual size when done.=

The 'notrunc' switch doesn't work for this purp= ose as its meaning is overloaded (it really specifies multiple behaviors) b= oth the non-truncation effect, as well as a directive to preserve any block= s not written during that specific invocation of 'dd'.=C2=A0 A poss= ible 3rd behavior arises from a vague definition of a block.=C2=A0 Do they = mean to preserve the data in the block, or do they mean to preserve the pos= ition of the block on disk?=C2=A0 It seems they mean to preserve data, but = whether or not that also preserves a place on disk isn't specified.
=
There really needs to be something to specify that writes occur= "in-place" such that no "_RE_-allocation" of blocks oc= curs (except to extend the file, if needed)=C2=A0 A second option would be = to truncate the file to the last position written.


Maybe= a oflag=3Doverwrite, and a 'ftrunc' for 'final trunc' to t= he position of the final byte written?





--00000000000098134805a8d9e0b4-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 24 16:07:30 2020 Received: (at 42034) by debbugs.gnu.org; 24 Jun 2020 20:07:30 +0000 Received: from localhost ([127.0.0.1]:39191 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joBgA-0004PN-L9 for submit@debbugs.gnu.org; Wed, 24 Jun 2020 16:07:30 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:48438) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joBg6-0004PC-W4 for 42034@debbugs.gnu.org; Wed, 24 Jun 2020 16:07:29 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 49sZ1n4SJnz1qs3l; Wed, 24 Jun 2020 22:07:25 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 49sZ1n2rTCz1r56k; Wed, 24 Jun 2020 22:07:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id wDPbB4bNrctQ; Wed, 24 Jun 2020 22:07:24 +0200 (CEST) X-Auth-Info: 0m5nsxui9KTi1w+PNcJiGiIRhO3xIrqDIlhD3cyEjikRuu62ALyD14H270N3JrQo Received: from igel.home (ppp-46-244-162-219.dynamic.mnet-online.de [46.244.162.219]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Wed, 24 Jun 2020 22:07:24 +0200 (CEST) Received: by igel.home (Postfix, from userid 1000) id BCD3F2C0B37; Wed, 24 Jun 2020 22:07:23 +0200 (CEST) From: Andreas Schwab To: L A Walsh Subject: Re: bug#42034: option to truncate at end or should that be default? References: X-Yow: Was my SOY LOAF left out in th'RAIN? It tastes REAL GOOD!! Date: Wed, 24 Jun 2020 22:07:23 +0200 In-Reply-To: (L. A. Walsh's message of "Wed, 24 Jun 2020 12:56:41 -0700") Message-ID: <87h7v0ut1g.fsf@igel.home> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 42034 Cc: 42034@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Jun 24 2020, L A Walsh wrote: > A second option would be to truncate the file to the last position > written. $ truncate -r $src $dest Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 24 18:30:30 2020 Received: (at 42034) by debbugs.gnu.org; 24 Jun 2020 22:30:30 +0000 Received: from localhost ([127.0.0.1]:39315 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joDuX-00020T-PM for submit@debbugs.gnu.org; Wed, 24 Jun 2020 18:30:30 -0400 Received: from havoc.proulx.com ([96.88.95.61]:35126) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joDuV-0001t4-HG for 42034@debbugs.gnu.org; Wed, 24 Jun 2020 18:30:28 -0400 Received: from joseki.proulx.com (localhost [127.0.0.1]) by havoc.proulx.com (Postfix) with ESMTP id EF8074A for <42034@debbugs.gnu.org>; Wed, 24 Jun 2020 16:30:20 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proulx.com; s=dkim2048; t=1593037821; bh=aWOITAfKs2VLqlcsV5ybJL7BPJShaZrypBUV2lgvUHA=; h=Date:From:To:Subject:References:In-Reply-To:From; b=mph/XnOhTE3V5qEZ8ceS7CXmcG43rnu5/g9Jj1PO8boninNZf5HDjYrRYFPqqdfSY vltXOd79CeM0b/BVhp1V89GdyIpz5eZLvvNK0Bdu9fHQc1YOERzzTZRTkWGMmHNF+k S+VzXGkyo4YDYKfSqDoLVZrEhZyoq3CEBVJiqC6/HL3p9nLJP/wIZJ2zR3LBJpdaKJ fQscoct+yEPji416ZNKRjo91X6hBGJTda3+7ctEUdlI1K2Ibsx49hzX524dNWOZxZ2 5wpr/pAgs1oxcHfdLo2EXxZOWKBNJhe4/2vJtGtRcW2me2kTXd4AWw40Y1a9qI7MOd qcct0O5FzUwRw== Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id E97AC21185 for <42034@debbugs.gnu.org>; Wed, 24 Jun 2020 16:30:19 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 7878B2DC8B; Wed, 24 Jun 2020 16:30:18 -0600 (MDT) Date: Wed, 24 Jun 2020 16:30:18 -0600 From: Bob Proulx To: 42034@debbugs.gnu.org Subject: Re: bug#42034: option to truncate at end or should that be default? Message-ID: <20200624162015168087241@bob.proulx.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 42034 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) L A Walsh wrote: > I allocated a large file of contiguous space (~3.6T), the size of a disk > image I was going to copy into it with 'dd'. I have the disk image > 'overwrite' the existing file, in place ... It's possible that you might want to be rescuing data from a failing disk or doing other surgery upon it. Therefore I want to mention ddrescue here. https://www.gnu.org/software/ddrescue/ Of course it all depends upon the use case but ddrescue is a good tool to have in the toolbox. It might be just the right tool. Take for example a RAID1 image on two failing drives that should be identical but both are reporting errors. If the failures do not overlap then ddrescue can be used to merge the successful reads from those two images producing one fully correct image. Bob From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 24 20:09:56 2020 Received: (at 42034) by debbugs.gnu.org; 25 Jun 2020 00:09:56 +0000 Received: from localhost ([127.0.0.1]:39342 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joFSj-0008FH-PC for submit@debbugs.gnu.org; Wed, 24 Jun 2020 20:09:56 -0400 Received: from mail-io1-f41.google.com ([209.85.166.41]:42916) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1joFSf-0008F0-FD for 42034@debbugs.gnu.org; Wed, 24 Jun 2020 20:09:53 -0400 Received: by mail-io1-f41.google.com with SMTP id c16so4167882ioi.9 for <42034@debbugs.gnu.org>; Wed, 24 Jun 2020 17:09:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=w0KQ5ncnaVaaFKkvFCghXYJY17TahrVfE9H2JYOMqkA=; b=hq3Q30ox4FOJwuHXn12DvMAPKCfzJf2pVkT95zfpgYIGPha8NtTqqOgd0tfeJMs+LF +roXTJRFMdQ6q8sK8aip2umlrYXiGOXwbhvG01nrnXHc0F50bnmaNjc8A6XJ9SB9wNS0 zmiPQ29OnzCZS23dYZIn0OpnRNdg/JCS6vIANKe/wRyY52HGsIAzIbawnxCtCsNWtBmw kZJf75qGzu7317uq3qgBMN42EG1EpgaOOlg4Ecg8UNC4Uh7nPC8625uqc5e4ApB0Sj0m YXt7I/ca/Ybe1bdYURGbZDQGnf5kaKyfSf/eUf+HIAr0ViweaEcTyhB8N/xnrz3LX1XZ hTKQ== X-Gm-Message-State: AOAM532N3a2RXE3xBEH/43vmFfsLjmOvzfYJLkXF8NFnYSM1vW6uv4ob RVczGARJmOraZuf17YevoxO2REoDlFFQNU+zgdNVPg== X-Google-Smtp-Source: ABdhPJzcq/KaC5xGGTHYtX4y0fNSDcBqKhnJfzaNM7CYaw0lX8+d9RrbFXdagSUZFqJEp/oYbzjGSQJVSDUiWz51EHU= X-Received: by 2002:a6b:c388:: with SMTP id t130mr17830525iof.89.1593043784019; Wed, 24 Jun 2020 17:09:44 -0700 (PDT) MIME-Version: 1.0 References: <87h7v0ut1g.fsf@igel.home> In-Reply-To: <87h7v0ut1g.fsf@igel.home> From: L A Walsh Date: Wed, 24 Jun 2020 17:09:30 -0700 Message-ID: Subject: Re: bug#42034: option to truncate at end or should that be default? To: Andreas Schwab Content-Type: multipart/alternative; boundary="000000000000dbc41d05a8dd684a" X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 42034 Cc: 42034@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --000000000000dbc41d05a8dd684a Content-Type: text/plain; charset="UTF-8" I think that would work in my specific use case. I can think of other use cases that it probably wouldn't, but I'm not going to worry about those right now. :-) Thanks! -l On Wed, Jun 24, 2020 at 1:07 PM Andreas Schwab wrote: > On Jun 24 2020, L A Walsh wrote: > > > A second option would be to truncate the file to the last position > > written. > > $ truncate -r $src $dest > > Andreas. > > -- > Andreas Schwab, schwab@linux-m68k.org > GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > "And now for something completely different." > --000000000000dbc41d05a8dd684a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I think that would work in my specific use case.=C2=A0 <= br>
I can think of other use cases that it probably wouldn't, bu= t I'm not going to worry
about those right now.=C2=A0 :-)
Thanks!
-l




On = Wed, Jun 24, 2020 at 1:07 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
On Jun 24 2020, L A Walsh wrote:

> A second option would be to truncate the file to the last position
> written.

$ truncate -r $src $dest

Andreas.

--
Andreas Schwab, = schwab@linux-m68k.org
GPG Key fingerprint =3D 7578 EB47 D4E5 4D69 2510=C2=A0 2552 DF73 E780 A9DA = AEC1
"And now for something completely different."
--000000000000dbc41d05a8dd684a--