Subject: bug#51482: closed (Re: bug#51482: dd with status=progress does not update its output when the main writing is finished)

From: [email-masked] (GNU bug Tracking System)
Date: Fri, 28 Jan 2022 02:39:01 +0000

Your bug report

#51482: dd with status=progress does not update its output when the main writing is finished

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
51482: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=51482
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#51482: dd with status=progress does not update its output when the main writing is finished
From: Paul Eggert <[email-masked]>
To: Sworddragon <[email-masked]>
Cc: [email-masked]
Date: Thu, 27 Jan 2022 18:38:44 -0800
List-Id: <debbugs-submit.debbugs.gnu.org>

On 10/29/21 07:22, Sworddragon wrote:

> When dd is being used with status=progress it appears to update the status
> every second but does not do a final update when dd finished its main
> writing task (e.g. when dd starts flushing via conv=fsync and it still
> blocks for like over a minute) causing the output to be incorrect and
> inconsistent across multiple tries.

Thanks for mentioning that. I installed the attached to implement your 
suggestion, and am boldly closing the bug report. Please give it a try 
when you have the chance.

From 4cda71156464d20789bac5b31f6a1ea36b183edd Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Thu, 27 Jan 2022 18:34:09 -0800
Subject: [PATCH] dd: output final progress before syncing

Problem reported by Sworddragon (Bug#51482).
* src/dd.c (reported_w_bytes): New var.
(print_xfer_stats): Set it.
(dd_copy): Print a final progress report if useful before
synchronizing output data.
---
 NEWS     |  4 ++++
 src/dd.c | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/NEWS b/NEWS
index 561087ccc..15c9428bd 100644
--- a/NEWS
+++ b/NEWS
@@ -58,6 +58,10 @@ GNU coreutils NEWS                                    -*- outline -*-
 
   The new 'date' option --resolution outputs the timestamp resolution.
 
+  With conv=fdatasync or conv=fsync, dd status=progress now reports
+  any extra final progress just before synchronizing output data,
+  since synchronizing can take a long time.
+
   sort --debug now diagnoses issues with --field-separator characters
   that conflict with characters possibly used in numbers.
 
diff --git a/src/dd.c b/src/dd.c
index a6a3708f1..957ad129e 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -196,6 +196,9 @@ static intmax_t r_full = 0;
 /* Number of bytes written.  */
 static intmax_t w_bytes = 0;
 
+/* Last-reported number of bytes written, or negative if never reported.  */
+static intmax_t reported_w_bytes = -1;
+
 /* Time that dd started.  */
 static xtime_t start_time;
 
@@ -815,6 +818,8 @@ print_xfer_stats (xtime_t progress_time)
     }
   else
     fputc ('\n', stderr);
+
+  reported_w_bytes = w_bytes;
 }
 
 static void
@@ -2365,6 +2370,13 @@ dd_copy (void)
         }
     }
 
+  /* fdatasync/fsync can take a long time, so issue a final progress
+     indication now if progress has been made since the previous indication.  */
+  if (conversions_mask & (C_FDATASYNC | C_FSYNC)
+      && status_level == STATUS_PROGRESS
+      && 0 <= reported_w_bytes && reported_w_bytes < w_bytes)
+    print_xfer_stats (0);
+
   if ((conversions_mask & C_FDATASYNC) && ifdatasync (STDOUT_FILENO) != 0)
     {
       if (errno != ENOSYS && errno != EINVAL)
-- 
2.32.0

Subject: dd with status=progress does not update its output when the main writing is finished
From: Sworddragon <[email-masked]>
To: [email-masked]
Date: Fri, 29 Oct 2021 14:22:02 +0000
List-Id: <debbugs-submit.debbugs.gnu.org>

Originally found as additional minor bug in ticket #51345 (Knoppix 9.1 with
GNU Coreutils 8.32), I think it is a good idea to create a dedicated report
for it to avoid getting it lost.

When dd is being used with status=progress it appears to update the status
every second but does not do a final update when dd finished its main
writing task (e.g. when dd starts flushing via conv=fsync and it still
blocks for like over a minute) causing the output to be incorrect and
inconsistent across multiple tries.