GNU bug report logs - #71017
fill-flowed-encode

Previous Next

To reply to this bug, email your comments to 71017 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: bug-gnu-emacs <at> gnu.org
Subject: fill-flowed-encode
Date: Fri, 17 May 2024 22:23:50 +0200

Hi y'all.

flow-fill.el.gz has a pair of functions, fill-flowed-encode and 
fill-flowed-fill-buffer (the latter is only called from the 
former).

Here is a fixed version of the former that then also deprecates 
the latter (I have signed FSF copyright papers):

(defun fill-flowed-encode (&optional buffer)
 (with-current-buffer (or buffer (current-buffer))
   ;; No point in doing this unless hard newlines is used.
   (when use-hard-newlines
     (let ((start (point-min)) end)
	;; Go through each paragraph, filling it and adding SPC
	;; as the last character on each line.
	(while (and (< start (point-max))
		    (setq end (or (text-property-any start (point-max) 'hard 't)
				  (point-max))))
	  (save-restriction
	    (narrow-to-region start end)
	    (let ((fill-column (eval fill-flowed-encode-column t))
		  (prefix
		   (concat "\n"
			   (or (and (looking-at ">[> ]*")
				    (match-string 0)) ""))))
	      (while (search-forward prefix nil t)
		(replace-match " " t t))
	      (goto-char start)
	      (while (< (+ (point) fill-column) (point-max))
		(forward-char fill-column)
		(search-backward " ")
		(forward-char)
		(insert prefix)))
	    (setq start (1+ (point-max))))))
     t)))

This fixes two bugs when sending RFC 2646–formatted email.

First, the old code didn't refill or encode the last paragraph at 
all unless there was at least one hard newline EOF.

Second, the old code borked up code indented with tabs and spaces 
(iff that code had overly long lines), such as the Lisp code in 
this email. It could sometimes insert extra in the middle of such 
long lines.

Here is an example of what it would do. It would turn this:

(defun lorem (ipsum)
 (dolor sit amet)
 (consectetur adipiscing elit (sed do eiusmod tempor incididunt 
ut labore et dolore magna aliqua))
 (ut enim ad minim veniam
     (quis nostrud exercitation ullamco laboris nisi ut aliquip 
ex ea commodo consequat
	    (duis aute irure dolor in reprehenderit in
		  voluptate velit esse cillum dolore
		  eu fugiat nulla pariatur)
	    excepteur sint occaecat cupidatat non proident
	    (sunt in culpa qui officia deserunt mollit anim id est 
laborumd))))

into this:

(defun lorem (ipsum)
 (dolor sit amet)
 (consectetur adipiscing elit (sed do eiusmod tempor incididunt
 ut labore et dolore magna aliqua))
 (ut enim ad minim veniam
     (quis nostrud exercitation ullamco laboris nisi ut aliquip
     ex ea commodo consequat
	    (duis aute irure dolor in reprehenderit in
		  voluptate velit esse cillum dolore
		  eu fugiat nulla pariatur)
	    excepteur sint occaecat cupidatat non proident
	    (sunt in culpa qui officia deserunt mollit anim id est
	    laborumd))))

It was breaking lines awkwardly so when they're reconnected they 
have extra whitespace in the mkddle of lines. 

Message #8 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sat, 29 Jun 2024 22:44:09 -0700

Sandra Snan <sandra.snan <at> idiomdrottning.org> writes:

> Hi y'all.

Hi Sandra,

> flow-fill.el.gz has a pair of functions, fill-flowed-encode and
> fill-flowed-fill-buffer (the latter is only called from the former).
>
> Here is a fixed version of the former that then also deprecates the latter (I
> have signed FSF copyright papers):
>
> (defun fill-flowed-encode (&optional buffer)
>  (with-current-buffer (or buffer (current-buffer))
>    ;; No point in doing this unless hard newlines is used.
>    (when use-hard-newlines
>      (let ((start (point-min)) end)
> 	;; Go through each paragraph, filling it and adding SPC
> 	;; as the last character on each line.
> 	(while (and (< start (point-max))
> 		    (setq end (or (text-property-any start (point-max) 'hard 't)
> 				  (point-max))))
> 	  (save-restriction
> 	    (narrow-to-region start end)
> 	    (let ((fill-column (eval fill-flowed-encode-column t))
> 		  (prefix
> 		   (concat "\n"
> 			   (or (and (looking-at ">[> ]*")
> 				    (match-string 0)) ""))))
> 	      (while (search-forward prefix nil t)
> 		(replace-match " " t t))
> 	      (goto-char start)
> 	      (while (< (+ (point) fill-column) (point-max))
> 		(forward-char fill-column)
> 		(search-backward " ")
> 		(forward-char)
> 		(insert prefix)))
> 	    (setq start (1+ (point-max))))))
>      t)))
>
> This fixes two bugs when sending RFC 2646–formatted email.
>
> First, the old code didn't refill or encode the last paragraph at all unless
> there was at least one hard newline EOF.
>
> Second, the old code borked up code indented with tabs and spaces (iff that code
> had overly long lines), such as the Lisp code in this email. It could sometimes
> insert extra in the middle of such long lines.
>
> Here is an example of what it would do. It would turn this:
>
> (defun lorem (ipsum)
>  (dolor sit amet)
>  (consectetur adipiscing elit (sed do eiusmod tempor incididunt ut labore et
> dolore magna aliqua))
>  (ut enim ad minim veniam
>      (quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
>     consequat
> 	    (duis aute irure dolor in reprehenderit in
> 		  voluptate velit esse cillum dolore
> 		  eu fugiat nulla pariatur)
> 	    excepteur sint occaecat cupidatat non proident
> 	    (sunt in culpa qui officia deserunt mollit anim id est laborumd))))
>
> into this:
>
> (defun lorem (ipsum)
>  (dolor sit amet)
>  (consectetur adipiscing elit (sed do eiusmod tempor incididunt
>  ut labore et dolore magna aliqua))
>  (ut enim ad minim veniam
>      (quis nostrud exercitation ullamco laboris nisi ut aliquip
>      ex ea commodo consequat
> 	    (duis aute irure dolor in reprehenderit in
> 		  voluptate velit esse cillum dolore
> 		  eu fugiat nulla pariatur)
> 	    excepteur sint occaecat cupidatat non proident
> 	    (sunt in culpa qui officia deserunt mollit anim id est
> 	    laborumd))))
>
> It was breaking lines awkwardly so when they're reconnected they have extra
> whitespace in the mkddle of lines.

Could you please resend the above as patches formatted by

    git format-patch -1

?

It will make it easier for us to review and install the proposed
changes.  Thanks in advance.

Message #11 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sun, 30 Jun 2024 10:32:34 +0200

Stefan Kangas <stefankangas <at> gmail.com> writes:

> Could you please resend the above as patches formatted by
>
>     git format-patch -1
>
> ?
>

I'm glad I'm gonna get a chance to resend because I've fixed some 
more bugs in the version I've been dogfooding since I first sent 
that so I wanna send an updated version that has that.

Normally when I'm sending Emacs lisp patches, it's on packages and 
they're usually in git already. Here it's a core file so the 
question I've got is what is the git repo I should make the change 
in? Then I could git send-email to this In-Reply-To thread id.

Stefan Kangas <stefankangas <at> gmail.com> writes:

> Sandra Snan <sandra.snan <at> idiomdrottning.org> writes:
>
>> Hi y'all.
>
> Hi Sandra,
>
>> flow-fill.el.gz has a pair of functions, fill-flowed-encode and 
>> fill-flowed-fill-buffer (the latter is only called from the 
>> former).
>> Here is a fixed version of the former that then also 
>> deprecates the latter (I have signed FSF copyright papers):
>> 
>> (defun fill-flowed-encode (&optional buffer)  (with-current-buffer 
>> (or buffer (current-buffer))    ;; No point in doing this unless 
>> hard newlines is used.    (when use-hard-newlines      (let ((start 
>> (point-min)) end) 	;; Go through each paragraph, filling it and 
>> adding SPC 	;; as the last character on each line. 	(while (and (< 
>> start (point-max)) 		    (setq end (or (text-property-any start 
>> (point-max) 'hard 't) 				  (point-max)))) 	  (save-restriction 	   
>>  (narrow-to-region start end) 	    (let ((fill-column (eval 
>> fill-flowed-encode-column t)) 		  (prefix 		   (concat "\n" 			   
>> (or (and (looking-at ">[> ]*") 				    (match-string 0)) "")))) 	   
>>    (while (search-forward prefix nil t) 		(replace-match " " t t)) 
>> 	      (goto-char start) 	      (while (< (+ (point) fill-column) 
>> (point-max)) 		(forward-char fill-column) 		(search-backward " ") 
>> 		(forward-char) 		(insert prefix))) 	    (setq start (1+ 
>> (point-max))))))      t)))
>> This fixes two bugs when sending RFC 
>> 2646–formatted email.
>> First, the old code didn't refill or 
>> encode the last paragraph at all unless there was at least one hard 
>> newline EOF.
>> Second, the old code borked up code indented with 
>> tabs and spaces (iff that code had overly long lines), such as the 
>> Lisp code in this email. It could sometimes insert extra in the 
>> middle of such long lines.
>> Here is an example of what it would 
>> do. It would turn this:
>> (defun lorem (ipsum)  (dolor sit amet)  
>> (consectetur adipiscing elit (sed do eiusmod tempor incididunt ut 
>> labore et dolore magna aliqua))  (ut enim ad minim veniam      
>> (quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea 
>> commodo     consequat 	    (duis aute irure dolor in reprehenderit 
>> in 		  voluptate velit esse cillum dolore 		  eu fugiat nulla 
>> pariatur) 	    excepteur sint occaecat cupidatat non proident 	    
>> (sunt in culpa qui officia deserunt mollit anim id est 
>> laborumd))))
>> into this:
>> (defun lorem (ipsum)  (dolor sit 
>> amet)  (consectetur adipiscing elit (sed do eiusmod tempor 
>> incididunt  ut labore et dolore magna aliqua))  (ut enim ad minim 
>> veniam      (quis nostrud exercitation ullamco laboris nisi ut 
>> aliquip      ex ea commodo consequat 	    (duis aute irure dolor in 
>> reprehenderit in 		  voluptate velit esse cillum dolore 		  eu 
>> fugiat nulla pariatur) 	    excepteur sint occaecat cupidatat non 
>> proident 	    (sunt in culpa qui officia deserunt mollit anim id 
>> est 	    laborumd))))
>> It was breaking lines awkwardly so when 
>> they're reconnected they have extra whitespace in the mkddle of 
>> lines.
>
> Could you please resend the above as patches formatted by
>
>     git format-patch -1
>
> ?
>
> It will make it easier for us to review and install the proposed 
> changes.  Thanks in advance.

Message #14 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sun, 30 Jun 2024 07:34:21 -0700

Sandra Snan <sandra.snan <at> idiomdrottning.org> writes:

> I'm glad I'm gonna get a chance to resend because I've fixed some
> more bugs in the version I've been dogfooding since I first sent
> that so I wanna send an updated version that has that.

Great, thanks.

> Normally when I'm sending Emacs lisp patches, it's on packages and
> they're usually in git already. Here it's a core file so the
> question I've got is what is the git repo I should make the change
> in? Then I could git send-email to this In-Reply-To thread id.

https://git.savannah.gnu.org/cgit/emacs.git

Please have a look at the CONTRIBUTE file in the Emacs source
distribution.

https://git.savannah.gnu.org/cgit/emacs.git/tree/CONTRIBUTE

Message #17 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: 71017 <at> debbugs.gnu.org
Cc: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Subject: [PATCH] Flow single-paragraph messages
Date: Sat,  6 Jul 2024 22:49:50 +0200

This fixes two bugs when sending RFC 2646–formatted email.

First, the old code didn't refill or encode the last paragraph at all
unless there was at least one hard newline EOF. This was a bee to track
down because there were two separate issues at play. One was a a bug in
flow-fill.el where every paragraph except the last paragraph was
reflowed, but the last paragraph would stay hardwrapped. Manually
placing a hard newline at the end of the file was a workaround but I
don't always remember to do that. I managed to fix that bug a few months
ago.

Second, the old code borked up code indented with tabs and spaces
(iff that code had overly long lines), such as Lisp code. It could
sometimes insert extra whitespace in the middle of such long lines.
I fixed that bug shortly after the first one.

But for months dogfooding those two changes, sometimes a hardwrapped
email would still be sent. I finally managed to debug and figure it out
and it took all day. Turns out mml just plain didn't call the
fill-flowed-encode function if the message doesn't have any hard
newlines (newlines with the hard text property). Well, of course a
single-paragraph email isn't gonna have any hard newlines! But it still
needs reflowing!

So I've now changed that and updated the documentation to match those
news semantics. I went all the way, but a possible compromise might be
to not-flow a message that has \n\n but no hard text props, since that's
a sign that something is wrong.

Since the use-hard-newlines variable is buffer local and all this
reflowing is being done in a temp buffer, that variable is more than
useless so I've removed references to it.
---
 doc/misc/emacs-mime.texi |  7 ++---
 lisp/gnus/mml.el         | 29 ++++++++----------
 lisp/mail/flow-fill.el   | 65 +++++++++++++++++-----------------------
 3 files changed, 41 insertions(+), 60 deletions(-)

diff --git a/doc/misc/emacs-mime.texi b/doc/misc/emacs-mime.texi
index ef7ea61..7621a9a 100644
--- a/doc/misc/emacs-mime.texi
+++ b/doc/misc/emacs-mime.texi
@@ -1087,13 +1087,10 @@ terminated by soft newline characters are filled together and wrapped
 after the column decided by @code{fill-flowed-encode-column}.
 Quotation marks (matching @samp{^>* ?}) are respected.  The variable
 controls how the text will look in a client that does not support
-flowed text, the default is to wrap after 66 characters.  If hard
-newline characters are not present in the buffer, no flow encoding
-occurs.
+flowed text, the default is to wrap after 66 characters.
 
 You can customize the value of the @code{mml-enable-flowed} variable
-to enable or disable the flowed encoding usage when newline
-characters are present in the buffer.
+to enable or disable the flowed encoding usage.
 
 On decoding flowed text, lines with soft newline characters are filled
 together and wrapped after the column decided by
diff --git a/lisp/gnus/mml.el b/lisp/gnus/mml.el
index e3bc393..2db39dc 100644
--- a/lisp/gnus/mml.el
+++ b/lisp/gnus/mml.el
@@ -691,23 +691,18 @@ type detected."
 		   (t
 		    ;; Only perform format=flowed filling on text/plain
 		    ;; parts where there either isn't a format parameter
-		    ;; in the mml tag or it says "flowed" and there
-		    ;; actually are hard newlines in the text.
-		    (let (use-hard-newlines)
-		      (when (and mml-enable-flowed
-				 (string= type "text/plain")
-				 (not (string= (cdr (assq 'sign cont)) "pgp"))
-				 (or (null (assq 'format cont))
-				     (string= (cdr (assq 'format cont))
-					      "flowed"))
-				 (setq use-hard-newlines
-				       (text-property-any
-					(point-min) (point-max) 'hard 't)))
-			(fill-flowed-encode)
-			;; Indicate that `mml-insert-mime-headers' should
-			;; insert a "; format=flowed" string unless the
-			;; user has already specified it.
-			(setq flowed (null (assq 'format cont)))))
+		    ;; in the mml tag or it says "flowed".
+		    (when (and mml-enable-flowed
+			       (string= type "text/plain")
+			       (not (string= (cdr (assq 'sign cont)) "pgp"))
+			       (or (null (assq 'format cont))
+				   (string= (cdr (assq 'format cont))
+					    "flowed")))
+		      (fill-flowed-encode)
+		      ;; Indicate that `mml-insert-mime-headers' should
+		      ;; insert a "; format=flowed" string unless the
+		      ;; user has already specified it.
+		      (setq flowed (null (assq 'format cont))))
 		    ;; Prefer `utf-8' for text/calendar parts.
 		    (if (or charset
 			    (not (string= type "text/calendar")))
diff --git a/lisp/mail/flow-fill.el b/lisp/mail/flow-fill.el
index 919490e..5c9ae21 100644
--- a/lisp/mail/flow-fill.el
+++ b/lisp/mail/flow-fill.el
@@ -73,50 +73,39 @@ RFC 2646 suggests 66 characters for readability."
 ;;;###autoload
 (defun fill-flowed-encode (&optional buffer)
   (with-current-buffer (or buffer (current-buffer))
-    ;; No point in doing this unless hard newlines is used.
-    (when use-hard-newlines
-      (let ((start (point-min)) end)
-	;; Go through each paragraph, filling it and adding SPC
-	;; as the last character on each line.
-	(while (setq end (text-property-any start (point-max) 'hard 't))
-	  (save-restriction
-	    (narrow-to-region start end)
-	    (let ((fill-column (eval fill-flowed-encode-column t)))
-	      (fill-flowed-fill-buffer))
-	    (goto-char (point-min))
-	    (while (re-search-forward "\n" nil t)
-	      (replace-match " \n" t t))
-	    (goto-char (setq start (1+ (point-max)))))))
-      t)))
-
-(defun fill-flowed-fill-buffer ()
-  (let ((prefix nil)
-	(prev-prefix nil)
-	(start (point-min)))
-    (goto-char (point-min))
-    (while (not (eobp))
-      (setq prefix (and (looking-at "[> ]+")
-			(match-string 0)))
-      (if (equal prefix prev-prefix)
-	  (forward-line 1)
+    (let ((fill-column (eval fill-flowed-encode-column t))
+	  (start (point-min))
+	  end)
+      ;; Go through each paragraph, filling it and adding SPC
+      ;; as the last character on each line.
+      (while (and (< start (point-max))
+		  (setq end (or (text-property-any start (point-max) 'hard 't)
+				(point-max))))
 	(save-restriction
-	  (narrow-to-region start (point))
-	  (let ((fill-prefix prev-prefix))
-	    (fill-region (point-min) (point-max) t 'nosqueeze 'to-eop))
-	  (goto-char (point-max)))
-	(setq prev-prefix prefix
-	      start (point))))
-    (save-restriction
-      (narrow-to-region start (point))
-      (let ((fill-prefix prev-prefix))
-	(fill-region (point-min) (point-max) t 'nosqueeze 'to-eop)))))
+	  (narrow-to-region start end)
+	  (let ((prefix
+		 (concat "\n"
+			 (or (and (looking-at ">[> ]*")
+				  (match-string 0)) ""))))
+	    (goto-char start)
+	    (while (search-forward prefix nil t)
+	      (replace-match " " t t))
+	    (goto-char start)
+	    (while (< (+ (point) fill-column) (point-max))
+	      (let ((start (point)))
+		(forward-char fill-column)
+		(when (search-backward " " start t)
+		  (forward-char)
+		  (insert prefix)))))
+	  (setq start (1+ (point-max))))))
+    t))
 
 ;;;###autoload
 (defun fill-flowed (&optional buffer delete-space)
   "Apply RFC2646 decoding to BUFFER.
 If BUFFER is nil, default to the current buffer.
 
-If DELETE-SPACE, delete RFC2646 spaces padding at the end of
+If DELETE-SPACE, delete RFC3676 spaces padding at the end of
 lines."
   (with-current-buffer (or buffer (current-buffer))
     (let ((fill-column  (eval fill-flowed-display-column t)))
@@ -154,7 +143,7 @@ lines."
           ;; Delete the newline.
           (when (eq (following-char) ?\s)
             (delete-char 1))
-          ;; Hack: Don't do the flowing on the signature line.
+          ;; As per RFC3767: Don't do the flowing on the signature line.
           (when (and (not (looking-at "-- $"))
                      (eq (char-before (line-end-position)) ?\s))
             (while (and (not (eobp))
-- 
2.39.2

Message #20 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>,
 Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: [PATCH] Flow single-paragraph messages
Date: Sun, 07 Jul 2024 08:43:57 +0300

> Cc: Sandra Snan <sandra.snan <at> idiomdrottning.org>
> Date: Sat,  6 Jul 2024 22:49:50 +0200
> From:  Sandra Snan via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> This fixes two bugs when sending RFC 2646–formatted email.

Thanks.  (I also see RFC 3676 mentioned -- what is this about?)

> First, the old code didn't refill or encode the last paragraph at all
> unless there was at least one hard newline EOF.

Isn't this the documented behavior?

> But for months dogfooding those two changes, sometimes a hardwrapped
> email would still be sent. I finally managed to debug and figure it out
> and it took all day. Turns out mml just plain didn't call the
> fill-flowed-encode function if the message doesn't have any hard
> newlines (newlines with the hard text property). Well, of course a
> single-paragraph email isn't gonna have any hard newlines! But it still
> needs reflowing!
> 
> So I've now changed that and updated the documentation to match those
> news semantics. I went all the way, but a possible compromise might be
> to not-flow a message that has \n\n but no hard text props, since that's
> a sign that something is wrong.

The change seems to be an incompatible behavior change, so I wonder
whether we'd need some way for users to get back old behavior.  Eric,
WDYT?

I'll leave it to Eric to comment on the code changes.

Message #23 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: [PATCH] Flow single-paragraph messages
Date: Sun, 07 Jul 2024 10:34:01 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> Thanks.  (I also see RFC 3676 mentioned -- what is this about?)

I mentioned RFC 3676 twice for different reasons.

First, it's introduced the delsp parameter which wasn't in 2646 so 
code related to that parameter shouldn't talk about 2646 but rather 
3676.

Second, I saw a comment referring to not tampering with the sig 
line as a hack. I happened to have the section of RFC 3676 open 
that mandates that behavior so I changed the comment, however, this 
might be in RFC 2646 also, I don't know that, so in that case that 
comment might better change to refer to 2646 instead.

>> First, the old code didn't refill or encode the last paragraph 
>> at all unless there was at least one hard newline EOF.
>
> Isn't this the documented behavior?

Re multi-paragraph messages:

No, it's not the documented behavior, it was an unrelated bug in 
fill-flowed-encode. It would refill all the other paragraphs, 
separated by hard newlines, just not the last one. That was a bug 
and broke documented behavior. I fixed that bug + another unrelated 
reflow bug. That was in fill-flowed-encode and that bugfix doesn't 
rely on the change in mml. With my fix in fill-flowed-encode, 
multi-paragraph-messages started working fine.

Re single-paragraph messages:

However, according to the old documented behavior, a message that 
contained no hard newlines should not be refilled. This documented 
behavior in mml-generate-mime-1 meant that single-paragraph 
messages would not be filled even with the fill-flowed-encode bug 
fixed. That is an unintended bad consequence of the documented 
behavior, a "bug in the design".

I did change that but I updated the documentation to match.

> The change seems to be an incompatible behavior change, so I 
> wonder whether we'd need some way for users to get back old 
> behavior.

There is still the (neglected) defcustom mml-enable-flowed which 
now becomes more relevant since it's a way to turn off all this 
meddling and reflowing in the first place.

The old documented behavior was bugged-by-design. It's not right 
that single paragraph messages are hardwrapped and not reflowed. In 
my day-to-day I write many messages in Emacs that I later see in 
threads in another MUA (Delta Chat) and these messages stand out in 
a way that something is wrong with them.

However, one intent behind the old behavior was, in spirit, good: 
It'd be good to detect whether or not users with mml-enable-flowed 
on have remembered to also turn on use-hard-newlines, which is 
important for users with that on to do, especially since 
mml-enable-flowed defaults to t. The old attempt at doing that was 
flawed since it only worked reliably for multi-paragraph messages.

Unfortunately there's currently no way to detect in a 
single-paragraph message whether or not use-hard-newlines have been 
turned on, since the variable it sets is buffer local. (One 
extremely klugy workaround would be to change the 
message-send-and-exit command to check whether use-hard-newlines is 
on and if it is, add an extra hard newline EOT just for detecting 
this. Not super into that solution so hopefully there are other 
ways.)

With this patch, the defcustom mml-enable-flowed becomes _the_ 
setting for this, which does match a lot of documentation on the 
books. Perhaps it shall no longer default to t though since it 
completely borks messages up if it's t but use-hard-newlines are 
not on!

So here we are:

The new behavior has a problem: messages will get reflowed if 
mml-enable-flowed is t (the default!) even when use-hard-newlines 
is off, meaning that even separate "\n\n" paragraphs will get 
flowed together which is not what people want. Use-hard-newlines 
should be mandatory whenever mml-enable-flowed is on.

The old behavior is not OK since single-paragraph messages will get 
messed up, hardwrapped even when those newlines were advertised as 
"soft", or not softwrapped even when the paragraph consists of just 
one single super long unbroken line.

I also have sent a patch to the messages-are-flowing project 
highlighting the importance of this variable:

https://github.com/legoscia/messages-are-flowing/pull/15/commits/ae432723c2565ceced5d01d9aa2d314bd42aaa3c

So how about this idea:

If mml-enable-flowed is on but fill-flowed-encode is asked to flow 
a message that doesn't have any hard newlines, assume Markdown 
semantics, i.e. special treatment for "\n\n+", "  $", and "^    ". 
I'll see if I have time to implement that this morning. I think I'd 
place that change in fill-flowed-encode.

That wouldn't affect people with mml-enable-flowed off, or people 
with both mml-enable-flowed and use-hard-newlines on, it'd just be 
a sort of DWIM fallback based on the guess that markdown semantics 
are somewhat widely known or expected in 2024, to prevent separate 
paragraphs to be flowed together for people with "incorrect" 
settings.

Again, if there were a cross-buffer way to reliably detect whether 
use-hard-newlines is on, that dwimmy fallback wouldn't be needed.

I thought about whether it'd have been better if it was instead 
soft newlines that were marked with a text property, not hard ones, 
but in the end that wouldn't properly softwrap messages with just 
one single overly long line. 

Message #26 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: 71017 <at> debbugs.gnu.org
Cc: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Subject: [PATCH] Flow single-paragraph messages
Date: Sun,  7 Jul 2024 11:04:48 +0200

See discussion in #71017.

In short:

if mml-flowed-enable is nil: no change to behavior

if mml-flowed-enable is t and use-hard-newlines is on: several bugs
fixed, some (multi-paragraph messages) of which were counter to
documentation and some (single-paragraph messages) that were
along documentation lines but which messed up messages royally.

if mml-flowed-enable is t but use-hard-newlines was accidentally
forgotten by user: new behavior; reflow but don't reflow together
separate paragraphs nor reflow across lines that markdown would've
considered hard.
---
 doc/misc/emacs-mime.texi |  7 +---
 lisp/gnus/mml.el         | 29 ++++++--------
 lisp/mail/flow-fill.el   | 86 ++++++++++++++++++++++------------------
 3 files changed, 62 insertions(+), 60 deletions(-)

diff --git a/doc/misc/emacs-mime.texi b/doc/misc/emacs-mime.texi
index ef7ea61..7621a9a 100644
--- a/doc/misc/emacs-mime.texi
+++ b/doc/misc/emacs-mime.texi
@@ -1087,13 +1087,10 @@ terminated by soft newline characters are filled together and wrapped
 after the column decided by @code{fill-flowed-encode-column}.
 Quotation marks (matching @samp{^>* ?}) are respected.  The variable
 controls how the text will look in a client that does not support
-flowed text, the default is to wrap after 66 characters.  If hard
-newline characters are not present in the buffer, no flow encoding
-occurs.
+flowed text, the default is to wrap after 66 characters.
 
 You can customize the value of the @code{mml-enable-flowed} variable
-to enable or disable the flowed encoding usage when newline
-characters are present in the buffer.
+to enable or disable the flowed encoding usage.
 
 On decoding flowed text, lines with soft newline characters are filled
 together and wrapped after the column decided by
diff --git a/lisp/gnus/mml.el b/lisp/gnus/mml.el
index e3bc393..2db39dc 100644
--- a/lisp/gnus/mml.el
+++ b/lisp/gnus/mml.el
@@ -691,23 +691,18 @@ type detected."
 		   (t
 		    ;; Only perform format=flowed filling on text/plain
 		    ;; parts where there either isn't a format parameter
-		    ;; in the mml tag or it says "flowed" and there
-		    ;; actually are hard newlines in the text.
-		    (let (use-hard-newlines)
-		      (when (and mml-enable-flowed
-				 (string= type "text/plain")
-				 (not (string= (cdr (assq 'sign cont)) "pgp"))
-				 (or (null (assq 'format cont))
-				     (string= (cdr (assq 'format cont))
-					      "flowed"))
-				 (setq use-hard-newlines
-				       (text-property-any
-					(point-min) (point-max) 'hard 't)))
-			(fill-flowed-encode)
-			;; Indicate that `mml-insert-mime-headers' should
-			;; insert a "; format=flowed" string unless the
-			;; user has already specified it.
-			(setq flowed (null (assq 'format cont)))))
+		    ;; in the mml tag or it says "flowed".
+		    (when (and mml-enable-flowed
+			       (string= type "text/plain")
+			       (not (string= (cdr (assq 'sign cont)) "pgp"))
+			       (or (null (assq 'format cont))
+				   (string= (cdr (assq 'format cont))
+					    "flowed")))
+		      (fill-flowed-encode)
+		      ;; Indicate that `mml-insert-mime-headers' should
+		      ;; insert a "; format=flowed" string unless the
+		      ;; user has already specified it.
+		      (setq flowed (null (assq 'format cont))))
 		    ;; Prefer `utf-8' for text/calendar parts.
 		    (if (or charset
 			    (not (string= type "text/calendar")))
diff --git a/lisp/mail/flow-fill.el b/lisp/mail/flow-fill.el
index 919490e..932aa8f 100644
--- a/lisp/mail/flow-fill.el
+++ b/lisp/mail/flow-fill.el
@@ -73,50 +73,60 @@ RFC 2646 suggests 66 characters for readability."
 ;;;###autoload
 (defun fill-flowed-encode (&optional buffer)
   (with-current-buffer (or buffer (current-buffer))
-    ;; No point in doing this unless hard newlines is used.
-    (when use-hard-newlines
-      (let ((start (point-min)) end)
-	;; Go through each paragraph, filling it and adding SPC
-	;; as the last character on each line.
-	(while (setq end (text-property-any start (point-max) 'hard 't))
-	  (save-restriction
-	    (narrow-to-region start end)
-	    (let ((fill-column (eval fill-flowed-encode-column t)))
-	      (fill-flowed-fill-buffer))
-	    (goto-char (point-min))
-	    (while (re-search-forward "\n" nil t)
-	      (replace-match " \n" t t))
-	    (goto-char (setq start (1+ (point-max)))))))
-      t)))
-
-(defun fill-flowed-fill-buffer ()
-  (let ((prefix nil)
-	(prev-prefix nil)
-	(start (point-min)))
-    (goto-char (point-min))
-    (while (not (eobp))
-      (setq prefix (and (looking-at "[> ]+")
-			(match-string 0)))
-      (if (equal prefix prev-prefix)
-	  (forward-line 1)
+    (let ((fill-column (eval fill-flowed-encode-column t))
+	  (start (point-min))
+	  (end (point-max)))
+      ;; Only when we've been called to reflow a buffer that doesn't
+      ;; have any hard newlines:
+      (unless (text-property-any start end 'hard 't)
+	(save-excursion
+	  ;; Harden between paras:
+	  (goto-char start)
+	  (while (search-forward "\n\n" end t)
+	    (set-hard-newline-properties
+	     (- (point) 2) (point)))
+	  ;; Harden "  $"
+	  (goto-char start)
+	  (while (search-forward "  \n" end t)
+	    (backward-char)
+	    (backward-delete-char 2)
+	    (set-hard-newline-properties
+	     (point) (1+ (point))))
+	  ;; Harden "^    "
+	  (goto-char start)
+	  (while (search-forward "\n    " end t)
+	    (set-hard-newline-properties
+	     (- (point) 5) (- (point) 4)))))
+      ;; Go through each paragraph, filling it and adding SPC
+      ;; as the last character on each line.
+      (while (and (< start (point-max))
+		  (setq end (or (text-property-any start (point-max) 'hard 't)
+				(point-max))))
 	(save-restriction
-	  (narrow-to-region start (point))
-	  (let ((fill-prefix prev-prefix))
-	    (fill-region (point-min) (point-max) t 'nosqueeze 'to-eop))
-	  (goto-char (point-max)))
-	(setq prev-prefix prefix
-	      start (point))))
-    (save-restriction
-      (narrow-to-region start (point))
-      (let ((fill-prefix prev-prefix))
-	(fill-region (point-min) (point-max) t 'nosqueeze 'to-eop)))))
+	  (narrow-to-region start end)
+	  (let ((prefix
+		 (concat "\n"
+			 (or (and (looking-at ">[> ]*")
+				  (match-string 0)) ""))))
+	    (goto-char start)
+	    (while (search-forward prefix nil t)
+	      (replace-match " " t t))
+	    (goto-char start)
+	    (while (< (+ (point) fill-column) (point-max))
+	      (let ((start (point)))
+		(forward-char fill-column)
+		(when (search-backward " " start t)
+		  (forward-char)
+		  (insert prefix)))))
+	  (setq start (1+ (point-max))))))
+    t))
 
 ;;;###autoload
 (defun fill-flowed (&optional buffer delete-space)
   "Apply RFC2646 decoding to BUFFER.
 If BUFFER is nil, default to the current buffer.
 
-If DELETE-SPACE, delete RFC2646 spaces padding at the end of
+If DELETE-SPACE, delete RFC3676 spaces padding at the end of
 lines."
   (with-current-buffer (or buffer (current-buffer))
     (let ((fill-column  (eval fill-flowed-display-column t)))
@@ -154,7 +164,7 @@ lines."
           ;; Delete the newline.
           (when (eq (following-char) ?\s)
             (delete-char 1))
-          ;; Hack: Don't do the flowing on the signature line.
+          ;; As per RFC3767: Don't do the flowing on the signature line.
           (when (and (not (looking-at "-- $"))
                      (eq (char-before (line-end-position)) ?\s))
             (while (and (not (eobp))
-- 
2.39.2

Message #29 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: 71017 <at> debbugs.gnu.org
Subject: Re: [PATCH] Flow single-paragraph messages
Date: Sun, 07 Jul 2024 11:16:16 +0200

Eww, I forgot about quote handling for this autoharden fallback. 

Message #32 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: 71017 <at> debbugs.gnu.org
Cc: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Subject: [PATCH] Flow single-paragraph messages
Date: Sun,  7 Jul 2024 12:02:52 +0200

See discussion in #71017.

In short:

if mml-flowed-enable is nil: no change to behavior

if mml-flowed-enable is t and use-hard-newlines is on: several bugs
fixed, some (multi-paragraph messages) of which were counter to
documentation and some (single-paragraph messages) that were
along documentation lines but which messed up messages royally.

if mml-flowed-enable is t but use-hard-newlines was accidentally
forgotten by user: new behavior; reflow but don't reflow together
separate paragraphs nor reflow across lines that markdown would've
considered hard.
---
 doc/misc/emacs-mime.texi |  7 +--
 lisp/gnus/mml.el         | 29 ++++++-------
 lisp/mail/flow-fill.el   | 93 ++++++++++++++++++++++++----------------
 3 files changed, 69 insertions(+), 60 deletions(-)

diff --git a/doc/misc/emacs-mime.texi b/doc/misc/emacs-mime.texi
index ef7ea61..7621a9a 100644
--- a/doc/misc/emacs-mime.texi
+++ b/doc/misc/emacs-mime.texi
@@ -1087,13 +1087,10 @@ terminated by soft newline characters are filled together and wrapped
 after the column decided by @code{fill-flowed-encode-column}.
 Quotation marks (matching @samp{^>* ?}) are respected.  The variable
 controls how the text will look in a client that does not support
-flowed text, the default is to wrap after 66 characters.  If hard
-newline characters are not present in the buffer, no flow encoding
-occurs.
+flowed text, the default is to wrap after 66 characters.
 
 You can customize the value of the @code{mml-enable-flowed} variable
-to enable or disable the flowed encoding usage when newline
-characters are present in the buffer.
+to enable or disable the flowed encoding usage.
 
 On decoding flowed text, lines with soft newline characters are filled
 together and wrapped after the column decided by
diff --git a/lisp/gnus/mml.el b/lisp/gnus/mml.el
index e3bc393..2db39dc 100644
--- a/lisp/gnus/mml.el
+++ b/lisp/gnus/mml.el
@@ -691,23 +691,18 @@ type detected."
 		   (t
 		    ;; Only perform format=flowed filling on text/plain
 		    ;; parts where there either isn't a format parameter
-		    ;; in the mml tag or it says "flowed" and there
-		    ;; actually are hard newlines in the text.
-		    (let (use-hard-newlines)
-		      (when (and mml-enable-flowed
-				 (string= type "text/plain")
-				 (not (string= (cdr (assq 'sign cont)) "pgp"))
-				 (or (null (assq 'format cont))
-				     (string= (cdr (assq 'format cont))
-					      "flowed"))
-				 (setq use-hard-newlines
-				       (text-property-any
-					(point-min) (point-max) 'hard 't)))
-			(fill-flowed-encode)
-			;; Indicate that `mml-insert-mime-headers' should
-			;; insert a "; format=flowed" string unless the
-			;; user has already specified it.
-			(setq flowed (null (assq 'format cont)))))
+		    ;; in the mml tag or it says "flowed".
+		    (when (and mml-enable-flowed
+			       (string= type "text/plain")
+			       (not (string= (cdr (assq 'sign cont)) "pgp"))
+			       (or (null (assq 'format cont))
+				   (string= (cdr (assq 'format cont))
+					    "flowed")))
+		      (fill-flowed-encode)
+		      ;; Indicate that `mml-insert-mime-headers' should
+		      ;; insert a "; format=flowed" string unless the
+		      ;; user has already specified it.
+		      (setq flowed (null (assq 'format cont))))
 		    ;; Prefer `utf-8' for text/calendar parts.
 		    (if (or charset
 			    (not (string= type "text/calendar")))
diff --git a/lisp/mail/flow-fill.el b/lisp/mail/flow-fill.el
index 919490e..793088a 100644
--- a/lisp/mail/flow-fill.el
+++ b/lisp/mail/flow-fill.el
@@ -73,50 +73,67 @@ RFC 2646 suggests 66 characters for readability."
 ;;;###autoload
 (defun fill-flowed-encode (&optional buffer)
   (with-current-buffer (or buffer (current-buffer))
-    ;; No point in doing this unless hard newlines is used.
-    (when use-hard-newlines
-      (let ((start (point-min)) end)
-	;; Go through each paragraph, filling it and adding SPC
-	;; as the last character on each line.
-	(while (setq end (text-property-any start (point-max) 'hard 't))
-	  (save-restriction
-	    (narrow-to-region start end)
-	    (let ((fill-column (eval fill-flowed-encode-column t)))
-	      (fill-flowed-fill-buffer))
-	    (goto-char (point-min))
-	    (while (re-search-forward "\n" nil t)
-	      (replace-match " \n" t t))
-	    (goto-char (setq start (1+ (point-max)))))))
-      t)))
-
-(defun fill-flowed-fill-buffer ()
-  (let ((prefix nil)
-	(prev-prefix nil)
-	(start (point-min)))
-    (goto-char (point-min))
-    (while (not (eobp))
-      (setq prefix (and (looking-at "[> ]+")
-			(match-string 0)))
-      (if (equal prefix prev-prefix)
-	  (forward-line 1)
+    (let ((fill-column (eval fill-flowed-encode-column t))
+	  (start (point-min))
+	  (end (point-max)))
+      ;; Only when we've been called to reflow a buffer that doesn't
+      ;; have any hard newlines:
+      (unless (text-property-any start end 'hard 't)
+	(save-excursion
+	  ;; Harden between paras:
+	  (goto-char start)
+	  (while (re-search-forward "\n[> ]*\n" end t)
+	    (set-hard-newline-properties
+	     (match-beginning 0) (point))
+	    (backward-char))
+	  ;; Harden before a quote starts:
+	  (goto-char start)
+	  (while (re-search-forward "\n[^>][^\n]*\n>" end t)
+	    (backward-char)
+	    (set-hard-newline-properties
+	     (1- (point)) (point)))
+	  ;; Harden "  $"
+	  (goto-char start)
+	  (while (search-forward "  \n" end t)
+	    (backward-char)
+	    (backward-delete-char 2)
+	    (set-hard-newline-properties
+	     (point) (1+ (point))))
+	  ;; Harden "^    "
+	  (goto-char start)
+	  (while (search-forward "\n    " end t)
+	    (set-hard-newline-properties
+	     (- (point) 5) (- (point) 4)))))
+      ;; Go through each paragraph, filling it and adding SPC
+      ;; as the last character on each line.
+      (while (and (< start (point-max))
+		  (setq end (or (text-property-any start (point-max) 'hard 't)
+				(point-max))))
 	(save-restriction
-	  (narrow-to-region start (point))
-	  (let ((fill-prefix prev-prefix))
-	    (fill-region (point-min) (point-max) t 'nosqueeze 'to-eop))
-	  (goto-char (point-max)))
-	(setq prev-prefix prefix
-	      start (point))))
-    (save-restriction
-      (narrow-to-region start (point))
-      (let ((fill-prefix prev-prefix))
-	(fill-region (point-min) (point-max) t 'nosqueeze 'to-eop)))))
+	  (narrow-to-region start end)
+	  (let ((prefix
+		 (concat "\n"
+			 (or (and (looking-at ">[> ]*")
+				  (match-string 0)) ""))))
+	    (goto-char start)
+	    (while (search-forward prefix nil t)
+	      (replace-match " " t t))
+	    (goto-char start)
+	    (while (< (+ (point) fill-column) (point-max))
+	      (let ((start (point)))
+		(forward-char fill-column)
+		(when (search-backward " " start t)
+		  (forward-char)
+		  (insert prefix)))))
+	  (setq start (1+ (point-max))))))
+    t))
 
 ;;;###autoload
 (defun fill-flowed (&optional buffer delete-space)
   "Apply RFC2646 decoding to BUFFER.
 If BUFFER is nil, default to the current buffer.
 
-If DELETE-SPACE, delete RFC2646 spaces padding at the end of
+If DELETE-SPACE, delete RFC3676 spaces padding at the end of
 lines."
   (with-current-buffer (or buffer (current-buffer))
     (let ((fill-column  (eval fill-flowed-display-column t)))
@@ -154,7 +171,7 @@ lines."
           ;; Delete the newline.
           (when (eq (following-char) ?\s)
             (delete-char 1))
-          ;; Hack: Don't do the flowing on the signature line.
+          ;; As per RFC3767: Don't do the flowing on the signature line.
           (when (and (not (looking-at "-- $"))
                      (eq (char-before (line-end-position)) ?\s))
             (while (and (not (eobp))
-- 
2.39.2

Message #37 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Henrik Ahlgren <pablo <at> seestieto.com>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Tue, 04 Mar 2025 12:01:49 +0200

Thank you, Sandra, for addressing this bug. It’s one reason I 
hesitate to send f=f messages with Gnus, despite the clear 
advantages of a properly functioning f=f implementation.

I’m curious if others have experienced the following behavior: when
sending a f=f message with use-hard-newlines enabled, the last paragraph
sometimes gets filled into the previous one, with two or three spaces in
between. I’m unsure how to consistently reproduce this issue.

Message #40 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: <71017 <at> debbugs.gnu.org>
Subject: Re: bug#71017: fill-flowed-encode
Date: Tue, 4 Mar 2025 14:14:18 +0000

[Message part 1 (text/plain, inline)]

That's something I haven't seen since I switched to my own patch

Message #43 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Henrik Ahlgren <pablo <at> seestieto.com>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Tue, 04 Mar 2025 23:08:37 +0200

Sandra Snan <sandra.snan <at> idiomdrottning.org> writes:

> -If DELETE-SPACE, delete RFC2646 spaces padding at the end of 
> +If DELETE-SPACE, delete RFC3676 spaces padding at the end of 
>  lines." 
>    (with-current-buffer (or buffer (current-buffer)) 
>      (let ((fill-column  (eval fill-flowed-display-column t))) 
> @@ -154,7 +164,7 @@ lines." 
>            ;; Delete the newline.  (when (eq (following-char) 
>            ?\s) 
>              (delete-char 1)) 
> -          ;; Hack: Don't do the flowing on the signature line. 
> +          ;; As per RFC3767: Don't do the flowing on the 
> signature line. 

I believe this is a typo; the comment should refer to RFC3676.

Message #46 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Stefan Kangas <stefankangas <at> gmail.com>,
 <71017 <at> debbugs.gnu.org>, Eric Abrahamsen <eric <at> ericabrahamsen.net>
Subject: Re: Re: bug#71017: fill-flowed-encode
Date: Tue, 4 Mar 2025 21:14:45 +0000

[Message part 1 (text/plain, inline)]

That's right.

Message #49 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Henrik Ahlgren <pablo <at> seestieto.com>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Tue, 04 Mar 2025 23:49:47 +0200

Henrik Ahlgren <pablo <at> seestieto.com> writes: 

> Sandra Snan <sandra.snan <at> idiomdrottning.org> writes: 
> 
>> -If DELETE-SPACE, delete RFC2646 spaces padding at the end of 
>> +If DELETE-SPACE, delete RFC3676 spaces padding at the end of 
>> lines."  (with-current-buffer (or buffer (current-buffer)) 
>> (let ((fill-column (eval fill-flowed-display-column t))) @@ 
>> -154,7 +164,7 @@ lines."  ;; Delete the newline.  (when (eq 
>> (following-char)            ?\s) (delete-char 1)) -          ;; 
>> Hack: Don't do the flowing on the signature line. +          ;; 
>> As per RFC3767: Don't do the flowing on the signature line.  

And here we have a clear example of another issue: I quoted (with 
C-c C-y) a hard-wrapped git-send-email message when replying, but 
it was soft-wrapped upon sending, making the patch nearly 
unreadable. While editing, the quoted part appeared normal. Not 
sure how to determine the correct behaviour and how to ensure it, 
but this surely isn't ideal.

The `messages-are-flowing` package has an open PR with a utility
function aimed at "hardening" the region. However, users must be 
aware of the issue and remember to manually apply the fix 
manually. Since most users don't have the package installed, and 
it apparently is no longer actively maintained (last commit was 8 
years ago), it may be beneficial to incorporate similar 
functionality directly into Emacs.

Message #52 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Stefan Kangas <stefankangas <at> gmail.com>,
 <71017 <at> debbugs.gnu.org>, Eric Abrahamsen <eric <at> ericabrahamsen.net>
Subject: Re: Re: bug#71017: fill-flowed-encode
Date: Tue, 4 Mar 2025 21:50:54 +0000

[Message part 1 (text/plain, inline)]

That PR was made by me.

Message #55 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Stefan Kangas <stefankangas <at> gmail.com>,
 <71017 <at> debbugs.gnu.org>, Eric Abrahamsen <eric <at> ericabrahamsen.net>
Subject: Re: Re: bug#71017: fill-flowed-encode
Date: Tue, 4 Mar 2025 21:51:26 +0000

[Message part 1 (text/plain, inline)]

And I have signed FSF papers so feel free to use it.

Message #58 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Henrik Ahlgren <pablo <at> seestieto.com>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Thu, 13 Mar 2025 12:41:53 +0100

Even though my setup works well with this "harden-region" stuff 
(and I keep double-checking how what I send from Emacs look in 
e.g. Delta Chat), I've been thinking that maybe this entire 
hard/soft newlines as invisibly delineated by props might 
fundamentally not be the best approach. Let's call it "approach 
one". This is what I'm still currently working for email and I 
haven't had any mishaps in a while thanks my patched 
messages-are-flowing. I hacked it so that newlines that are 
paragraph breaks do NOT show those annoying li'l return signs, 
but hard newlines that are in the middle of a paragraph (for code 
or poetry) do show them. That was very scary to deploy (as hiding 
information from myself always is) but it has been working.

Now, for all three approaches, I'm talking about the local 
writing format in the buffer for the Emacs user. The wire format 
shall the same for all three approaches i.e. the current approach 
which follows the RFC without stuffing > marks. 
https://idiomdrottning.org/format-flowed

One idea, "approach two", might be to use an idea from markdown: 
Paragraphs (i.e. \n\n+) are always hard, lines that start with 
four spaces or end with two spaces have those spaces removed and 
are also made hard, and all other newlines are soft. With 
approach two you still need to remember to add spaces to code 
sections (for example with markdown-pre-region) so it keeps some 
of the same "harden-region" problem of the approach one. This 
approach two is what I am and have been using for many years to 
write for my blog. I personally only ever use the four spaces 
with markdown-pre-region rather than the three backtick "code 
fence" method but if we were to go with approach two, we'd 
probably better support both.

And then "approach three" would be... All newlines are hard. I've 
been using approach three to post on Fediverse and I've been 
using visual-fill-column-mode together with some hooks that make 
it and visual-line-mode toggle in lockstep so there's never one 
without the other. I need to remember to have autofill *off* when 
using this, but it's been so great! I know, I know, this is a 
very "Windows and Mac" approach to text (and don't worry, the 
wire format will still be wrapped using the flowed RFC, we're not 
actually *sending* these hyper long lines here) but it works. 
I've always thought visual-line-mode was completely useless and 
ugly but with visual-fill-column-mode it works really well. It's 
not as fire-and-forget as the other two approaches since the one 
drawback to approach three is that if I'm not sure I accidentally 
made a newline, I won't see it. So when that creeping insecurity 
starts rearing, I toggle visual-line-mode (which also toggles 
visual-fill-column-mode) to double check. Or if it's just one 
line I can delete the "is it a space or a newline" character and 
insert a space, but, I'm more likely to do the toggle check since 
I have it on the keymap & on the mode line.

So, hmm, all three approaches have their drawbacks. The first two 
requires deliberately hardening and softening, the third matches 
many other modern apps but can (rarely) accidentally insert stray 
hards. A harder-to-detect but less-catastrophic error than 
accidentally softening a huge code patch or poem.

So what do y'all think, keep on polishing the first approach or 
switch to two or three (and maybe polish them further)? For me 
this itch has been well scratched with my patches both here and 
to messages-are-flowing ( 
https://github.com/legoscia/messages-are-flowing/pull/15/ ). 

Message #61 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Henrik Ahlgren <pablo <at> seestieto.com>
Cc: 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sat, 22 Mar 2025 08:54:30 +0100

I said all that but then yesterday, just a little over a week 
after I was boasting that I never mess things up, I too forgot to 
"harden-region" a paste of a shell output. Ergo, approach three 
is starting to looking like the winner as far as my vote goes!

I.e. the idea user writes super long lines and there are no 
"soft" newlines, instead relying on visual-line-mode [which, I 
dunno, maybe that uses  soft newlines under the hood, I haven't 
looked] and visual-fill-column-mode (or however else the user 
wants to do it, like a few years ago I also made 
https://idiomdrottning.org/gmihw that people could pipe their 
text through), and that then we or I write code that breaks those 
long lines according to format flowed so that the wire format is 
still ff wrapped.

Message #64 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sandra Snan <sandra.snan <at> idiomdrottning.org>
Cc: pablo <at> seestieto.com, 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sat, 22 Mar 2025 13:00:30 +0200

> Cc: 71017 <at> debbugs.gnu.org
> Date: Sat, 22 Mar 2025 08:54:30 +0100
> From:  Sandra Snan via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> I.e. the idea user writes super long lines and there are no 
> "soft" newlines, instead relying on visual-line-mode [which, I 
> dunno, maybe that uses  soft newlines under the hood, I haven't 
> looked]

It doesn't.

Message #67 received at 71017 <at> debbugs.gnu.org (full text, mbox):

From: Sandra Snan <sandra.snan <at> idiomdrottning.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: pablo <at> seestieto.com, 71017 <at> debbugs.gnu.org
Subject: Re: bug#71017: fill-flowed-encode
Date: Sat, 22 Mar 2025 18:34:43 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

Sandra wrote:
>> maybe visual-line-mode uses soft newlines under the hood, I 
>> haven't  looked
>
> It doesn't.

Okay, that's great news then 👍🏻

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.