Subject: bug#63941: [PATCH] ; always CRLF before non-first boundary in multipart form

From: ozzloy <[email-masked]>
Date: Tue, 6 Jun 2023 22:25:40 -0700

When I POST a file ending with a newline using EWW, the final newline
gets chomped off of the content.  This is because
mm-url-encode-multipart-form-data inserts CRLF unless it is at the
beginning of a line.  For file uploads, this behavior is incorrect.

To reproduce,

0. Upload a file ending in "\n" using EWW.

1. Observe the POST does not have CRLF between the file content and the
boundary.

Refer to https://www.rfc-editor.org/rfc/rfc2046#section-5.1.1 for
details.

I have not tested other kinds of html form posts.  I am not familiar
with what should be tested.  I did include tests for this specific bug
though.

In GNU Emacs 29.0.91 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2023-06-06 built on trent-reznor
Repository revision: b5eb43ba289519704c6cb0fe456038dcaec172c3
Repository branch: CRLF-before-noninitial-boundary
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Ubuntu 22.04.2 LTS

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
emacs)

Memory information:
((conses 16 36910 11079)
 (symbols 48 5128 0)
 (strings 32 13160 1100)
 (string-bytes 1 372301)
 (vectors 16 9331)
 (vector-slots 8 149255 18597)
 (floats 8 33 21)
 (intervals 56 326 0)
 (buffers 976 11))
From 46da4c9d9367aaf4bd3ce2faf118845f3930dabf Mon Sep 17 00:00:00 2001
From: Daniel Watson <[email-masked]>
Date: Sat, 3 Jun 2023 21:15:25 -0700
Subject: [PATCH] ; always CRLF before non-first boundary in multipart form
 data

; Insert CRLF after file contents and before boundary,
; in accordance with the syntax description here
; https://www.rfc-editor.org/rfc/rfc2046#section-5.1.1
; The CRLF is attached to the boundary, and not the preceding part.
---
 lisp/gnus/mm-url.el            |  3 +-
 test/lisp/gnus/mm-url-tests.el | 53 ++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 2 deletions(-)
 create mode 100644 test/lisp/gnus/mm-url-tests.el

diff --git a/lisp/gnus/mm-url.el b/lisp/gnus/mm-url.el
index 11847a79f17..022762a7799 100644
--- a/lisp/gnus/mm-url.el
+++ b/lisp/gnus/mm-url.el
@@ -438,8 +438,7 @@ mm-url-encode-multipart-form-data
 	  (insert (format "Content-Disposition: form-data; name=%S\r\n\r\n"
 			  name))
 	  (insert value)))
-	(unless (bolp)
-	  (insert "\r\n"))))
+	(insert "\r\n")))
     (insert "--" boundary "--\r\n")
     (buffer-string)))
 
diff --git a/test/lisp/gnus/mm-url-tests.el b/test/lisp/gnus/mm-url-tests.el
new file mode 100644
index 00000000000..ed51cb7d086
--- /dev/null
+++ b/test/lisp/gnus/mm-url-tests.el
@@ -0,0 +1,53 @@
+;;; mm-url-tests.el ---  -*- lexical-binding:t -*-
+
+;; Copyright (C) 2021-2023 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;; Code:
+
+(require 'ert)
+(require 'mm-url)
+
+(ert-deftest test-mm-url-encode-multipart-form-data ()
+  (letrec
+      ((boundary "====-=-=")
+       (make-data (lambda (count)
+                    `(("file"
+                       ("filedata" . ,(make-string count ?\n))
+                       ("name"     . "file")
+                       ("filename" . "g")))))
+       (template
+        (concat
+         "--" boundary "\r\n"
+         "Content-Disposition:"
+         " form-data; name=\"file\"; filename=\"g\"\r\n"
+         "Content-Transfer-Encoding: binary\r\n"
+         "Content-Type: text/plain\r\n"
+         "\r\n"
+         "%s" ;; here's the file content
+         "\r\n" ;; \r\n attaches to boundary below, not file content
+         ;; ref: https://www.rfc-editor.org/rfc/rfc2046#section-5.1.1
+         "--" boundary "--" "\r\n")))
+    (dotimes (count 3)
+      (let ((data (funcall make-data count))
+            (expected (format template (make-string count ?\n))))
+        (should (equal (mm-url-encode-multipart-form-data data boundary)
+                       expected))))))
+
+;;; mm-url-tests.el ends here
-- 
2.34.1