Subject: Re: bug#13808: Gnus gratuiously messes with the encoding of attachments

From: David Kastrup <[email-masked]>
Date: Mon, 08 Jul 2013 16:14:40 +0200

First a note and apology: due to a misconfiguration of my mail server,
mail to debbugs.gnu.org has instead been sent (and seen as being sent
to) to gnu.org instead.  As a consequence, a significant part of our
communication might never have made it to the bug tracker.

I apology for that and hope that my system's configuration will now be
fixed in that regard.

Lars Magne Ingebrigtsen <[email-masked]> writes:

> David Kastrup <[email-masked]> writes:
>
>> You are confusing the technical implementation with the intended
>> semantics.  An inline file is expected to be displayed in the message,
>> but indeed it _can_ be saved conveniently as a single file, so
>> reencoding it is not a good idea either.
>>
>> The only purpose of an attachment is to be saved as a file.  If this
>> file is not identical to the file that has been sent, the purpose is not
>> met.
>
> There is no technical difference between Content-Disposition: inline or
> attachment.  text/plain parts have to adhere to certain standards (be
> encoded with a valid charset etc) to be valid.
>
> If you want to just send a sequence of bytes, you have to send something
> other than text/*.

But we are not talking about "text/plain" here, but rather
"text/x-lilypond".  Gnus offers, among others, the content types

text/cache-manifest 	text/calendar
text/css 	text/csv
text/dns 	text/enriched
text/h323 	text/html
text/iuls 	text/mathml
text/plain 	text/richtext
text/scriptlet 	text/spreadsheet
text/tab-separated-values 	text/texmacs
text/vnd.sun.j2me.app-descriptor 	text/vnd.wap.wml
text/vnd.wap.wmlscript 	text/x-bibtex
text/x-boo 	text/x-c++hdr
text/x-c++src 	text/x-chdr
text/x-component 	text/x-csh
text/x-csrc 	text/x-diff
text/x-dsrc 	text/x-haskell
text/x-java 	text/x-lilypond
text/x-literate-haskell 	text/x-moc
text/x-org 	text/x-pascal
text/x-patch 	text/x-pcs-gcd
text/x-perl 	text/x-python
text/x-scala 	text/x-setext
text/x-sfv 	text/x-sh
text/x-tcl 	text/x-tex
text/x-vcalendar 	text/x-vcard

And when you use C-c m f for attaching a file of extension, say, .ly, it
will _default_ to using text/x-lilypond.  Of the above content types, it
would obviously be a really bad idea to silently reencode an attached
file to have file contents with a different encoding for the
overwhelming majority of the above file types as they either expect a
certain encoding _or_ declare the desired encoding in instructions in
the file that will not get changed by Gnus when it it changes the
original encoding.

>> A file that compiled fine at the site of the sender does no longer
>> compile when received.  Only ASCII characters survive unchanged.  That
>> makes an attachment not better for transfering program files than just
>> pasting them in the middle of the mail message would.
>
> That Gnus re-encoded a text/plain part that was utf-8 on disk to
> Latin-1 sounds like a bug, yes.

I would tend to agree here, but my complaint actually was that Gnus
reencoded a text/x-lilypond part that was utf-8 on disk.

> But that has nothing to do with whether the parts are attachments or
> not.

I think this behavior less defensible when attachments are involved, but
I'd be perfectly happy if it was fixed for both attachments as well as
inline parts.

> Unless you're looking at different standards than what I'm looking at.

Not likely.  I was just trying to create an example of least defensible
behavior by illustrating with an attachment of type text/x-lilypond.

I'd be perfectly content with preserving file encoding for inline
passages of text/plain: the fewer chances there are for things going
wrong or being surprising, the better.

Now part of the scenario for reproducing the problem might be the
following variable:

mm-coding-system-priorities is a variable defined in `mm-util.el'.
Its value is (iso-8859-1 iso-8859-15 utf-8)
Original value was nil

Documentation:
Preferred coding systems for encoding outgoing messages.

More than one suitable coding system may be found for some text.
By default, the coding system with the highest priority is used
to encode outgoing messages (see `sort-coding-systems').  If this
variable is set, it overrides the default priority.

You can customize this variable.

This variable was introduced, or its default value was changed, in
version 21.2 of Emacs.

[back]

I think that this variable should likely not be consulted at all when
attaching/sending a file.  When attaching/sending a _buffer_ that is
_not_ visiting a file, the choice might be more difficult to make.

But I'd consider it an "element of least surprise" strategy to restrict
the effect of mm-coding-system-priorities to the actual original text in
the message buffer.

-- 
David Kastrup