Subject: Re: bug#77539: 31.0.50; nnatom: missing content in articles.

From: Daniel Semyonov <[email-masked]>
Date: Tue, 17 Jun 2025 17:05:12 +0300

>>>>> Daniel Semyonov writes:

>>>>> Eli Zaretskii writes:
    >> Ping! Ping! Ping! Daniel, please respond.

    > Sorry, I'm very sick for a few weeks now (getting better hopefully),
    > this issue is specific to nnatom, I did test but need to do more
    > research, I think I'll be able to do this soon.
    > Did not expect to be away for so long, sorry again, I'll make sure to
    > write soon whether or not I made progress

Not sure how no one noticed this for so long, but nnatom/nnfeed don't
take into account the fact that MIME messages can only contain ASCII
text (in the case of Gnus, it seems non-ASCII text is simply not shown
when an article is displayed); Non-ASCII text must encoded in one of a
two ways specific to MIME to be displayed correctly.

However, I made some simple changes to encode the body of articles using
'quoted-printable' encoding to test this, and yet non-ASCII characters
still seem to be missing, even though looking at the raw article I can
see it was encoded correctly and all the appropriate MIME headers are
set correctly (I even tested with an HTML email with identical content
where non-ASCII characters are visible through Gnus, and it seems the
raw articles are nearly identical, barring some headers specific to the
email unrelated to encoding).

I'm honestly not sure how to proceed from here, I don't know what could
be causing this difference at this point.

>From a cursory look at how nnrss handles this, it seems that encoding
the body is handled automatically there, as the message is structured
with MML and then converted to MIME using 'mml-to-mime' (which also
encodes the message body); if I don't figure it out soon I'll guess I'll
convert nnatom to use this method too.

    >>> Cc: [email-masked], [email-masked]
    >>> Date: Sat, 24 May 2025 11:50:34 +0300
    >>> From: Eli Zaretskii <[email-masked]>
    >>> 
    >>> Ping! Ping! Can we make some further progress here?
    >>> 
    >>> > Cc: [email-masked], [email-masked]
    >>> > Date: Sat, 10 May 2025 12:18:45 +0300
    >>> > From: Eli Zaretskii <[email-masked]>
    >>> > 
    >>> > Ping!  Any progress here?
    >>> > 
    >>> > > From: Daniel Semyonov <[email-masked]>
    >>> > > Cc: Fernando de Morais <[email-masked]>,  [email-masked]
    >>> > > Date: Wed, 23 Apr 2025 18:04:08 +0300
    >>> > > 
    >>> > > >>>>> Eli Zaretskii writes:
    >>> > > 
    >>> > >     > Ping! How should we proceed with this issue?
    >>> > > 
    >>> > > I haven't had time to look into it properly yet, sorry.
    >>> > > I plan to test with HTML emails in Gnus (to figure out whether the issue
    >>> > > occurs in general when Gnus renders an HTML article, or just nnatom HTML
    >>> > > articles).
    >>> > > 
    >>> > >     >> Cc: [email-masked]
    >>> > >     >> From: Fernando de Morais <[email-masked]>
    >>> > >     >> Date: Sat, 05 Apr 2025 10:33:52 -0300
    >>> > >     >> 
    >>> > >     >> Hello Daniel,
    >>> > >     >> 
    >>> > >     >> Daniel Semyonov <[email-masked]> writes:
    >>> > >     >> 
    >>> > >     >> > IIRC HTML articles are displayed by Gnus using shr.el, I assume this
    >>> > >     >> > would need to be fixed there (though I'm not sure if this is really a
    >>> > >     >> > "bug" or expected behavior for an HTML renderer).
    >>> > >     >> 
    >>> > >     >> `shr' is also the default HTML render for EWW, if I'm not mistaken.  I
    >>> > >     >> did a test: I repeated the same steps from my original report and
    >>> > >     >> navigated, via the article link, to the original page using EWW, and the
    >>> > >     >> characters appeared normally.
    >>> > >     >> 
    >>> > >     >> I also navigated to the feed page (planet.emacslife.com) via EWW, and
    >>> > >     >> everything is displayed normally there.  Maybe is not related to `shr',
    >>> > >     >> then?
    >>> > >     >> 
    >>> > >     >> > Thank you for reporting these issues,
    >>> > >     >> 
    >>> > >     >> You're welcome!
    >>> > >     >> 
    >>> > >     >> -- 
    >>> > >     >> Regards,
    >>> > >     >> Fernando de Morais.
    >>> > >     >> 
    >>> > >     >> 
    >>> > >     >> 
    >>> > >     >> 
    >>> > > 
    >>> > 
    >>> > 
    >>> > 
    >>> > 
    >>> 
    >>> 
    >>> 
    >>>