GNU bug report logs - #75397
29.4; GNUS message mode lacks Cyrillic support (GNU/Linux)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 75397 in the body.
You can then email your comments to 75397 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.4; GNUS message mode lacks Cyrillic support (GNU/Linux)
Date: Mon, 06 Jan 2025 01:49:38 +0100

1. Type M-gnus to start GNUS.
2. Start composing a message by pressing 'm'.
3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
4. Press C-c C-d to save the draft.
5. Press S D e to resume editing.
6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".

Is this even real?? By the way, this does not occur with ordinary text
files. And everything works fine in the view mode inside the draft
group. Pressing S D e, on the other hand, makes things go haywire.

Incidentally, a similar problem, though not as bad, occurs with German
characters (e.g. ä) which get replaced with \number after pressing S D e.

Many thanks.

Message #8 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
Cc: 75397 <at> debbugs.gnu.org
Subject: Re: bug#75397: 29.4; GNUS message mode lacks Cyrillic support
 (GNU/Linux)
Date: Mon, 06 Jan 2025 10:36:34 +0100

On Jan 06 2025, Nickolai Dobrynin wrote:

> 1. Type M-gnus to start GNUS.
> 2. Start composing a message by pressing 'm'.
> 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
> 4. Press C-c C-d to save the draft.
> 5. Press S D e to resume editing.
> 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
>
> Is this even real?? By the way, this does not occur with ordinary text
> files. And everything works fine in the view mode inside the draft
> group. Pressing S D e, on the other hand, makes things go haywire.

The problem is that message-dont-send doesn't encode the message when it
saves it.  gnus-summary-resend-message-edit assumes that the stored
message is encoded and goes through an unibyte buffer while trying to
decode it, which then strips off all non-ascii bits from the characters.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #11 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
To: 75397 <at> debbugs.gnu.org
Subject: Any workarounds for this?
Date: Tue, 07 Jan 2025 19:13:49 +0100

Hi Andreas,

1. Can you suggest a workaround for this?

2. For clarity's sake, would it be better to change the title of the
bug report to, e.g., "messages get corrupted after being saved to "drafts" b/c
message-dont-send doesn't encode them before saving"? This would
probably summarize the problem much better to the rest of the Emacs team.

Message #14 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
Cc: 75397 <at> debbugs.gnu.org
Subject: Re: bug#75397: 29.4;
 GNUS message mode lacks Cyrillic support (GNU/Linux)
Date: Wed, 08 Jan 2025 14:47:22 +0200

> From: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
> Date: Tue, 07 Jan 2025 19:13:49 +0100
> 
> 1. Can you suggest a workaround for this?
> 
> 2. For clarity's sake, would it be better to change the title of the
> bug report to, e.g., "messages get corrupted after being saved to "drafts" b/c
> message-dont-send doesn't encode them before saving"? This would
> probably summarize the problem much better to the rest of the Emacs team.

No need.  The Emacs team, such as it is, took good note of your
report, and will investigate the ways to fix this soon.  At that time,
we may have suggestions for workarounds or patches for fixing the
problem.  It's just that we didn't have yet time to do so.

For now, the only workaround to try I can suggest is to explicitly
bind the coding-system while saving the draft, like this:

  C-x RET c utf-8-emacs RET C-c C-d

If after that "S D e" doesn't restore the original Cyrillic text, try
this instead:

  C-x RET c utf-8-emacs RET S D e

P.S. Please don't change the Subject line when you respond to messages
in this thread: it makes it a bit harder to find the previous/next
messages when reading the discussions.  I've restored the original
Subject on my response.

Message #17 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: 75397 <at> debbugs.gnu.org, ndobrynin <at> gmail.com
Subject: Re: bug#75397: 29.4;
 GNUS message mode lacks Cyrillic support (GNU/Linux)
Date: Wed, 08 Jan 2025 17:30:18 +0200

> Cc: 75397 <at> debbugs.gnu.org
> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Date: Mon, 06 Jan 2025 10:36:34 +0100
> 
> On Jan 06 2025, Nickolai Dobrynin wrote:
> 
> > 1. Type M-gnus to start GNUS.
> > 2. Start composing a message by pressing 'm'.
> > 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
> > 4. Press C-c C-d to save the draft.
> > 5. Press S D e to resume editing.
> > 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
> >
> > Is this even real?? By the way, this does not occur with ordinary text
> > files. And everything works fine in the view mode inside the draft
> > group. Pressing S D e, on the other hand, makes things go haywire.
> 
> The problem is that message-dont-send doesn't encode the message when it
> saves it.

I must be missing something here, because I don't see what you
describe above:

  (defun message-dont-send ()
    "Don't send the message you have been editing.
  Instead, just auto-save the buffer and then bury it."
    (interactive nil message-mode)
    (set-buffer-modified-p t)
    (save-buffer)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
    (let ((actions message-postpone-actions))
      (message-bury (current-buffer))
      (message-do-actions actions)))

So message-dont-send saves the buffer.  And that buffer has a
buffer-file-coding-system set in message-set-auto-save-file-name,
which is called when message-mode is turned on (which is supposed to
happen when you press 'm' in Gnus):

    (setq buffer-file-coding-system message-draft-coding-system)))

And message-draft-coding-system is:


  (defvar message-draft-coding-system mm-auto-save-coding-system
    "Coding system to compose mail.")

which in my testing, starting from "emacs -Q", produces utf-8-emacs,
as I'd expect.

In my testing, the draft is saved using utf-8-emacs, and if I them
visit it, I get the Cyrillic text back.  As expected.

So where does the above breaks?  Is the value of
mm-auto-save-coding-system different?  Otherwise, save-buffer should
have encoded the draft's text.  What did I miss?

Message #20 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 75397 <at> debbugs.gnu.org, ndobrynin <at> gmail.com
Subject: Re: bug#75397: 29.4; GNUS message mode lacks Cyrillic support
 (GNU/Linux)
Date: Wed, 08 Jan 2025 17:53:16 +0100

On Jan 08 2025, Eli Zaretskii wrote:

>> Cc: 75397 <at> debbugs.gnu.org
>> From: Andreas Schwab <schwab <at> linux-m68k.org>
>> Date: Mon, 06 Jan 2025 10:36:34 +0100
>> 
>> On Jan 06 2025, Nickolai Dobrynin wrote:
>> 
>> > 1. Type M-gnus to start GNUS.
>> > 2. Start composing a message by pressing 'm'.
>> > 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
>> > 4. Press C-c C-d to save the draft.
>> > 5. Press S D e to resume editing.
>> > 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
>> >
>> > Is this even real?? By the way, this does not occur with ordinary text
>> > files. And everything works fine in the view mode inside the draft
>> > group. Pressing S D e, on the other hand, makes things go haywire.
>> 
>> The problem is that message-dont-send doesn't encode the message when it
>> saves it.
>
> I must be missing something here, because I don't see what you
> describe above:

It is about MIME encoding.

>   (defun message-dont-send ()
>     "Don't send the message you have been editing.
>   Instead, just auto-save the buffer and then bury it."
>     (interactive nil message-mode)
>     (set-buffer-modified-p t)
>     (save-buffer)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>     (let ((actions message-postpone-actions))
>       (message-bury (current-buffer))
>       (message-do-actions actions)))

This just saves the raw buffer contents, but
gnus-summary-resend-message-edit expects it to be a proper RFC2822 mail,
with suitable MIME headers.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #23 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: 75397 <at> debbugs.gnu.org, ndobrynin <at> gmail.com, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#75397: 29.4; GNUS message mode lacks Cyrillic support
 (GNU/Linux)
Date: Wed, 08 Jan 2025 18:12:32 +0100

>>>>> On Wed, 08 Jan 2025 17:53:16 +0100, Andreas Schwab <schwab <at> linux-m68k.org> said:

    Andreas> On Jan 08 2025, Eli Zaretskii wrote:
    >>> Cc: 75397 <at> debbugs.gnu.org
    >>> From: Andreas Schwab <schwab <at> linux-m68k.org>
    >>> Date: Mon, 06 Jan 2025 10:36:34 +0100
    >>> 
    >>> On Jan 06 2025, Nickolai Dobrynin wrote:
    >>> 
    >>> > 1. Type M-gnus to start GNUS.
    >>> > 2. Start composing a message by pressing 'm'.
    >>> > 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
    >>> > 4. Press C-c C-d to save the draft.
    >>> > 5. Press S D e to resume editing.
    >>> > 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
    >>> >
    >>> > Is this even real?? By the way, this does not occur with ordinary text
    >>> > files. And everything works fine in the view mode inside the draft
    >>> > group. Pressing S D e, on the other hand, makes things go haywire.
    >>> 
    >>> The problem is that message-dont-send doesn't encode the message when it
    >>> saves it.
    >> 
    >> I must be missing something here, because I don't see what you
    >> describe above:

    Andreas> It is about MIME encoding.

    >> (defun message-dont-send ()
    >> "Don't send the message you have been editing.
    >> Instead, just auto-save the buffer and then bury it."
    >> (interactive nil message-mode)
    >> (set-buffer-modified-p t)
    >> (save-buffer)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
    >> (let ((actions message-postpone-actions))
    >> (message-bury (current-buffer))
    >> (message-do-actions actions)))

    Andreas> This just saves the raw buffer contents, but
    Andreas> gnus-summary-resend-message-edit expects it to be a proper RFC2822 mail,
    Andreas> with suitable MIME headers.

Yes. Which is why the command to edit an unsent draft is 'e' (or 'D
e').

Robert
-- 

Message #26 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: 75397 <at> debbugs.gnu.org, ndobrynin <at> gmail.com
Subject: Re: bug#75397: 29.4; GNUS message mode lacks Cyrillic support
 (GNU/Linux)
Date: Wed, 08 Jan 2025 20:42:39 +0200

> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Cc: ndobrynin <at> gmail.com,  75397 <at> debbugs.gnu.org
> Date: Wed, 08 Jan 2025 17:53:16 +0100
> 
> On Jan 08 2025, Eli Zaretskii wrote:
> 
> >> Cc: 75397 <at> debbugs.gnu.org
> >> From: Andreas Schwab <schwab <at> linux-m68k.org>
> >> Date: Mon, 06 Jan 2025 10:36:34 +0100
> >> 
> >> On Jan 06 2025, Nickolai Dobrynin wrote:
> >> 
> >> > 1. Type M-gnus to start GNUS.
> >> > 2. Start composing a message by pressing 'm'.
> >> > 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
> >> > 4. Press C-c C-d to save the draft.
> >> > 5. Press S D e to resume editing.
> >> > 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
> >> >
> >> > Is this even real?? By the way, this does not occur with ordinary text
> >> > files. And everything works fine in the view mode inside the draft
> >> > group. Pressing S D e, on the other hand, makes things go haywire.
> >> 
> >> The problem is that message-dont-send doesn't encode the message when it
> >> saves it.
> >
> > I must be missing something here, because I don't see what you
> > describe above:
> 
> It is about MIME encoding.
> 
> >   (defun message-dont-send ()
> >     "Don't send the message you have been editing.
> >   Instead, just auto-save the buffer and then bury it."
> >     (interactive nil message-mode)
> >     (set-buffer-modified-p t)
> >     (save-buffer)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> >     (let ((actions message-postpone-actions))
> >       (message-bury (current-buffer))
> >       (message-do-actions actions)))
> 
> This just saves the raw buffer contents, but
> gnus-summary-resend-message-edit expects it to be a proper RFC2822 mail,
> with suitable MIME headers.

OK, but then where is it documented that "S D e" in Gnus Summary is
supposed to read the draft saved by "C-c C-d" in the mail buffer?
Could it be that these two are not supposed to work together?

The node "Drafts" in the Gnus manual describesa different procedure.
If I follow it, then after typing "D e" in the nndraft:drafts group, I
get placed in the *unsent mail* buffer with the correct Cyrillic text.

Nickolai, did you try to use these different commands in your
scenario?

Message #33 received at 75397-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>,
 Michael Albinus <michael.albinus <at> gmx.de>
Cc: 75397-done <at> debbugs.gnu.org, rpluim <at> gmail.com, schwab <at> linux-m68k.org
Subject: Re: [Nickolai Dobrynin] Re: bug#75397: 29.4; GNUS message mode lacks
 Cyrillic support (GNU/Linux)
Date: Sun, 26 Jan 2025 11:05:29 +0200

> From: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
> Date: Sun, 26 Jan 2025 09:51:28 +0100
> 
> Eli, Andreas, Robert,
> 
> Apologies for contacting you directly outside the bug tracking system.
> I did reply to Eli's question, but my message never made it to the
> discussion thread at
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=753971
> 
> (Why?)

Maybe Michael (CC'ed) could explain that.

> Anyway, please find my response below.
> 
> Best regards.
> 
> -------------------- Start of forwarded message --------------------
> From: Nickolai Dobrynin <ndobrynin <at> gmail.com>
> To: 75397 <at> debbugs.gnu.org
> Subject: Re: bug#75397: 29.4; GNUS message mode lacks Cyrillic support
>  (GNU/Linux)
> Date: Wed, 08 Jan 2025 21:38:34 +0100
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Andreas Schwab <schwab <at> linux-m68k.org>
> >> Cc: ndobrynin <at> gmail.com,  75397 <at> debbugs.gnu.org
> >> Date: Wed, 08 Jan 2025 17:53:16 +0100
> >> 
> >> On Jan 08 2025, Eli Zaretskii wrote:
> >> 
> >> >> Cc: 75397 <at> debbugs.gnu.org
> >> >> From: Andreas Schwab <schwab <at> linux-m68k.org>
> >> >> Date: Mon, 06 Jan 2025 10:36:34 +0100
> >> >> 
> >> >> On Jan 06 2025, Nickolai Dobrynin wrote:
> >> >> 
> >> >> > 1. Type M-gnus to start GNUS.
> >> >> > 2. Start composing a message by pressing 'm'.
> >> >> > 3. Enter the first, say, 5 letters of the Cyrillic alphabet, i.e. абвгд.
> >> >> > 4. Press C-c C-d to save the draft.
> >> >> > 5. Press S D e to resume editing.
> >> >> > 6. Bizarrely enough, instead of "абвгд", you will see a sequence of digits "01234".
> >> >> >
> >> >> > Is this even real?? By the way, this does not occur with ordinary text
> >> >> > files. And everything works fine in the view mode inside the draft
> >> >> > group. Pressing S D e, on the other hand, makes things go haywire.
> >> >> 
> >> >> The problem is that message-dont-send doesn't encode the message when it
> >> >> saves it.
> >> >
> >> > I must be missing something here, because I don't see what you
> >> > describe above:
> >> 
> >> It is about MIME encoding.
> >> 
> >> >   (defun message-dont-send ()
> >> >     "Don't send the message you have been editing.
> >> >   Instead, just auto-save the buffer and then bury it."
> >> >     (interactive nil message-mode)
> >> >     (set-buffer-modified-p t)
> >> >     (save-buffer)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> >> >     (let ((actions message-postpone-actions))
> >> >       (message-bury (current-buffer))
> >> >       (message-do-actions actions)))
> >> 
> >> This just saves the raw buffer contents, but
> >> gnus-summary-resend-message-edit expects it to be a proper RFC2822 mail,
> >> with suitable MIME headers.
> >
> > OK, but then where is it documented that "S D e" in Gnus Summary is
> > supposed to read the draft saved by "C-c C-d" in the mail buffer?
> > Could it be that these two are not supposed to work together?
> >
> > The node "Drafts" in the Gnus manual describesa different procedure.
> > If I follow it, then after typing "D e" in the nndraft:drafts group, I
> > get placed in the *unsent mail* buffer with the correct Cyrillic text.
> >
> > Nickolai, did you try to use these different commands in your
> > scenario?
> 
> I just tried them now, and I can confirm that both "e" and "D e"
> work. Based on the recent discussion, it appears I've made a newbie
> mistake by using "S D e".

Thanks for telling us.  So I'm now closing this bug.

Message #36 received at 75397 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: "Nickolai Dobrynin" <ndobrynin <at> gmail.com>
Cc: 75397 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, schwab <at> linux-m68k.org,
 rpluim <at> gmail.com
Subject: Re: [Nickolai Dobrynin] Re: bug#75397: 29.4; GNUS message mode
 lacks Cyrillic support (GNU/Linux)
Date: Sun, 26 Jan 2025 10:30:03 +0100

Hi Nickolai,

>> Eli, Andreas, Robert,
>>
>> Apologies for contacting you directly outside the bug tracking system.
>> I did reply to Eli's question, but my message never made it to the
>> discussion thread at
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=753971
>>
>> (Why?)
>
> Maybe Michael (CC'ed) could explain that.

Perhaps it helps to use the correct bug number (75397 instead of
753971)?

The debbugs mailman logs show two posts from you on Jan 5 and Jan
7. Nothing on Jan 8; perhaps a reject due to an error of the moderator?

Otherwise I remember a report of Bob Proulx about GNU servers (including
debbugs) being under hard spam attcks, some weeks ago. Perhaps this is related?

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.