GNU bug report logs - #76655
30.1; MH-E: "unseen" folder broken on OpenBSD

Previous Next

To reply to this bug, email your comments to 76655 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <kupfer <at> rawbw.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Fri, 28 Feb 2025 16:56:41 -0800

A user running 30.1 reported that when he runs mh-index-new-messages,
the resulting index folder (+mhe-index/sequence/unseen) does not list
the source folder(s) for the found messages.  The folder should look
something like

  +readLater/linux
    4  02/28 [Van Snyder     ]  Re: Hardware question
    6  02/28   [Greg Wooledge  ]  Re: Hardware question
    8 %02/28 [Salvatore Bonac]  [SECURITY] [DSA 5872-1] xorg-server security upda
  
  +readLater/misc
   22+t02/28 [Substack       ]  In-app video publishing, Notes stats, and a new t
   23 %02/28 [Palo Alto Farme]  Shop the Market, Support Our Community

But this user is missing the "+readLater/linux" and "+readLater/misc"
lines.

Furthermore, if the user views or deletes the message in the index
folder, a corresponding action is supposed to be performed on the
message in the source folder.  That's not happening.

Normally, messages in an MH-E index folder should have an
"X-MHE-Checksum" header line.  The user reports that those headers are
missing from the emails in the index folder.  MH-E uses this header to
tie the entry in the index folder to the entry in the source folder, so
their absence explains the above two problems.

The relevant code in MH-E is rather twisty, but it looks like the
headers are missing due to a bug in mh-index-sequenced-messages.  MH-E
calls out to an external program to compute the MD5 checksum for a
message.  mh-index-sequenced-messages sets
mh-search-next-result-function to mh-mairix-next-result, which
works if the external program is md5sum, which is what Linux users
usually get.

But the user reports that mh-checksum-cmd is set to invoke openssl,
and it looks like MH-E is unable to parse the openssl output.

The reason for using an external program are historical.  (These
problems are not a regression in Emacs 30.  The user also saw them with
Emacs 29.)  MH-E should get rid of the current framework
(mh-checksum-cmd, mh-checksum-parser, mh-checksum-choose, etc.) and just
use Emacs's md5 function.

Message #8 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <kupfer <at> rawbw.com>
To: 76655 <at> debbugs.gnu.org
Subject: more investigation needed
Date: Fri, 28 Feb 2025 17:59:06 -0800

I looked at the code some more and ran an additional test.  I now think
that the thing that I noticed about mh-search-next-result-function is
unrelated to the checksum problem.  If I force Emacs on my Linux system
to use openssl, I still get the proper results in the index folder, and
I see a checksum header in a test email.

So there's something else peculiar about OpenBSD that I need to track
down.  I'll try standing up an OpenBSD VM to reproduce the problem.

It would still be nice to replace the calls to an external program with
the Emacs md5 function.

mike

Message #11 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mike Kupfer <kupfer <at> rawbw.com>, Bill Wohler <wohler <at> newt.com>
Cc: 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sat, 01 Mar 2025 10:59:19 +0200

> From: Mike Kupfer <kupfer <at> rawbw.com>
> Date: Fri, 28 Feb 2025 16:56:41 -0800
> 
> A user running 30.1 reported that when he runs mh-index-new-messages,
> the resulting index folder (+mhe-index/sequence/unseen) does not list
> the source folder(s) for the found messages.  The folder should look
> something like
> 
>   +readLater/linux
>     4  02/28 [Van Snyder     ]  Re: Hardware question
>     6  02/28   [Greg Wooledge  ]  Re: Hardware question
>     8 %02/28 [Salvatore Bonac]  [SECURITY] [DSA 5872-1] xorg-server security upda
>   
>   +readLater/misc
>    22+t02/28 [Substack       ]  In-app video publishing, Notes stats, and a new t
>    23 %02/28 [Palo Alto Farme]  Shop the Market, Support Our Community
> 
> But this user is missing the "+readLater/linux" and "+readLater/misc"
> lines.
> 
> Furthermore, if the user views or deletes the message in the index
> folder, a corresponding action is supposed to be performed on the
> message in the source folder.  That's not happening.
> 
> Normally, messages in an MH-E index folder should have an
> "X-MHE-Checksum" header line.  The user reports that those headers are
> missing from the emails in the index folder.  MH-E uses this header to
> tie the entry in the index folder to the entry in the source folder, so
> their absence explains the above two problems.
> 
> The relevant code in MH-E is rather twisty, but it looks like the
> headers are missing due to a bug in mh-index-sequenced-messages.  MH-E
> calls out to an external program to compute the MD5 checksum for a
> message.  mh-index-sequenced-messages sets
> mh-search-next-result-function to mh-mairix-next-result, which
> works if the external program is md5sum, which is what Linux users
> usually get.
> 
> But the user reports that mh-checksum-cmd is set to invoke openssl,
> and it looks like MH-E is unable to parse the openssl output.
> 
> The reason for using an external program are historical.  (These
> problems are not a regression in Emacs 30.  The user also saw them with
> Emacs 29.)  MH-E should get rid of the current framework
> (mh-checksum-cmd, mh-checksum-parser, mh-checksum-choose, etc.) and just
> use Emacs's md5 function.

Bill, any comments or suggestions?

Message #14 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mike Kupfer <kupfer <at> rawbw.com>, Bill Wohler <wohler <at> newt.com>
Cc: 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sat, 01 Mar 2025 11:08:30 +0200

> From: Mike Kupfer <kupfer <at> rawbw.com>
> Date: Fri, 28 Feb 2025 17:59:06 -0800
> 
> I looked at the code some more and ran an additional test.  I now think
> that the thing that I noticed about mh-search-next-result-function is
> unrelated to the checksum problem.  If I force Emacs on my Linux system
> to use openssl, I still get the proper results in the index folder, and
> I see a checksum header in a test email.
> 
> So there's something else peculiar about OpenBSD that I need to track
> down.  I'll try standing up an OpenBSD VM to reproduce the problem.
> 
> It would still be nice to replace the calls to an external program with
> the Emacs md5 function.

Thanks, but please don't change the Subject when you reply to a bug
discussion.  Doing so makes it harder to find related messages, at
least in Rmail.

I've restored the original Subject in this response message.

Message #17 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Bill Wohler <wohler <at> newt.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76655 <at> debbugs.gnu.org, Mike Kupfer <kupfer <at> rawbw.com>
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sat, 01 Mar 2025 09:40:52 -0800

Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Mike Kupfer <kupfer <at> rawbw.com>
> > Date: Fri, 28 Feb 2025 16:56:41 -0800
> > 
> > A user running 30.1 reported that when he runs mh-index-new-messages,
> > the resulting index folder (+mhe-index/sequence/unseen) does not list
> > the source folder(s) for the found messages.  The folder should look
> > something like
> > 
> >   +readLater/linux
> >     4  02/28 [Van Snyder     ]  Re: Hardware question
> >     6  02/28   [Greg Wooledge  ]  Re: Hardware question
> >     8 %02/28 [Salvatore Bonac]  [SECURITY] [DSA 5872-1] xorg-server security upda
> >   
> >   +readLater/misc
> >    22+t02/28 [Substack       ]  In-app video publishing, Notes stats, and a new t
> >    23 %02/28 [Palo Alto Farme]  Shop the Market, Support Our Community
> > 
> > But this user is missing the "+readLater/linux" and "+readLater/misc"
> > lines.
> > 
> > Furthermore, if the user views or deletes the message in the index
> > folder, a corresponding action is supposed to be performed on the
> > message in the source folder.  That's not happening.
> > 
> > Normally, messages in an MH-E index folder should have an
> > "X-MHE-Checksum" header line.  The user reports that those headers are
> > missing from the emails in the index folder.  MH-E uses this header to
> > tie the entry in the index folder to the entry in the source folder, so
> > their absence explains the above two problems.
> > 
> > The relevant code in MH-E is rather twisty, but it looks like the
> > headers are missing due to a bug in mh-index-sequenced-messages.  MH-E
> > calls out to an external program to compute the MD5 checksum for a
> > message.  mh-index-sequenced-messages sets
> > mh-search-next-result-function to mh-mairix-next-result, which
> > works if the external program is md5sum, which is what Linux users
> > usually get.
> > 
> > But the user reports that mh-checksum-cmd is set to invoke openssl,
> > and it looks like MH-E is unable to parse the openssl output.
> > 
> > The reason for using an external program are historical.  (These
> > problems are not a regression in Emacs 30.  The user also saw them with
> > Emacs 29.)  MH-E should get rid of the current framework
> > (mh-checksum-cmd, mh-checksum-parser, mh-checksum-choose, etc.) and just
> > use Emacs's md5 function.
> 
> Bill, any comments or suggestions?

Mike and I have been discussing this, and I agree that minimizing
dependencies and using the the internal (md5) function is a good thing.
As Mike mentioned in another email, there may be more going on on our
user's OpenBSD system that Mike is going to try to untangle.

-- 
Bill Wohler <wohler <at> newt.com> aka <Bill.Wohler <at> nasa.gov>
http://www.newt.com/wohler/, GnuPG ID:610BD9AD

Message #20 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Mike Kupfer <kupfer <at> rawbw.com>, 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sun, 2 Mar 2025 01:09:11 +0000

Mike Kupfer <kupfer <at> rawbw.com> writes:

> The reason for using an external program are historical.  (These
> problems are not a regression in Emacs 30.  The user also saw them with
> Emacs 29.)  MH-E should get rid of the current framework
> (mh-checksum-cmd, mh-checksum-parser, mh-checksum-choose, etc.) and just
> use Emacs's md5 function.

FWIW, I'd use `secure-hash' instead, if possible:

    (secure-hash 'md5 "foo")

Message #23 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <kupfer <at> rawbw.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sat, 01 Mar 2025 17:51:43 -0800

Stefan Kangas wrote:

> FWIW, I'd use `secure-hash' instead, if possible:
> 
>     (secure-hash 'md5 "foo")

What's the advantage of using 'secure-hash' instead of 'md5'?

mike

Message #26 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Mike Kupfer <kupfer <at> rawbw.com>
Cc: 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sun, 2 Mar 2025 02:24:40 +0000

Mike Kupfer <kupfer <at> rawbw.com> writes:

> Stefan Kangas wrote:
>
>> FWIW, I'd use `secure-hash' instead, if possible:
>>
>>     (secure-hash 'md5 "foo")
>
> What's the advantage of using 'secure-hash' instead of 'md5'?

I just think it's a better API, e.g. it doesn't have NOERROR and it has
BINARY.  It also supports other hashing algorithms.

Just my two cents, of course.

Message #29 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <kupfer <at> rawbw.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 76655 <at> debbugs.gnu.org, wohler <at> newt.com
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Sun, 02 Mar 2025 14:11:26 -0800

Stefan Kangas wrote:

> Mike Kupfer <kupfer <at> rawbw.com> writes:
> 
> > Stefan Kangas wrote:
> >
> >> FWIW, I'd use `secure-hash' instead, if possible:
> >>
> >>     (secure-hash 'md5 "foo")
> >
> > What's the advantage of using 'secure-hash' instead of 'md5'?
> 
> I just think it's a better API, e.g. it doesn't have NOERROR and it has
> BINARY.  It also supports other hashing algorithms.
> 
> Just my two cents, of course.

Okay.  Thanks for pointing out that there's an alternative API to
consider.  I doubt I would have found it on my own.

mike

Message #32 received at 76655 <at> debbugs.gnu.org (full text, mbox):

From: Mike Kupfer <kupfer <at> rawbw.com>
To: Bill Wohler <wohler <at> newt.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76655 <at> debbugs.gnu.org
Subject: Re: bug#76655: 30.1; MH-E: "unseen" folder broken on OpenBSD
Date: Wed, 26 Mar 2025 15:05:33 -0700

Bill Wohler wrote:

> As Mike mentioned in another email, there may be more going on on our
> user's OpenBSD system that Mike is going to try to untangle.

Status update: I've got an OpenBSD VM set up, but I'm unable to
reproduce the problem that the user reported.  I'll think about next
steps.

mike

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.