GNU bug report logs - #74395
'mumi send-email' fails to detect created issue number

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 74395 in the body.
You can then email your comments to 74395 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: bug-guix <bug-guix <at> gnu.org>
Cc: Arun Isaac <arunisaac <at> systemreboot.net>
Subject: 'mumi send-email' fails to detect created issue number
Date: Sun, 17 Nov 2024 21:21:04 +0900

Hello,

I've been using 'mumi send-email', and one annoyance is that it doesn't
seem to reliably (or at all?) detect the created issue in the tracker,
even after the issue was synced to the MUMI database and visible through
the web interface.

For example, the issue https://issues.guix.gnu.org/74394 took about 3
minutes to be processed by GNU Debbugs and was available in the web
interface of Mumi, while the 'mumi send-email' client kept printing:

--8<---------------cut here---------------start------------->8---
Server has not yet received our email. Will retry in 60 seconds. 14 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 13 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 12 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 11 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 10 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 9 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 8 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 7 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 6 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 5 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 4 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 3 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 2 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 1 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 0 retries remaining.
Mail not acknowledged by issue tracker. Giving up.
--8<---------------cut here---------------end--------------->8---

It'd be nice to investigate and fix that.

-- 
Thanks,
Maxim

Message #8 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 74395 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Wed, 12 Feb 2025 21:04:49 +0900

Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hello,
>
> I've been using 'mumi send-email', and one annoyance is that it doesn't
> seem to reliably (or at all?) detect the created issue in the tracker,
> even after the issue was synced to the MUMI database and visible through
> the web interface.
>
> For example, the issue https://issues.guix.gnu.org/74394 took about 3
> minutes to be processed by GNU Debbugs and was available in the web
> interface of Mumi, while the 'mumi send-email' client kept printing:
>
> Server has not yet received our email. Will retry in 60 seconds. 14 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 13 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 12 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 11 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 10 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 9 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 8 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 7 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 6 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 5 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 4 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 3 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 2 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 1 retries remaining.
> Server has not yet received our email. Will retry in 60 seconds. 0 retries remaining.
> Mail not acknowledged by issue tracker. Giving up.
>
> It'd be nice to investigate and fix that.

The reason is that mumi on the server side attempts to reindex the
complete 20 GiB something of mail archives every 5 minutes, which takes
20 minutes to complete.

We need a finer grained mechanism to process only newly added mails and
avoid doing a full reindex, which is too costly.

-- 
Thanks,
Maxim

Message #11 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Wed, 12 Feb 2025 12:59:45 +0000

Hi Maxim,

> The reason is that mumi on the server side attempts to reindex the
> complete 20 GiB something of mail archives every 5 minutes, which takes
> 20 minutes to complete.
>
> We need a finer grained mechanism to process only newly added mails and
> avoid doing a full reindex, which is too costly.

Thanks for investigating this. I've been thinking along the same lines
as well. I can come up with something, but do you have any suggestions
off the bat?

Regards,
Arun

Message #14 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Wed, 12 Feb 2025 14:49:08 +0000

> I can come up with something, but do you have any suggestions off the
> bat?

My first instinct would be to look at the timestamps of changed log
files and only reindex those. Maybe that could work. Do you wish to work
on this, or do you prefer I did it? Let me know.

Thanks!

Message #17 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Wed, 12 Feb 2025 23:57:32 +0900

Hi Arun,

Arun Isaac <arunisaac <at> systemreboot.net> writes:

> Hi Maxim,
>
>> The reason is that mumi on the server side attempts to reindex the
>> complete 20 GiB something of mail archives every 5 minutes, which takes
>> 20 minutes to complete.
>>
>> We need a finer grained mechanism to process only newly added mails and
>> avoid doing a full reindex, which is too costly.
>
> Thanks for investigating this. I've been thinking along the same lines
> as well. I can come up with something, but do you have any suggestions
> off the bat?

There's already some code to avoid doing a full re-indexing of the
archived messages, but it wasn't actually used until commit 0e8852b3
("scripts: Do not loose arguments when looping in `update-state!'.") in
mumi, which is recent.

I think reconfiguring mumi at its latest commit could already help,
although the update-state! code currently will run a full update every
10 times, and it's currently configured to run every 30 seconds although
our rsync of debbugs data is every 5 minutes.

We'll have to tweak these values so that a full reindex doesn't happen
so often (perhaps once per day would be sufficient).  The rsync could be
made a bit more aggressively (every minute perhaps) to help reactivity.

-- 
Thanks,
Maxim

Message #20 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Thu, 13 Feb 2025 14:05:43 +0900

Hi Arun,

Arun Isaac <arunisaac <at> systemreboot.net> writes:

>> I can come up with something, but do you have any suggestions off the
>> bat?
>
> My first instinct would be to look at the timestamps of changed log
> files and only reindex those. Maybe that could work. Do you wish to work
> on this, or do you prefer I did it? Let me know.

It sounds like a fun fix to come up with, but I'm already swamped trying
ot finish tens of unfinished things, so by all means, go ahead if you
have time/interest in fixing it!

Some thoughts I had:

Not sure if rsync produces some kind of lock file while they're busy
updating the tree; if they do it could be useful to avoid re-indexing
while it's actively syncing.

It would be even nicer to send some signal to mumi when rsync completes,
which would notify it to re-index.

Other things that may be useful: inotify to keep track of new/updated
files.

Happy hacking!

-- 
Maxim

Message #27 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: 'mumi send-email' fails to detect created issue number
Date: Thu, 13 Feb 2025 22:21:25 +0000

> It sounds like a fun fix to come up with, but I'm already swamped trying
> ot finish tens of unfinished things, so by all means, go ahead if you
> have time/interest in fixing it!

Sure, happy to work on this! I have some spare capacity at the moment
and this is very important to make `mumi send-email' more usable.

> Some thoughts I had:
>
> Not sure if rsync produces some kind of lock file while they're busy
> updating the tree; if they do it could be useful to avoid re-indexing
> while it's actively syncing.
>
> It would be even nicer to send some signal to mumi when rsync completes,
> which would notify it to re-index.
>
> Other things that may be useful: inotify to keep track of new/updated
> files.

All good leads to look into. Thanks! Will keep you posted here.

Message #32 received at 74395-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 74395-done <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: Re: 'mumi send-email' fails to detect created issue number
Date: Fri, 14 Mar 2025 01:55:45 +0000

Hi Maxim,

These changes should fix the indexing speed issue. We now only index
modified issues.

https://git.savannah.gnu.org/cgit/guix/mumi.git/commit/?id=1e4228d78ca82bf146f6eb318f6cfbc1558a1a0a
https://git.savannah.gnu.org/cgit/guix/mumi.git/commit/?id=3df61afdd0c1a869f4328c17573797157c4583c8

These changes are not deployed to issues.guix.gnu.org yet, but I will do
so by next week. Please reopen this issue if the problem persists after
that.

Cheers!
Arun

Message #40 received at 74395-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 74395-done <at> debbugs.gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: Re: 'mumi send-email' fails to detect created issue number
Date: Sun, 16 Mar 2025 06:49:41 +0000

Hi Maxim,

> These changes are not deployed to issues.guix.gnu.org yet, but I will
> do so by next week.

These changes have been deployed now. When next you use mumi send-email,
please let me know if it works reliably.

Thanks,
Arun

Message #43 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: closed (Re: 'mumi send-email' fails to detect
 created issue number)
Date: Sun, 16 Mar 2025 20:40:24 +0900

Hi Arun,

help-debbugs <at> gnu.org (GNU bug Tracking System) writes:

[...]

> These changes should fix the indexing speed issue. We now only index
> modified issues.
>
> https://git.savannah.gnu.org/cgit/guix/mumi.git/commit/?id=1e4228d78ca82bf146f6eb318f6cfbc1558a1a0a
> https://git.savannah.gnu.org/cgit/guix/mumi.git/commit/?id=3df61afdd0c1a869f4328c17573797157c4583c8
>
> These changes are not deployed to issues.guix.gnu.org yet, but I will do
> so by next week. Please reopen this issue if the problem persists after
> that.

Woohoo!  Great news!  Thank you for fixing this old issue.  Should we
also raise the frequency of the rsync job?  Currently it's every 5
minutes; we could make try perhaps every minute, but it should block
when it's already running.  I think the new Shepherd timer API would
allow defining it as such.

-- 
Thanks,
Maxim

Message #46 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: closed (Re: 'mumi send-email' fails to detect
 created issue number)
Date: Sun, 16 Mar 2025 16:47:45 +0000

Hi Maxim,

> Should we also raise the frequency of the rsync job? Currently it's
> every 5 minutes; we could make try perhaps every minute, but it should
> block when it's already running. I think the new Shepherd timer API
> would allow defining it as such.

Ah, good point! I didn't think of that. That would be worth doing to
further improve responsiveness.

Is there a way to prevent multiple rsync instances from running at the
same time? I understand it can be dangerous.
https://serverfault.com/questions/332577/is-it-safe-to-running-multiple-instances-of-rsync-together

Did you mean that the new Shepherd timer API has a way to achieve this
blocking?

Regards,
Arun

Message #49 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: closed (Re: 'mumi send-email' fails to detect
 created issue number)
Date: Wed, 19 Mar 2025 21:43:10 +0900

Hi Arun,

I tried to use 'mumi send-email' today, and unfortunately it didn't
work:

--8<---------------cut here---------------start------------->8---
$ mumi send-email libvirt-uefi/*.patch
git send-email --to=guix-patches <at> gnu.org libvirt-uefi/0000-cover-letter.patch
libvirt-uefi/0000-cover-letter.patch

[...]

Result: 250 
Server has not yet received our email. Will retry in 60 seconds. 14 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 13 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 12 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 11 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 10 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 9 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 8 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 7 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 6 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 5 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 4 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 3 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 2 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 1 retries remaining.
Server has not yet received our email. Will retry in 60 seconds. 0 retries remaining.
Mail not acknowledged by issue tracker. Giving up.
--8<---------------cut here---------------end--------------->8---


[...]

>> Should we also raise the frequency of the rsync job? Currently it's
>> every 5 minutes; we could make try perhaps every minute, but it should
>> block when it's already running. I think the new Shepherd timer API
>> would allow defining it as such.
>
> Ah, good point! I didn't think of that. That would be worth doing to
> further improve responsiveness.
>
> Is there a way to prevent multiple rsync instances from running at the
> same time? I understand it can be dangerous.
> https://serverfault.com/questions/332577/is-it-safe-to-running-multiple-instances-of-rsync-together
>
> Did you mean that the new Shepherd timer API has a way to achieve this
> blocking?

Yes; the make-timer-constructor has a #:wait-for-termination? argument
(defaults to #f), which would need to be set to #t.  It's documented in
(info "(shepherd) Timers").

I'll try converting the mcron in a bit, I'll let you know how it went.

-- 
Thanks,
Maxim

Message #52 received at 74395 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 74395 <at> debbugs.gnu.org
Subject: Re: bug#74395: closed (Re: 'mumi send-email' fails to detect
 created issue number)
Date: Wed, 19 Mar 2025 23:17:14 +0000

Hi Maxim,

> I tried to use 'mumi send-email' today, and unfortunately it didn't
> work:

Oops, I think I introduced a regression with all the recent code churn.
I have fixed it now.
https://git.savannah.gnu.org/cgit/guix/mumi.git/commit/?id=5386332eab96009e315bda6682bb16102c092e73

I have also reconfigured berlin with the new code, and tested a run of
`mumi send-email', and it works! It still takes about 5 minutes due to
the rsync interval, but we're getting somewhere now! :-) Please confirm
that it works for you too.

> Yes; the make-timer-constructor has a #:wait-for-termination? argument
> (defaults to #f), which would need to be set to #t.  It's documented in
> (info "(shepherd) Timers").
>
> I'll try converting the mcron in a bit, I'll let you know how it went.

That's great, thank you!

Regards,
Arun

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.