GNU bug report logs - #11599
New Gnus reads full group information too often

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11599 in the body.
You can then email your comments to 11599 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Wiegley <jwiegley <at> gmail.com>
To: submit <at> debbugs.gnu.org (The Gnus Bugfixing Girls + Boys)
Subject: New Gnus reads full group information too often
Date: Thu, 31 May 2012 20:01:37 -0500

I have many mailing lists in IMAP, newsgroups in NNTP, and virtual groups that
aggregate these two into topical categories.

With the latest Ma Gnus from Git:

  1. Hit 'g' to update all my Groups.  This is fast.

  2. Hit 'g' again, this is very fast.

  3. Press RET to visit any IMAP group.

  4. Hit 'g', things are still very fast.

  5. Press RET to visit any IMAP group.  Then press RET to visit any other
     IMAP group.

  6. Hit 'g'.  Now I see this in the minibuffer:

     nnimap read 876k from localhost (initial sync of 1 group; please wait)

  7. The real problem is that if I visit several groups (some upwards of
     200,000 messages on the IMAP server), the next time I hit 'g' it will
     read many tens of megabytes from every one of those groups, taking more
     than a minute and completely locking up Emacs.

This is new behavior, as I always run the following after hitting 'g':

--8<---------------cut here---------------start------------->8---
(defun gnus-score-groups (&optional arg)
  (interactive)
  (save-excursion
    (dolist (info (cdr gnus-newsrc-alist))
      ;; Only consider this group if it's at or below the current level
      (when (<= (gnus-info-level info)
                (if (numberp arg)
                    arg
                  (or (gnus-group-default-level nil t)
                      (funcall
                       (symbol-function 'gnus-group-default-list-level))
                      gnus-level-subscribed)))
        (let* ((group (gnus-info-group info))
               (unread (gnus-group-unread group)))
          (when (and (string-match "^\\(list\\.\\|nntp\\+LocalNews:\\)" group)
                     (not (string= "INBOX" group))
                     (numberp unread) (> unread 0))
            (ignore-errors
              (gnus-summary-read-group group nil t))
            (when (and gnus-summary-buffer
                       (buffer-live-p gnus-summary-buffer)
                       (eq (current-buffer)
                           (get-buffer gnus-summary-buffer)))
              (message "Exiting summary for group %s ..." group)
              (gnus-summary-exit))))))))
--8<---------------cut here---------------end--------------->8---

This visits every IMAP or NNTP group that uses adaptive scoring, so that the
resulting unread counts reflect reality.  Yet after I run this, Gnus does an
"initial sync" of 33 or more groups, bringing Emacs to its knees every time.

I started noticing this behavior in the last couple days.  I've updated Gnus
twice in that time (I think).  Right now I'm at commit 1bd6e87.

Thanks,
  John Wiegley

Ma Gnus v0.6
GNU Emacs 23.4.1 (x86_64-apple-darwin11.2.0, Carbon Version 1.6.0 AppKit 1138.23)
 of 2012-01-31 on vulcan
200 Leafnode NNTP daemon, version 2.0.0.alpha20110815a.luascript at newartisans.com 
500 Unknown command

Message #8 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: John Wiegley <jwiegley <at> gmail.com>
Cc: 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Mon, 11 Jun 2012 23:24:13 +0200

John Wiegley <jwiegley <at> gmail.com> writes:

>   5. Press RET to visit any IMAP group.  Then press RET to visit any other
>      IMAP group.
>
>   6. Hit 'g'.  Now I see this in the minibuffer:
>
>      nnimap read 876k from localhost (initial sync of 1 group; please wait)

Right.  Something happens to the `unexist' tracking that makes Gnus
think that it needs to do a full resync, because it's gotten totally out
of sync.

I've had the same report from another user who's done extensive tracing
to try to find out why this happens, but so far I haven't been able to
reproduce or figure out why this happens.

Could you try to `g' until it stops doing the "initial sync" thing.  And
then `G E' on a group when it's "ok", and then `G E' the group again
after you've triggered the "initial sync" thing?  I think nnimap messes
up the group data somehow...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #11 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: John Wiegley <jwiegley <at> gmail.com>
To: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
Cc: Dave Abrahams <dave <at> boostpro.com>, 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Mon, 11 Jun 2012 17:45:41 -0500

>>>>> Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> Right.  Something happens to the `unexist' tracking that makes Gnus think
> that it needs to do a full resync, because it's gotten totally out of sync.

> I've had the same report from another user who's done extensive tracing to
> try to find out why this happens, but so far I haven't been able to
> reproduce or figure out why this happens.

> Could you try to `g' until it stops doing the "initial sync" thing.  And
> then `G E' on a group when it's "ok", and then `G E' the group again after
> you've triggered the "initial sync" thing?  I think nnimap messes up the
> group data somehow...

You know, I think this was Dovecot's fault, not Gnus.  I noticed that in the
groups where this was occurring, the IMAP message counts got doubled (in some
cases, many times over).  So I killed all the dups, did a force-resync on
everything, and since then Gnus has never said "initial sync" to me.

So, all is back to normal here, but I'm still in the dark as to what caused
the message duplication, or the re-syncing in Gnus.  Which one came first is a
mystery to me.

John

Message #14 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: John Wiegley <jwiegley <at> gmail.com>
Cc: Dave Abrahams <dave <at> boostpro.com>, 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Wed, 20 Jun 2012 19:29:19 +0200

John Wiegley <jwiegley <at> gmail.com> writes:

> You know, I think this was Dovecot's fault, not Gnus.  I noticed that
> in the groups where this was occurring, the IMAP message counts got
> doubled (in some cases, many times over).  So I killed all the dups,
> did a force-resync on everything, and since then Gnus has never said
> "initial sync" to me.

Right.  But since this is something that Dovecot seems to do
sporadically, it's probably a bug in Gnus that triggers it somehow.  I
still haven't been able to reproduce it, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #17 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: "John Wiegley" <johnw <at> boostpro.com>
To: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
Cc: Dave Abrahams <dave <at> boostpro.com>, 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Fri, 22 Jun 2012 06:07:22 -0500

>>>>> Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> Right.  But since this is something that Dovecot seems to do
> sporadically, it's probably a bug in Gnus that triggers it somehow.  I
> still haven't been able to reproduce it, though.

It's come back again, somewhat.  When I hit '6 g', I see "initial sync of 1
group(s)", every time.  No message duplication happening in Dovecot this time,
though.

-- 
John Wiegley
BoostPro Computing
http://www.boostpro.com

Message #26 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "John Wiegley" <johnw <at> boostpro.com>
Cc: Dave Abrahams <dave <at> boostpro.com>, 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Wed, 25 Jan 2017 15:31:32 +0100

"John Wiegley" <johnw <at> boostpro.com> writes:

>>>>>> Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:
>
>> Right.  But since this is something that Dovecot seems to do
>> sporadically, it's probably a bug in Gnus that triggers it somehow.  I
>> still haven't been able to reproduce it, though.
>
> It's come back again, somewhat.  When I hit '6 g', I see "initial sync of 1
> group(s)", every time.  No message duplication happening in Dovecot this time,
> though.

I think the message duplication issue has been fixed?  But I still see
the "Initial sync" thing from time on some servers, mainly Gmail.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #29 received at 11599 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "John Wiegley" <johnw <at> boostpro.com>
Cc: Dave Abrahams <dave <at> boostpro.com>, 11599 <at> debbugs.gnu.org
Subject: Re: bug#11599: New Gnus reads full group information too often
Date: Sun, 19 Jul 2020 18:14:37 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> I think the message duplication issue has been fixed?  But I still see
> the "Initial sync" thing from time on some servers, mainly Gmail.

I haven't seen this is some years now, so I'm going to go ahead and
guess that this problem has gone away, and I'm closing this bug report.

Please reopen if it's still an issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.