GNU bug report logs - #28502
25.3; list-packages ends with "error in process filter: End of file during parsing"

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28502 in the body.
You can then email your comments to 28502 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Mon, 18 Sep 2017 12:48:15 -0500

Hello,

I seem to have encountered a bug. From emacs -Q:

(require 'package)
(setq package-archives
      '(("melpa" . "https://melpa.org/packages/")
        ("melpa-stable" . "https://stable.melpa.org/packages/")
        ("gnu" . "https://elpa.gnu.org/packages/")))
(package-initialize)

Then do M-x list-packages. I get a message "error in process filter: End of file during parsing."

In GNU Emacs 25.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.19)
 of 2017-09-16 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.11903000
Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Message #8 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 28502 <at> debbugs.gnu.org
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Tue, 26 Sep 2017 20:45:25 -0400

tags 28502 + unreproducible moreinfo
quit

Alex Branham <alex.branham <at> gmail.com> writes:

> Hello,
>
> I seem to have encountered a bug. From emacs -Q:
>
> (require 'package)
> (setq package-archives
>       '(("melpa" . "https://melpa.org/packages/")
>         ("melpa-stable" . "https://stable.melpa.org/packages/")
>         ("gnu" . "https://elpa.gnu.org/packages/")))
> (package-initialize)
>
> Then do M-x list-packages. I get a message "error in process filter: End of file during parsing."

Works for me.  Does it happen if you remove the (package-initialize)
call?  I think this could enable some of the packages you have
installed, so despite using 'emacs -Q' it's not entirely a clean
session.

If you M-x toggle-debug-on-error do you get a backtrace?

Message #13 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Alex Branham <alex.branham <at> gmail.com>, 28502 <at> debbugs.gnu.org
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Wed, 27 Sep 2017 15:43:47 +0200

Noam Postavsky <npostavs <at> users.sourceforge.net> writes:

> tags 28502 + unreproducible moreinfo

AFAIR I also saw this once.  Then I deleted

  "~/.emacs.d/elpa/archives/melpa/archive-contents"

(this was on July 24th), and the problem was gone.

FWIW I have the deleted file still on my hard disc.  No fun to visit it
with Emacs however since it's one of these one-huge-line-only files.  At
least, `read' on position 1 succeeds, so I'm not sure what the problem
was with it.


Michael.

Message #16 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 28502 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Wed, 27 Sep 2017 09:06:43 -0500

Thanks for the replies.

After nuking ~/.emacs.d/elpa, the problem went away and hasn't recurred.
Perhaps there should be some sort of package-force-archive-refresh
function to force overwriting the archive contents?

Alex

On Wed 27 Sep 2017 at 13:43, Michael Heerdegen <michael_heerdegen <at> web.de> wrote:

> Noam Postavsky <npostavs <at> users.sourceforge.net> writes:
>
>> tags 28502 + unreproducible moreinfo
>
> AFAIR I also saw this once.  Then I deleted
>
>   "~/.emacs.d/elpa/archives/melpa/archive-contents"
>
> (this was on July 24th), and the problem was gone.
>
> FWIW I have the deleted file still on my hard disc.  No fun to visit it
> with Emacs however since it's one of these one-huge-line-only files.  At
> least, `read' on position 1 succeeds, so I'm not sure what the problem
> was with it.
>
>
> Michael.

Message #19 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 28502 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Wed, 27 Sep 2017 16:47:23 +0200

Alex Branham <alex.branham <at> gmail.com> writes:

> After nuking ~/.emacs.d/elpa, the problem went away and hasn't recurred.
> Perhaps there should be some sort of package-force-archive-refresh
> function to force overwriting the archive contents?

No, normally this should never happen, there seems to be a problem with
inconsistent or broken package caches.  We need to find out what goes
wrong.

Have you made a backup of the files you deleted btw?  They could be
useful for debugging.


Michael.

Message #22 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 28502 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Wed, 27 Sep 2017 09:50:41 -0500

On Wed 27 Sep 2017 at 14:47, Michael Heerdegen <michael_heerdegen <at> web.de> wrote:

>> Perhaps there should be some sort of package-force-archive-refresh
>> function to force overwriting the archive contents?
>
> No, normally this should never happen, there seems to be a problem with
> inconsistent or broken package caches.  We need to find out what goes
> wrong.

I'm betting that during a package-refresh-contents call the internet disconnected and that this somehow messed up the package cache.

>
> Have you made a backup of the files you deleted btw?  They could be
> useful for debugging.

No, unfortunately I didn't. If it happens again I will do so.

Message #25 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Alex Branham <alex.branham <at> gmail.com>, 28502 <at> debbugs.gnu.org
Subject: Re: bug#28502: 25.3;
 list-packages ends with "error in process filter: End of file during
 parsing"
Date: Fri, 29 Sep 2017 16:34:33 +0200

[Message part 1 (text/plain, inline)]

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> Noam Postavsky <npostavs <at> users.sourceforge.net> writes:
>
> > tags 28502 + unreproducible moreinfo
>
> AFAIR I also saw this once.  Then I deleted
>
>   "~/.emacs.d/elpa/archives/melpa/archive-contents"
>
> (this was on July 24th), and the problem was gone.
>
> FWIW I have the deleted file still on my hard disc.  No fun to visit it
> with Emacs however since it's one of these one-huge-line-only files.  At
> least, `read' on position 1 succeeds, so I'm not sure what the problem
> was with it.

FWIW, I see now that my error was a bit different; I can still reproduce
it with the copy of the file I had removed (it is attached):

Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
  package--add-to-archive-contents(nil "melpa")
  package-read-archive-contents("melpa")
  package-read-all-archive-contents()
  package-initialize()
  byte-code("..." 10)
  load("~/gnu-emacs/.gnu-emacs")
  eval-buffer(#<buffer  *load*> nil "/home/micha/.emacs" nil t)  ; Reading at buffer position 261
  load-with-code-conversion("/home/micha/.emacs" "/home/micha/.emacs" t t)
  load("~/.emacs" t t)
  #f(compiled-function () #<bytecode 0x2567f5>)()
  command-line()
  normal-top-level()


Michael.

[archive-contents (application/octet-stream, attachment)]

Message #32 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Alex Branham <alex.branham <at> gmail.com>, 28502 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3; list-packages ends with "error in process
 filter: End of file during parsing"
Date: Mon, 21 Oct 2019 00:05:57 +0200

[Message part 1 (text/plain, inline)]

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> FWIW, I see now that my error was a bit different; I can still reproduce
> it with the copy of the file I had removed (it is attached):
>
> Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
>   package--add-to-archive-contents(nil "melpa")
>   package-read-archive-contents("melpa")
>   package-read-all-archive-contents()
>   package-initialize()
>   byte-code("..." 10)
>   load("~/gnu-emacs/.gnu-emacs")
>   eval-buffer(#<buffer  *load*> nil "/home/micha/.emacs" nil t)  ; Reading at buffer position 261
>   load-with-code-conversion("/home/micha/.emacs" "/home/micha/.emacs" t t)
>   load("~/.emacs" t t)
>   #f(compiled-function () #<bytecode 0x2567f5>)()
>   command-line()
>   normal-top-level()

Thanks for the archive-contents file and backtrace.

The file you saw contained a package entry that was nil.  My guess is
that this was due to an intermittent error on MELPA, since it was
well-formed in all other respects.

I don't think it's necessary to signal an error in this case.  We should
just ignore nil entries.  On the other hand, it might indicate that
something is wrong with the package archive, so it's nice to warn about
it.

I've installed the attached patch on master which does that.

Best regards,
Stefan Kangas

[0001-Don-t-try-to-add-nil-packages-on-refresh.patch (application/octet-stream, attachment)]

Message #35 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 28502 <at> debbugs.gnu.org
Subject: Re: bug#28502: 25.3; list-packages ends with "error in process
 filter: End of file during parsing"
Date: Mon, 21 Oct 2019 03:27:25 +0200

Alex Branham <alex.branham <at> gmail.com> writes:

> Hello,
>
> I seem to have encountered a bug. From emacs -Q:
>
> (require 'package)
> (setq package-archives
>       '(("melpa" . "https://melpa.org/packages/")
>         ("melpa-stable" . "https://stable.melpa.org/packages/")
>         ("gnu" . "https://elpa.gnu.org/packages/")))
> (package-initialize)
>
> Then do M-x list-packages. I get a message "error in process filter: End of file during parsing."

Without a backtrace, it's hard to say exactly, but I strongly suspect
that the error you're seeing here happens when we try to `read' an
incompletely downloaded archive-contents file.

Normally, you should get the error saying "Failed to download `%s'
archive."  But if debug-on-error is non-nil, then you will get a
backtrace instead.  I think that is the correct behaviour.

Having read this thread and investigated this issue a bit, I have a hard
time figuring out exactly what to do.  I think we have too little to go
on: no backtrace and no recipe for reproduction.

- When you saw this, did you have debug-on-error set to a non-nil value?
- Have you seen this issue since, and if so, are you able to reproduce it?
- Alternatively, could you provide a backtrace?

In summary, I think we need more information in order to be able to make
any progress here.

If I don't hear back from you within a couple of weeks, I'll just close
this as unreproducible.  If anyone has any other ideas, please speak up.

Best regards,
Stefan Kangas

Message #38 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Alex Branham <alex.branham <at> gmail.com>, 28502 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3; list-packages ends with "error in process
 filter: End of file during parsing"
Date: Mon, 21 Oct 2019 12:14:32 +0200

Stefan Kangas <stefan <at> marxist.se> writes:

> The file you saw contained a package entry that was nil.  My guess is
> that this was due to an intermittent error on MELPA, since it was
> well-formed in all other respects.
>
> I don't think it's necessary to signal an error in this case.  We should
> just ignore nil entries.  On the other hand, it might indicate that
> something is wrong with the package archive, so it's nice to warn about
> it.
>
> I've installed the attached patch on master which does that.

If your assumption is correct, this seems like a reasonable fix.  Thank
you!

Michael.

Message #41 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Alex Branham <alex.branham <at> gmail.com>, 28502 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#28502: 25.3; list-packages ends with "error in process
 filter: End of file during parsing"
Date: Mon, 21 Oct 2019 13:50:31 +0200

Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> > The file you saw contained a package entry that was nil.  My guess is
> > that this was due to an intermittent error on MELPA, since it was
> > well-formed in all other respects.
> >
> > I don't think it's necessary to signal an error in this case.  We should
> > just ignore nil entries.  On the other hand, it might indicate that
> > something is wrong with the package archive, so it's nice to warn about
> > it.
> >
> > I've installed the attached patch on master which does that.
>
> If your assumption is correct, this seems like a reasonable fix.  Thank
> you!

I'm pretty confident: if you open the archive-contents file and run
M-x end-of-buffer, you see the nil entry at the end there.  You can
also reproduce the backtrace you saw by reverting the changes to
package.el and running the test that I add I added.

Best regards,
Stefan Kangas

Message #44 received at 28502 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 28502 <at> debbugs.gnu.org
Subject: Re: bug#28502: 25.3; list-packages ends with "error in process
 filter: End of file during parsing"
Date: Fri, 29 Nov 2019 14:00:26 +0100

tags 28502 + unreproducible
close 28502
thanks

Stefan Kangas <stefan <at> marxist.se> writes:

> If I don't hear back from you within a couple of weeks, I'll just close
> this as unreproducible.  If anyone has any other ideas, please speak up.

More information was requested, but none was given within 5 weeks, so
I'm closing this bug.  If this is still an issue, please reply to this
email (use "Reply to all" in your email client) and we can reopen the
bug report.

Best regards,
Stefan Kangas

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.