GNU bug report logs - #18860
24.4; packages don't download consistently from https

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18860 in the body.
You can then email your comments to 18860 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nic Ferrier <nferrier <at> ferrier.me.uk>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.4; packages don't download consistently from https
Date: Mon, 27 Oct 2014 23:16:47 +0000

Emacs 24.4's package system does something odd when the archive is on
HTTPS.

It seems as if dependencies are downloaded through HTTPS but the main
package is attempted to be downloaded through HTTP.

Here's how to reproduce:

$ cat > test.el <<HERE
(let ((package-user-dir (make-temp-name "/tmp/emacs-package-bug")))
  (package-initialize)
  (add-to-list
   (quote package-archives)
   (quote ("marmalade" . "https://marmalade-repo.org/packages/")))
  (package-refresh-contents)
  (package-install (quote elpakit)))
HERE

$ emacs -batch -l test.el
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: marmalade-repo.org:443
Contacting host: marmalade-repo.org:443
Contacting host: marmalade-repo.org:443
Making version-control local to s-autoloads.el while let-bound!
Generating autoloads for s.el...
Generating autoloads for s.el...done
Saving file /tmp/emacs-package-bug2503RFt/s-1.9.0/s-autoloads.el...
Wrote /tmp/emacs-package-bug2503RFt/s-1.9.0/s-autoloads.el
Checking /tmp/emacs-package-bug2503RFt/s-1.9.0...
Compiling /tmp/emacs-package-bug2503RFt/s-1.9.0/s-autoloads.el...
Compiling /tmp/emacs-package-bug2503RFt/s-1.9.0/s-pkg.el...
Wrote /tmp/emacs-package-bug2503RFt/s-1.9.0/s-pkg.elc
Compiling /tmp/emacs-package-bug2503RFt/s-1.9.0/s.el...
Wrote /tmp/emacs-package-bug2503RFt/s-1.9.0/s.elc
Done (Total of 2 files compiled, 1 skipped)
https://marmalade-repo.org/packages/noflet-0.0.14.el: Bad Request


It seems random which of these requests fail. But as soon as one is sent
over HTTP it fails (obviously).


marmalade-repo (which is currently the only repo doing https package
archives) could fix this problem, partially, on the server side. But we
can't protect the user that way. As soon as emacs makes an HTTP request
for something that should be signed the user is vulnerable to attack.

This is particularly egregious for a packaging system.




In GNU Emacs 24.4.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.23)
 of 2014-10-20 on nicferrier-XPS13-9333
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description:	Ubuntu 14.04.1 LTS

Configured using:
 `configure --prefix=/home/nicferrier/emacs-24-4'

Important settings:
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Message #8 received at 18860 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Nic Ferrier <nferrier <at> ferrier.me.uk>
Cc: 18860 <at> debbugs.gnu.org
Subject: Re: bug#18860: 24.4; packages don't download consistently from https
Date: Mon, 27 Oct 2014 20:46:55 -0400

> Emacs 24.4's package system does something odd when the archive is on
> HTTPS.
> It seems as if dependencies are downloaded through HTTPS but the main
> package is attempted to be downloaded through HTTP.

I did

   M-x trace-function RET url-retrieve-synchronously RET

before running your test.  On my first test, it failed at
shadchen-1.2.el but the request still had the right "https:" prefix.
IOW the trace shows that the url passed from package.el does include the
"https:" and the problem is somewhere in the URL package.

This said, I haven't managed to reproduce the problem since, so
I haven't been able to dig any further.  And in any case I'm not
familiar enough with the HTTP protocol to be of much use there.


        Stefan

Message #11 received at 18860 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 18860 <at> debbugs.gnu.org, Nic Ferrier <nferrier <at> ferrier.me.uk>
Subject: Re: bug#18860: 24.4; packages don't download consistently from https
Date: Tue, 28 Oct 2014 09:28:06 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> This said, I haven't managed to reproduce the problem since, so
> I haven't been able to dig any further.  And in any case I'm not
> familiar enough with the HTTP protocol to be of much use there.

I don't think this has anything to do with the HTTP protocol.  We just
need to find the place and condition where a non-TLS connection is
opened to the https port.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #14 received at 18860 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 18860 <at> debbugs.gnu.org, Nic Ferrier <nferrier <at> ferrier.me.uk>
Subject: Re: bug#18860: 24.4; packages don't download consistently from https
Date: Tue, 28 Oct 2014 09:42:44 -0400

>> This said, I haven't managed to reproduce the problem since, so
>> I haven't been able to dig any further.  And in any case I'm not
>> familiar enough with the HTTP protocol to be of much use there.
> I don't think this has anything to do with the HTTP protocol.  We just
> need to find the place and condition where a non-TLS connection is
> opened to the https port.

Right.  Maybe M-x trace-function RET open-network-stream RET
will give better hints.


        Stefan

Message #17 received at 18860 <at> debbugs.gnu.org (full text, mbox):

From: Ted Zlatanov <tzz <at> lifelogs.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 18860 <at> debbugs.gnu.org, Andreas Schwab <schwab <at> suse.de>,
 Nic Ferrier <nferrier <at> ferrier.me.uk>
Subject: Re: bug#18860: 24.4; packages don't download consistently from https
Date: Tue, 04 Nov 2014 16:20:11 -0500

On Tue, 28 Oct 2014 09:42:44 -0400 Stefan Monnier <monnier <at> iro.umontreal.ca> wrote: 

>>> This said, I haven't managed to reproduce the problem since, so
>>> I haven't been able to dig any further.  And in any case I'm not
>>> familiar enough with the HTTP protocol to be of much use there.
>> I don't think this has anything to do with the HTTP protocol.  We just
>> need to find the place and condition where a non-TLS connection is
>> opened to the https port.

SM> Right.  Maybe M-x trace-function RET open-network-stream RET
SM> will give better hints.

I ran the test with "emacs -q" and only saw connections to
"marmalade-repo.org:443". I'd love to find a way to replicate.

Thanks
Ted

Message #22 received at 18860 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 18860 <at> debbugs.gnu.org, Andreas Schwab <schwab <at> suse.de>,
 Nic Ferrier <nferrier <at> ferrier.me.uk>
Subject: Re: bug#18860: 24.4; packages don't download consistently from https
Date: Tue, 24 Sep 2019 10:01:08 +0200

Ted Zlatanov <tzz <at> lifelogs.com> writes:

> On Tue, 28 Oct 2014 09:42:44 -0400 Stefan Monnier
> <monnier <at> iro.umontreal.ca> wrote:
>
>>>> This said, I haven't managed to reproduce the problem since, so
>>>> I haven't been able to dig any further.  And in any case I'm not
>>>> familiar enough with the HTTP protocol to be of much use there.
>>> I don't think this has anything to do with the HTTP protocol.  We just
>>> need to find the place and condition where a non-TLS connection is
>>> opened to the https port.
>
> SM> Right.  Maybe M-x trace-function RET open-network-stream RET
> SM> will give better hints.
>
> I ran the test with "emacs -q" and only saw connections to
> "marmalade-repo.org:443". I'd love to find a way to replicate.

This old bug seemed to be unreproducible and things seem to work, so I'm
closing this bug report.  Please reopen if this is still an issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.