GNU bug report logs - #16220
url-http.el: Not conforming to HTTP spec

Previous Next

Full log

Message #32 received at 16220 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Jarosław Rzeszótko <sztywny <at> gmail.com>
Cc: 16220 <at> debbugs.gnu.org
Subject: Re: bug#16220: url-http.el: Not conforming to HTTP spec
Date: Wed, 01 Jan 2014 21:21:04 -0500

> The problem is that url-http.el sets a lot of headers by default that
> can not be overwritten in any other way then dynamically overshadowing
> some variables.

Indeed, this is ugly.  Improvements welcome.

> For example, all connections are keep-alive by
> default, which is confusing in itself already,

Not sure why it should be a problem.

> (let ((url-request-method "GET")
>       (url-request-extra-headers '(("Connection" . "close"))))
>   (url-retrieve-synchronously "http://www.google.com/"))
> And what is sent is this:
> GET / HTTP/1.1
> Connection: keep-alive
> ...
> Connection: close

> Which again isn't valid HTTP and the behaviour of the HTTP server in
> this case is undefined and implementation specific. The only way to
> workaround this is doing this:

> (let ((url-http-attempt-keepalives nil)
>       (url-request-method "GET")
>       (url-request-extra-headers '(("Connection" . "close"))))
>   (url-retrieve-synchronously "http://www.google.com/"))

Yuck!  We can probably fix this fairly easily by letting
url-request-extra-headers override (rather than just add to)
other headers.

> This is all the more irritating so many of the headers are set by
> default using the variables url-vars.el. Why those things are at all
> variables is a mystery to me.

Probably partly historical evolution (there was no place to add new
"parameters", so adding dynamic vars was an easy way to add more control
without breaking existing code).

> In the end it is much easier to do HTTP requests manually using
> make-network-process then it is with the url library,

I think that's misleading: the URL library is supposed to deal with
things like proxies and redirections, which "manual requests via
make-network-process" probably won't handle.

> Didn't anyone else run into problems with it?

Apparently not yet.  But I agree that the API might deserve a redesign
(IIRC another problem is in the way headers in the answer are returned
to the caller, which does not work consistently across different kinds
of URLs (ftp, http, file, imap, ...)).


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.