GNU bug report logs -
#8622
24.0.50; url-parse does not implement RFC3986 5.2
Previous Next
Reported by: Julien Danjou <julien <at> danjou.info>
Date: Thu, 5 May 2011 16:15:02 UTC
Severity: normal
Tags: fixed
Found in version 24.0.50
Done: Lars Ingebrigtsen <larsi <at> gnus.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8622 in the body.
You can then email your comments to 8622 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Thu, 05 May 2011 16:15:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Julien Danjou <julien <at> danjou.info>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Thu, 05 May 2011 16:15:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
`url-generic-parse-url' from url-parse does not implement correctly
relative path handling as described in the RFC3986 and subsections of
section 5.2.
URL listed in 5.4.1 and 5.4.2 does not work such as:
"../../../g" = "http://a/g"
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 04:48:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> URL listed in 5.4.1 and 5.4.2 does not work such as:
>
> "../../../g" = "http://a/g"
Where does the "a" come from?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 12:08:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Lars Magne Ingebrigtsen wrote:
> Julien Danjou <julien <at> danjou.info> writes:
>
>> URL listed in 5.4.1 and 5.4.2 does not work such as:
>>
>> "../../../g" = "http://a/g"
>
> Where does the "a" come from?
Read:
"http://a/../../../g" = "http://a/g"
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 14:05:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> Read:
>
> "http://a/../../../g" = "http://a/g"
ELISP> (url-generic-parse-url "http://a/../../../g")
[cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t]
How is that wrong?
Andreas.
--
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 14:45:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Andreas Schwab wrote:
> ELISP> (url-generic-parse-url "http://a/../../../g")
> [cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t]
>
> How is that wrong?
Just tested with telnet:
GET /url/ HTTP/1.1
Host: domain.com
HTTP/1.1 200 OK
GET /../../../url/ HTTP/1.1
Host: domain.com
HTTP/1.1 400 Bad Request
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 15:26:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> On Sun, Sep 11 2011, Andreas Schwab wrote:
>
>> ELISP> (url-generic-parse-url "http://a/../../../g")
>> [cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t]
>>
>> How is that wrong?
>
> Just tested with telnet:
>
> GET /url/ HTTP/1.1
> Host: domain.com
>
> HTTP/1.1 200 OK
>
>
>
> GET /../../../url/ HTTP/1.1
> Host: domain.com
>
> HTTP/1.1 400 Bad Request
What are you trying to prove?
Andreas.
--
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 15:29:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
>> Julien Danjou <julien <at> danjou.info> writes:
>>
>>> URL listed in 5.4.1 and 5.4.2 does not work such as:
>>>
>>> "../../../g" = "http://a/g"
>>
>> Where does the "a" come from?
>
> Read:
>
> "http://a/../../../g" = "http://a/g"
5.2 talks about relative URLs, though, so "http://a/../../../g" seems
irrelevant...
5.4.1 also talks about relative URLs, and how to resolve them if you
have a base address. But `url-generic-parse-url' does not, as far as I
can tell, deal with the concept of "relative URLs and bases", but only
parses full, non-relative URLs.
And for a full URL, this looks correct to me:
(url-generic-parse-url "http://a/../../../g")
=>
[cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t nil]
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 15:32:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Andreas Schwab wrote:
> What are you trying to prove?
That your example:
ELISP> (url-generic-parse-url "http://a/../../../g")
[cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t]
will not work.
Could you give a try, maybe you'll understand?
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 15:51:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> On Sun, Sep 11 2011, Andreas Schwab wrote:
>
>> What are you trying to prove?
>
> That your example:
>
> ELISP> (url-generic-parse-url "http://a/../../../g")
> [cl-struct-url "http" nil nil "a" 80 "/../../../g" nil nil t]
>
> will not work.
In which way does it not work? There is the scheme "http", the host
"a", the port 80 and the local part "/../../../g". Everything is fully
correct according to the rules of a URI. How the local part is
interpreted is only defined by the remote host.
Andreas.
--
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 15:58:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Andreas Schwab wrote:
> In which way does it not work? There is the scheme "http", the host
> "a", the port 80 and the local part "/../../../g". Everything is fully
> correct according to the rules of a URI. How the local part is
> interpreted is only defined by the remote host.
Men, this is getting me crazy. Let me rephrase the whole thing.
You got an URL of the form:
http://a/../../../foobar.png
If you use Firefox, Chromium, wget, or whatever to retrieve it, the
program will act according to RFC3986 and transform that URL to:
http://a/foobar.png
`url' from Emacs will not, and will fail to retrieve the image.
Now I may be mistaken about where in the code the bug is, but there's a
bug: the `url' functions are unable to fetch such an URL, whereas any
other tool is able to.
In Lisp:
(switch-to-buffer (url-retrieve-synchronously
"http://www.gnu.org/graphics/t-desktop-4-small.jpg"))
=> That works
(switch-to-buffer (url-retrieve-synchronously
"http://www.gnu.org/../graphics/t-desktop-4-small.jpg"))
=> Show a 400 bad request
% wget http://www.gnu.org/../graphics/t-desktop-4-small.jpg
--2011-09-11 17:52:56-- http://www.gnu.org/graphics/t-desktop-4-small.jpg
Resolving www.gnu.org (www.gnu.org)... 140.186.70.148
Connecting to www.gnu.org (www.gnu.org)|140.186.70.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30195 (29K) [image/jpeg]
Saving to: `t-desktop-4-small.jpg'
100%[=======================================================================================================================================================================================================================================>] 30,195 66.5K/s in 0.4s
2011-09-11 17:52:56 (66.5 KB/s) - `t-desktop-4-small.jpg' saved [30195/30195]
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:12:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> Men, this is getting me crazy. Let me rephrase the whole thing.
>
> You got an URL of the form:
>
> http://a/../../../foobar.png
>
> If you use Firefox, Chromium, wget, or whatever to retrieve it, the
> program will act according to RFC3986 and transform that URL to:
>
> http://a/foobar.png
Where in RFC3986 does it say that you're supposed to do that? 5.2
(etc.) only talks about relative URLs.
You may be totally correct that url.el should do this, ../-stripping in
absolute URLs, but does the RFC actually say so?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:14:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> You got an URL of the form:
>
> http://a/../../../foobar.png
This is a URI, not a relative-ref, so the rules for . and .. don't apply.
Andreas.
--
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:21:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Lars Magne Ingebrigtsen wrote:
> Where in RFC3986 does it say that you're supposed to do that? 5.2
> (etc.) only talks about relative URLs.
>
> You may be totally correct that url.el should do this, ../-stripping in
> absolute URLs, but does the RFC actually say so?
It seems to me that it's what 5.4 talks about.
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:28:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> It seems to me that it's what 5.4 talks about.
This is how 5.4 starts:
5.4. Reference Resolution Examples
Within a representation with a well defined base URI of
http://a/b/c/d;p?q
a relative reference is transformed to its target URI as follows.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:31:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Lars Magne Ingebrigtsen wrote:
> Julien Danjou <julien <at> danjou.info> writes:
>
>> It seems to me that it's what 5.4 talks about.
>
> This is how 5.4 starts:
>
> 5.4. Reference Resolution Examples
>
> Within a representation with a well defined base URI of
>
> http://a/b/c/d;p?q
>
> a relative reference is transformed to its target URI as follows.
Then go to 5.4.2:
Although the following abnormal examples are unlikely to occur in
normal practice, all URI parsers should be capable of resolving them
consistently. Each example uses the same base as that above.
Parsers must be careful in handling cases where there are more ".."
segments in a relative-path reference than there are hierarchical
levels in the base URI's path. Note that the ".." syntax cannot be
used to change the authority component of a URI.
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:38:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> Then go to 5.4.2:
>
> Although the following abnormal examples are unlikely to occur in
> normal practice, all URI parsers should be capable of resolving them
> consistently. Each example uses the same base as that above.
>
> Parsers must be careful in handling cases where there are more ".."
> segments in a relative-path reference than there are hierarchical
> levels in the base URI's path. Note that the ".." syntax cannot be
> used to change the authority component of a URI.
Still talking about "base" and "relative". Nothing about parsing a full
URL.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:43:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 8622 <at> debbugs.gnu.org (full text, mbox):
However, as a practical issue, it might make sense to just do what
Firefox does, which is probably "best practise". If for no other reason
than that Firefox does it.
It will strip "../" and "./" from the front of the local part of all
URLs, according to tcpdump.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:44:01 GMT)
Full text and
rfc822 format available.
Message #56 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Lars Magne Ingebrigtsen wrote:
> Julien Danjou <julien <at> danjou.info> writes:
>
>> Then go to 5.4.2:
>>
>> Although the following abnormal examples are unlikely to occur in
>> normal practice, all URI parsers should be capable of resolving them
>> consistently. Each example uses the same base as that above.
>>
>> Parsers must be careful in handling cases where there are more ".."
>> segments in a relative-path reference than there are hierarchical
>> levels in the base URI's path. Note that the ".." syntax cannot be
>> used to change the authority component of a URI.
>
> Still talking about "base" and "relative". Nothing about parsing a full
> URL.
I'm sorry, I am missing why this does not apply to the example I gave.
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:47:02 GMT)
Full text and
rfc822 format available.
Message #59 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Julien Danjou <julien <at> danjou.info> writes:
> I'm sorry, I am missing why this does not apply to the example I gave.
http://a/../../b is not a relative URL. :-)
The entire thing is about how to glue a base URL, like "http://a/b/c"
together with a relative URL, like "../../d". If you're not gluing URLs
together, then the section does not apply. At all.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 16:52:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 8622 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Sep 11 2011, Lars Magne Ingebrigtsen wrote:
> Julien Danjou <julien <at> danjou.info> writes:
>
>> I'm sorry, I am missing why this does not apply to the example I gave.
>
> http://a/../../b is not a relative URL. :-)
>
> The entire thing is about how to glue a base URL, like "http://a/b/c"
> together with a relative URL, like "../../d". If you're not gluing URLs
> together, then the section does not apply. At all.
Oh ok. But if you consider the base URL being http://a/, you clearly
match this case. I admit we are not doing relative anyhow, but I don't
think absolute and relative are supposed to have different behaviour in
such a case.
And it seems that 5.2.4 clearly explains the algo in our (absolute) case,
don't you think?
--
Julien Danjou
❱ http://julien.danjou.info
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 11 Sep 2011 19:01:01 GMT)
Full text and
rfc822 format available.
Message #65 received at submit <at> debbugs.gnu.org (full text, mbox):
However, as a practical issue, it might make sense to just do what
Firefox does, which is probably "best practise". If for no other reason
than that Firefox does it.
It will strip "../" and "./" from the front of the local part of all
URLs, according to tcpdump.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog http://lars.ingebrigtsen.no/
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Fri, 25 Dec 2015 22:06:01 GMT)
Full text and
rfc822 format available.
Message #68 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:
> However, as a practical issue, it might make sense to just do what
> Firefox does, which is probably "best practise". If for no other reason
> than that Firefox does it.
>
> It will strip "../" and "./" from the front of the local part of all
> URLs, according to tcpdump.
Hey, I wonder what Alain's new URL parsing function does here. :-)
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#8622; Package
emacs.
(Sun, 15 Apr 2018 20:15:01 GMT)
Full text and
rfc822 format available.
Message #71 received at 8622 <at> debbugs.gnu.org (full text, mbox):
Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:
> However, as a practical issue, it might make sense to just do what
> Firefox does, which is probably "best practise". If for no other reason
> than that Firefox does it.
>
> It will strip "../" and "./" from the front of the local part of all
> URLs, according to tcpdump.
I've checked Chromium, too, and it does the same.
I think it might make sense for eww to strip the "../" things, and leave
url-parse at it is, since it should be standards-compliant, and the
standards don't seem to say anything about this.
I'll tweak eww.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
Added tag(s) fixed.
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Sun, 15 Apr 2018 20:21:02 GMT)
Full text and
rfc822 format available.
bug closed, send any further explanations to
8622 <at> debbugs.gnu.org and Julien Danjou <julien <at> danjou.info>
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Sun, 15 Apr 2018 20:21:02 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Mon, 14 May 2018 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 7 years and 121 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.