From unknown Sun Sep 07 23:15:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#12244: [patch] URI encoding bugs Resent-From: Ian Price Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 20 Aug 2012 22:20:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 12244 X-GNU-PR-Package: guile X-GNU-PR-Keywords: patch To: 12244@debbugs.gnu.org X-Debbugs-Original-To: bug-guile@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.134550119021307 (code B ref -1); Mon, 20 Aug 2012 22:20:02 +0000 Received: (at submit) by debbugs.gnu.org; 20 Aug 2012 22:19:50 +0000 Received: from localhost ([127.0.0.1]:40534 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T3aK5-0005Xb-VC for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:50 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33154) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T3aK3-0005XT-Rg for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1T3aJn-0002ng-5A for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:32 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:49722) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJn-0002nc-2B for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:31 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56969) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJl-0003WN-Tv for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1T3aJk-0002nE-Is for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:29 -0400 Received: from plane.gmane.org ([80.91.229.3]:56323) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJk-0002mn-7u for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:28 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1T3aJj-0001Vk-8t for bug-guile@gnu.org; Tue, 21 Aug 2012 00:19:27 +0200 Received: from host86-182-156-79.range86-182.btcentralplus.com ([86.182.156.79]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Aug 2012 00:19:27 +0200 Received: from ianprice90 by host86-182-156-79.range86-182.btcentralplus.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Aug 2012 00:19:27 +0200 X-Injected-Via-Gmane: http://gmane.org/ From: Ian Price Date: Mon, 20 Aug 2012 23:19:12 +0100 Lines: 129 Message-ID: <87ehn1ryu7.fsf@Kagami.home> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: host86-182-156-79.range86-182.btcentralplus.com User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) Cancel-Lock: sha1:NCcdE6Y5pizpDc36gIy87qbXGpc= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 208.118.235.17 X-Spam-Score: -6.9 (------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hi, There are two bugs in the current uri-encode procedure in (web uri). Firstly, if you have an octet less than 16 it only gets encoded to % HEXDIGIT instead of % HEXDIGIT HEXDIGIT. scheme@(guile−user)> (uri-encode "foo\nbar") $30 = "foo%abar" Secondly, if you have a string with no unreserved characters, nothing gets encoded. scheme@(guile−user)> (uri-encode "<>\\^") $31 = "<>\\∧" scheme@(guile−user)> (uri-encode "<>\\^a") $32 = "%3c%3e%5c%5ea" Patches attached. Cheers, -- Ian Price -- shift-reset.com "Programming is like pinball. The reward for doing it well is the opportunity to do it again" - from "The Wizardy Compiled" --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Fix-uri-encoding-for-octets-0-15.patch Content-Description: octets 0-15 patch >From 11f56bd6a4fdf1331ea30cd68b4d77e35215b4a5 Mon Sep 17 00:00:00 2001 From: Ian Price Date: Mon, 20 Aug 2012 23:03:38 +0100 Subject: [PATCH 1/2] Fix uri-encoding for octets 0-15 * module/web/uri.scm (uri-encode): All encoded octets should be of the form % HEXDIGIT HEXDIGIT. * test-suite/tests/web-uri.test ("encode"): Add test. --- module/web/uri.scm | 2 ++ test-suite/tests/web-uri.test | 3 ++- 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/module/web/uri.scm b/module/web/uri.scm index 109118b..3816d02 100644 --- a/module/web/uri.scm +++ b/module/web/uri.scm @@ -377,6 +377,8 @@ the byte." (if (< i len) (let ((byte (bytevector-u8-ref bv i))) (display #\% port) + (when (< byte 16) + (display #\0 port)) (display (number->string byte 16) port) (lp (1+ i)))))))) str))) diff --git a/test-suite/tests/web-uri.test b/test-suite/tests/web-uri.test index 4621a19..a9ded46 100644 --- a/test-suite/tests/web-uri.test +++ b/test-suite/tests/web-uri.test @@ -258,4 +258,5 @@ (equal? "foo bar" (uri-decode "foo+bar")))) (with-test-prefix "encode" - (pass-if (equal? "foo%20bar" (uri-encode "foo bar")))) + (pass-if (equal? "foo%20bar" (uri-encode "foo bar"))) + (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar")))) -- 1.7.7.6 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-Fix-uri-encoding-for-strings-with-no-unreserved-char.patch Content-Description: no unreserved chars patch >From ae4fa3f65c1d49822b5a284a065017673c81e65e Mon Sep 17 00:00:00 2001 From: Ian Price Date: Mon, 20 Aug 2012 23:12:23 +0100 Subject: [PATCH 2/2] Fix uri-encoding for strings with no unreserved chars * module/web/uri.scm (uri-encode): Change test to check for unreserved chars instead of reserved chars. * test-suite/tests/web-uri.test ("encode"): Add test. --- module/web/uri.scm | 4 +++- test-suite/tests/web-uri.test | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/module/web/uri.scm b/module/web/uri.scm index 3816d02..78614a5 100644 --- a/module/web/uri.scm +++ b/module/web/uri.scm @@ -364,7 +364,9 @@ Percent-encoding first writes out the given character to a bytevector within the given @var{encoding}, then encodes each byte as @code{%@var{HH}}, where @var{HH} is the hexadecimal representation of the byte." - (if (string-index str unescaped-chars) + (define (needs-escaped? ch) + (not (char-set-contains? unescaped-chars ch))) + (if (string-index str needs-escaped?) (call-with-output-string* (lambda (port) (string-for-each diff --git a/test-suite/tests/web-uri.test b/test-suite/tests/web-uri.test index a9ded46..3f6e7e3 100644 --- a/test-suite/tests/web-uri.test +++ b/test-suite/tests/web-uri.test @@ -259,4 +259,5 @@ (with-test-prefix "encode" (pass-if (equal? "foo%20bar" (uri-encode "foo bar"))) - (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar")))) + (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar"))) + (pass-if (equal? "%3c%3e%5c%5e" (uri-encode "<>\\^")))) -- 1.7.7.6 --=-=-=-- From unknown Sun Sep 07 23:15:46 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.428 (Entity 5.428) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Ian Price Subject: bug#12244: closed (Re: bug#12244: [patch] URI encoding bugs) Message-ID: References: <87lih1mj1b.fsf@gnu.org> <87ehn1ryu7.fsf@Kagami.home> X-Gnu-PR-Message: they-closed 12244 X-Gnu-PR-Package: guile X-Gnu-PR-Keywords: patch Reply-To: 12244@debbugs.gnu.org Date: Sun, 26 Aug 2012 21:39:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1346017142-3123-1" This is a multi-part message in MIME format... ------------=_1346017142-3123-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #12244: [patch] URI encoding bugs which was filed against the guile package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 12244@debbugs.gnu.org. --=20 12244: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D12244 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1346017142-3123-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 12244-done) by debbugs.gnu.org; 26 Aug 2012 21:38:30 +0000 Received: from localhost ([127.0.0.1]:51237 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T5kXO-0000ne-IF for submit@debbugs.gnu.org; Sun, 26 Aug 2012 17:38:30 -0400 Received: from xanadu.aquilenet.fr ([88.191.123.111]:37824) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T5kXM-0000nV-OI for 12244-done@debbugs.gnu.org; Sun, 26 Aug 2012 17:38:29 -0400 Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id EE80C931A; Sun, 26 Aug 2012 23:37:36 +0200 (CEST) Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9mPe2H6iYv1C; Sun, 26 Aug 2012 23:37:36 +0200 (CEST) Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id 905E49319; Sun, 26 Aug 2012 23:37:36 +0200 (CEST) From: ludo@gnu.org (Ludovic =?iso-8859-1?Q?Court=E8s?=) To: Ian Price Subject: Re: bug#12244: [patch] URI encoding bugs References: <87ehn1ryu7.fsf@Kagami.home> Date: Sun, 26 Aug 2012 23:37:36 +0200 In-Reply-To: <87ehn1ryu7.fsf@Kagami.home> (Ian Price's message of "Mon, 20 Aug 2012 23:19:12 +0100") Message-ID: <87lih1mj1b.fsf@gnu.org> User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.2 (-) X-Debbugs-Envelope-To: 12244-done Cc: 12244-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.2 (-) Hi Ian, Applied both, thanks! Ludo=E2=80=99. ------------=_1346017142-3123-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 20 Aug 2012 22:19:50 +0000 Received: from localhost ([127.0.0.1]:40534 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T3aK5-0005Xb-VC for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:50 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33154) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T3aK3-0005XT-Rg for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1T3aJn-0002ng-5A for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:32 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:49722) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJn-0002nc-2B for submit@debbugs.gnu.org; Mon, 20 Aug 2012 18:19:31 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56969) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJl-0003WN-Tv for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1T3aJk-0002nE-Is for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:29 -0400 Received: from plane.gmane.org ([80.91.229.3]:56323) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3aJk-0002mn-7u for bug-guile@gnu.org; Mon, 20 Aug 2012 18:19:28 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1T3aJj-0001Vk-8t for bug-guile@gnu.org; Tue, 21 Aug 2012 00:19:27 +0200 Received: from host86-182-156-79.range86-182.btcentralplus.com ([86.182.156.79]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Aug 2012 00:19:27 +0200 Received: from ianprice90 by host86-182-156-79.range86-182.btcentralplus.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Aug 2012 00:19:27 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: bug-guile@gnu.org From: Ian Price Subject: [patch] URI encoding bugs Date: Mon, 20 Aug 2012 23:19:12 +0100 Lines: 129 Message-ID: <87ehn1ryu7.fsf@Kagami.home> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: host86-182-156-79.range86-182.btcentralplus.com User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) Cancel-Lock: sha1:NCcdE6Y5pizpDc36gIy87qbXGpc= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 208.118.235.17 X-Spam-Score: -6.9 (------) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hi, There are two bugs in the current uri-encode procedure in (web uri). Firstly, if you have an octet less than 16 it only gets encoded to % HEXDIGIT instead of % HEXDIGIT HEXDIGIT. scheme@(guile−user)> (uri-encode "foo\nbar") $30 = "foo%abar" Secondly, if you have a string with no unreserved characters, nothing gets encoded. scheme@(guile−user)> (uri-encode "<>\\^") $31 = "<>\\∧" scheme@(guile−user)> (uri-encode "<>\\^a") $32 = "%3c%3e%5c%5ea" Patches attached. Cheers, -- Ian Price -- shift-reset.com "Programming is like pinball. The reward for doing it well is the opportunity to do it again" - from "The Wizardy Compiled" --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Fix-uri-encoding-for-octets-0-15.patch Content-Description: octets 0-15 patch >From 11f56bd6a4fdf1331ea30cd68b4d77e35215b4a5 Mon Sep 17 00:00:00 2001 From: Ian Price Date: Mon, 20 Aug 2012 23:03:38 +0100 Subject: [PATCH 1/2] Fix uri-encoding for octets 0-15 * module/web/uri.scm (uri-encode): All encoded octets should be of the form % HEXDIGIT HEXDIGIT. * test-suite/tests/web-uri.test ("encode"): Add test. --- module/web/uri.scm | 2 ++ test-suite/tests/web-uri.test | 3 ++- 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/module/web/uri.scm b/module/web/uri.scm index 109118b..3816d02 100644 --- a/module/web/uri.scm +++ b/module/web/uri.scm @@ -377,6 +377,8 @@ the byte." (if (< i len) (let ((byte (bytevector-u8-ref bv i))) (display #\% port) + (when (< byte 16) + (display #\0 port)) (display (number->string byte 16) port) (lp (1+ i)))))))) str))) diff --git a/test-suite/tests/web-uri.test b/test-suite/tests/web-uri.test index 4621a19..a9ded46 100644 --- a/test-suite/tests/web-uri.test +++ b/test-suite/tests/web-uri.test @@ -258,4 +258,5 @@ (equal? "foo bar" (uri-decode "foo+bar")))) (with-test-prefix "encode" - (pass-if (equal? "foo%20bar" (uri-encode "foo bar")))) + (pass-if (equal? "foo%20bar" (uri-encode "foo bar"))) + (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar")))) -- 1.7.7.6 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-Fix-uri-encoding-for-strings-with-no-unreserved-char.patch Content-Description: no unreserved chars patch >From ae4fa3f65c1d49822b5a284a065017673c81e65e Mon Sep 17 00:00:00 2001 From: Ian Price Date: Mon, 20 Aug 2012 23:12:23 +0100 Subject: [PATCH 2/2] Fix uri-encoding for strings with no unreserved chars * module/web/uri.scm (uri-encode): Change test to check for unreserved chars instead of reserved chars. * test-suite/tests/web-uri.test ("encode"): Add test. --- module/web/uri.scm | 4 +++- test-suite/tests/web-uri.test | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/module/web/uri.scm b/module/web/uri.scm index 3816d02..78614a5 100644 --- a/module/web/uri.scm +++ b/module/web/uri.scm @@ -364,7 +364,9 @@ Percent-encoding first writes out the given character to a bytevector within the given @var{encoding}, then encodes each byte as @code{%@var{HH}}, where @var{HH} is the hexadecimal representation of the byte." - (if (string-index str unescaped-chars) + (define (needs-escaped? ch) + (not (char-set-contains? unescaped-chars ch))) + (if (string-index str needs-escaped?) (call-with-output-string* (lambda (port) (string-for-each diff --git a/test-suite/tests/web-uri.test b/test-suite/tests/web-uri.test index a9ded46..3f6e7e3 100644 --- a/test-suite/tests/web-uri.test +++ b/test-suite/tests/web-uri.test @@ -259,4 +259,5 @@ (with-test-prefix "encode" (pass-if (equal? "foo%20bar" (uri-encode "foo bar"))) - (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar")))) + (pass-if (equal? "foo%0a%00bar" (uri-encode "foo\n\x00bar"))) + (pass-if (equal? "%3c%3e%5c%5e" (uri-encode "<>\\^")))) -- 1.7.7.6 --=-=-=-- ------------=_1346017142-3123-1--