From unknown Wed Jun 18 23:07:15 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#24117 <24117@debbugs.gnu.org> To: bug#24117 <24117@debbugs.gnu.org> Subject: Status: 25.1; url-http-create-request: Multibyte text in HTTP request Reply-To: bug#24117 <24117@debbugs.gnu.org> Date: Thu, 19 Jun 2025 06:07:15 +0000 retitle 24117 25.1; url-http-create-request: Multibyte text in HTTP request reassign 24117 emacs submitter 24117 Sho Takemori severity 24117 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 31 04:27:08 2016 Received: (at submit) by debbugs.gnu.org; 31 Jul 2016 08:27:08 +0000 Received: from localhost ([127.0.0.1]:51931 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bTm5S-0007Ig-EA for submit@debbugs.gnu.org; Sun, 31 Jul 2016 04:27:08 -0400 Received: from eggs.gnu.org ([208.118.235.92]:57124) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bTm5P-0007IB-Gf for submit@debbugs.gnu.org; Sun, 31 Jul 2016 04:27:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bTm5E-0007RR-2i for submit@debbugs.gnu.org; Sun, 31 Jul 2016 04:26:58 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,T_DKIM_INVALID,WEIRD_PORT autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:35247) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bTm5D-0007RN-Ut for submit@debbugs.gnu.org; Sun, 31 Jul 2016 04:26:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48093) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bTm57-0006oK-Ds for bug-gnu-emacs@gnu.org; Sun, 31 Jul 2016 04:26:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bTm51-0007Qy-A7 for bug-gnu-emacs@gnu.org; Sun, 31 Jul 2016 04:26:44 -0400 Received: from mail-oi0-x234.google.com ([2607:f8b0:4003:c06::234]:34913) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bTm50-0007Qs-W4 for bug-gnu-emacs@gnu.org; Sun, 31 Jul 2016 04:26:39 -0400 Received: by mail-oi0-x234.google.com with SMTP id l72so157984024oig.2 for ; Sun, 31 Jul 2016 01:26:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=1cStMDGtB3epXJphuMAUB2z4hBtl9c/shE20mHdre2k=; b=Nm+z3js403WybuskhmtzZO4H7/8JwVrSuw6NNR6LJkspxTod8EHQYX5CuK0ThJkayz +u62WanPBYmmQgfujfszN9gx6cucCc735ZY+lL62FjBYyX/w3xzXFwf3ijqFn4NKAkC8 zs5lffe2eqUdnQmaGEVdRQ2P+1JHeyyZcZWnBN1rbKHlwocZgiCyigWdGWYlbV3tyL4N dQYMSnbZJks6gJHaOgs/di9hFBMQSJbaJDDJx24QoiAZQut2Ks2nLpRnpQUOp8cbFbvh BulT/XS6T/k08JgdpX/UlhQ5IsgPeZXQRQJeGhOEevXG+mXIOL3cgE9SB47Z1DBDD/v0 IeJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1cStMDGtB3epXJphuMAUB2z4hBtl9c/shE20mHdre2k=; b=mHXzEioO+lJWuJwT9wSqyLZHpP7rK8J5GCDH0G1guwPcvqO8NVjmbNpXiS+3LhnoW3 6ww8WfUUcrW76Q2ZgSLCtaTuXbvdNSS+JItu54qFb+jda8hu8nW+g0G2ThAPchE/z+Zm uYM91sTtMzT0kQLaD6daCX9sjA82P+zaVLuSUVv/E3xEGh9XA6wgUsPnjLBi6kU72JrB AkUuu5e5GaSYFa7JBwAKL41BAoiTaHaU2X/V3HRccPxbT4+E7YqBK5aMvZMqPVohHWea bLwBnNMYtmI8Ys4rgfWvEzhjrLBA2HoXd9HHcc4cku0o5flOoffvyf5U1QoPfWjDKZfs qBcg== X-Gm-Message-State: AEkoouvOokfyJHfFZUI29uWX78ZvHeNoM4Hy9IxspM55U/dCU7SfRS+PjmryrXj9+/WUERYjGJBlcITQa4ue2Q== X-Received: by 10.202.236.71 with SMTP id k68mr27411857oih.162.1469953597642; Sun, 31 Jul 2016 01:26:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.33.77 with HTTP; Sun, 31 Jul 2016 01:26:37 -0700 (PDT) From: Sho Takemori Date: Sun, 31 Jul 2016 17:26:37 +0900 Message-ID: Subject: 25.1; url-http-create-request: Multibyte text in HTTP request To: bug-gnu-emacs@gnu.org Content-Type: multipart/alternative; boundary=001a1134e8d40621d60538ea3e48 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --001a1134e8d40621d60538ea3e48 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I got an error "error in process sentinel: url-http-create-request: Multibyte text in HTTP request" when I visited a Python file which contains a multibyte character with `anaconda-eldoc-mode' turned on. At first, I thought this was a bug of anaconda-mode. So I opened an issue in github (https://github.com/proofit404/anaconda-mode/issues/189). I guess `(=3D (string-bytes request) (length request))` in `url-http-create-request' should be `(=3D (string-bytes url-http-data) (length url-http-data))`, because `(=3D (string-bytes request) (length request))` may be `nil' even if `(=3D (string-bytes url-http-data) (length url-http-data))` is `t'. Sho Takemori In GNU Emacs 25.1.1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9) of 2016-07-26 built on HP-500-270jp Repository revision: 0f0b191a5324115fe9e8c438eceef4043decf209 Windowing system distributor 'The X.Org Foundation', version 11.0.11803000 System Description: Ubuntu 16.04.1 LTS Configured using: 'configure --with-sound=3Dno --with-modules' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK GPM DBUS GCONF GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES Important settings: value of $LC_MONETARY: ja_JP.UTF-8 value of $LC_NUMERIC: ja_JP.UTF-8 value of $LC_TIME: ja_JP.UTF-8 value of $LANG: ja_JP.UTF-8 value of $XMODIFIERS: @im=3Dibus locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: diff-auto-refine-mode: t shell-dirtrack-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent messages: MIME-Version: 1.0 Connection: keep-alive Extension: Security/Digest Security/SSL Host: 127.0.0.1:9001 Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Content-length: 20552 {"jsonrpc":"2.0","id":1,"method":"eldoc","params":{"source":"# -*- coding: utf-8 -*-\nimport itertools\nfrom itertools import groupby\n\nfrom sage.all import mul\nfrom sage.arith.all import kronecker_symbol\nfrom sage.functions.all import ceil, floor, sgn\nfrom sage.matrix.all import (block_diagonal_matrix, block_matrix,\n diagonal_matrix, identity_matrix, matrix)\nfrom sage.misc.all import cached_function\nfrom sage.quadratic_forms.all import QuadraticForm, least_quadratic_nonresidue\nfrom sage.rings.all import QQ, ZZ, CyclotomicField, FiniteField, PolynomialRing\n\n\ndef _index_of_gamma_0_gl_n(alpha, p):\n =E2=80=99=E2=80=99=E2=80=99\n Ret= urns delta(a1, ..., an) defined in Shimura, Euler products and Eisenstein\n series, pp 118, (15.1.7).\n =E2=80=99=E2=80=99=E2=80=99\n if p in ZZ:\n p =3D = ZZ(p)\n\n def _bn(n):\n return mul(1 - p ** (-i) for i in xrange(1, n + 1))\n\n e_r_ls = =3D [(k, len(list(v)))\n for k, v in groupby(sorted(alpha), lambda x: x)]\n res =3D _bn(len(alpha)) / mul(_bn(r) for _, r in e_r_ls)\n f= or i, (ei, ri) in enumerate(e_r_ls):\n for j, (ej, rj) in enumerate(e_r_ls):\n if i < j:\n res *=3D p ** ((= ej - ei) * ri * rj)\n return res\n\n\ndef _gl2_coset_gamma0(a, p):\n w = =3D matrix([[0, -1],\n [1, 0]])\n for m12 in range(p ** a):\n yield matrix([[1, m12],\n [0, 1]])\n for m21 in range(p ** (a - 1)):\n m =3D matrix([[1, 0],\n [p * m21, 1]])\n yield w * m\n\n\ndef _gl3_coset_gamma0(alpha, p):\n r=E2=80=99=E2=80=99=E2=80=99\n Let alpha =3D [a0, a1, a2] with = a0 <=3D a1 <=3D a2,\n g =3D diag([p^a0, p^a1, p^a2]), and Gamma0 =3D g^(-1) GL3(Z) g =E2=88=A7 GL3(Z).\= n Return a complete set Gamma0 \\ GL3(Z).\n =E2=80=99=E2=80=99=E2=80=99\n= if p in ZZ:\n p =3D ZZ(p)\n a0, a1, a2 =3D alpha\n if a0 < a1 < a2:\n retur= n list(__gl3_coset_gamma0_distinct(a0, a1, a2, p))\n elif a0 =3D=3D a1 and= a1 < a2:\n return list(__gl3_coset_gamma0_2_1(a0, a2, p))\n elif a0 < a1 and a1 =3D=3D a2:\n return list(__gl3_coset_gamma0_1_2(a0, a2, p))\n elif a0 =3D=3D a1 =3D=3D a2:\n return [identity_matrix(ZZ, = 3)]\n else:\n raise ValueError\n\n\ndef __gl3_coset_gamma0_2_1(a1, a3, p):\n w23 =3D matrix([[1, 0, 0],\n [0, 0, 1],\n [0, 1, 0]])\n for m13 in range(p ** (a3 - a1 - 1)):\n for m23 in range(p ** (a3 - a1 - 1)):\n m =3D matrix([[1, 0, p * m13],\n [0, 1, p * m23],\n [0, 0, 1]])\n yield m\n\n for m32 in range(p ** (a3 - a1)):\n m =3D matrix([[1, 0, 0],\n [0, 1, 0],\n [0, m32, 1]])\n for g in _gl2_coset_gamma0(a3 - a1, p):\n n =3D block_diagonal_matrix(g, matrix([[1]]))\n yield w23 * m * n\n\n\ndef __gl3_coset_gamma0_1_2(a1, a2, p):\n w12 =3D matrix([[0, 1, 0],\n [1, 0, 0],\n [0, 0, 1]])\n\n for m12 in range(p ** (a2 - a1 - 1)):\n for m13 in range(p ** (a2 - a1 - 1)):\n m =3D matrix([[1, p * m12, p * m13],\n [0, 1, 0],\n [0, 0, 1]])\n yield m\n for m21 in range(p ** (a2 - a1)):\n m =3D matrix([[1, 0, 0],\n [m21, 1, 0],\n [0, 0, 1]])\n for g in _gl2_coset_gamma0(a2 - a1, p):\n n =3D block_diagonal_matrix(matrix([[1]]), g)\n yield w12 * m * n\n\n\ndef __gl3_coset_gamma0_distinct(a1, a2, a3, p):\n\n w12 =3D matrix([[0, 1, 0],\n [1, 0, 0],\n [0, 0, 1]])\n\n w23 =3D matrix([[1, 0, 0],\n [0, 0, 1],\n [0, 1, 0]])\n\n w13 =3D matrix([[0, 0, 1],\n [0, 1, 0],\n [1, 0, 0]])\n\n w123 =3D matrix([[0, 1, 0],\n [0, 0, 1],\n [1, 0, 0]])\n\n w132 =3D matrix([[0, 0, 1],\n [1, 0, 0],\n [0, 1, 0]])\n\n # w =3D 1\n for m12 in range(p ** (a2 - a1 - 1)):\n for m13 in range(p ** (a3 - a1 - 1)):\n for m23 in range(p ** (a3 - a2 - 1)):\n yield matrix([[1, p * m12, p * m13],\n [0, 1, p * m23],\n [0, 0, 1]])\n # w =3D (12)\n for m13 in range(p ** (a3 - a2 - 1)):\n for m21 in range(p ** (a2 - a1)):\n for m23 in range(p ** (a3 - a1 - 1)):\n m =3D matrix([[1, 0, p * m13],\n [m21, 1, p * m23],\n [0, 0, 1]])\n yield w12 * m\n # w =3D (23)\n for m12 in range= (p ** (a3 - a1 - 1)):\n for m13 in range(p ** (a2 - a1 - 1)):\n for m32 in range(p ** (a3 - a2)):\n m =3D matrix([[1, p * m12, p * m13],\n [0, 1, 0],\n [0, m32, 1]])\n yield w23 * m\n\n # w =3D (13)\n for m21 in range(p ** (a3 - a2)):\n for m31 in range(p ** (a3 - a1)):\n for m32 in range(p ** (a2 - a1)):\n m =3D matrix([[1, 0, 0],\n [m21, 1, 0],\n [m31, m32, 1]])\n yield w13 * m\n\n # w = =3D (123)\n for m21 in range(p ** (a3 - a1)):\n for m23 in range(p ** (a2 - a1 - 1)):\n for m31 in range(p ** (a3 - a2)):\n m =3D matrix([[1, 0, 0],\n [m21, 1, p * m23],\n [m31, 0, 1]])\n yield w123 * m\n # w =3D (132)\n for m12 in range(p ** (a3 - a2 - 1)):\n for m31 in range(p ** (a2 - a1)):\n for m32 in range(p ** (a3 - a1)):\n m =3D matrix([[1, p * m12, 0],\n [0, 1, 0],\n [m31, m32, 1]])\n yield w132 * m\n\n\nclass HalfIntMatElement(object):\n\n def __init__(self, T):\n =E2=80=99=E2=80=99=E2=80=99\n :params T:= half integral matrix of size 3 or a list\n =E2=80=99=E2=80=99=E2=80=99\n if isinstanc= e(T, list):\n a, b, c, d, e, f =3D [ZZ(x) for x in T]\n mat =3D matrix([[a, f= / 2, e / 2],\n [f / 2, b, d / 2],\n [e / 2, d / 2, c]])\n else:\n mat =3D T\n self.__entries =3D tuple(mat.list())\n\n def __eq__(self, other):\n if isinstance(other, HalfIntMatElement):\n return self.__entries =3D=3D other.__entries\n else:\n raise NotImplementedError\n\n def __repr__(self):\n return self.T.__repr__()\n\n def __hash__(self):\n return hash(self.__entries)\n\n @property\n def T(self):\n return matrix(3, self.__entries)\n\n def right_action(self, g):\n =E2=80= =99=E2=80=99=E2=80=99\n :param g: matrix of size n\n return self[g] (Siegel=E2=80=99s notation)\n =E2=80=99=E2=80=99=E2=80=99\n S =3D g.transpose()= * self.T * g\n return HalfIntMatElement(S)\n\n def satisfy_cong_condition_tp(self, p, alpha):\n =E2=80=99=E2=80=99=E2=80=99\n Test if sum_{B mod D}= exp(2pi T B D^(-1)) is zero, where D =3D diag(p^a1, p^a2, a^a3),\n a1, a2, a3 =3D alpha.\n =E2=80=99=E2=80=99=E2=80=99\n return (all(ZZ(self.T[i, i]) % p **= alpha[i] =3D=3D 0 for i in range(3)) and\n all(ZZ(self.T[i, j] * 2) % p ** alpha[i] =3D= =3D 0\n for i in range(3) for j in range(i + 1, 3)))\n\n def is_divisible_by(self, m):\n =E2=80=99=E2=80=99=E2=80=99\n = Test if self is divisible by m\n :param m: integer\n =E2=80=99=E2=80=99=E2=80= =99\n return _half_int_mat_is_div_by(self.T, m)\n\n def __floordiv__(self, other):\n S =3D matrix(QQ, 3)\n for i in range(3):\n S[i, i]= =3D ZZ(self.T[i, i]) // other\n for i in range(3):\n for j in range(i + 1, 3):\n S[i, j] =3D S[j, i] =3D (ZZ(self.T[i, j] = * 2) // other) / 2\n return HalfIntMatElement(S)\n\n\ndef alpha_list(dl):\n =E2=80=99=E2=80=99=E2=80=99\n Return a list of (a0,= a1, a2) with 0 <=3D a0 <=3D a1 <=3D a2 <=3D dl\n =E2=80=99=E2=80=99=E2=80=99\n return [(a0, a1, a= 2) for a0 in range(dl + 1)\n for a1 in range(a0, dl + 1) for a2 in range(a1, dl + 1)]\n\n\ndef tp_action_fourier_coeff(p, T, F):\n =E2=80=99=E2=80=99=E2= =80=99\n Return the Tth Fourier coefficient of F|T(p), where F is a modular form.\n :param p: a prime number\n :param T: a half integral matrix or an instance of HalfIntMatElement\n :param F: a dictionary or a Siegel modular form of degree 3\n =E2=80=99=E2=80=99=E2=80=99\n p =3D ZZ(p)\n return _action_fc_base(tp_action_fc_alist(p, T), F, T)\n\n\ndef tp2_action_fourier_coeff(p, i, T, F):\n =E2=80=99=E2=80=99=E2=80=99\n = Similar to tp_action_fourier_coeff for T_i(p^2).\n =E2=80=99=E2=80=99=E2=80=99\n = p =3D ZZ(p)\n return _action_fc_base(tp2_action_fc_alist(p, T, i), F, T)\n\n\ndef _action_fc_base(ls, F, T):\n if not isinstance(T, HalfIntMatElement):\n T =3D HalfIntMatElement(T)\n res =3D 0\n for s, a, g in ls:\n res =3D a * F[s].left_action(g) + res\n return res\n\n\ndef hecke_eigenvalue_tp(p, F, T=3DNone):\n =E2=80=99=E2=80=99=E2=80=99\n = p, F, T: same as aruments of tp_action_fourier_coeff.\n Assuming F is an eigenform, return the eigenvalue for T(p),\n T is used for the computation of Fourier coefficients.\n If T is omitted, T will be set to\n matrix([[1, 1/2, 1/2], [1/2, 1, 1/2], [1/2, 1/2, 1]]).\n =E2=80=99=E2=80=99=E2=80=99\n = return _hecke_eigenvalue_base(lambda s: tp_action_fourier_coeff(p, s, F), F, T=3DT)\n\n\ndef hecke_eigenvalue_tp2(p, i, F, T=3DNone):\n =E2=80=99=E2= =80=99=E2=80=99\n Similar to hecke_eigenvalue_tp for T(p^2).\n =E2=80=99=E2=80=99=E2=80=99\n re= turn _hecke_eigenvalue_base(lambda s: tp2_action_fourier_coeff(p, i, s, F), F, T=3DT)\n\n\ndef spinor_l_euler_factor(p, F, t=3DNone, T=3DNone):\n =E2= =80=99=E2=80=99=E2=80=99\n F: a dict or Siegel modular form of degree 3.\n Return a polynomial G(t) of degree 8, s.t.\n G(p^(-s))^(-1) is the p-Euler factor of the spinor L function of F.\n =E2=80=99=E2=80=99=E2=80=99\n p =3D ZZ(p)\n if t = is None:\n t =3D PolynomialRing(QQ, 1, names=3D=E2=80=99t=E2=80=99, order=3D\"neglex\").gens= ()[0]\n c =3D {}\n tp =3D hecke_eigenvalue_tp(p, F, T=3DT)\n tpp1, tpp2, tpp3 =3D [hecke_eigenvalue_tp2(p, i, F, T=3DT) for i in [1, 2, 3]]\n c[0] =3D ZZ(= 1)\n c[1] =3D tp\n c[2] =3D p * (tpp1 + (p**2 + 1) * tpp2 + (p**2 + 1)**2 = * tpp3)\n c[3] =3D p**3 * tp * (tpp2 + tpp3)\n c[4] =3D p**6 * (tp**2 * tpp3 + tpp2**2 - 2 * p * tpp1 * tpp3 -\n 2 * (p - 1) * tpp2 * tpp3 -\n (p**6 + 2 * p**5 + 2 * p**3 + 2 * p - 1) * tpp3**2)\n c[5] =3D p**6 * tpp3 * c[3]\n c[6] =3D p**12 * tpp3 ** 2= * c[2]\n c[7] =3D p**18 * tpp3 ** 3 * c[1]\n c[8] =3D p**24 * tpp3 ** 4= \n return sum((-1)**k * v * t**k for k, v in c.items())\n\n\ndef rankin_convolution_degree1(f, g, p, name=3DNone):\n u=E2=80=99=E2=80=99= =E2=80=99\n f, g: primitive forms of degree 1 and level 1.\n Return p-euler factor of the Rankin convolution of f and g as\n a polynomial.\n =E2=80=99=E2=80=99= =E2=80=99\n k1 =3D f.weight()\n k2 =3D g.weight()\n ap =3D f[p]\n bp =3D g[p]\n t = =3D PolynomialRing(QQ, 1, names=3D=E2=80=99t=E2=80=99 if name is None else name= ,\n order=3D\"neglex\").gens()[0]\n return (1 - ap * bp * t +\n (ap**2 * p**(k2 - 1) + bp**2 * p**(k1 - 1) - 2 * p**(k1 + k2 - 2)) * t**2 -\n ap * bp * p**(k1 + k2 - 2) * t**3 + p**(2 * (k1 + k2 - 2)) * t**4)\n\n\ndef _hecke_eigenvalue_base(fc_func, F, T=3DNone):\n if = T is None:\n T =3D HalfIntMatElement(matrix([[ZZ(1), ZZ(1) / ZZ(2), ZZ(1) / ZZ(2)],\n [ZZ(1) / ZZ(2), ZZ(1), ZZ(1) / ZZ(2)],\n [ZZ(1) / ZZ(2), ZZ(1) / ZZ(2), ZZ(1)]]))\n if not isinstance(T, HalfIntMatElement):\n T =3D HalfIntMatElement(T)\n v1 =3D fc_func(T).vector\n v =3D F[T].vector\n if v =3D=3D 0:\n raise ZeroDivisionError\n else:\n i =3D next(i for i in range(len(v)) i= f v[i] !=3D 0)\n return v1[i] / v[i]\n\n\n@cached_function\ndef tp_action_fc_alist(p, T):\n =E2=80=99=E2=80=99=E2=80=99\n return a li= st of tuples (S, a, g) s.t.\n S: an instance of HalfIntMatElement\n a: integer\n g: 3 by 3 matrix s.t.\n F|T(p) =3D sum(a rho(g) F[S] | (a, g, S)).\n =E2=80= =99=E2=80=99=E2=80=99\n res1 =3D []\n for alpha in alpha_list(1):\n D =3D diagonal_matri= x([p ** a for a in alpha])\n for V in _gl3_coset_gamma0(alpha, p):\n M =3D D * V\n S =3D T.right_action(M.transpose())\n if S.is_divisible_by(p):\n S =3D S // p\n if S.satisfy_cong_condition_tp(p, alpha):\n # p**(-6) and p in the third item are for normalization.\n res1.append(\n (S, p ** (-6) * mul(p ** alpha[i] for i in range(3) for j in range(i, 3)),\n M ** (-1) * p))\n return __convert_reduced_nonisom_matrices(res1)\n\n\ndef __convert_reduced_nonisom_matrices(alst):\n red_res =3D []\n for s, a= , g in alst:\n u =3D _minkowski_reduction_transform_matrix(s.T)\n t =3D s.right_action(u)\n red_res.append((t, a, g * u.transpose() *= * (-1)))\n\n non_isoms =3D []\n\n for s, a, g in red_res:\n q = =3D QuadraticForm(ZZ, 2 * s.T)\n u =3D None\n for t, _, _ in non_isoms:\n q1 =3D QuadraticForm(ZZ, 2 * t.T)\n if q.det() =3D=3D q1.det():\n u =3D q.is_globally_equivalent_to= (q1, return_matrix=3DTrue)\n if u and u.transpose() * q.Gram_matrix_rational() * u =3D=3D q1.Gram_matrix_rational():\n break\n if u:\n non_isoms.append((s.right_action(u), a, g * u.transpose() ** (-1)))\n else:\n non_isoms.append((s, a, g))\n return non_isoms\n\n\n@cached_function\nd= ef tp2_action_fc_alist(p, T, i):\n =E2=80=99=E2=80=99=E2=80=99\n similar= to tp_action_fc_alist for T_i(p^2) for i =3D 0, 1, 2, 3.\n =E2=80=99=E2=80=99=E2=80=99\n re= s1 =3D []\n\n for alpha in alpha_list(2):\n D =3D diagonal_matrix([p ** a for a in alpha])\n for V in _gl3_coset_gamma0(alpha, p):\n M =3D D * V\n S =3D T.right_action(M.transpose())\n if S.is_divisible_by(p = ** 2):\n S =3D S // (p ** 2)\n res1.append((S, p ** (-12) * _expt_sum(S, p, alpha, i),\n M ** (-1) * p ** 2))\n\n return __convert_reduced_nonisom_matrices([(a, b, c) for a, b, c in res1 if b !=3D 0])\n\n\ndef _nearest_integer(x):\n r =3D floor(x)\n if x - r > 0.5:\n return r + 1\n else:\n return r\n\n\ndef _gaussian_reduction(b1, b2, S):\n =E2=80=99=E2=80=99= =E2=80=99\n b1, b2: vectors of length 3\n S: symmetric matrix of size 3\n =E2=80=99=E2=80= =99=E2=80=99\n while True:\n nb1 =3D b1 * S * b1\n nb2 =3D b2 * S * b2\n if= nb2 < nb1:\n b1, b2 =3D b2, b1\n x =3D (b2 * S * b1) / (b1 * = S * b1)\n r =3D _nearest_integer(x)\n a =3D b2 - r * b1\n = if a * S * a >=3D b2 * S * b2:\n return (b1, b2)\n else:\n b1, b2 =3D a, b1\n\n\ndef _sym_mat_gen(p, n):\n if n =3D=3D 1:\n for a in range(p):\n yield matrix([[a]])\n else:\n for s in _sym_mat_gen(p, n - 1):\n ls =3D [range(p) for _ in range(n)]\n for a in itertools.product(*ls):\n v =3D matrix([a[:-1]])\n yield block_matrix([[s, v.transpose()= ], [v, matrix([[a[-1]]])]])\n\n\ndef _gen_gauss_sum_direct_way(N, p, r):\n res =3D 0\n K =3D CyclotomicField(p)\n zeta =3D K.gen()\n for S i= n _sym_mat_gen(p, N.ncols()):\n if S.change_ring(FiniteField(p)).rank() =3D=3D r:\n res +=3D zeta *= * ((N * S).trace())\n try:\n return QQ(res)\n except TypeError:\n return res\n\n\ndef _generalized_gauss_sum(N, p, r):\n if r =3D=3D = 0:\n return 1\n if p =3D=3D 2:\n return _gen_gauss_sum_direct_way(N, p, r)\n else:\n N_mp =3D N.change_ring(FiniteField(p))\n d, _, v =3D N_mp.smith_form()\n t =3D d.rank()\n N1 =3D (v.transpose() * N_mp *\n v).matrix_from_rows_and_columns(range(t), range(t))\n eps =3D kronecker_symbol(N1.det(), p)\n return _gen_gauss_sum_non_dyadic(p, eps, N.ncols(), t, r)\n\n\ndef _half_int_mat_is_div_by(S, m):\n n =3D S.ncols()\n return (all(ZZ(S[i, i]) % m =3D=3D 0 for i in range(n)) and\= n all(ZZ(2 * S[i, j]) % m =3D=3D 0 for i in range(n) for j in range(= i + 1, n)))\n\n\n@cached_function\ndef _gen_gauss_sum_non_dyadic(p, eps, n, t, r):\n =E2=80=99=E2=80=99=E2=80=99\n cf. H. Saito, a generalization of= Gauss sums\n =E2=80=99=E2=80=99=E2=80=99\n\n def parenthesis_prod(a, b, m):\n if m =3D=3D 0:\n retu= rn 1\n else:\n return mul(1 - a * b ** i for i in range(m))\n\n if (n - t) % 2 =3D=3D 0:\n m =3D (n - t) // 2\n else:\n m =3D (n - t + 1) // 2\n\n if n =3D=3D r:\n if n = % 2 =3D=3D 1:\n return ((-1) ** ((n - 2 * m + 1) // 2) * p ** ((n ** 2 + (2 * m) ** 2 - 1) // 4) *\n parenthesis_prod(p ** (-1), p ** (-2), m))\n elif n % 2 =3D=3D t % 2 =3D=3D 0:\n return ((-kronecker_symbol(-1, p)) ** ((n - 2 * m) // 2) *\n eps * p ** ((n ** 2 + (2 * m + 1) ** 2 - 1) // 4) *\n parenthesis_prod(p ** (-1), p ** (-2), m))\n else:\n return 0\n else:\n diag =3D [1 for _ in range(t)]\n if ep= s =3D=3D -1:\n diag[-1] =3D least_quadratic_nonresidue(p)\n = diag =3D diag + [0 for _ in range(n - t)]\n N =3D diagonal_matrix(diag).change_ring(FiniteField(p))\n return _gen_gauss_sum_direct_way(N, p, r)\n\n\ndef _expt_sum(S, p, alpha, i):\n =E2=80=99=E2=80=99=E2=80=99\n Return the exponential sum in Miyawaki=E2= =80=99s paper, where alpha[-1] <=3D 2, for T_i(p^2).\n =E2=80=99=E2=80=99=E2=80=99\n a, b, c =3D [al= pha.count(_i) for _i in range(3)]\n S33 =3D S.T.matrix_from_rows_and_columns(range(a + b, 3), range(a + b, 3))\n S22 =3D S.T.matrix_from_rows_and_columns(range(a, a + b), range(a, a + b))\n S32 =3D S.T.matrix_from_rows_and_columns(range(a = + b, 3), range(a))\n\n if c > 0 and not _half_int_mat_is_div_by(S33, p ** 2):\n return 0\n if c > 0 and b > 0 and any(x % p !=3D 0 for x in (S32 * ZZ(2)).change_ring(ZZ).list()):\n return 0\n\n if b =3D=3D= 0 and a + c =3D=3D 3 - i:\n return p ** (c * (c + 1))\n elif b =3D= =3D 0:\n return 0\n else:\n return p ** (c * (c + 1)) * p ** (b * c) * _generalized_gauss_sum(S22, p, b - i)\n\n\ndef _minkowski_reduction(b1, b2, b3, S):\n\n def inner_prod(x, y):\n return x * S * y\n\n while True:\n b1, b2, b3 =3D sorted([b1, b2= , b3], key=3Dlambda b: b * S * b)\n\n b1, b2 =3D _gaussian_reduction(b= 1, b2, S)\n\n b11 =3D inner_prod(b1, b1)\n b12 =3D inner_prod(b1= , b2)\n b13 =3D inner_prod(b1, b3)\n b22 =3D inner_prod(b2, b2)= \n b23 =3D inner_prod(b2, b3)\n b33 =3D inner_prod(b3, b3)\n\n y1 =3D - (b13 / b11 - b12 * b23 / (b11 * b22)) / \\\n (1 - b12 = ** 2 / (b11 * b22))\n y2 =3D - (b23 / b22 - b12 * b13 / (b11 * b22)) / \\\n (1 - b12 ** 2 / (b11 * b22))\n\n # Find integers x1, x2 so that norm(b3 + x2 * b2 + x1 * b1) is minimal.\n a_norms_alst = =3D []\n\n for x1 in [floor(y1), ceil(y1)]:\n for x2 in [floor(y2), ceil(y2)]:\n a =3D b3 + x2 * b2 + x1 * b1\n a_norms_alst.append((x1, x2, a, inner_prod(a, a)))\n _inner_prod_a =3D min(x[-1] for x in a_norms_alst)\n x1, x2, a, _ = =3D next(x for x in a_norms_alst if x[-1] =3D=3D _inner_prod_a)\n\n if _inner_prod_a >=3D b33:\n # Change sings of b1, b2, b3 and terminate the alogrithm\n sngs =3D [sgn(b12), sgn(b13), sgn(b23)]\n bs =3D [b1, b2, b3]\n try:\n # If b12, b13 or b23 is zero, change sgns of b1, b2, b3 so that\n # b12, b13, b23 >=3D 0.\n zero_i =3D sngs.index(0)\n set_ls =3D [set([1, 2]), set([1, 3]), set([2, 3])]\n t =3D set_ls[zero_i]\n _other =3D [x for x in [1, 2, 3] if = x not in t][0]\n for x in t:\n i =3D set_ls.index(set([x, _other]))\n if sngs[i] < 0:\n bs[x - 1] *=3D -1\n b1, b2, b3 =3D bs\n except ValueError:\n # Else change sgns so that b12, b13 > 0\n if b12 < 0:\n b2 =3D -b2\n if b13 < 0:\n b3 =3D -b3\n return (b1, b= 2, b3)\n else:\n b3 =3D a\n\n\ndef _minkowski_reduction_transform_matrix(S):\n =E2=80=99=E2=80=99=E2=80=99\= n Return a unimodular matrix u such that u^t * S * u is reduced in Minkowski=E2=80=99s sense.\n = =E2=80=99=E2=80=99=E2=80=99\n b1, b2, b3 =3D identity_matrix(QQ, 3).columns()\n c1, c2, c3 =3D _minkowski_reduction(b1, b2, b3, S)\n return matrix([c1, c2, c3]).transpose()\n","line":10,"column":0,"path":"/home/sho/work/sage_packag= es/e8theta_degree3/hecke_module.py"}} Quit [2 times] Load-path shadows: /home/sho/.emacs.d/elpa/helm-20160723.2238/helm-multi-match hides /home/sho/.emacs.d/elpa/helm-core-20160723.944/helm-multi-match /home/sho/.emacs.d/elpa/scala-mode-20160519.731/ob-scala hides /usr/local/share/emacs/25.1/lisp/org/ob-scala /home/sho/.emacs.d/elpa/seq-2.16/seq hides /usr/local/share/emacs/25.1/lisp/emacs-lisp/seq Features: (shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec epg mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader sendmail mail-utils network-stream nsm starttls url-cache url-http tls gnutls mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-gw url-auth anaconda-mode pythonic f url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util mailcap vc-git diff-mode easy-mmode python tramp-sh tramp tramp-compat tramp-loaddefs trampver shell pcomplete format-spec comint ring ansi-color finder-inf tex-site advice edmacro kmacro gh-common gh-profile url-parse auth-source gnus-util mm-util help-fns mail-prsvr password-cache url-vars s ucs-normalize marshal eieio-compat cl-seq json map dash eieio eieio-core cl-macs go-mode-autoloads rx info package epg-config seq byte-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util japan-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify dynamic-setting system-font-setting font-render-setting move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 279045 5729) (symbols 48 28763 0) (miscs 40 591 109) (strings 32 47201 10304) (string-bytes 1 1610049) (vectors 16 45065) (vector-slots 8 885950 3678) (floats 8 389 220) (intervals 56 445 0) (buffers 976 23) (heap 1024 44595 2062)) --001a1134e8d40621d60538ea3e48 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I got an error "error in process sentinel: url-h= ttp-create-request: Multibyte text in HTTP request" when I visited a P= ython file which contains a multibyte character with `anaconda-eldoc-mode&#= 39; turned on.

At first, I thought this was a bug = of anaconda-mode. So I opened an issue in github (https://github.com/proofit404/ana= conda-mode/issues/189).

I guess `(=3D (string-= bytes request) (length request))` in `url-http-create-request' should b= e `(=3D (string-bytes url-http-data) (length url-http-data))`, because `(= =3D (string-bytes request) (length request))` may be `nil' even if `(= =3D (string-bytes url-http-data) (length url-http-data))` is `t'.
=

Sho Takemori

In GNU Emacs 25.1= .1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
=C2=A0of 2016-07-26= built on HP-500-270jp
Repository revision: 0f0b191a5324115fe9e8c= 438eceef4043decf209
Windowing system distributor 'The X.Org F= oundation', version 11.0.11803000
System Description: Ubuntu 16.04= .1 LTS

Configured using:
=C2=A0'conf= igure --with-sound=3Dno --with-modules'

Config= ured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK GPM DBUS GC= ONF GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_= FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
=C2=A0 value of $LC_MONETARY= : ja_JP.UTF-8
=C2=A0 value of $LC_NUMERIC: ja_JP.UTF-8
= =C2=A0 value of $LC_TIME: ja_JP.UTF-8
=C2=A0 value of $LANG: ja_J= P.UTF-8
=C2=A0 value of $XMODIFIERS: @im=3Dibus
=C2=A0 = locale-coding-system: utf-8-unix

Major mode: Lisp = Interaction

Minor modes in effect:
=C2= =A0 diff-auto-refine-mode: t
=C2=A0 shell-dirtrack-mode: t
<= div>=C2=A0 tooltip-mode: t
=C2=A0 global-eldoc-mode: t
= =C2=A0 electric-indent-mode: t
=C2=A0 mouse-wheel-mode: t
=C2=A0 tool-bar-mode: t
=C2=A0 menu-bar-mode: t
=C2= =A0 file-name-shadow-mode: t
=C2=A0 global-font-lock-mode: t
=C2=A0 font-lock-mode: t
=C2=A0 blink-cursor-mode: t
<= div>=C2=A0 auto-composition-mode: t
=C2=A0 auto-encryption-mode: = t
=C2=A0 auto-compression-mode: t
=C2=A0 line-number-mo= de: t
=C2=A0 transient-mark-mode: t

Rece= nt messages:
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Accept-encoding: gzip<= /div>
Accept: */*
User-Agent: URL/Emacs
Content-len= gth: 20552

{"jsonrpc":"2.0",&q= uot;id":1,"method":"eldoc","params":{&qu= ot;source":"# -*- coding: utf-8 -*-\nimport itertools\nfrom itert= ools import groupby\n\nfrom sage.all import mul\nfrom sage.arith.all import= kronecker_symbol\nfrom sage.functions.all import ceil, floor, sgn\nfrom sa= ge.matrix.all import (block_diagonal_matrix, block_matrix,\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 diagonal_matrix, identity_matrix, matrix)\nfrom sage.misc.all im= port cached_function\nfrom sage.quadratic_forms.all import QuadraticForm, l= east_quadratic_nonresidue\nfrom sage.rings.all import QQ, ZZ, CyclotomicFie= ld, FiniteField, PolynomialRing\n\n\ndef _index_of_gamma_0_gl_n(alpha, p):\= n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Returns delta(a1,= ..., an) defined in Shimura, Euler products and Eisenstein\n =C2=A0 =C2=A0= series, pp 118, (15.1.7).\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2= =A0 =C2=A0if p in ZZ:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0p =3D ZZ(p)\n\n =C2=A0 = =C2=A0def _bn(n):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return mul(1 - p ** (-i) for= i in xrange(1, n + 1))\n\n =C2=A0 =C2=A0e_r_ls =3D [(k, len(list(v)))\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for k, v in groupby(sorted(= alpha), lambda x: x)]\n =C2=A0 =C2=A0res =3D _bn(len(alpha)) / mul(_bn(r) f= or _, r in e_r_ls)\n =C2=A0 =C2=A0for i, (ei, ri) in enumerate(e_r_ls):\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0for j, (ej, rj) in enumerate(e_r_ls):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if i < j:\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res *=3D p ** ((ej - ei) * ri * rj)\n =C2=A0= =C2=A0return res\n\n\ndef _gl2_coset_gamma0(a, p):\n =C2=A0 =C2=A0w =3D ma= trix([[0, -1],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1,= 0]])\n =C2=A0 =C2=A0for m12 in range(p ** a):\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0yield matrix([[1, m12],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1]])\n =C2=A0 =C2=A0for m21 in range(p *= * (a - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0],\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[p * m21, 1]])\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w * m\n\n\ndef _gl3_coset_gamma0(alpha, p= ):\n =C2=A0 =C2=A0r=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Let alpha =3D= [a0, a1, a2] with a0 <=3D a1 <=3D a2,\n =C2=A0 =C2=A0g =3D diag([p^a= 0, p^a1, p^a2]), and Gamma0 =3D g^(-1) GL3(Z) g =E2=88=A7 GL3(Z).\n =C2=A0 = =C2=A0Return a complete set Gamma0 \\ GL3(Z).\n =C2=A0 =C2=A0=E2=80=99=E2= =80=99=E2=80=99\n =C2=A0 =C2=A0if p in ZZ:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0p = =3D ZZ(p)\n =C2=A0 =C2=A0a0, a1, a2 =3D alpha\n =C2=A0 =C2=A0if a0 < a1 = < a2:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return list(__gl3_coset_gamma0_distin= ct(a0, a1, a2, p))\n =C2=A0 =C2=A0elif a0 =3D=3D a1 and a1 < a2:\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0return list(__gl3_coset_gamma0_2_1(a0, a2, p))\n = =C2=A0 =C2=A0elif a0 < a1 and a1 =3D=3D a2:\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0return list(__gl3_coset_gamma0_1_2(a0, a2, p))\n =C2=A0 =C2=A0elif a0 = =3D=3D a1 =3D=3D a2:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return [identity_matrix(Z= Z, 3)]\n =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0raise ValueError\n= \n\ndef __gl3_coset_gamma0_2_1(a1, a3, p):\n =C2=A0 =C2=A0w23 =3D matrix([[= 1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[= 0, 0, 1],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[= 0, 1, 0]])\n =C2=A0 =C2=A0for m13 in range(p ** (a3 - a1 - 1)):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0for m23 in range(p ** (a3 - a1 - 1)):\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, p * m13],\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1,= p * m23],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0yield m\n\n =C2=A0 =C2=A0for m32 in range(p ** (a3 - a1)):\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, m32, 1]])\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0for g in _gl2_coset_gamma0(a3 - a1, p):\n =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0n =3D block_diagonal_matrix(g, matrix([[1]]))\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w23 * m * n\n\n\ndef __gl3_coset_g= amma0_1_2(a1, a2, p):\n =C2=A0 =C2=A0w12 =3D matrix([[0, 1, 0],\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0],\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\n\n =C2= =A0 =C2=A0for m12 in range(p ** (a2 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0for m13 in range(p ** (a2 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0m =3D matrix([[1, p * m12, p * m13],\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0[0, 0, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield m\n =C2= =A0 =C2=A0for m21 in range(p ** (a2 - a1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m = =3D matrix([[1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0[m21, 1, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for= g in _gl2_coset_gamma0(a2 - a1, p):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0n =3D block_diagonal_matrix(matrix([[1]]), g)\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0yield w12 * m * n\n\n\ndef __gl3_coset_gamma0_distinct(= a1, a2, a3, p):\n\n =C2=A0 =C2=A0w12 =3D matrix([[0, 1, 0],\n =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0],\n =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\n\n =C2=A0 =C2= =A0w23 =3D matrix([[1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0[0, 0, 1],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0[0, 1, 0]])\n\n =C2=A0 =C2=A0w13 =3D matrix([[0, 0, 1],= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0]]= )\n\n =C2=A0 =C2=A0w123 =3D matrix([[0, 1, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [0, 0, 1],\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [1, 0, 0]])\n\n =C2=A0 =C2=A0w132= =3D matrix([[0, 0, 1],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 [1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 [0, 1, 0]])\n\n =C2=A0 =C2=A0# w =3D 1\n =C2=A0 =C2=A0for m1= 2 in range(p ** (a2 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m13 in ran= ge(p ** (a3 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m23 = in range(p ** (a3 - a2 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0yield matrix([[1, p * m12, p * m13],\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0[0, 1, p * m23],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\n = =C2=A0 =C2=A0# w =3D (12)\n =C2=A0 =C2=A0for m13 in range(p ** (a3 - a2 - 1= )):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m21 in range(p ** (a2 - a1)):\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m23 in range(p ** (a3 - a1 - 1)):\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0,= p * m13],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m21, 1, p * m23],\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0[0, 0, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= yield w12 * m\n =C2=A0 =C2=A0# w =3D (23)\n =C2=A0 =C2=A0for m12 in range(p= ** (a3 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m13 in range(p ** (a2 = - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m32 in range(p *= * (a3 - a2)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m = =3D matrix([[1, p * m12, p * m13],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0[0, m32, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0yield w23 * m\n\n =C2=A0 =C2=A0# w =3D (13)\n =C2=A0 =C2= =A0for m21 in range(p ** (a3 - a2)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m31 i= n range(p ** (a3 - a1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m32= in range(p ** (a2 - a1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0m =3D matrix([[1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m21, 1, 0],\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[m31, m32, 1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0yield w13 * m\n\n =C2=A0 =C2=A0# w =3D (123)\n =C2= =A0 =C2=A0for m21 in range(p ** (a3 - a1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0fo= r m23 in range(p ** (a2 - a1 - 1)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0for m31 in range(p ** (a3 - a2)):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m= 21, 1, p * m23],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m31, 0, 1]])\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w123 * m\n =C2=A0 =C2=A0# w =3D= (132)\n =C2=A0 =C2=A0for m12 in range(p ** (a3 - a2 - 1)):\n =C2=A0 =C2=A0= =C2=A0 =C2=A0for m31 in range(p ** (a2 - a1)):\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0for m32 in range(p ** (a3 - a1)):\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, p * m12, 0],\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0[0, 1, 0],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m31, m32, 1]])\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w132 * m\n\n\ncla= ss HalfIntMatElement(object):\n\n =C2=A0 =C2=A0def __init__(self, T):\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0:params T: half integral matrix of size 3 or a list\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if is= instance(T, list):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0a, b, c, d, e= , f =3D [ZZ(x) for x in T]\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mat = =3D matrix([[a, f / 2, e / 2],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[f / 2, b, d / 2],\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0[e / 2, d / 2, c]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mat =3D T\n =C2=A0 =C2=A0 =C2=A0 =C2=A0se= lf.__entries =3D tuple(mat.list())\n\n =C2=A0 =C2=A0def __eq__(self, other)= :\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if isinstance(other, HalfIntMatElement):\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return self.__entries =3D=3D other= .__entries\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0raise NotImplementedError\n\n =C2=A0 =C2=A0def __repr__(self)= :\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return self.T.__repr__()\n\n =C2=A0 =C2=A0de= f __hash__(self):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return hash(self.__entries)\= n\n =C2=A0 =C2=A0@property\n =C2=A0 =C2=A0def T(self):\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0return matrix(3, self.__entries)\n\n =C2=A0 =C2=A0def right_actio= n(self, g):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0:param g: matrix of size n\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0return self[g] (Siegel=E2=80=99s notation)\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D g.transpo= se() * self.T * g\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return HalfIntMatElement(S)\= n\n =C2=A0 =C2=A0def satisfy_cong_condition_tp(self, p, alpha):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0Test if sum_{B mod D} exp(2pi T B D^(-1)) is zero, where D =3D diag(p^a1= , p^a2, a^a3),\n =C2=A0 =C2=A0 =C2=A0 =C2=A0a1, a2, a3 =3D alpha.\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0return (all(ZZ(self.T[i, i]) % p ** alpha[i] =3D=3D 0 for i in range(3))= and\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0all(ZZ(self.T= [i, j] * 2) % p ** alpha[i] =3D=3D 0\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for i in range(3) for j in range(i + 1, 3= )))\n\n =C2=A0 =C2=A0def is_divisible_by(self, m):\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2=A0Test if self= is divisible by m\n =C2=A0 =C2=A0 =C2=A0 =C2=A0:param m: integer\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0return _half_int_mat_is_div_by(self.T, m)\n\n =C2=A0 =C2=A0def __floordi= v__(self, other):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D matrix(QQ, 3)\n =C2=A0= =C2=A0 =C2=A0 =C2=A0for i in range(3):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0S[i, i] =3D ZZ(self.T[i, i]) // other\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0for i in range(3):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for j in r= ange(i + 1, 3):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0S[= i, j] =3D S[j, i] =3D (ZZ(self.T[i, j] * 2) // other) / 2\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0return HalfIntMatElement(S)\n\n\ndef alpha_list(dl):\n =C2=A0 = =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Return a list of (a0, a1, = a2) with 0 <=3D a0 <=3D a1 <=3D a2 <=3D dl\n =C2=A0 =C2=A0=E2= =80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0return [(a0, a1, a2) for a0 in rang= e(dl + 1)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for a1 in range(a0, dl= + 1) for a2 in range(a1, dl + 1)]\n\n\ndef tp_action_fourier_coeff(p, T, F= ):\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Return the Tth= Fourier coefficient of F|T(p), where F is a modular form.\n =C2=A0 =C2=A0:= param p: a prime number\n =C2=A0 =C2=A0:param T: a half integral matrix or = an instance of HalfIntMatElement\n =C2=A0 =C2=A0:param F: a dictionary or a= Siegel modular form of degree 3\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99= \n =C2=A0 =C2=A0p =3D ZZ(p)\n =C2=A0 =C2=A0return _action_fc_base(tp_action= _fc_alist(p, T), F, T)\n\n\ndef tp2_action_fourier_coeff(p, i, T, F):\n =C2= =A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Similar to tp_action_f= ourier_coeff for T_i(p^2).\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2= =A0 =C2=A0p =3D ZZ(p)\n =C2=A0 =C2=A0return _action_fc_base(tp2_action_fc_a= list(p, T, i), F, T)\n\n\ndef _action_fc_base(ls, F, T):\n =C2=A0 =C2=A0if = not isinstance(T, HalfIntMatElement):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D Ha= lfIntMatElement(T)\n =C2=A0 =C2=A0res =3D 0\n =C2=A0 =C2=A0for s, a, g in l= s:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0res =3D a * F[s].left_action(g) + res\n =C2= =A0 =C2=A0return res\n\n\ndef hecke_eigenvalue_tp(p, F, T=3DNone):\n =C2=A0= =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0p, F, T: same as aruments= of tp_action_fourier_coeff.\n =C2=A0 =C2=A0Assuming F is an eigenform, ret= urn the eigenvalue for T(p),\n =C2=A0 =C2=A0T is used for the computation o= f Fourier coefficients.\n =C2=A0 =C2=A0If T is omitted, T will be set to\n = =C2=A0 =C2=A0matrix([[1, 1/2, 1/2], [1/2, 1, 1/2], [1/2, 1/2, 1]]).\n =C2= =A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0return _hecke_eigenval= ue_base(lambda s: tp_action_fourier_coeff(p, s, F), F, T=3DT)\n\n\ndef heck= e_eigenvalue_tp2(p, i, F, T=3DNone):\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2= =80=99\n =C2=A0 =C2=A0Similar to hecke_eigenvalue_tp for T(p^2).\n =C2=A0 = =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0return _hecke_eigenvalue_b= ase(lambda s: tp2_action_fourier_coeff(p, i, s, F), F, T=3DT)\n\n\ndef spin= or_l_euler_factor(p, F, t=3DNone, T=3DNone):\n =C2=A0 =C2=A0=E2=80=99=E2=80= =99=E2=80=99\n =C2=A0 =C2=A0F: a dict or Siegel modular form of degree 3.\n= =C2=A0 =C2=A0Return a polynomial G(t) of degree 8, s.t.\n =C2=A0 =C2=A0G(p= ^(-s))^(-1) is the p-Euler factor of the spinor L function of F.\n =C2=A0 = =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0p =3D ZZ(p)\n =C2=A0 =C2= =A0if t is None:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0t =3D PolynomialRing(QQ, 1, n= ames=3D=E2=80=99t=E2=80=99, order=3D\"neglex\").gens()[0]\n =C2= =A0 =C2=A0c =3D {}\n =C2=A0 =C2=A0tp =3D hecke_eigenvalue_tp(p, F, T=3DT)\n= =C2=A0 =C2=A0tpp1, tpp2, tpp3 =3D [hecke_eigenvalue_tp2(p, i, F, T=3DT) fo= r i in [1, 2, 3]]\n =C2=A0 =C2=A0c[0] =3D ZZ(1)\n =C2=A0 =C2=A0c[1] =3D tp\= n =C2=A0 =C2=A0c[2] =3D p * (tpp1 + (p**2 + 1) * tpp2 + (p**2 + 1)**2 * tpp= 3)\n =C2=A0 =C2=A0c[3] =3D p**3 * tp * (tpp2 + tpp3)\n =C2=A0 =C2=A0c[4] = =3D p**6 * (tp**2 * tpp3 + tpp2**2 - 2 * p * tpp1 * tpp3 -\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2 * (p - 1) * tpp2 * tpp3 = -\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (p**6 + = 2 * p**5 + 2 * p**3 + 2 * p - 1) * tpp3**2)\n =C2=A0 =C2=A0c[5] =3D p**6 * = tpp3 * c[3]\n =C2=A0 =C2=A0c[6] =3D p**12 * tpp3 ** 2 * c[2]\n =C2=A0 =C2= =A0c[7] =3D p**18 * tpp3 ** 3 * c[1]\n =C2=A0 =C2=A0c[8] =3D p**24 * tpp3 *= * 4\n =C2=A0 =C2=A0return sum((-1)**k * v * t**k for k, v in c.items())\n\n= \ndef rankin_convolution_degree1(f, g, p, name=3DNone):\n =C2=A0 =C2=A0u=E2= =80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0f, g: primitive forms of degree 1 a= nd level 1.\n =C2=A0 =C2=A0Return p-euler factor of the Rankin convolution = of f and g as\n =C2=A0 =C2=A0a polynomial.\n =C2=A0 =C2=A0=E2=80=99=E2=80= =99=E2=80=99\n =C2=A0 =C2=A0k1 =3D f.weight()\n =C2=A0 =C2=A0k2 =3D g.weigh= t()\n =C2=A0 =C2=A0ap =3D f[p]\n =C2=A0 =C2=A0bp =3D g[p]\n =C2=A0 =C2=A0t = =3D PolynomialRing(QQ, 1, names=3D=E2=80=99t=E2=80=99 if name is None else = name,\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 order=3D\"neglex\").gens()[0]\n =C2=A0 =C2=A0return (1= - ap * bp * t +\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(ap**2 * p**(k2= - 1) + bp**2 * p**(k1 - 1) - 2 * p**(k1 + k2 - 2)) * t**2 -\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ap * bp * p**(k1 + k2 - 2) * t**3 + p**(2 * = (k1 + k2 - 2)) * t**4)\n\n\ndef _hecke_eigenvalue_base(fc_func, F, T=3DNone= ):\n =C2=A0 =C2=A0if T is None:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D HalfIntM= atElement(matrix([[ZZ(1), ZZ(1) / ZZ(2), ZZ(1) / ZZ(2)],\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[ZZ(1) / ZZ(2), ZZ(1), ZZ(1) /= ZZ(2)],\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[ZZ(1)= / ZZ(2), ZZ(1) / ZZ(2), ZZ(1)]]))\n =C2=A0 =C2=A0if not isinstance(T, Half= IntMatElement):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D HalfIntMatElement(T)\n = =C2=A0 =C2=A0v1 =3D fc_func(T).vector\n =C2=A0 =C2=A0v =3D F[T].vector\n = =C2=A0 =C2=A0if v =3D=3D 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0raise ZeroDivision= Error\n =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0i =3D next(i for i = in range(len(v)) if v[i] !=3D 0)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return v1[i] = / v[i]\n\n\n@cached_function\ndef tp_action_fc_alist(p, T):\n =C2=A0 =C2=A0= =E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0return a list of tuples (S, a, g= ) s.t.\n =C2=A0 =C2=A0S: an instance of HalfIntMatElement\n =C2=A0 =C2=A0a:= integer\n =C2=A0 =C2=A0g: 3 by 3 matrix s.t.\n =C2=A0 =C2=A0F|T(p) =3D sum= (a rho(g) F[S] | (a, g, S)).\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n = =C2=A0 =C2=A0res1 =3D []\n =C2=A0 =C2=A0for alpha in alpha_list(1):\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0D =3D diagonal_matrix([p ** a for a in alpha])\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0for V in _gl3_coset_gamma0(alpha, p):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0M =3D D * V\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0S =3D T.right_action(M.transpose())\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0if S.is_divisible_by(p):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D S // p\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0if S.satisfy_cong_condition_tp(p, alpha):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# p**(-6) and= p in the third item are for normalization.\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res1.append(\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(S, p ** = (-6) * mul(p ** alpha[i] for i in range(3) for j in range(i, 3)),\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 M ** (-1) * p))\n =C2=A0 =C2=A0return __convert_reduced_nonisom_matrice= s(res1)\n\n\ndef __convert_reduced_nonisom_matrices(alst):\n =C2=A0 =C2=A0r= ed_res =3D []\n =C2=A0 =C2=A0for s, a, g in alst:\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0u =3D _minkowski_reduction_transform_matrix(s.T)\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0t =3D s.right_action(u)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0red_res.appe= nd((t, a, g * u.transpose() ** (-1)))\n\n =C2=A0 =C2=A0non_isoms =3D []\n\n= =C2=A0 =C2=A0for s, a, g in red_res:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0q =3D Qu= adraticForm(ZZ, 2 * s.T)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0u =3D None\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0for t, _, _ in non_isoms:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0q1 =3D QuadraticForm(ZZ, 2 * t.T)\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0if q.det() =3D=3D q1.det():\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0u =3D q.is_globally_equivalent_to(q1, return_mat= rix=3DTrue)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if u a= nd u.transpose() * q.Gram_matrix_rational() * u =3D=3D q1.Gram_matrix_ratio= nal():\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0break\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if u:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0non_isoms.append((s.right_action(u), a, g * u.transpose() ** (= -1)))\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0non_isoms.append((s, a, g))\n =C2=A0 =C2=A0return non_isoms\n\n\n= @cached_function\ndef tp2_action_fc_alist(p, T, i):\n =C2=A0 =C2=A0=E2=80= =99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0similar to tp_action_fc_alist for T_i(= p^2) for i =3D 0, 1, 2, 3.\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2= =A0 =C2=A0res1 =3D []\n\n =C2=A0 =C2=A0for alpha in alpha_list(2):\n =C2=A0= =C2=A0 =C2=A0 =C2=A0D =3D diagonal_matrix([p ** a for a in alpha])\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0for V in _gl3_coset_gamma0(alpha, p):\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0M =3D D * V\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0S =3D T.right_action(M.transpose())\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0if S.is_divisible_by(p ** 2):\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D S // (p ** 2)\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res1.append((S, p ** (-12) * _expt_sum(S,= p, alpha, i),\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 M ** (-1) * p ** 2))\n\n =C2=A0 = =C2=A0return __convert_reduced_nonisom_matrices([(a, b, c) for a, b, c in r= es1 if b !=3D 0])\n\n\ndef _nearest_integer(x):\n =C2=A0 =C2=A0r =3D floor(= x)\n =C2=A0 =C2=A0if x - r > 0.5:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return r = + 1\n =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return r\n\n\ndef _ga= ussian_reduction(b1, b2, S):\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n = =C2=A0 =C2=A0b1, b2: vectors of length 3\n =C2=A0 =C2=A0S: symmetric matrix= of size 3\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0while = True:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0nb1 =3D b1 * S * b1\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0nb2 =3D b2 * S * b2\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if nb2 < nb1:= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2 =3D b2, b1\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0x =3D (b2 * S * b1) / (b1 * S * b1)\n =C2=A0 =C2=A0 =C2=A0= =C2=A0r =3D _nearest_integer(x)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0a =3D b2 - r = * b1\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if a * S * a >=3D b2 * S * b2:\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return (b1, b2)\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2 =3D a, b1= \n\n\ndef _sym_mat_gen(p, n):\n =C2=A0 =C2=A0if n =3D=3D 1:\n =C2=A0 =C2=A0= =C2=A0 =C2=A0for a in range(p):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0yield matrix([[a]])\n =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0fo= r s in _sym_mat_gen(p, n - 1):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0l= s =3D [range(p) for _ in range(n)]\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0for a in itertools.product(*ls):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0v =3D matrix([a[:-1]])\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0yield block_matrix([[s, v.transpose()], [v, matr= ix([[a[-1]]])]])\n\n\ndef _gen_gauss_sum_direct_way(N, p, r):\n =C2=A0 =C2= =A0res =3D 0\n =C2=A0 =C2=A0K =3D CyclotomicField(p)\n =C2=A0 =C2=A0zeta = =3D K.gen()\n =C2=A0 =C2=A0for S in _sym_mat_gen(p, N.ncols()):\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0if S.change_ring(FiniteField(p)).rank() =3D=3D r:\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res +=3D zeta ** ((N * S).trace())\n = =C2=A0 =C2=A0try:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return QQ(res)\n =C2=A0 =C2= =A0except TypeError:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return res\n\n\ndef _gene= ralized_gauss_sum(N, p, r):\n =C2=A0 =C2=A0if r =3D=3D 0:\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0return 1\n =C2=A0 =C2=A0if p =3D=3D 2:\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0return _gen_gauss_sum_direct_way(N, p, r)\n =C2=A0 =C2=A0else:\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0N_mp =3D N.change_ring(FiniteField(p))\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0d, _, v =3D N_mp.smith_form()\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0t =3D d.rank()\n =C2=A0 =C2=A0 =C2=A0 =C2=A0N1 =3D (v.transpose() * N_mp= *\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0v).matrix_from_rows_an= d_columns(range(t), range(t))\n =C2=A0 =C2=A0 =C2=A0 =C2=A0eps =3D kronecke= r_symbol(N1.det(), p)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return _gen_gauss_sum_no= n_dyadic(p, eps, N.ncols(), t, r)\n\n\ndef _half_int_mat_is_div_by(S, m):\n= =C2=A0 =C2=A0n =3D S.ncols()\n =C2=A0 =C2=A0return (all(ZZ(S[i, i]) % m = =3D=3D 0 for i in range(n)) and\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= all(ZZ(2 * S[i, j]) % m =3D=3D 0 for i in range(n) for j in range(i + 1, n)= ))\n\n\n@cached_function\ndef _gen_gauss_sum_non_dyadic(p, eps, n, t, r):\n= =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0cf. H. Saito, a ge= neralization of Gauss sums\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n\n = =C2=A0 =C2=A0def parenthesis_prod(a, b, m):\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if= m =3D=3D 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return= mul(1 - a * b ** i for i in range(m))\n\n =C2=A0 =C2=A0if (n - t) % 2 =3D= =3D 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D (n - t) // 2\n =C2=A0 =C2=A0else:= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D (n - t + 1) // 2\n\n =C2=A0 =C2=A0if n = =3D=3D r:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if n % 2 =3D=3D 1:\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0return ((-1) ** ((n - 2 * m + 1) // 2) * p ** ((= n ** 2 + (2 * m) ** 2 - 1) // 4) *\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0parenthesis_prod(p ** (-1), p ** (-2), m))\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0elif n % 2 =3D=3D t % 2 =3D=3D 0:\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return ((-kronecker_symbol(-1, p)) ** ((n - = 2 * m) // 2) *\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0eps * p ** ((n ** 2 + (2 * m + 1) ** 2 - 1) // 4) *\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0parenthesis_p= rod(p ** (-1), p ** (-2), m))\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\n =C2=A0 =C2=A0else:\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0diag =3D [1 for _ in range(t)]\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0if eps =3D=3D -1:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0diag[-1]= =3D least_quadratic_nonresidue(p)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0diag =3D di= ag + [0 for _ in range(n - t)]\n =C2=A0 =C2=A0 =C2=A0 =C2=A0N =3D diagonal_= matrix(diag).change_ring(FiniteField(p))\n =C2=A0 =C2=A0 =C2=A0 =C2=A0retur= n _gen_gauss_sum_direct_way(N, p, r)\n\n\ndef _expt_sum(S, p, alpha, i):\n = =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0Return the exponent= ial sum in Miyawaki=E2=80=99s paper, where alpha[-1] <=3D 2, for T_i(p^2= ).\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 =C2=A0a, b, c =3D [a= lpha.count(_i) for _i in range(3)]\n =C2=A0 =C2=A0S33 =3D S.T.matrix_from_r= ows_and_columns(range(a + b, 3), range(a + b, 3))\n =C2=A0 =C2=A0S22 =3D S.= T.matrix_from_rows_and_columns(range(a, a + b), range(a, a + b))\n =C2=A0 = =C2=A0S32 =3D S.T.matrix_from_rows_and_columns(range(a + b, 3), range(a))\n= \n =C2=A0 =C2=A0if c > 0 and not _half_int_mat_is_div_by(S33, p ** 2):\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\n =C2=A0 =C2=A0if c > 0 and b > = 0 and any(x % p !=3D 0 for x in (S32 * ZZ(2)).change_ring(ZZ).list()):\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\n\n =C2=A0 =C2=A0if b =3D=3D 0 and a + = c =3D=3D 3 - i:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return p ** (c * (c + 1))\n = =C2=A0 =C2=A0elif b =3D=3D 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\n =C2= =A0 =C2=A0else:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return p ** (c * (c + 1)) * p = ** (b * c) * _generalized_gauss_sum(S22, p, b - i)\n\n\ndef _minkowski_redu= ction(b1, b2, b3, S):\n\n =C2=A0 =C2=A0def inner_prod(x, y):\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0return x * S * y\n\n =C2=A0 =C2=A0while True:\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0b1, b2, b3 =3D sorted([b1, b2, b3], key=3Dlambda b: b * S = * b)\n\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2 =3D _gaussian_reduction(b1, b2, = S)\n\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b11 =3D inner_prod(b1, b1)\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0b12 =3D inner_prod(b1, b2)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b1= 3 =3D inner_prod(b1, b3)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b22 =3D inner_prod(b2= , b2)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b23 =3D inner_prod(b2, b3)\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0b33 =3D inner_prod(b3, b3)\n\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= y1 =3D - (b13 / b11 - b12 * b23 / (b11 * b22)) / \\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0(1 - b12 ** 2 / (b11 * b22))\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0y2 =3D - (b23 / b22 - b12 * b13 / (b11 * b22)) / \\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0(1 - b12 ** 2 / (b11 * b22))\n\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0# Find integers x1, x2 so that norm(b3 + x2 * b2 + x1 * b1) is mi= nimal.\n =C2=A0 =C2=A0 =C2=A0 =C2=A0a_norms_alst =3D []\n\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0for x1 in [floor(y1), ceil(y1)]:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0for x2 in [floor(y2), ceil(y2)]:\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0a =3D b3 + x2 * b2 + x1 * b1\n =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0a_norms_alst.append((x1, x2, a, i= nner_prod(a, a)))\n =C2=A0 =C2=A0 =C2=A0 =C2=A0_inner_prod_a =3D min(x[-1] = for x in a_norms_alst)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0x1, x2, a, _ =3D next(x= for x in a_norms_alst if x[-1] =3D=3D _inner_prod_a)\n\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0if _inner_prod_a >=3D b33:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0# Change sings of b1, b2, b3 and terminate the alogrithm\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sngs =3D [sgn(b12), sgn(b13), sgn(b23)]\= n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bs =3D [b1, b2, b3]\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0try:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0# If b12, b13 or b23 is zero, change sgns of b1, b2, b3= so that\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# b12, b1= 3, b23 >=3D 0.\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= zero_i =3D sngs.index(0)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0set_ls =3D [set([1, 2]), set([1, 3]), set([2, 3])]\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0t =3D set_ls[zero_i]\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0_other =3D [x for x in [1, 2, = 3] if x not in t][0]\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0for x in t:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0i =3D set_ls.index(set([x, _other]))\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if sngs[i] < 0:\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0bs[x - 1] *=3D -1\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0b1, b2, b3 =3D bs\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0except Valu= eError:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Else cha= nge sgns so that b12, b13 > 0\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0if b12 < 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0b2 =3D -b2\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0if b13 < 0:\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b3 =3D -b3\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0return (b1, b2, b3)\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b3 =3D a\n\n\ndef _minkowski_reductio= n_transform_matrix(S):\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 = =C2=A0Return a unimodular matrix u such that u^t * S * u is reduced in Mink= owski=E2=80=99s sense.\n =C2=A0 =C2=A0=E2=80=99=E2=80=99=E2=80=99\n =C2=A0 = =C2=A0b1, b2, b3 =3D identity_matrix(QQ, 3).columns()\n =C2=A0 =C2=A0c1, c2= , c3 =3D _minkowski_reduction(b1, b2, b3, S)\n =C2=A0 =C2=A0return matrix([= c1, c2, c3]).transpose()\n","line":10,"column":0,&= quot;path":"/home/sho/work/sage_packages/e8theta_degree3/hecke_mo= dule.py"}}
Quit [2 times]

Load-path= shadows:
/home/sho/.emacs.d/elpa/helm-20160723.2238/helm-multi-m= atch hides /home/sho/.emacs.d/elpa/helm-core-20160723.944/helm-multi-match<= /div>
/home/sho/.emacs.d/elpa/scala-mode-20160519.731/ob-scala hides /u= sr/local/share/emacs/25.1/lisp/org/ob-scala
/home/sho/.emacs.d/el= pa/seq-2.16/seq hides /usr/local/share/emacs/25.1/lisp/emacs-lisp/seq
=

Features:
(shadow sort mail-extr emacsbug mes= sage dired rfc822 mml mml-sec epg
mm-decode mm-bodies mm-encode m= ailabbrev gmm-utils mailheader sendmail
mail-utils network-stream= nsm starttls url-cache url-http tls gnutls
mail-parse rfc2231 rf= c2047 rfc2045 ietf-drums url-gw url-auth
anaconda-mode pythonic f= url url-proxy url-privacy url-expand
url-methods url-history url= -cookie url-domsuf url-util mailcap vc-git
diff-mode easy-mmode p= ython tramp-sh tramp tramp-compat tramp-loaddefs
trampver shell p= complete format-spec comint ring ansi-color finder-inf
tex-site a= dvice edmacro kmacro gh-common gh-profile url-parse
auth-source g= nus-util mm-util help-fns mail-prsvr password-cache
url-vars s uc= s-normalize marshal eieio-compat cl-seq json map dash eieio
eieio= -core cl-macs go-mode-autoloads rx info package epg-config seq
by= te-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv
cl-loaddefs pcase cl-lib time-date mule-util japan-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe<= /div>
tabulated-list newcomment elisp-mode lisp-mode prog-mode register= page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock= font-lock
syntax facemenu font-core frame cl-generic cham georgi= an utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao kore= an japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech e= uropean ethiopic indian
cyrillic chinese charscript case-table ep= a-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded n= advice loaddefs button faces
cus-face macroexp files text-propert= ies overlay sha1 md5 base64 format
env code-pages mule custom wid= get hashtable-print-readable backquote
dbusbind inotify dynamic-s= etting system-font-setting font-render-setting
move-toolbar gtk x= -toolkit x multi-tty make-network-process emacs)

M= emory information:
((conses 16 279045 5729)
=C2=A0(symb= ols 48 28763 0)
=C2=A0(miscs 40 591 109)
=C2=A0(strings= 32 47201 10304)
=C2=A0(string-bytes 1 1610049)
=C2=A0(= vectors 16 45065)
=C2=A0(vector-slots 8 885950 3678)
= =C2=A0(floats 8 389 220)
=C2=A0(intervals 56 445 0)
=C2= =A0(buffers 976 23)
=C2=A0(heap 1024 44595 2062))

<= /div>
--001a1134e8d40621d60538ea3e48-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 31 10:31:46 2016 Received: (at 24117) by debbugs.gnu.org; 31 Jul 2016 14:31:46 +0000 Received: from localhost ([127.0.0.1]:52272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bTrmL-0002Jd-TN for submit@debbugs.gnu.org; Sun, 31 Jul 2016 10:31:46 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39158) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bTrmK-0002JO-L0 for 24117@debbugs.gnu.org; Sun, 31 Jul 2016 10:31:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bTrmC-00024u-8J for 24117@debbugs.gnu.org; Sun, 31 Jul 2016 10:31:39 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:58408) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bTrmC-00024q-58; Sun, 31 Jul 2016 10:31:36 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3434 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bTrmA-0007ni-9t; Sun, 31 Jul 2016 10:31:34 -0400 Date: Sun, 31 Jul 2016 17:31:33 +0300 Message-Id: <83d1ltq3p6.fsf@gnu.org> From: Eli Zaretskii To: Sho Takemori In-reply-to: (message from Sho Takemori on Sun, 31 Jul 2016 17:26:37 +0900) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.3 (------) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.3 (------) > From: Sho Takemori > Date: Sun, 31 Jul 2016 17:26:37 +0900 > > I got an error "error in process sentinel: url-http-create-request: Multibyte text in HTTP request" when I visited a > Python file which contains a multibyte character with `anaconda-eldoc-mode' turned on. That file name should have been encoded by the time it is passed to url-http.el, so the problem should not have happened, because encoded strings are unibyte strings. > At first, I thought this was a bug of anaconda-mode. So I opened an issue in github > (https://github.com/proofit404/anaconda-mode/issues/189). > > I guess `(= (string-bytes request) (length request))` in `url-http-create-request' should be `(= (string-bytes > url-http-data) (length url-http-data))`, because `(= (string-bytes request) (length request))` may be `nil' even if > `(= (string-bytes url-http-data) (length url-http-data))` is `t'. I don't think I agree in general: all the strings that are used by url-http-create-request should be unibyte strings. if they all are unibyte strings, then I think the situation you describe should not happen. However, you didn't provide enough details to analyze the situation, so perhaps I'm missing something. Could you please show all the details, specifically, what were the values of the various variables used by url-http-create-request to generate the request? For each value that is a string, please also tell whether it's a unibyte or a multibyte string. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 31 19:21:51 2016 Received: (at 24117) by debbugs.gnu.org; 31 Jul 2016 23:21:51 +0000 Received: from localhost ([127.0.0.1]:52416 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bU03J-0006T0-PZ for submit@debbugs.gnu.org; Sun, 31 Jul 2016 19:21:51 -0400 Received: from mail-oi0-f54.google.com ([209.85.218.54]:34709) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bU03G-0006Sl-7T for 24117@debbugs.gnu.org; Sun, 31 Jul 2016 19:21:47 -0400 Received: by mail-oi0-f54.google.com with SMTP id l65so171742115oib.1 for <24117@debbugs.gnu.org>; Sun, 31 Jul 2016 16:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=D9pogyrhVeawbnOAPOuQZpiP3rCU4kK9VRUZXGCAKbA=; b=seEIZzeeWYShR+4Bn9Wh4zWTZCT/x0QafISzN+o3g+LNDxakfynbIZ0VZsQguQNtIo +quXpBHVvInJf3OA3fK8hu4c+OyMixqZvVMJGVBgTkzN50wJ9ZGY2mXXjr95Ym3MhdhP CC0aiBVXTIOghgW2EDeaELndWRdBPJBMiGQzFMicScq1ZVXMrFXROfpSefOQIJkl204Q 5cIqxt9KIR55gqhYNo59Ga+ijVq4QpWYyy8Sqc6YB4I+uGkjXhNZ4ZQ5aoPr0qSsB2ux cXyfXMtnBNc9cV8TSjeXdC245z+I4GuBsRUehiDCPAOIlFCpSvQqOzO9+sMMp/x2Tlbz V/Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=D9pogyrhVeawbnOAPOuQZpiP3rCU4kK9VRUZXGCAKbA=; b=dbCrsXS54Sdq8j41HqOSZ2wDriL8IemSu6XFvzxw4XZa/4VkQEXlTmGCzq9Y1S5kL3 XEGrSchyIth0aiLNHV7IAYi2ghJ1CgduocAMS+R/JO9oijAVv1WvbdmltMiACYnOH01x S+Wj1MLPFoO49ud+5cEAhT1AVFcviT+o54ZeU5dmDagx5FMgL2PIRqENjYpt4Bd9XNwL ajo9eaLOmyGXGkrwvfCV3MSxtSWJhLFX+CBUj77djh2w2sSItisWpvQ/mxYsOJdfQdTh W1Bfx+Fl5CBhbZiNQJamRYGjdSQz8e7nce89kYKkUO7O0yooKqWmhddt1JN8lalNqjSx b+mg== X-Gm-Message-State: AEkoouuMey0lDTmBYY24whLnBgreZIKKW/f4w5lvlBJ0PVOizPBaNf3WvfFTZ+k3CbINFGHbVLmg9oykEfrPZA== X-Received: by 10.157.9.6 with SMTP id 6mr33487197otp.132.1470007300215; Sun, 31 Jul 2016 16:21:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.33.77 with HTTP; Sun, 31 Jul 2016 16:21:39 -0700 (PDT) In-Reply-To: <83d1ltq3p6.fsf@gnu.org> References: <83d1ltq3p6.fsf@gnu.org> From: Sho Takemori Date: Mon, 1 Aug 2016 08:21:39 +0900 Message-ID: Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii Content-Type: multipart/alternative; boundary=94eb2c0481aef25b020538f6be7f X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --94eb2c0481aef25b020538f6be7f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable It seems that anaconda-mode use two global variables (url-request-method and url-request-data) to make generate the request. https://github.com/proofit404/anaconda-mode/blob/master/anaconda-mode.el#L3= 49 url-request-method is bound to an ASCII string "POST". In my situation, url-request-data is bound to a unibyte string as below. "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"eldoc\",\"params\":{\"source\":= \"# -*- coding: utf-8 -*-\\nimport itertools\\nfrom itertools import groupby\\n\\nfrom sage.all import mul\\nfrom sage.arith.all import kronecker_symbol\\nfrom sage.functions.all import ceil, floor, sgn\\nfrom sage.matrix.all import (block_diagonal_matrix, block_matrix,\\n diagonal_matrix, identity_matrix, matrix)\\nfrom sage.misc.all import cached_function\\nfrom sage.quadratic_forms.all import QuadraticForm, least_quadratic_nonresidue\\nfrom sage.rings.all import QQ, ZZ, CyclotomicField, FiniteField, PolynomialRing\\n\\n\\ndef _index_of_gamma_0_gl_n(alpha, p):\\n '''\\n Returns delta(a1, ..., an) defined in Shimura, Euler products and Eisenstein\\n series, pp 118, (15.1.7).\\n '''\\n if p in ZZ:\\n p =3D ZZ(p)\\n\\n def _bn(n):\\n return mul(1 - p ** (-i) for i in xrange(1, n + 1))\\n\\n e_r_ls =3D [(k, len(list(v)))\\n for k, v in groupby(sorted(alpha), lambda x: x)]\\n res =3D _bn(len(alpha)) / mul(_bn(r) for _, r in e_r_ls)\\n for i, (ei, ri) in enumerate(e_r_ls):\\n for j, (ej, rj) in enumerate(e_r_ls):\\n if i < j:\\n res *=3D p ** ((ej - ei) * ri * rj)\\n return res\\n\\n\\ndef _gl2_coset_gamma0(a, p):\\n w =3D matrix([[0, -1],\\n [1, 0]])\\n for m12 in range(p ** a):\\n yield matrix([[1, m12],\\n [0, 1]])\\n for m21 in range(p ** (a - 1)):\\n m =3D matrix([[1, 0],\\n = [p * m21, 1]])\\n yield w * m\\n\\n\\ndef _gl3_coset_gamma0(alpha, p):\\n r'''\\n Let alpha =3D [a0, a1, a2] with a0 <=3D a1 <=3D a2,\\n= g =3D diag([p^a0, p^a1, p^a2]), and Gamma0 =3D g^(-1) GL3(Z) g \342\210\247 GL3(Z).\\n Return a complete set Gamma0 \\\\ GL3(Z).\\n '''\\n if p in ZZ:\\n p =3D ZZ(p)\\n a0, a1, a2 =3D alpha\\n if a0 < a1 = < a2:\\n return list(__gl3_coset_gamma0_distinct(a0, a1, a2, p))\\n elif a0 =3D=3D a1 and a1 < a2:\\n return list(__gl3_coset_gamma0_2_1(a0, a2, p))\\n elif a0 < a1 and a1 =3D=3D a2= :\\n return list(__gl3_coset_gamma0_1_2(a0, a2, p))\\n elif a0 =3D=3D = a1 =3D=3D a2:\\n return [identity_matrix(ZZ, 3)]\\n else:\\n raise ValueError\\n\\n\\ndef __gl3_coset_gamma0_2_1(a1, a3, p):\\n w23 =3D matrix([[1, 0, 0],\\n [0, 0, 1],\\n [= 0, 1, 0]])\\n for m13 in range(p ** (a3 - a1 - 1)):\\n for m23 in range(p ** (a3 - a1 - 1)):\\n m =3D matrix([[1, 0, p * m13],\\n [0, 1, p * m23],\\n [0, 0, 1]])\\n yield m\\n\\n for m32 in range(p ** (a3 - a1)):\\n m =3D matrix([[1, 0, 0],\\n [0, 1, 0],\\n [0, m32, 1]])\\n for g in _gl2_coset_gamma0(a3 - a1, p):\\n n =3D block_diagonal_matrix(g, matrix([[1]]))\\n yield w23 * m * n\\n\\n\\ndef __gl3_coset_gamma0_1_2(a1, a2, p):\\n w12 =3D matrix([[0, 1, 0],\\n [1, 0, 0],\\n [0, 0, 1]])\\n\\n for m12 in range(p ** (a2 - a1 - 1)):\\n for m13 in range(p ** (a2 - a1 - 1)):\\n m =3D matrix([[1, p * m12, p * m13],\\n [0, 1, 0],\\n [0, 0, 1]])\\n yield m\\n for m21 in range(p ** (a2 - a1)):\\n m =3D matrix([[1, 0, 0],\\n [m21, 1, 0],\\n [0, 0, 1]])\\n for g in _gl2_coset_gamma0(a2 - a1, p):\\n n =3D block_diagonal_matrix(matrix([[1]]), g)\\n yield w12 = * m * n\\n\\n\\ndef __gl3_coset_gamma0_distinct(a1, a2, a3, p):\\n\\n w12 =3D matrix([[0, 1, 0],\\n [1, 0, 0],\\n [= 0, 0, 1]])\\n\\n w23 =3D matrix([[1, 0, 0],\\n [0, 0, 1],\= \n [0, 1, 0]])\\n\\n w13 =3D matrix([[0, 0, 1],\\n [0, 1, 0],\\n [1, 0, 0]])\\n\\n w123 =3D matrix([[0, 1, 0],\\n [0, 0, 1],\\n [1, 0, 0]])\\n\\n w132 =3D matrix([[0, 0, 1],\\n [1, 0, 0],\\n [0, 1, 0]])\\n\\n # w =3D 1\\n for m12 in range(p ** (a2 - a1 - 1)):\\n for m13 in range(p ** (a3 - a1 - 1)):\\n for m23 in range(p ** (a3 - a2 - 1)):\\n yield matrix([[1, p * m12, p * m13],\\n [0, 1, p * m23],\\n [0, 0, 1]])\\n # w =3D (12)\\n for m13 in range(p ** (a3 - a2 - 1)):\\n for m21 in range(p ** (a2 - a1)):\\n for m23 in range(p ** (a3 - a1 - 1)):\\n m =3D matrix([[1, 0, p * m13],\\n [m21, 1, p * m23],\\n [0, 0, 1]])\\n yield w12 * m\\n # w =3D (23)\\n for m12 in range(p ** (a3= - a1 - 1)):\\n for m13 in range(p ** (a2 - a1 - 1)):\\n for m32 in range(p ** (a3 - a2)):\\n m =3D matrix([[1, p * m12, = p * m13],\\n [0, 1, 0],\\n [0, m32, 1]])\\n yield w23 * m\\n\\n # w =3D (13)\\n for m21 in range(p ** (a3 - a2)):\\n for m31 in range(p ** (a3 - a1)):\\n for m32 in range(p ** (a2 - a1)):\\n m = =3D matrix([[1, 0, 0],\\n [m21, 1, 0],\\n [m31, m32, 1]])\\n yield w13 * m\\n\\n # w =3D (123)\\n for m21 in range(p ** (a3 - a1)):\\n for m23 in range(p ** (a2 - a1 - 1)):\\n for m31 in range(p ** (a3 - a2)):\\n m =3D matrix([[1, 0, 0],\\n [m21, 1, p * m23],\\n [m31, 0, 1]])\\n yield w123 * m\\n # w =3D (132)\\n for m12 in range(p ** (a3 - a2 - 1)):\\n for m31 in range(p ** (a2 - a1)):\\n for m32 in range(p ** (a3 - a1)):\\n m =3D matrix([[1, p * m12, 0],\= \n [0, 1, 0],\\n [m31, m32, 1]])\\n yield w132 * m\\n\\n\\nclass HalfIntMatElement(object):\\n\\n def __init__(self, T):\\n '''\\n :params T: half integral matrix of size 3 or a list\\n '''\\n if isinstance(T, list):\\n a, b, c, d, e, f =3D [ZZ(x) fo= r x in T]\\n mat =3D matrix([[a, f / 2, e / 2],\\n [f / 2, b, d / 2],\\n [e / 2, d / 2, c]])\\n else:\\n mat =3D T\\n self.__entries =3D tuple(mat.list())\\n\\n def __eq__(self, other):\\n if isinstance(other, HalfIntMatElement):\\n return self.__entries =3D=3D other.__entries\\n else:\\n raise NotImplementedError\\n\\n def __repr__(self):\\n return self.T.__repr__()\\n\\n def __hash__(self):\\n return hash(self.__entries)\\n\\n @property\\n def T(self):\\n return matrix(3, self.__entries)\\n\\n def right_action(self, g):\\n '''\\n :param g: matrix of size n\\n return self[g] (Siegel's notation)\\n '''\\n S =3D g.transpose() * self.T * g\\n return HalfIntMatElement(S)\\n\\n def satisfy_cong_condition_tp(self, p, alpha):\\n '''\\n Test if sum_{B mod D} exp(2pi T B D^(-1)) is zero, where D =3D diag(p^a1, p^a2, a^a3),\\n a1, a2, a3 =3D alpha.\\n '''\\n return (all(ZZ(self.T[i, i]) % p ** alpha[i] =3D=3D 0 for i in range(3)) and\\n all(ZZ(self.T[i, j] * 2) % p ** alpha[i] =3D=3D 0\\n for i in range(3) for j in range(i + 1, 3)))\\n\\n def is_divisible_by(self, m):\\n '''\\n Test if self is divisible by m\\n :param m: integer\\n '''\\n return _half_int_mat_is_div_by(self.T, m)\\n\\n def __floordiv__(self, other):\\n S =3D matrix(QQ, 3)\\n for i in range(3):\\n S[i, i] =3D ZZ(self.T[i, i]) // other\\n for i in range(3):\\n for j in range(i + 1, 3):\\n S[i, j] =3D S[j, i] =3D (ZZ(self.T[i, j] * 2) // other) / 2\\n return HalfIntMatElement(S)\\n\\n\\ndef alpha_list(dl):\\n '''\\n Return a list of (a0, a1, a2) with 0 <=3D a0 <=3D a1 <=3D a2 <=3D dl\\n '''\\n = return [(a0, a1, a2) for a0 in range(dl + 1)\\n for a1 in range(a0, dl + 1) for a2 in range(a1, dl + 1)]\\n\\n\\ndef tp_action_fourier_coeff(p, T, F):\\n '''\\n Return the Tth Fourier coefficient of F|T(p), where F is a modular form.\\n :param p: a prime number\\n :param T: a half integral matrix or an instance of HalfIntMatElement\\n :param F: a dictionary or a Siegel modular form of degree 3\\n '''\\n p =3D ZZ(p)\\n return _action_fc_base(tp_action_fc_alist(p, T), F, T)\\n\\n\\ndef tp2_action_fourier_coeff(p, i, T, F):\\n '''\\n Similar to tp_action_fourier_coeff for T_i(p^2).\\n '''\\n p =3D ZZ(p)\\n return _action_fc_base(tp2_action_fc_alist(p, T, i), F, T)\\n\\n\\ndef _action_fc_base(ls, F, T):\\n if not isinstance(T, HalfIntMatElement):\\n T =3D HalfIntMatElement(T)\\n res =3D 0\\n for s, a, g in ls:\\n res =3D a * F[s].left_action(g) + res\\n return res\\n\\n\\ndef hecke_eigenvalue_tp(p, F, T=3DNone):\\n '''\\n p, F, T: same as aruments of tp_action_fourier_coeff.\\n Assuming F is an eigenform, return the eigenvalue for T(p),\\n T is used for the computation of Fourier coefficients.\\n If T is omitted, T will be set to\\n matrix([[1, 1/2, 1/2], [1/2, 1, 1/2], [1/2, 1/2, 1]]).\\n '''\\n return _hecke_eigenvalue_base(lambda s: tp_action_fourier_coeff(p, s, F), F, T=3DT)\\n\\n\\ndef hecke_eigenvalue_tp2(p, i, F, T=3DNone):\\n '''\\n Similar to hecke_eigenvalue_tp for T(p^2).\\n '''\\n return _hecke_eigenvalue_base(lambda s: tp2_action_fourier_coeff(p, i, s, F), F, T=3DT)\\n\\n\\ndef spinor_l_euler_factor(p, F, t=3DNone, T=3DNone):\\n '= ''\\n F: a dict or Siegel modular form of degree 3.\\n Return a polynomial G(t) of degree 8, s.t.\\n G(p^(-s))^(-1) is the p-Euler factor of the spinor L function of F.\\n '''\\n p =3D ZZ(p)\\n if t is None:\\n t =3D PolynomialRing(QQ, 1, names=3D't', order=3D\\\"neglex\\\").gens()[0]\\n c =3D {}\\n tp =3D hecke_eigenvalue_tp(p, F, T=3DT)\\n tpp1, tpp2, tpp3 =3D [hecke_eigenvalue_tp2(p, i, F, T=3DT) for i in [1, 2, 3]]\\n c[0] =3D ZZ(1)\\n c[1] =3D tp\\n c[2] =3D p * (tpp1 + (p**2 + 1) * tpp2 + (p**= 2 + 1)**2 * tpp3)\\n c[3] =3D p**3 * tp * (tpp2 + tpp3)\\n c[4] =3D p**6 = * (tp**2 * tpp3 + tpp2**2 - 2 * p * tpp1 * tpp3 -\\n 2 * (p - 1) * tpp2 * tpp3 -\\n (p**6 + 2 * p**5 + 2 * p**3 + 2 * p - 1) * tpp3**2)\\n c[5] =3D p**6 * tpp3 * c[3]\\n c[6] =3D p**12 * = tpp3 ** 2 * c[2]\\n c[7] =3D p**18 * tpp3 ** 3 * c[1]\\n c[8] =3D p**24 * = tpp3 ** 4\\n return sum((-1)**k * v * t**k for k, v in c.items())\\n\\n\\ndef rankin_convolution_degree1(f, g, p, name=3DNone):\\n u'''\\n f, g: primitive forms of degree 1 and level 1.\\n Return p-euler factor of the Rankin convolution of f and g as\\n a polynomial.\\n '''\\n k1 =3D f.weight()\\n k2 =3D g.weight()\\n ap =3D f[p]\\n bp =3D g[p]\\n = t =3D PolynomialRing(QQ, 1, names=3D't' if name is None else name,\\n order=3D\\\"neglex\\\").gens()[0]\\n return (1 - ap * bp * t +\\= n (ap**2 * p**(k2 - 1) + bp**2 * p**(k1 - 1) - 2 * p**(k1 + k2 - 2)) * t**2 -\\n ap * bp * p**(k1 + k2 - 2) * t**3 + p**(2 * (k1 + k2 - 2)) * t**4)\\n\\n\\ndef _hecke_eigenvalue_base(fc_func, F, T=3DNone):\\n if T is None:\\n T =3D HalfIntMatElement(matrix([[ZZ(1), ZZ(1) / ZZ(2), ZZ(1) / ZZ(2)],\\n [ZZ(1) / ZZ(2), ZZ(1), ZZ(1) / ZZ(2)],\\n [ZZ(1) / ZZ(2), ZZ(1) / ZZ(2), ZZ(1)]]))\\n if not isinstance(T, HalfIntMatElement):\\n T =3D HalfIntMatElement(T)\\n v1 =3D fc_func(T).vector\\n v =3D F[T].vector= \\n if v =3D=3D 0:\\n raise ZeroDivisionError\\n else:\\n i= =3D next(i for i in range(len(v)) if v[i] !=3D 0)\\n return v1[i] / v[i]\\n\\n\\n@cached_function\\ndef tp_action_fc_alist(p, T):\\n '''\\n return a list of tuples (S, a, g) s.t.\\n S: an instance of HalfIntMatElement\\n a: integer\\n g: 3 by 3 matrix s.t.\\n F|T(p) =3D sum(a rho(g) F[S] | (a, g, S)).\\n '''\\n res1 =3D []\\n for a= lpha in alpha_list(1):\\n D =3D diagonal_matrix([p ** a for a in alpha])\= \n for V in _gl3_coset_gamma0(alpha, p):\\n M =3D D * V\\n S =3D T.right_action(M.transpose())\\n if S.is_divisible_by(p):\\n S =3D S // p\\n if S.satisfy_cong_condition_tp(p, alpha):\\n # p**(-6) and p in the third item are for normalization.\\n res1.append(\\n (S, p ** (-6) * mul(p ** alpha[i] for i in range(3) for j in range(i, 3)),\\n M ** (-1) * p))\\n return __convert_reduced_nonisom_matrices(res1)\\n\\n\\ndef __convert_reduced_nonisom_matrices(alst):\\n red_res =3D []\\n for s, a, g in alst:\\n u =3D _minkowski_reduction_transform_matrix(s.T)\\n t =3D s.right_action(u)\\n red_res.append((t, a, g * u.transpose() ** (-1)))\\n\\n non_isoms =3D []\\n\\n for s, a, g in red_res:\\n q =3D QuadraticForm(ZZ, 2 * s.T)\\n u =3D None\\n for t, _, _ in non_isoms:\\n q1 =3D QuadraticForm(ZZ, 2 * t.T)\\n if q.det() =3D=3D q1.det():\\n u =3D q.is_globally_equivalent_to(q1, return_matrix=3DTrue)\\n if = u and u.transpose() * q.Gram_matrix_rational() * u =3D=3D q1.Gram_matrix_rational():\\n break\\n if u:\\n non_isoms.append((s.right_action(u), a, g * u.transpose() ** (-1)))\\n else:\\n non_isoms.append((s, a, g))\\n return non_isoms\\n\\n\\n@cached_function\\ndef tp2_action_fc_alist(p, T, i):\\n '''\\n similar to tp_action_fc_alist for T_i(p^2) for i =3D 0, 1, 2, 3.\\n '''\\n res1 =3D []\\n\\n for alpha in alpha_list(2):\\= n D =3D diagonal_matrix([p ** a for a in alpha])\\n for V in _gl3_coset_gamma0(alpha, p):\\n M =3D D * V\\n S =3D T.right_action(M.transpose())\\n if S.is_divisible_by(p ** 2):\\n S =3D S // (p ** 2)\\n res1.append((S,= p ** (-12) * _expt_sum(S, p, alpha, i),\\n M ** (-1) * p ** 2))\\n\\n return __convert_reduced_nonisom_matrices([(a, b, c) for a, b, c in res1 if b !=3D 0])\\n\\n\\ndef _nearest_integer(x):\\n = r =3D floor(x)\\n if x - r > 0.5:\\n return r + 1\\n else:\\n return r\\n\\n\\ndef _gaussian_reduction(b1, b2, S):\\n '''\\n b1, b2: vectors of length 3\\n S: symmetric matrix of size 3\\n '''\\n while True:\\n nb1 =3D b1 * S * b1\\n nb2 =3D b2 * S * b2\\n if nb2 < nb1:\\n b1, b2 =3D b2, b1\\n x =3D (b2 * S * = b1) / (b1 * S * b1)\\n r =3D _nearest_integer(x)\\n a =3D b2 - r = * b1\\n if a * S * a >=3D b2 * S * b2:\\n return (b1, b2)\\= n else:\\n b1, b2 =3D a, b1\\n\\n\\ndef _sym_mat_gen(p, n):\\n if n =3D=3D 1:\\n for a in range(p):\\n yield matrix([[a]])\\n else:\\n for s in _sym_mat_gen(p, n - 1):\\n ls =3D [range(p) for _ in range(n)]\\n for a in itertools.product(*ls):\\n v =3D matrix([a[:-1]])\\n yield block_matrix([[s, v.transpose()], [v, matrix([[a[-1]]])]])\\n\\n\\ndef _gen_gauss_sum_direct_way(N, p, r):\\n res =3D 0\\n K =3D CyclotomicField(p)\\n zeta =3D K.gen()\\n for = S in _sym_mat_gen(p, N.ncols()):\\n if S.change_ring(FiniteField(p)).rank() =3D=3D r:\\n res +=3D zeta = ** ((N * S).trace())\\n try:\\n return QQ(res)\\n except TypeError:\\n return res\\n\\n\\ndef _generalized_gauss_sum(N, p, r):\\n if r =3D=3D 0:\\n return 1\\n if p =3D=3D 2:\\n = return _gen_gauss_sum_direct_way(N, p, r)\\n else:\\n N_mp =3D N.change_ring(FiniteField(p))\\n d, _, v =3D N_mp.smith_form()\\n t =3D d.rank()\\n N1 =3D (v.transpose() * N_mp *\\n v).matrix_from_rows_and_columns(range(t), range(t))\\n eps =3D kronecker_symbol(N1.det(), p)\\n return _gen_gauss_sum_non_dyadic(p, eps, N.ncols(), t, r)\\n\\n\\ndef _half_int_mat_is_div_by(S, m):\\n n = =3D S.ncols()\\n return (all(ZZ(S[i, i]) % m =3D=3D 0 for i in range(n)) and= \\n all(ZZ(2 * S[i, j]) % m =3D=3D 0 for i in range(n) for j in rang= e(i + 1, n)))\\n\\n\\n@cached_function\\ndef _gen_gauss_sum_non_dyadic(p, eps, n, t, r):\\n '''\\n cf. H. Saito, a generalization of Gauss sums\\n '''\\n\\n def parenthesis_prod(a, b, m):\\n if m =3D=3D 0:\\n return 1\\n else:\\n return mul(1 - a * b ** i for i in range(m))\\n\\n if (n - t) % 2 =3D=3D 0:\\n m =3D (n - t) // 2= \\n else:\\n m =3D (n - t + 1) // 2\\n\\n if n =3D=3D r:\\n i= f n % 2 =3D=3D 1:\\n return ((-1) ** ((n - 2 * m + 1) // 2) * p ** ((n= ** 2 + (2 * m) ** 2 - 1) // 4) *\\n parenthesis_prod(p ** (-1), p ** (-2), m))\\n elif n % 2 =3D=3D t % 2 =3D=3D 0:\\n return ((-kronecker_symbol(-1, p)) ** ((n - 2 * m) // 2) *\\n eps * p ** ((n ** 2 + (2 * m + 1) ** 2 - 1) // 4) *\\n parenthesis_prod(p ** (-1), p ** (-2), m))\\n else:\\n return 0\\n else:\\n diag =3D [1 for _ in range(t)]\\n = if eps =3D=3D -1:\\n diag[-1] =3D least_quadratic_nonresidue(p)\\n diag =3D diag + [0 for _ in range(n - t)]\\n N =3D diagonal_matrix(diag).change_ring(FiniteField(p))\\n return _gen_gauss_sum_direct_way(N, p, r)\\n\\n\\ndef _expt_sum(S, p, alpha, i):\\n '''\\n Return the exponential sum in Miyawaki's paper, where alpha[-1] <=3D 2, for T_i(p^2).\\n '''\\n a, b, c =3D [alpha.count(_i= ) for _i in range(3)]\\n S33 =3D S.T.matrix_from_rows_and_columns(range(a = + b, 3), range(a + b, 3))\\n S22 =3D S.T.matrix_from_rows_and_columns(range(a, a + b), range(a, a + b))\\n S32 =3D S.T.matrix_from_rows_and_columns(range(a + b, 3), range(a))\\n\\n if c > 0 and not _half_int_mat_is_div_by(S33, p ** 2):\\n return 0\\n if c > 0 and b > 0 and any(x % p !=3D 0 for x in (S32 * ZZ(2)).change_ring(ZZ).list()):\\n return 0\\n\\n if b =3D=3D 0 a= nd a + c =3D=3D 3 - i:\\n return p ** (c * (c + 1))\\n elif b =3D=3D 0= :\\n return 0\\n else:\\n return p ** (c * (c + 1)) * p ** (b * c) * _generalized_gauss_sum(S22, p, b - i)\\n\\n\\ndef _minkowski_reduction(b1, b2, b3, S):\\n\\n def inner_prod(x, y):\\n return x * S * y\\n\\n while True:\\n b1, b2, b3 =3D sorted([b= 1, b2, b3], key=3Dlambda b: b * S * b)\\n\\n b1, b2 =3D _gaussian_reduction(b1, b2, S)\\n\\n b11 =3D inner_prod(b1, b1)\\n b12 =3D inner_prod(b1, b2)\\n b13 =3D inner_prod(b1, b3)\\n b22 =3D inner_prod(b2, b2)\\n b23 =3D inner_prod(b2, b3)\\n = b33 =3D inner_prod(b3, b3)\\n\\n y1 =3D - (b13 / b11 - b12 * b23 / (b11 = * b22)) / \\\\\\n (1 - b12 ** 2 / (b11 * b22))\\n y2 =3D - (b23 / b22 - b12 * b13 / (b11 * b22)) / \\\\\\n (1 - b12 ** 2 / (b11 * b22))\\n\\n # Find integers x1, x2 so that norm(b3 + x2 * b2 + x1 * b1) is minimal.\\n a_norms_alst =3D []\\n\\n for x1 in [floor(y1), ceil(y1)]:\\n for x2 in [floor(y2), ceil(y2)]:\\n a =3D b3 + x2 * b2 + x1 * b1\\n a_norms_alst.append((x1, x2, a, inner_prod(a, a)))\\n _inner_prod_a =3D min(x[-1] for x in a_norms_alst)\\n x1, x2, a, _ =3D next(x for = x in a_norms_alst if x[-1] =3D=3D _inner_prod_a)\\n\\n if _inner_prod_a >= =3D b33:\\n # Change sings of b1, b2, b3 and terminate the alogrithm\\n sngs =3D [sgn(b12), sgn(b13), sgn(b23)]\\n bs =3D [b1, b2, b3]\\n try:\\n # If b12, b13 or b23 is zero, change sgns of b1, b2, b3 so that\\n # b12, b13, b23 >=3D 0.\\n zero_i =3D sngs.index(0)\\n set_ls =3D [set([1, 2]), set([1, 3]), set([2, 3])]\\n t =3D set_ls[zero_i]\\n _other =3D [x for x in [1, 2, 3] if x not = in t][0]\\n for x in t:\\n i =3D set_ls.index(set([x, _other]))\\n if sngs[i] < 0:\\n bs[x - 1] *=3D -1\\n b1, b2, b3 =3D bs\\n except ValueError:\\n # Else change sgns so that b12, b13 > 0\\n if b12 < 0:\\n b2 =3D -b2\\n if b13 < 0:\\n b3 =3D -b3\\n retur= n (b1, b2, b3)\\n else:\\n b3 =3D a\\n\\n\\ndef _minkowski_reduction_transform_matrix(S):\\n '''\\n Return a unimodular matrix u such that u^t * S * u is reduced in Minkowski's sense.\\n '''\\n b1, b2, b3 =3D identity_matrix(QQ, 3).columns()\\n c1, c2, c3 =3D _minkowski_reduction(b1, b2, b3, S)\\n return matrix([c1= , c2, c3]).transpose()\\n\",\"line\":52,\"column\":41,\"path\":\"/home/sho/work/s= age_packages/e8theta_degree3/hecke_module.py\"}}" The file contains a multibyte string "=E2=88=A7" and anaconda-mode converts= it to "\342\210\247". Sho Takemori 2016-07-31 23:31 GMT+09:00 Eli Zaretskii : > > From: Sho Takemori > > Date: Sun, 31 Jul 2016 17:26:37 +0900 > > > > I got an error "error in process sentinel: url-http-create-request: > Multibyte text in HTTP request" when I visited a > > Python file which contains a multibyte character with > `anaconda-eldoc-mode' turned on. > > That file name should have been encoded by the time it is passed to > url-http.el, so the problem should not have happened, because encoded > strings are unibyte strings. > > > At first, I thought this was a bug of anaconda-mode. So I opened an > issue in github > > (https://github.com/proofit404/anaconda-mode/issues/189). > > > > I guess `(=3D (string-bytes request) (length request))` in > `url-http-create-request' should be `(=3D (string-bytes > > url-http-data) (length url-http-data))`, because `(=3D (string-bytes > request) (length request))` may be `nil' even if > > `(=3D (string-bytes url-http-data) (length url-http-data))` is `t'. > > I don't think I agree in general: all the strings that are used by > url-http-create-request should be unibyte strings. if they all are > unibyte strings, then I think the situation you describe should not > happen. However, you didn't provide enough details to analyze the > situation, so perhaps I'm missing something. Could you please show > all the details, specifically, what were the values of the various > variables used by url-http-create-request to generate the request? > For each value that is a string, please also tell whether it's a > unibyte or a multibyte string. > > Thanks. > --94eb2c0481aef25b020538f6be7f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
It seems that anaconda-mode use two global variables = (url-request-method and url-request-data)
to make generate the re= quest.


<= div>url-request-method is bound to an ASCII string "POST".
<= div>In my situation, url-request-data is bound to a unibyte string as below= .

"{\"jsonrpc\":\"2.0\",\= "id\":1,\"method\":\"eldoc\",\"params\&q= uot;:{\"source\":\"# -*- coding: utf-8 -*-\\nimport itertool= s\\nfrom itertools import groupby\\n\\nfrom sage.all import mul\\nfrom sage= .arith.all import kronecker_symbol\\nfrom sage.functions.all import ceil, f= loor, sgn\\nfrom sage.matrix.all import (block_diagonal_matrix, block_matri= x,\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 diagonal_matrix, identity_matrix, matrix)\\nfr= om sage.misc.all import cached_function\\nfrom sage.quadratic_forms.all imp= ort QuadraticForm, least_quadratic_nonresidue\\nfrom sage.rings.all import = QQ, ZZ, CyclotomicField, FiniteField, PolynomialRing\\n\\n\\ndef _index_of_= gamma_0_gl_n(alpha, p):\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Ret= urns delta(a1, ..., an) defined in Shimura, Euler products and Eisenstein\\= n =C2=A0 =C2=A0series, pp 118, (15.1.7).\\n =C2=A0 =C2=A0'''\\n= =C2=A0 =C2=A0if p in ZZ:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0p =3D ZZ(p)\\n\\n = =C2=A0 =C2=A0def _bn(n):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return mul(1 - p ** = (-i) for i in xrange(1, n + 1))\\n\\n =C2=A0 =C2=A0e_r_ls =3D [(k, len(list= (v)))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for k, v in groupb= y(sorted(alpha), lambda x: x)]\\n =C2=A0 =C2=A0res =3D _bn(len(alpha)) / mu= l(_bn(r) for _, r in e_r_ls)\\n =C2=A0 =C2=A0for i, (ei, ri) in enumerate(e= _r_ls):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for j, (ej, rj) in enumerate(e_r_ls):= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if i < j:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res *=3D p ** ((ej - ei) * ri * rj= )\\n =C2=A0 =C2=A0return res\\n\\n\\ndef _gl2_coset_gamma0(a, p):\\n =C2=A0= =C2=A0w =3D matrix([[0, -1],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0[1, 0]])\\n =C2=A0 =C2=A0for m12 in range(p ** a):\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0yield matrix([[1, m12],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1]])\\n =C2=A0 =C2=A0fo= r m21 in range(p ** (a - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[= 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0[p * m21, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w * m\\n\\n\\ndef = _gl3_coset_gamma0(alpha, p):\\n =C2=A0 =C2=A0r'''\\n =C2=A0 =C2= =A0Let alpha =3D [a0, a1, a2] with a0 <=3D a1 <=3D a2,\\n =C2=A0 =C2= =A0g =3D diag([p^a0, p^a1, p^a2]), and Gamma0 =3D g^(-1) GL3(Z) g \342\210\= 247 GL3(Z).\\n =C2=A0 =C2=A0Return a complete set Gamma0 \\\\ GL3(Z).\\n = =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0if p in ZZ:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0p =3D ZZ(p)\\n =C2=A0 =C2=A0a0, a1, a2 =3D alpha\\n =C2=A0 =C2= =A0if a0 < a1 < a2:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return list(__gl3_c= oset_gamma0_distinct(a0, a1, a2, p))\\n =C2=A0 =C2=A0elif a0 =3D=3D a1 and = a1 < a2:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return list(__gl3_coset_gamma0_2_= 1(a0, a2, p))\\n =C2=A0 =C2=A0elif a0 < a1 and a1 =3D=3D a2:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return list(__gl3_coset_gamma0_1_2(a0, a2, p))\\n =C2= =A0 =C2=A0elif a0 =3D=3D a1 =3D=3D a2:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return= [identity_matrix(ZZ, 3)]\\n =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0raise ValueError\\n\\n\\ndef __gl3_coset_gamma0_2_1(a1, a3, p):\\n =C2= =A0 =C2=A0w23 =3D matrix([[1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0]])\\n =C2=A0 =C2=A0for m13 in range(p = ** (a3 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m23 in range(p ** (a3 = - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0= , p * m13],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[0, 1, p * m23],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield m\\n\\n =C2=A0 =C2=A0for m32 in ran= ge(p ** (a3 - a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, 0],\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0,= 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0[0, m32, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for g in _gl2_coset_gamma= 0(a3 - a1, p):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0n =3D block_diag= onal_matrix(g, matrix([[1]]))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0y= ield w23 * m * n\\n\\n\\ndef __gl3_coset_gamma0_1_2(a1, a2, p):\\n =C2=A0 = =C2=A0w12 =3D matrix([[0, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\\n\\n =C2=A0 =C2=A0for m12 in range(= p ** (a2 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m13 in range(p ** (a= 2 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1,= p * m12, p * m13],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield m\\n =C2=A0 =C2=A0for m21 in ra= nge(p ** (a2 - a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, 0],= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m= 21, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0[0, 0, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for g in _gl2_coset_gam= ma0(a2 - a1, p):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0n =3D block_di= agonal_matrix(matrix([[1]]), g)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0yield w12 * m * n\\n\\n\\ndef __gl3_coset_gamma0_distinct(a1, a2, a3, p)= :\\n\\n =C2=A0 =C2=A0w12 =3D matrix([[0, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\\n\\n =C2=A0 =C2=A0w23= =3D matrix([[1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0[0, 0, 1],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0[0, 1, 0]])\\n\\n =C2=A0 =C2=A0w13 =3D matrix([[0, 0, 1],\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[1, 0, 0]= ])\\n\\n =C2=A0 =C2=A0w123 =3D matrix([[0, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [0, 0, 1],\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [1, 0, 0]])\\n\\n =C2=A0 =C2= =A0w132 =3D matrix([[0, 0, 1],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 [1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 [0, 1, 0]])\\n\\n =C2=A0 =C2=A0# w =3D 1\\n =C2=A0= =C2=A0for m12 in range(p ** (a2 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= for m13 in range(p ** (a3 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0for m23 in range(p ** (a3 - a2 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0yield matrix([[1, p * m12, p * m13],\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[0, 1, p * m23],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0= , 0, 1]])\\n =C2=A0 =C2=A0# w =3D (12)\\n =C2=A0 =C2=A0for m13 in range(p *= * (a3 - a2 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m21 in range(p ** (a2 -= a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m23 in range(p ** (a= 3 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m = =3D matrix([[1, 0, p * m13],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m21, 1, p * m23],\\= n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[0, 0, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0yield w12 * m\\n =C2=A0 =C2=A0# w =3D (23)\\n =C2=A0 = =C2=A0for m12 in range(p ** (a3 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0f= or m13 in range(p ** (a2 - a1 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0for m32 in range(p ** (a3 - a2)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, p * m12, p * m13],\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0[0, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, m32, 1]])\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w23 * m\\n\\n =C2=A0 = =C2=A0# w =3D (13)\\n =C2=A0 =C2=A0for m21 in range(p ** (a3 - a2)):\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0for m31 in range(p ** (a3 - a1)):\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0for m32 in range(p ** (a2 - a1)):\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix([[1, 0, 0],\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[m21, 1, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m31, m32, 1]= ])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield w13 * m\= \n\\n =C2=A0 =C2=A0# w =3D (123)\\n =C2=A0 =C2=A0for m21 in range(p ** (a3 = - a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m23 in range(p ** (a2 - a1 - 1)):= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m31 in range(p ** (a3 - a2= )):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m =3D matrix(= [[1, 0, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[m21, 1, p * m23],\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0[m31, 0, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0yield w123 * m\\n =C2=A0 =C2=A0# w =3D (132)\\n =C2=A0 =C2=A0for m12= in range(p ** (a3 - a2 - 1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for m31 in ran= ge(p ** (a2 - a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for m32 in = range(p ** (a3 - a1)):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0m =3D matrix([[1, p * m12, 0],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[0, 1, 0],\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0[m31, m32, 1]])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0yield w132 * m\\n\\n\\nclass HalfIntMatElement(obje= ct):\\n\\n =C2=A0 =C2=A0def __init__(self, T):\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0'''\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0:params T: half integral m= atrix of size 3 or a list\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0if isinstance(T, list):\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0a, b, c, d, e, f =3D [ZZ(x) for x in T]\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mat =3D matrix([[a, f / 2, e / 2],\\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0[f / 2, b, d / 2],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[e / 2, d / 2, c]])\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m= at =3D T\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0self.__entries =3D tuple(mat.list())= \\n\\n =C2=A0 =C2=A0def __eq__(self, other):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= if isinstance(other, HalfIntMatElement):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0return self.__entries =3D=3D other.__entries\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise NotImplem= entedError\\n\\n =C2=A0 =C2=A0def __repr__(self):\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0return self.T.__repr__()\\n\\n =C2=A0 =C2=A0def __hash__(self):\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0return hash(self.__entries)\\n\\n =C2=A0 =C2=A0@= property\\n =C2=A0 =C2=A0def T(self):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return = matrix(3, self.__entries)\\n\\n =C2=A0 =C2=A0def right_action(self, g):\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0:p= aram g: matrix of size n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return self[g] (Sieg= el's notation)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0S =3D g.transpose() * self.T * g\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0return HalfIntMatElement(S)\\n\\n =C2=A0 =C2=A0def satisfy_cong_c= ondition_tp(self, p, alpha):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0Test if sum_{B mod D} exp(2pi T B D^(-1)) is = zero, where D =3D diag(p^a1, p^a2, a^a3),\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0a1,= a2, a3 =3D alpha.\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return (all(ZZ(self.T[i, i]) % p ** alpha[i] =3D=3D 0 f= or i in range(3)) and\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0all(ZZ(self.T[i, j] * 2) % p ** alpha[i] =3D=3D 0\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for i in range(3) fo= r j in range(i + 1, 3)))\\n\\n =C2=A0 =C2=A0def is_divisible_by(self, m):\\= n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0= Test if self is divisible by m\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0:param m: inte= ger\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0return _half_int_mat_is_div_by(self.T, m)\\n\\n =C2=A0 =C2=A0def __fl= oordiv__(self, other):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D matrix(QQ, 3)\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0for i in range(3):\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0S[i, i] =3D ZZ(self.T[i, i]) // other\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0for i in range(3):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fo= r j in range(i + 1, 3):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0S[i, j] =3D S[j, i] =3D (ZZ(self.T[i, j] * 2) // other) / 2\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0return HalfIntMatElement(S)\\n\\n\\ndef alpha_list(= dl):\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Return a list of (a0, = a1, a2) with 0 <=3D a0 <=3D a1 <=3D a2 <=3D dl\\n =C2=A0 =C2=A0= '''\\n =C2=A0 =C2=A0return [(a0, a1, a2) for a0 in range(dl + 1= )\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for a1 in range(a0, dl + 1) f= or a2 in range(a1, dl + 1)]\\n\\n\\ndef tp_action_fourier_coeff(p, T, F):\\= n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Return the Tth Fourier coeff= icient of F|T(p), where F is a modular form.\\n =C2=A0 =C2=A0:param p: a pr= ime number\\n =C2=A0 =C2=A0:param T: a half integral matrix or an instance = of HalfIntMatElement\\n =C2=A0 =C2=A0:param F: a dictionary or a Siegel mod= ular form of degree 3\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0p =3D= ZZ(p)\\n =C2=A0 =C2=A0return _action_fc_base(tp_action_fc_alist(p, T), F, = T)\\n\\n\\ndef tp2_action_fourier_coeff(p, i, T, F):\\n =C2=A0 =C2=A0'&= #39;'\\n =C2=A0 =C2=A0Similar to tp_action_fourier_coeff for T_i(p^2).\= \n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0p =3D ZZ(p)\\n =C2=A0 =C2= =A0return _action_fc_base(tp2_action_fc_alist(p, T, i), F, T)\\n\\n\\ndef _= action_fc_base(ls, F, T):\\n =C2=A0 =C2=A0if not isinstance(T, HalfIntMatEl= ement):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D HalfIntMatElement(T)\\n =C2=A0 = =C2=A0res =3D 0\\n =C2=A0 =C2=A0for s, a, g in ls:\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0res =3D a * F[s].left_action(g) + res\\n =C2=A0 =C2=A0return res\\n\\= n\\ndef hecke_eigenvalue_tp(p, F, T=3DNone):\\n =C2=A0 =C2=A0'''= ;\\n =C2=A0 =C2=A0p, F, T: same as aruments of tp_action_fourier_coeff.\\n = =C2=A0 =C2=A0Assuming F is an eigenform, return the eigenvalue for T(p),\\n= =C2=A0 =C2=A0T is used for the computation of Fourier coefficients.\\n =C2= =A0 =C2=A0If T is omitted, T will be set to\\n =C2=A0 =C2=A0matrix([[1, 1/2= , 1/2], [1/2, 1, 1/2], [1/2, 1/2, 1]]).\\n =C2=A0 =C2=A0'''\\n = =C2=A0 =C2=A0return _hecke_eigenvalue_base(lambda s: tp_action_fourier_coef= f(p, s, F), F, T=3DT)\\n\\n\\ndef hecke_eigenvalue_tp2(p, i, F, T=3DNone):\= \n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Similar to hecke_eigenvalue= _tp for T(p^2).\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0return _hec= ke_eigenvalue_base(lambda s: tp2_action_fourier_coeff(p, i, s, F), F, T=3DT= )\\n\\n\\ndef spinor_l_euler_factor(p, F, t=3DNone, T=3DNone):\\n =C2=A0 = =C2=A0'''\\n =C2=A0 =C2=A0F: a dict or Siegel modular form of d= egree 3.\\n =C2=A0 =C2=A0Return a polynomial G(t) of degree 8, s.t.\\n =C2= =A0 =C2=A0G(p^(-s))^(-1) is the p-Euler factor of the spinor L function of = F.\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0p =3D ZZ(p)\\n =C2=A0 = =C2=A0if t is None:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0t =3D PolynomialRing(QQ, = 1, names=3D't', order=3D\\\"neglex\\\").gens()[0]\\n =C2= =A0 =C2=A0c =3D {}\\n =C2=A0 =C2=A0tp =3D hecke_eigenvalue_tp(p, F, T=3DT)\= \n =C2=A0 =C2=A0tpp1, tpp2, tpp3 =3D [hecke_eigenvalue_tp2(p, i, F, T=3DT) = for i in [1, 2, 3]]\\n =C2=A0 =C2=A0c[0] =3D ZZ(1)\\n =C2=A0 =C2=A0c[1] =3D= tp\\n =C2=A0 =C2=A0c[2] =3D p * (tpp1 + (p**2 + 1) * tpp2 + (p**2 + 1)**2 = * tpp3)\\n =C2=A0 =C2=A0c[3] =3D p**3 * tp * (tpp2 + tpp3)\\n =C2=A0 =C2=A0= c[4] =3D p**6 * (tp**2 * tpp3 + tpp2**2 - 2 * p * tpp1 * tpp3 -\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2 * (p - 1) * tpp2 = * tpp3 -\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = (p**6 + 2 * p**5 + 2 * p**3 + 2 * p - 1) * tpp3**2)\\n =C2=A0 =C2=A0c[5] = =3D p**6 * tpp3 * c[3]\\n =C2=A0 =C2=A0c[6] =3D p**12 * tpp3 ** 2 * c[2]\\n= =C2=A0 =C2=A0c[7] =3D p**18 * tpp3 ** 3 * c[1]\\n =C2=A0 =C2=A0c[8] =3D p*= *24 * tpp3 ** 4\\n =C2=A0 =C2=A0return sum((-1)**k * v * t**k for k, v in c= .items())\\n\\n\\ndef rankin_convolution_degree1(f, g, p, name=3DNone):\\n = =C2=A0 =C2=A0u'''\\n =C2=A0 =C2=A0f, g: primitive forms of degr= ee 1 and level 1.\\n =C2=A0 =C2=A0Return p-euler factor of the Rankin convo= lution of f and g as\\n =C2=A0 =C2=A0a polynomial.\\n =C2=A0 =C2=A0'= 9;'\\n =C2=A0 =C2=A0k1 =3D f.weight()\\n =C2=A0 =C2=A0k2 =3D g.weight()= \\n =C2=A0 =C2=A0ap =3D f[p]\\n =C2=A0 =C2=A0bp =3D g[p]\\n =C2=A0 =C2=A0t = =3D PolynomialRing(QQ, 1, names=3D't' if name is None else name,\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 order=3D\\\"neglex\\\").gens()[0]\\n =C2=A0 =C2=A0return (1 -= ap * bp * t +\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(ap**2 * p**(k2 = - 1) + bp**2 * p**(k1 - 1) - 2 * p**(k1 + k2 - 2)) * t**2 -\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ap * bp * p**(k1 + k2 - 2) * t**3 + p**(2 * = (k1 + k2 - 2)) * t**4)\\n\\n\\ndef _hecke_eigenvalue_base(fc_func, F, T=3DN= one):\\n =C2=A0 =C2=A0if T is None:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D Hal= fIntMatElement(matrix([[ZZ(1), ZZ(1) / ZZ(2), ZZ(1) / ZZ(2)],\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[ZZ(1) / ZZ(2), ZZ(1), ZZ(1= ) / ZZ(2)],\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[Z= Z(1) / ZZ(2), ZZ(1) / ZZ(2), ZZ(1)]]))\\n =C2=A0 =C2=A0if not isinstance(T,= HalfIntMatElement):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0T =3D HalfIntMatElement(= T)\\n =C2=A0 =C2=A0v1 =3D fc_func(T).vector\\n =C2=A0 =C2=A0v =3D F[T].vect= or\\n =C2=A0 =C2=A0if v =3D=3D 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0raise ZeroD= ivisionError\\n =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0i =3D next= (i for i in range(len(v)) if v[i] !=3D 0)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0ret= urn v1[i] / v[i]\\n\\n\\n@cached_function\\ndef tp_action_fc_alist(p, T):\\= n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0return a list of tuples (S, = a, g) s.t.\\n =C2=A0 =C2=A0S: an instance of HalfIntMatElement\\n =C2=A0 = =C2=A0a: integer\\n =C2=A0 =C2=A0g: 3 by 3 matrix s.t.\\n =C2=A0 =C2=A0F|T(= p) =3D sum(a rho(g) F[S] | (a, g, S)).\\n =C2=A0 =C2=A0'''\\n = =C2=A0 =C2=A0res1 =3D []\\n =C2=A0 =C2=A0for alpha in alpha_list(1):\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0D =3D diagonal_matrix([p ** a for a in alpha])\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0for V in _gl3_coset_gamma0(alpha, p):\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0M =3D D * V\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0S =3D T.right_action(M.transpose())\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0if S.is_divisible_by(p):\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D S // p\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if S.satisfy_cong_condition_tp(p, alpha):\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# p**= (-6) and p in the third item are for normalization.\\n =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res1.append(\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(S= , p ** (-6) * mul(p ** alpha[i] for i in range(3) for j in range(i, 3)),\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 M ** (-1) * p))\\n =C2=A0 =C2=A0return __convert_reduced_nonisom= _matrices(res1)\\n\\n\\ndef __convert_reduced_nonisom_matrices(alst):\\n = =C2=A0 =C2=A0red_res =3D []\\n =C2=A0 =C2=A0for s, a, g in alst:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0u =3D _minkowski_reduction_transform_matrix(s.T)\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0t =3D s.right_action(u)\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0red_res.append((t, a, g * u.transpose() ** (-1)))\\n\\n =C2=A0 =C2=A0non= _isoms =3D []\\n\\n =C2=A0 =C2=A0for s, a, g in red_res:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0q =3D QuadraticForm(ZZ, 2 * s.T)\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0u =3D None\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for t, _, _ in non_isoms:\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q1 =3D QuadraticForm(ZZ, 2 * t.T)\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if q.det() =3D=3D q1.det():\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0u =3D q.is_globally_= equivalent_to(q1, return_matrix=3DTrue)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0if u and u.transpose() * q.Gram_matrix_rational() *= u =3D=3D q1.Gram_matrix_rational():\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if u:= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0non_isoms.append((s.right_acti= on(u), a, g * u.transpose() ** (-1)))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\\= n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0non_isoms.append((s, a, g))\\n = =C2=A0 =C2=A0return non_isoms\\n\\n\\n@cached_function\\ndef tp2_action_fc_= alist(p, T, i):\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0similar to = tp_action_fc_alist for T_i(p^2) for i =3D 0, 1, 2, 3.\\n =C2=A0 =C2=A0'= ''\\n =C2=A0 =C2=A0res1 =3D []\\n\\n =C2=A0 =C2=A0for alpha in alph= a_list(2):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0D =3D diagonal_matrix([p ** a for = a in alpha])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for V in _gl3_coset_gamma0(alpha= , p):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0M =3D D * V\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D T.right_action(M.transpose())\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if S.is_divisible_by(p ** 2):\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0S =3D S // (p ** 2)\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res1.append((S, p **= (-12) * _expt_sum(S, p, alpha, i),\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 M ** (-1) * = p ** 2))\\n\\n =C2=A0 =C2=A0return __convert_reduced_nonisom_matrices([(a, = b, c) for a, b, c in res1 if b !=3D 0])\\n\\n\\ndef _nearest_integer(x):\\n= =C2=A0 =C2=A0r =3D floor(x)\\n =C2=A0 =C2=A0if x - r > 0.5:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return r + 1\\n =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0return r\\n\\n\\ndef _gaussian_reduction(b1, b2, S):\\n =C2=A0 = =C2=A0'''\\n =C2=A0 =C2=A0b1, b2: vectors of length 3\\n =C2=A0= =C2=A0S: symmetric matrix of size 3\\n =C2=A0 =C2=A0'''\\n =C2= =A0 =C2=A0while True:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0nb1 =3D b1 * S * b1\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0nb2 =3D b2 * S * b2\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0if nb2 < nb1:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2 =3D = b2, b1\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0x =3D (b2 * S * b1) / (b1 * S * b1)\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0r =3D _nearest_integer(x)\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0a =3D b2 - r * b1\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if a * S * a >= =3D b2 * S * b2:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return (b1, b2= )\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0b1, b2 =3D a, b1\\n\\n\\ndef _sym_mat_gen(p, n):\\n =C2=A0 =C2=A0if = n =3D=3D 1:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for a in range(p):\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield matrix([[a]])\\n =C2=A0 =C2=A0else:\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0for s in _sym_mat_gen(p, n - 1):\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ls =3D [range(p) for _ in range(n)]\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for a in itertools.product(*ls):\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0v =3D matrix([a[:-1]= ])\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0yield block_ma= trix([[s, v.transpose()], [v, matrix([[a[-1]]])]])\\n\\n\\ndef _gen_gauss_s= um_direct_way(N, p, r):\\n =C2=A0 =C2=A0res =3D 0\\n =C2=A0 =C2=A0K =3D Cyc= lotomicField(p)\\n =C2=A0 =C2=A0zeta =3D K.gen()\\n =C2=A0 =C2=A0for S in _= sym_mat_gen(p, N.ncols()):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if S.change_ring(F= initeField(p)).rank() =3D=3D r:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0res +=3D zeta ** ((N * S).trace())\\n =C2=A0 =C2=A0try:\\n =C2=A0 =C2=A0= =C2=A0 =C2=A0return QQ(res)\\n =C2=A0 =C2=A0except TypeError:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return res\\n\\n\\ndef _generalized_gauss_sum(N, p, r):= \\n =C2=A0 =C2=A0if r =3D=3D 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1\\n = =C2=A0 =C2=A0if p =3D=3D 2:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return _gen_gauss= _sum_direct_way(N, p, r)\\n =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0N_mp =3D N.change_ring(FiniteField(p))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0d, = _, v =3D N_mp.smith_form()\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0t =3D d.rank()\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0N1 =3D (v.transpose() * N_mp *\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0v).matrix_from_rows_and_columns(range(t),= range(t))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0eps =3D kronecker_symbol(N1.det(),= p)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return _gen_gauss_sum_non_dyadic(p, eps, = N.ncols(), t, r)\\n\\n\\ndef _half_int_mat_is_div_by(S, m):\\n =C2=A0 =C2= =A0n =3D S.ncols()\\n =C2=A0 =C2=A0return (all(ZZ(S[i, i]) % m =3D=3D 0 for= i in range(n)) and\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0all(ZZ(2 * = S[i, j]) % m =3D=3D 0 for i in range(n) for j in range(i + 1, n)))\\n\\n\\n= @cached_function\\ndef _gen_gauss_sum_non_dyadic(p, eps, n, t, r):\\n =C2= =A0 =C2=A0'''\\n =C2=A0 =C2=A0cf. H. Saito, a generalization of= Gauss sums\\n =C2=A0 =C2=A0'''\\n\\n =C2=A0 =C2=A0def parenthe= sis_prod(a, b, m):\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if m =3D=3D 0:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0el= se:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return mul(1 - a * b ** i f= or i in range(m))\\n\\n =C2=A0 =C2=A0if (n - t) % 2 =3D=3D 0:\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0m =3D (n - t) // 2\\n =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0m =3D (n - t + 1) // 2\\n\\n =C2=A0 =C2=A0if n =3D=3D r:\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0if n % 2 =3D=3D 1:\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0return ((-1) ** ((n - 2 * m + 1) // 2) * p ** ((n ** 2 + (= 2 * m) ** 2 - 1) // 4) *\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0parenthesis_prod(p ** (-1), p ** (-2), m))\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0elif n % 2 =3D=3D t % 2 =3D=3D 0:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0return ((-kronecker_symbol(-1, p)) ** ((n - 2 * = m) // 2) *\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0eps * p ** ((n ** 2 + (2 * m + 1) ** 2 - 1) // 4) *\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0parenthesis_prod= (p ** (-1), p ** (-2), m))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\\n =C2=A0 =C2=A0else:\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0diag =3D [1 for _ in range(t)]\\n =C2=A0 =C2=A0 =C2=A0 = =C2=A0if eps =3D=3D -1:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0diag[-1= ] =3D least_quadratic_nonresidue(p)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0diag =3D = diag + [0 for _ in range(n - t)]\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0N =3D diagon= al_matrix(diag).change_ring(FiniteField(p))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0r= eturn _gen_gauss_sum_direct_way(N, p, r)\\n\\n\\ndef _expt_sum(S, p, alpha,= i):\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Return the exponential= sum in Miyawaki's paper, where alpha[-1] <=3D 2, for T_i(p^2).\\n = =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0a, b, c =3D [alpha.count(_i) f= or _i in range(3)]\\n =C2=A0 =C2=A0S33 =3D S.T.matrix_from_rows_and_columns= (range(a + b, 3), range(a + b, 3))\\n =C2=A0 =C2=A0S22 =3D S.T.matrix_from_= rows_and_columns(range(a, a + b), range(a, a + b))\\n =C2=A0 =C2=A0S32 =3D = S.T.matrix_from_rows_and_columns(range(a + b, 3), range(a))\\n\\n =C2=A0 = =C2=A0if c > 0 and not _half_int_mat_is_div_by(S33, p ** 2):\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return 0\\n =C2=A0 =C2=A0if c > 0 and b > 0 and a= ny(x % p !=3D 0 for x in (S32 * ZZ(2)).change_ring(ZZ).list()):\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return 0\\n\\n =C2=A0 =C2=A0if b =3D=3D 0 and a + c =3D= =3D 3 - i:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return p ** (c * (c + 1))\\n =C2= =A0 =C2=A0elif b =3D=3D 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0\\n =C2=A0= =C2=A0else:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0return p ** (c * (c + 1)) * p **= (b * c) * _generalized_gauss_sum(S22, p, b - i)\\n\\n\\ndef _minkowski_red= uction(b1, b2, b3, S):\\n\\n =C2=A0 =C2=A0def inner_prod(x, y):\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0return x * S * y\\n\\n =C2=A0 =C2=A0while True:\\n =C2= =A0 =C2=A0 =C2=A0 =C2=A0b1, b2, b3 =3D sorted([b1, b2, b3], key=3Dlambda b:= b * S * b)\\n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2 =3D _gaussian_reduction= (b1, b2, S)\\n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b11 =3D inner_prod(b1, b1)\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0b12 =3D inner_prod(b1, b2)\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0b13 =3D inner_prod(b1, b3)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b22 =3D = inner_prod(b2, b2)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b23 =3D inner_prod(b2, b3)= \\n =C2=A0 =C2=A0 =C2=A0 =C2=A0b33 =3D inner_prod(b3, b3)\\n\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0y1 =3D - (b13 / b11 - b12 * b23 / (b11 * b22)) / \\\\\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(1 - b12 ** 2 / (b11 * b22))\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0y2 =3D - (b23 / b22 - b12 * b13 / (b11 * b22)) /= \\\\\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(1 - b12 ** 2 / (b11 * b2= 2))\\n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0# Find integers x1, x2 so that norm(b3= + x2 * b2 + x1 * b1) is minimal.\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0a_norms_als= t =3D []\\n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0for x1 in [floor(y1), ceil(y1)]:\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for x2 in [floor(y2), ceil(y2)]= :\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0a =3D b3 + x2 *= b2 + x1 * b1\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0a_n= orms_alst.append((x1, x2, a, inner_prod(a, a)))\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0_inner_prod_a =3D min(x[-1] for x in a_norms_alst)\\n =C2=A0 =C2=A0 =C2= =A0 =C2=A0x1, x2, a, _ =3D next(x for x in a_norms_alst if x[-1] =3D=3D _in= ner_prod_a)\\n\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0if _inner_prod_a >=3D b33:\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Change sings of b1, b2, b3 an= d terminate the alogrithm\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sngs = =3D [sgn(b12), sgn(b13), sgn(b23)]\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0bs =3D [b1, b2, b3]\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0try:\= \n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# If b12, b13 or = b23 is zero, change sgns of b1, b2, b3 so that\\n =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# b12, b13, b23 >=3D 0.\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0zero_i =3D sngs.index(0)\\n =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0set_ls =3D [set([1, 2]), s= et([1, 3]), set([2, 3])]\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0t =3D set_ls[zero_i]\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0_other =3D [x for x in [1, 2, 3] if x not in t][0]\\n =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for x in t:\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0i =3D set_ls.ind= ex(set([x, _other]))\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0if sngs[i] < 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bs[x - 1] *=3D -1\\n = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0b1, b2, b3 =3D bs\\n= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0except ValueError:\\n =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Else change sgns so that b12= , b13 > 0\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if b= 12 < 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0b2 =3D -b2\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0if b13 < 0:\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0b3 =3D -b3\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= return (b1, b2, b3)\\n =C2=A0 =C2=A0 =C2=A0 =C2=A0else:\\n =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0b3 =3D a\\n\\n\\ndef _minkowski_reduction_transf= orm_matrix(S):\\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0Return a uni= modular matrix u such that u^t * S * u is reduced in Minkowski's sense.= \\n =C2=A0 =C2=A0'''\\n =C2=A0 =C2=A0b1, b2, b3 =3D identity_ma= trix(QQ, 3).columns()\\n =C2=A0 =C2=A0c1, c2, c3 =3D _minkowski_reduction(b= 1, b2, b3, S)\\n =C2=A0 =C2=A0return matrix([c1, c2, c3]).transpose()\\n\&q= uot;,\"line\":52,\"column\":41,\"path\":\&quo= t;/home/sho/work/sage_packages/e8theta_degree3/hecke_module.py\"}}&quo= t;

The file contains a multibyte string "=E2= =88=A7" and anaconda-mode converts it to "\342\210\247".

Sho Takemori


2016-07-31 23:31 GMT+09:00 Eli = Zaretskii <eliz@gnu.org>:
> = From: Sho Takemori <stakemorii@g= mail.com>
> Date: Sun, 31 Jul 2016 17:26:37 +0900
>
> I got an error "error in process sentinel: url-http-create-reques= t: Multibyte text in HTTP request" when I visited a
> Python file which contains a multibyte character with `anaconda-eldoc-= mode' turned on.

That file name should have been encoded by the time it is passed to
url-http.el, so the problem should not have happened, because encoded
strings are unibyte strings.

> At first, I thought this was a bug of anaconda-mode. So I opened an is= sue in github
> (https://github.com/proofit404/anaconda-m= ode/issues/189).
>
> I guess `(=3D (string-bytes request) (length request))` in `url-http-c= reate-request' should be `(=3D (string-bytes
> url-http-data) (length url-http-data))`, because `(=3D (string-bytes r= equest) (length request))` may be `nil' even if
> `(=3D (string-bytes url-http-data) (length url-http-data))` is `t'= .

I don't think I agree in general: all the strings that are used by
url-http-create-request should be unibyte strings.=C2=A0 if they all are unibyte strings, then I think the situation you describe should not
happen.=C2=A0 However, you didn't provide enough details to analyze the=
situation, so perhaps I'm missing something.=C2=A0 Could you please sho= w
all the details, specifically, what were the values of the various
variables used by url-http-create-request to generate the request?
For each value that is a string, please also tell whether it's a
unibyte or a multibyte string.

Thanks.

--94eb2c0481aef25b020538f6be7f-- From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 01 09:18:19 2016 Received: (at 24117) by debbugs.gnu.org; 1 Aug 2016 13:18:19 +0000 Received: from localhost ([127.0.0.1]:52745 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUD6l-0004ct-Ff for submit@debbugs.gnu.org; Mon, 01 Aug 2016 09:18:19 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47328) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUD6f-0004ca-Nx for 24117@debbugs.gnu.org; Mon, 01 Aug 2016 09:18:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bUD6W-0006h6-65 for 24117@debbugs.gnu.org; Mon, 01 Aug 2016 09:18:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:43274) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bUD6W-0006gu-2h; Mon, 01 Aug 2016 09:18:00 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4739 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bUD6U-0000MZ-8b; Mon, 01 Aug 2016 09:17:58 -0400 Date: Mon, 01 Aug 2016 16:17:43 +0300 Message-Id: <83popsocg8.fsf@gnu.org> From: Eli Zaretskii To: Sho Takemori In-reply-to: (message from Sho Takemori on Mon, 1 Aug 2016 08:21:39 +0900) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.2 (------) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.2 (------) > From: Sho Takemori > Date: Mon, 1 Aug 2016 08:21:39 +0900 > Cc: 24117@debbugs.gnu.org > > It seems that anaconda-mode use two global variables (url-request-method and url-request-data) > to make generate the request. > > https://github.com/proofit404/anaconda-mode/blob/master/anaconda-mode.el#L349 > > url-request-method is bound to an ASCII string "POST". > In my situation, url-request-data is bound to a unibyte string as below. I don't see any non-ASCII characters in that string. So how come it causes the error message? > The file contains a multibyte string "∧" I don't see this character in the string you show. > and anaconda-mode converts it to "\342\210\247". Which is a correct UTF-8 encoding of that character, and should produce a unibyte string. To summarize, I still don't understand how come the error happened. Could you perhaps step with Edebug into url-http-create-request, and see what is going on there? Or come up with a reproducible recipe of calling url-http-create-request that I could examine on my machine? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 01 20:52:35 2016 Received: (at 24117) by debbugs.gnu.org; 2 Aug 2016 00:52:35 +0000 Received: from localhost ([127.0.0.1]:53245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUNwh-0007RN-In for submit@debbugs.gnu.org; Mon, 01 Aug 2016 20:52:35 -0400 Received: from mail-lf0-f43.google.com ([209.85.215.43]:34844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUNwf-0007RA-Ma for 24117@debbugs.gnu.org; Mon, 01 Aug 2016 20:52:34 -0400 Received: by mail-lf0-f43.google.com with SMTP id f93so127167762lfi.2 for <24117@debbugs.gnu.org>; Mon, 01 Aug 2016 17:52:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=51WBSaetjLRS4H/3dNAwWJIjoULYOhiW1EiN+W8DSFo=; b=mNds6oTM0qH/0mAEQJ57s+iNCOscQKyicDm1U1+H34ppduOOYXASdMxI8o+FHBP9Xn 6wtShSZavPaSQpDNVIdXj0b6a9K9VeCACe49vql2GvECg9kqCfoGPOlFNfz9cTJnQBuA myJxbMcXowiiic/x+6NQ84lkX3MCeVjKGx+2LuamMcf34XltZLM1o2aQFing2VpQs17Z NvWhxzUI+89THgaXvoC26Ui/IVPsRpnKnpJQqidpjwSIBM+tmM6+/ZqiTCJShFjtTMNk Xol5EiZdNM6zOSiXt6OCewm3UqfUnfQ1u1FlcUb46XLPAG3dNWKhGgQlfNk3vj8ApZRT GBJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=51WBSaetjLRS4H/3dNAwWJIjoULYOhiW1EiN+W8DSFo=; b=BNuWjn+JnUY4JxJmcZc5MzIyLwBj7BKB/SVBd0Kbzaz59mjI9nI8P+ZwJ5qGJD+gTj 0pAYrNFNyEh+jltgsRVR/3OHHDLtLyQB/JJXCOUwuXUq7udV9OLDbFMUToZBE+47H1gT Km0TneHBdydyPCkVcsRNFMSt4qj0I1EirUkhldSReNlNT7NmbxC6MJh9boi1LgWlw8Bo +IFVL3mlCUMa70yYlO8xIEZFL+JV1SgNX6erc0C2kLv/xJXnGf/6oisCW+uFNRFxwdUd h85WxIBAaFMQEgR3PESQgzvi4JJctIZczVEdvvTnQ68RDIZyHBNJUIgv0DUxoJd3U2wG zX4g== X-Gm-Message-State: AEkoouuXP/5cp/pU+M0ov6iVdaWRxWU0TOyiFFc0tSrEEqa5oVJgY2mX9CnnIU3n7h+rHg== X-Received: by 10.25.87.2 with SMTP id l2mr21090481lfb.170.1470099147673; Mon, 01 Aug 2016 17:52:27 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.243]) by smtp.googlemail.com with ESMTPSA id r76sm5615618lfi.35.2016.08.01.17.52.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Aug 2016 17:52:26 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii , Sho Takemori References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> From: Dmitry Gutov Message-ID: <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> Date: Tue, 2 Aug 2016 03:52:25 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83popsocg8.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/01/2016 04:17 PM, Eli Zaretskii wrote: > To summarize, I still don't understand how come the error happened. > Could you perhaps step with Edebug into url-http-create-request, and > see what is going on there? Or come up with a reproducible recipe of > calling url-http-create-request that I could examine on my machine? Here's the essence of the problem: (length (concat (encode-coding-string "фыва" 'utf-8) (string-as-multibyte "abc"))) => 11 (string-bytes (concat (encode-coding-string "фыва" 'utf-8) (string-as-multibyte "abc"))) => 19 And (multibyte-string-p (url-host (url-generic-parse-url "http://127.0.0.1"))) => t Apparently, url-generic-parse-url creates a multibyte string for the host name because it performs its parsing in a buffer. And url-http-create-request uses the return value of (url-host url-http-target-url) to set the Location header. And all of that gets concatenated in the request. Some possible solutions: - Perform the "string-bytes = length" verification only for url-http-data, not the the whole request string. This strikes me as ugly, but apparently we've been living with using a multibyte string here for a while. - Call url-encode-url on the return value of (url-host url-http-target-url), and hope that no similar problem pops up with any of the related variables. This does solve the immediate problem with anaconda-mode, I've checked. - Something else? From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 01 23:26:11 2016 Received: (at 24117) by debbugs.gnu.org; 2 Aug 2016 03:26:11 +0000 Received: from localhost ([127.0.0.1]:53268 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUQLK-0002YG-Ui for submit@debbugs.gnu.org; Mon, 01 Aug 2016 23:26:11 -0400 Received: from mail-oi0-f41.google.com ([209.85.218.41]:35546) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUQLK-0002Y4-BJ for 24117@debbugs.gnu.org; Mon, 01 Aug 2016 23:26:10 -0400 Received: by mail-oi0-f41.google.com with SMTP id 4so9358343oih.2 for <24117@debbugs.gnu.org>; Mon, 01 Aug 2016 20:26:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Jhek0/DO7Rb9BNwhhIoRh/tQYImvDLkAhiKhd7A3OGc=; b=HC252aCgVcczM8ARqx62pzR0PD99Uf9C1eMzr7mViekLOPqgl4ujy6blRxwoAza5aP u4t8ebkIPVKEjHsjAm04VBAt0EOBDYf36tjaCBBF1a9JqaJ+NFjUlTNWBJKCbYQYyCVy SYkEpFplT30yoHq5fE8r/tBGID4F6AB3s44ANCbUAmxAVqtDxxMNc5g+zBAppC6BnbKU yYcuVL7H4VtyuUqq+yOFD/7nPzIL4mXLkFOs5Nn08dEhwcTKTvUFf4KZS9eeRrWJNru0 VpHsvYncKwvIVrDPd/1MSFMzpeA6kRdPm1CIlNI8nS2AsOryTX5m+wiV74hQ9mlgBibs QWKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Jhek0/DO7Rb9BNwhhIoRh/tQYImvDLkAhiKhd7A3OGc=; b=AoUdSZyfCE2wxkSokjlBpaLJckM3WpnkaB7RZ3+Sd6OIwOm50N6dsiJoVXqG8wuk44 1DH3sqT2ke+7RD8rP1cxzU//sQ64lnMxCfIP12YriXMCbaICvG756kJevEQw6q1jlqOe udcox9p1Fc2AqE/NgG/S9WgPPb7ejEu+4dQYG/xrJ+74Tqdy6iO5iYH+wqzzrr2sEGPA KC5Heg9VYLG2R7/Fa6A6TTQSsj9Ax+m0TUFPTFpRKvRSKBaybCkU0jXtgJuU93vtnHP1 UH19DKWlWl26LO5uXbI7g5qAP14x+yUkTPspBCCK9NFt/kP+OkduOgp7RCggJCVJcLbD r4Lw== X-Gm-Message-State: AEkooutNuI6l+FszA6XTZr8TOJ3GinHTRpMlYv0WgCzVozp1wvOAF6R9Qd1c4/618z4yUt7WQbTidWimHchhiw== X-Received: by 10.202.117.22 with SMTP id q22mr31510302oic.63.1470108364595; Mon, 01 Aug 2016 20:26:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.33.77 with HTTP; Mon, 1 Aug 2016 20:26:03 -0700 (PDT) In-Reply-To: <83popsocg8.fsf@gnu.org> References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> From: Sho Takemori Date: Tue, 2 Aug 2016 12:26:03 +0900 Message-ID: Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii Content-Type: multipart/alternative; boundary=001a11c1800cda672105390e46a1 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a11c1800cda672105390e46a1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable > I don't see this character in the string you show. It does not contain "=E2=88=A7" but it contains "\342\210\247". That example was unnecessarily big. I should have provided a minimal one. Sho Takemori 2016-08-01 22:17 GMT+09:00 Eli Zaretskii : > > From: Sho Takemori > > Date: Mon, 1 Aug 2016 08:21:39 +0900 > > Cc: 24117@debbugs.gnu.org > > > > It seems that anaconda-mode use two global variables (url-request-metho= d > and url-request-data) > > to make generate the request. > > > > > https://github.com/proofit404/anaconda-mode/blob/master/anaconda-mode.el#= L349 > > > > url-request-method is bound to an ASCII string "POST". > > In my situation, url-request-data is bound to a unibyte string as below= . > > I don't see any non-ASCII characters in that string. So how come it > causes the error message? > > > The file contains a multibyte string "=E2=88=A7" > > I don't see this character in the string you show. > > > and anaconda-mode converts it to "\342\210\247". > > Which is a correct UTF-8 encoding of that character, and should > produce a unibyte string. > > To summarize, I still don't understand how come the error happened. > Could you perhaps step with Edebug into url-http-create-request, and > see what is going on there? Or come up with a reproducible recipe of > calling url-http-create-request that I could examine on my machine? > > Thanks. > --001a11c1800cda672105390e46a1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
> I don't see this character in the string you= show.

It does not contain "=E2=88=A7" b= ut it contains "\342\210\247".
That example was unneces= sarily big. I should have provided a minimal one.

= Sho Takemori


2016-08-01 22:17 GMT+09:00 Eli Zaretskii <eliz@gnu.org>:
> From: Sho Takemori <= ;stakemorii@gmail.com>
> Date: Mon, 1 Aug 2016 08:21:39 +0900
> Cc: 24117@debbugs.gnu.org=
>
> It seems that anaconda-mode use two global variables (url-request-meth= od and url-request-data)
> to make generate the request.
>
> https://github.com= /proofit404/anaconda-mode/blob/master/anaconda-mode.el#L349
>
> url-request-method is bound to an ASCII string "POST".
> In my situation, url-request-data is bound to a unibyte string as belo= w.

I don't see any non-ASCII characters in that string.=C2=A0 So ho= w come it
causes the error message?

> The file contains a multibyte string "=E2=88=A7"

I don't see this character in the string you show.

> and anaconda-mode converts it to "\342\210\247".

Which is a correct UTF-8 encoding of that character, and should
produce a unibyte string.

To summarize, I still don't understand how come the error happened.
Could you perhaps step with Edebug into url-http-create-request, and
see what is going on there?=C2=A0 Or come up with a reproducible recipe of<= br> calling url-http-create-request that I could examine on my machine?

Thanks.

--001a11c1800cda672105390e46a1-- From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 02 11:26:03 2016 Received: (at 24117) by debbugs.gnu.org; 2 Aug 2016 15:26:03 +0000 Received: from localhost ([127.0.0.1]:53775 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUbZz-0004Hh-Ad for submit@debbugs.gnu.org; Tue, 02 Aug 2016 11:26:03 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUbZx-0004HE-Ii for 24117@debbugs.gnu.org; Tue, 02 Aug 2016 11:26:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bUbZp-0007r1-7R for 24117@debbugs.gnu.org; Tue, 02 Aug 2016 11:25:56 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33484) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bUbZp-0007qs-3o; Tue, 02 Aug 2016 11:25:53 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2486 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bUbZn-0008Nn-7J; Tue, 02 Aug 2016 11:25:51 -0400 Date: Tue, 02 Aug 2016 18:25:37 +0300 Message-Id: <83mvkvmbv2.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> (message from Dmitry Gutov on Tue, 2 Aug 2016 03:52:25 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.2 (------) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.2 (------) > Cc: 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 2 Aug 2016 03:52:25 +0300 > > (length (concat (encode-coding-string "фыва" 'utf-8) > (string-as-multibyte "abc"))) > > => 11 > > (string-bytes (concat (encode-coding-string "фыва" 'utf-8) > (string-as-multibyte "abc"))) > > => 19 > > And > > (multibyte-string-p (url-host (url-generic-parse-url "http://127.0.0.1"))) > > => t > > Apparently, url-generic-parse-url creates a multibyte string for the > host name because it performs its parsing in a buffer. And > url-http-create-request uses the return value of (url-host > url-http-target-url) to set the Location header. And all of that gets > concatenated in the request. Thanks for spelling this out. > Some possible solutions: > > - Perform the "string-bytes = length" verification only for > url-http-data, not the the whole request string. This strikes me as > ugly, but apparently we've been living with using a multibyte string > here for a while. > > - Call url-encode-url on the return value of (url-host > url-http-target-url), and hope that no similar problem pops up with any > of the related variables. This does solve the immediate problem with > anaconda-mode, I've checked. > > - Something else? How about making the temporary buffer parsed by url-generic-parse-url a unibyte buffer? Does that fix the problem? AFAIU, RFC 3986 doesn't allow non-ASCII characters, so we should be okay handling that in a unibyte buffer, right? I mean something like this: (with-temp-buffer ;; Don't let those temp-buffer modifications accidentally ;; deactivate the mark of the current-buffer. (let ((deactivate-mark nil)) (set-syntax-table url-parse-syntax-table) (erase-buffer) (set-buffer-multibyte nil) ;; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< (insert url) (goto-char (point-min)) ... As for other possible problems like that, are there any that could be expected already? If so, we could try fixing them now. Alternatively, we could just wait for them to come up; after all, catching those was the main rationale for introducing the length test, right? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 02 22:39:41 2016 Received: (at 24117) by debbugs.gnu.org; 3 Aug 2016 02:39:41 +0000 Received: from localhost ([127.0.0.1]:53977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUm5t-00082R-6W for submit@debbugs.gnu.org; Tue, 02 Aug 2016 22:39:41 -0400 Received: from mail-lf0-f52.google.com ([209.85.215.52]:33257) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUm5r-00082A-RM for 24117@debbugs.gnu.org; Tue, 02 Aug 2016 22:39:40 -0400 Received: by mail-lf0-f52.google.com with SMTP id b199so151457970lfe.0 for <24117@debbugs.gnu.org>; Tue, 02 Aug 2016 19:39:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=62gA331l84B1xJfL7amyvcS+5iRrmJgRk0GI73tCHcg=; b=v35LRPUy0ogFmjZYvkjnhqlZZSJNT5vv/GqgnH5w9SA2gMJZF3lgePdnByowycLbzE TOr6EQux3AmMV2/ZRBBThw71nNX8U9eefkzHlVLxUngrVExR5R44+KP5lSlj6pfS7b9i 8Lz+v8FsCGuvSvKuW8xerrL6Fs6Y5EW/x340x7SEU92Mgz2PzhdWo+zXIvvXN7gxvKx4 QiaKPm782xKaxuOylhQBxEJfKT0g3YiS4aIznWcuvRdqm9fNDDc1wZ9NbmbujWGG10Ww EsZKoSpTHMm+mS2itYMOW9WyKLxIjxS9CM3C2TQyJoMSBHULs/yJJmIk0WO/a8ers7Yw dsPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=62gA331l84B1xJfL7amyvcS+5iRrmJgRk0GI73tCHcg=; b=G+mhaYK8B7Dk76wrDuhgDIHl95QkfGCg0GluApuGA8jhsy1Ny6sfKgQQ/bTYYeqbM3 0kzlFCv8Wwhmorwwp8B/RqZxZcjPXmGe6Z34P3h7/rnL0K9MjygL5IhLG/9XD6uAjlwi /ip9+XMpIX309bFtvbepz2qK4m6bTqS/u6YC78H+cXgtrlyYn8hfWtsECYDWoVYsQVQ8 YmkALxEwxZbwhthPDYg7b025bEJCyzDMjngtvTYhR9g786f1p6Uy+ZMTLAC2cgxX9HGL hY8OKXt9PO0rYQXuLFFbUy5RKiVV3opYxcBH2fl+JQ6/nGSBPUBukT0NTGj5A10jL0WI ul9Q== X-Gm-Message-State: AEkooutCB7yujEfaiIQmpeTe85ZWkeSfKXBfvvLMPWZhkPDdx5w76GPq729c22/AHbD67A== X-Received: by 10.25.41.142 with SMTP id p136mr21750270lfp.32.1470191973718; Tue, 02 Aug 2016 19:39:33 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id y200sm997952lfd.12.2016.08.02.19.39.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Aug 2016 19:39:32 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> From: Dmitry Gutov Message-ID: <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> Date: Wed, 3 Aug 2016 05:39:31 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83mvkvmbv2.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Lars Magne Ingebrigtsen , 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/02/2016 06:25 PM, Eli Zaretskii wrote: > How about making the temporary buffer parsed by url-generic-parse-url > a unibyte buffer? Does that fix the problem? It does fix anaconda-mode, yes. > AFAIU, RFC 3986 doesn't > allow non-ASCII characters, so we should be okay handling that in a > unibyte buffer, right? I don't really know. RFC 3986 or not, I suppose in practice the url could be quoted before or after it's parsed. And url-parse-tests.el doesn't specify this case. Lars, what do you think? > I mean something like this: > > (with-temp-buffer > ;; Don't let those temp-buffer modifications accidentally > ;; deactivate the mark of the current-buffer. > (let ((deactivate-mark nil)) > (set-syntax-table url-parse-syntax-table) > (erase-buffer) > (set-buffer-multibyte nil) ;; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > (insert url) > (goto-char (point-min)) > ... Heh, that's exactly where I added the line, without looking at your code. > As for other possible problems like that, are there any that could be > expected already? If so, we could try fixing them now. Nothing else jumps out so far. The function depends on quite a few global variables. To be really certain, we'd have to trace how all of them are created, and for all that are not directly bound by the user, the chains of calls that produce them. > Alternatively, we could just wait for them to come up; I'm worried about having a problem crop up in some significant use case after we release 25.1. That doesn't feel very probable, but still. > after all, > catching those was the main rationale for introducing the length test, > right? The most important part was to make sure that the length of the body in bytes is equal to the value of the Content-Length header (the difference caused actual problems). But then we decided to make the check wider and test that the whole request string is unibyte-ish. Which made sense, but seems to be working out less well than we expected. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 04 13:03:34 2016 Received: (at 24117) by debbugs.gnu.org; 4 Aug 2016 17:03:34 +0000 Received: from localhost ([127.0.0.1]:55836 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bVM3S-0006XZ-3c for submit@debbugs.gnu.org; Thu, 04 Aug 2016 13:03:34 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40237) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bVM3Q-0006XN-30 for 24117@debbugs.gnu.org; Thu, 04 Aug 2016 13:03:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bVM3H-0000Pw-DD for 24117@debbugs.gnu.org; Thu, 04 Aug 2016 13:03:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37618) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bVM2z-0000Ko-Jp; Thu, 04 Aug 2016 13:03:05 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4264 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bVM2y-0005EK-C1; Thu, 04 Aug 2016 13:03:04 -0400 Date: Thu, 04 Aug 2016 20:02:55 +0300 Message-Id: <83twf0lb5s.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> (message from Dmitry Gutov on Wed, 3 Aug 2016 05:39:31 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.3 (------) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.3 (------) > Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, > Lars Magne Ingebrigtsen > From: Dmitry Gutov > Date: Wed, 3 Aug 2016 05:39:31 +0300 > > On 08/02/2016 06:25 PM, Eli Zaretskii wrote: > > > How about making the temporary buffer parsed by url-generic-parse-url > > a unibyte buffer? Does that fix the problem? > > It does fix anaconda-mode, yes. Hmm, but url-generic-parse-url is called from gazillion other places, so maybe this is not safe. > > AFAIU, RFC 3986 doesn't > > allow non-ASCII characters, so we should be okay handling that in a > > unibyte buffer, right? > > I don't really know. RFC 3986 or not, I suppose in practice the url > could be quoted before or after it's parsed. And url-parse-tests.el > doesn't specify this case. No, I meant that since RFC 3986 doesn't allow non-ASCII characters, and url-generic-parse-url doesn't do anything about that, it is either already broken for non-ASCII characters, or already copes with them. So we don't need to worry about that. However, a safer change would be to do something like this: (or (not (multibyte-string-p url-http-target-url)) (setq url-http-target-url (decode-coding-string url-http-target-url 'utf-8))) in url-http-create-request. Can you try this? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 07 21:57:10 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 01:57:10 +0000 Received: from localhost ([127.0.0.1]:59386 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWZoU-0008OL-EY for submit@debbugs.gnu.org; Sun, 07 Aug 2016 21:57:10 -0400 Received: from mail-lf0-f51.google.com ([209.85.215.51]:32775) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWZoT-0008O4-0o for 24117@debbugs.gnu.org; Sun, 07 Aug 2016 21:57:09 -0400 Received: by mail-lf0-f51.google.com with SMTP id b199so237556939lfe.0 for <24117@debbugs.gnu.org>; Sun, 07 Aug 2016 18:57:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=myHP5JOuDqHn3c53NTqTtCEsWIY7tz6bMEGebJU+YR4=; b=i+VWN1S5k8aOQ3NljlC78sPxaz/PvxY6gJTu2L15JwJxMcmrt0eAscpVwx30IRSWIM pkYntYfT7nfRHdR+maqDfqOSdcBf9LjgWdd62xTckiUUrK9adIpVKGZHK/Nbe6cIorR7 lckPDQEs2P72sreGZEHZlgg8lm1AbSUr/tc7gplw/XNfTdXJJL/nNdyFF56aqBi3PDvS 5lP9QSBq9S3XlBEPYmWxW8QMNbLwpJpm8yOo/Op0uF0f3Fno/jcKVP4gZeF1s8wsbreH fWdiuKRaSXwvSSqdsBcb0ry04lDbOPGYmHlHd2V2kBn27gS7Qn3WaQkTIlnWchguWvTT rFpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=myHP5JOuDqHn3c53NTqTtCEsWIY7tz6bMEGebJU+YR4=; b=GjPdDwoEe8b6EbH/vk8hwa5uiO08h86DWtssUznqKFo2tBXMKFGav3gawBAznCQL/3 /PfGnuf50TWZvv5W9/FVH3e8I5EVE90ctZiSBrk3bItoJ0Bp4gZBK6JDuezNhamgvHTf x/dxuPBYTtVG7+wn37PmzblbMOsk+JEZniqmhLb4s0bJNXm9IjXRYyQ6iLCvNd9SVysU 6MYMO3A+z2uvS9PhmppxotTjpzXbLYG5SbJavzca2/8qIY+0cney6vLCSHEtmroSwUfJ 7oleyQNMZyogHB37T8Y6/xI0DCwgeU+b4MLTiiYznvE8iotBhTa7sXJ7ssZNwEyJ1Vci Fb0g== X-Gm-Message-State: AEkoouvvphwa9bWoMzE8joKvsHFUksryn3PsoA6bA0DA69D4/myg82TQkJlPJIjM7Pt4Mg== X-Received: by 10.25.16.162 with SMTP id 34mr29874365lfq.127.1470621421514; Sun, 07 Aug 2016 18:57:01 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id r190sm5283196lfg.49.2016.08.07.18.56.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 07 Aug 2016 18:57:00 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Mon, 8 Aug 2016 04:56:58 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83twf0lb5s.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) Hi Eli, On 08/04/2016 08:02 PM, Eli Zaretskii wrote: > Hmm, but url-generic-parse-url is called from gazillion other places, > so maybe this is not safe. Only about 40 places, all of them either in lisp/url or lisp/gnus. Sadly, Lars is being silent on the matter. It might not be 100% safe, but maybe doing TRT could be enough. > No, I meant that since RFC 3986 doesn't allow non-ASCII characters, Indeed. > and url-generic-parse-url doesn't do anything about that, it is either > already broken for non-ASCII characters, or already copes with them. > So we don't need to worry about that. I imagined that some code that uses the return value of url-http-create-request might perform the escaping. But that doesn't seem to be the case, see below. > However, a safer change would be to do something like this: > > (or (not (multibyte-string-p url-http-target-url)) > (setq url-http-target-url > (decode-coding-string url-http-target-url 'utf-8))) > > in url-http-create-request. Can you try this? I'll try it if you insist, but that choice of encoding seems rather arbitrary. I think we should go with your previous suggestion: make the URL parsing buffer unibyte. But we do try to handle non-ASCII URLs on the level above url-generic-parse-url. See url-retrieve-internal: one of the first things it does is (setq url (url-encode-url url)). And only after that, (setq url (url-generic-parse-url url)). The URL package doesn't seem to support international domains anyway. This fails: (url-retrieve-synchronously "http://банки.рф") However, the error it fails with is a bit more comprehensible if the URL parsing buffer is unibyte: Debugger entered--Lisp error: (error "банки.рф/80 Name or service not known") Instead of: Debugger entered--Lisp error: (error "\301\220\300\261\301\220\300\260\301\220\300\275\301\220\300\272\301\220\300\270.\301\221\300\200\301\221\300\204/80 Name or service not known") From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 09:33:01 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 13:33:01 +0000 Received: from localhost ([127.0.0.1]:59663 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWkft-0004LQ-4x for submit@debbugs.gnu.org; Mon, 08 Aug 2016 09:33:01 -0400 Received: from mail-pa0-f41.google.com ([209.85.220.41]:36394) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWkfq-0004LA-PR for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 09:32:59 -0400 Received: by mail-pa0-f41.google.com with SMTP id pp5so114025979pac.3 for <24117@debbugs.gnu.org>; Mon, 08 Aug 2016 06:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version; bh=CswhTy/PPyEhp8e7G0Bn6f8F6d8xQBuitcl+8FHAdTc=; b=cE4e3h5f9G7zJryTfFR41nlU4IJemJ/2kFFGgldhvHi0U/VETEhj2fQl6RcnCMPZ2G 7WCu/+rpCJ2i/XCtrmNOc3p9RliYfoRZnWdj72q187IM6d25QC19helTbypn20zeuHof jP0yIyWDTgTNsRsR+tW7S1eqPS/mZewkE7YnA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version; bh=CswhTy/PPyEhp8e7G0Bn6f8F6d8xQBuitcl+8FHAdTc=; b=Cara4AXBjWVRvlTeZokB7NqbHZg80f6dLFn9nMyPVeXDfw5aY6lYLw3rOVJCI2+HHT VoBCdhxfgPy1s9lazIAPC0513sS7dDCGIOcbTYLwDNp9f/wpLu4hvkxpbboc+PoHLQ/L Je3AwEF3mM1PVbaX6AFdESc+yOB5ZBkATpt0NP2QmTEZgzw//sFtGJL7Lyd5JPCGiNIz ysbU7M3SRS7n0j1cASuBQisJQmmfKNVnF7xg3YkJsEEKljqDHoL0iYbyc11hhWeuu+kJ Ysx7yFkNTCqbvi/lG6ss5AaD9usg2ZyKNJnFHvoN+jLEgb2ix2yUYPozsEvgNER8rLed oLZw== X-Gm-Message-State: AEkoouszzbygdxw4ZAFNbDxEuv2pwdi9hw0VuIYaY6FgHOdvJskXkg3L4wfBu8XqWIp1vQ== X-Received: by 10.66.8.163 with SMTP id s3mr63955794paa.142.1470663172867; Mon, 08 Aug 2016 06:32:52 -0700 (PDT) Received: from flea (c-98-229-60-157.hsd1.ma.comcast.net. [98.229.60.157]) by smtp.gmail.com with ESMTPSA id p64sm48385906pfd.11.2016.08.08.06.32.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Aug 2016 06:32:52 -0700 (PDT) From: Ted Zlatanov To: Dmitry Gutov , Katsumi Yamaoka Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Mon, 08 Aug 2016 09:32:44 -0400 In-Reply-To: (Dmitry Gutov's message of "Mon, 8 Aug 2016 04:56:58 +0300") Message-ID: <871t1zcrnn.fsf@lifelogs.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Mon, 8 Aug 2016 04:56:58 +0300 Dmitry Gutov wrote: DG> Hi Eli, DG> On 08/04/2016 08:02 PM, Eli Zaretskii wrote: >> Hmm, but url-generic-parse-url is called from gazillion other places, >> so maybe this is not safe. DG> Only about 40 places, all of them either in lisp/url or lisp/gnus. Sadly, Lars DG> is being silent on the matter. DG> It might not be 100% safe, but maybe doing TRT could be enough. Lars tends to work in batches (from experience) so waiting on him can take a while. Looking at the discussion, I think the change is OK. I've CC-ed Katsumi Yamaoka since he may have some feedback as well. If this is pushed, since it's a fairly low-level change, it should include a test for the specific issue it fixes (I didn't see that in the discussion so far). Then if we need to tune the code further, we'll have something to keep us from creating a regression. Thanks Ted From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 11:34:07 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 15:34:07 +0000 Received: from localhost ([127.0.0.1]:60219 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmZ1-0007EL-NI for submit@debbugs.gnu.org; Mon, 08 Aug 2016 11:34:07 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60325) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmYx-0007Dq-Id for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 11:34:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWmYn-0005On-KG for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 11:33:54 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34284) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWmYn-0005OZ-H8; Mon, 08 Aug 2016 11:33:49 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4664 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWmYl-0006eB-8W; Mon, 08 Aug 2016 11:33:48 -0400 Date: Mon, 08 Aug 2016 18:33:30 +0300 Message-Id: <83r39zi8c5.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Mon, 8 Aug 2016 04:56:58 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Mon, 8 Aug 2016 04:56:58 +0300 > > > (or (not (multibyte-string-p url-http-target-url)) > > (setq url-http-target-url > > (decode-coding-string url-http-target-url 'utf-8))) > > > > in url-http-create-request. Can you try this? > > I'll try it if you insist, but that choice of encoding seems rather > arbitrary. I think we should go with your previous suggestion: make the > URL parsing buffer unibyte. OK, let's go with that. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 11:52:44 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 15:52:44 +0000 Received: from localhost ([127.0.0.1]:60224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmr6-0007fy-IE for submit@debbugs.gnu.org; Mon, 08 Aug 2016 11:52:44 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:35871) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmr4-0007fq-Jm for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 11:52:43 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWmr0-0004OU-Gw; Mon, 08 Aug 2016 17:52:40 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEXi0M2/tbpBHx/p29cL BQZ1TUOUhIGMCRZLAAACS0lEQVQ4jW2UwY7jIAyGQWhzDtOW8w4KOVOh7XmFSO9FMeeSCbz/I6xN 0jaHtSIF+LAB+wcmObOWCYfGmGXMbS3OuMQe9hkT/IkNmmIZz0xRb7PatylEKmNKkGezvP0IZMue XModNIcW60qNDurRJBqjZcSfCAHND+Fld0UbEzdAoLVGoGObMsrmcUsQ/AZgCDOSsRJw1/IGGgYK OLZQzp1bJI0xwNC4N5vHDshB+xCGcC9w9Bg8DYcJP0gfAIgANC4WYDYfD0OAjuIB6gpvgN2oDTEM U1fabgPeQIk6ERhrVSuUdnJ3NugPYGhntea86j3UV8LcQdyBlPMGhFOy1hJDA1lmqXVquXKCIRnC NOOupqxqA47xk7BcVcxE9JQWDzEmgTpQ6GOlWrSPk2418Qg6VEDGWqILUFKW4HEteNiaWRUIrCwl ehNHKm6ER4eCeToCXKVo8BBYb64h2UyiIFlwWRKKIgt7ukZIm4qa9mQtUCALnIGZE/YFbniUNIy8 1Lwg2EQs23Gkgpgg48YnBC3U+UTaO3VwqQjkAdwRdHKGSh5y3YEQa/hx7tcSokok0BdgbA3fCAYP aoSeH8AFTs6q4buCITAfPH5uaal9l0zqmRyOYB6vNd8MAXUEw3TN18LNWJ9HgLmehr+2PsyI6VrC pN8eQRsc+kYVGbwJJpg3+L3ohxrLUoCA/gAsdu6/LiskAq81ON1H2VMe8a/o9Oz/tt37rv+M9AJH LalN9fs7QWbfTwZqh94JsY0LK93L2nNj96cEa/UGWNl/hVHRJmJBMpkAAAAASUVORK5CYII= Date: Mon, 08 Aug 2016 17:52:02 +0200 In-Reply-To: (Dmitry Gutov's message of "Mon, 8 Aug 2016 04:56:58 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Dmitry Gutov writes: >> Hmm, but url-generic-parse-url is called from gazillion other places, >> so maybe this is not safe. > > Only about 40 places, all of them either in lisp/url or > lisp/gnus. Sadly, Lars is being silent on the matter. It's also called in ffap, eww and newst, according to grep. :-) > The URL package doesn't seem to support international domains > anyway. This fails: > > (url-retrieve-synchronously "http://=D0=B1=D0=B0=D0=BD=D0=BA=D0=B8.=D1=80= =D1=84") I think that's a bug... --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 11:55:14 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 15:55:14 +0000 Received: from localhost ([127.0.0.1]:60228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmtW-0007ji-0J for submit@debbugs.gnu.org; Mon, 08 Aug 2016 11:55:14 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:35908) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWmtT-0007ja-Uk for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 11:55:12 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWmtR-0004Rm-DM; Mon, 08 Aug 2016 17:55:11 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEXi0M2/tbpBHx/p29cL BQZ1TUOUhIGMCRZLAAACS0lEQVQ4jW2UwY7jIAyGQWhzDtOW8w4KOVOh7XmFSO9FMeeSCbz/I6xN 0jaHtSIF+LAB+wcmObOWCYfGmGXMbS3OuMQe9hkT/IkNmmIZz0xRb7PatylEKmNKkGezvP0IZMue XModNIcW60qNDurRJBqjZcSfCAHND+Fld0UbEzdAoLVGoGObMsrmcUsQ/AZgCDOSsRJw1/IGGgYK OLZQzp1bJI0xwNC4N5vHDshB+xCGcC9w9Bg8DYcJP0gfAIgANC4WYDYfD0OAjuIB6gpvgN2oDTEM U1fabgPeQIk6ERhrVSuUdnJ3NugPYGhntea86j3UV8LcQdyBlPMGhFOy1hJDA1lmqXVquXKCIRnC NOOupqxqA47xk7BcVcxE9JQWDzEmgTpQ6GOlWrSPk2418Qg6VEDGWqILUFKW4HEteNiaWRUIrCwl ehNHKm6ER4eCeToCXKVo8BBYb64h2UyiIFlwWRKKIgt7ukZIm4qa9mQtUCALnIGZE/YFbniUNIy8 1Lwg2EQs23Gkgpgg48YnBC3U+UTaO3VwqQjkAdwRdHKGSh5y3YEQa/hx7tcSokok0BdgbA3fCAYP aoSeH8AFTs6q4buCITAfPH5uaal9l0zqmRyOYB6vNd8MAXUEw3TN18LNWJ9HgLmehr+2PsyI6VrC pN8eQRsc+kYVGbwJJpg3+L3ohxrLUoCA/gAsdu6/LiskAq81ON1H2VMe8a/o9Oz/tt37rv+M9AJH LalN9fs7QWbfTwZqh94JsY0LK93L2nNj96cEa/UGWNl/hVHRJmJBMpkAAAAASUVORK5CYII= Date: Mon, 08 Aug 2016 17:54:33 +0200 In-Reply-To: (Dmitry Gutov's message of "Mon, 8 Aug 2016 04:56:58 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Dmitry Gutov writes: > I'll try it if you insist, but that choice of encoding seems rather > arbitrary. I think we should go with your previous suggestion: make > the URL parsing buffer unibyte. It's be sad if (url-generic-parse-url "http://g=C3=B3ogle.com/f=C3=B3o") stopped working. This function is Emacs' workhorse for chopping up URLs, and it's a very useful function. There's a bunch of hand-rolled URL parsing implementations that don't quite work right. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:14:55 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:14:55 +0000 Received: from localhost ([127.0.0.1]:60237 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnCY-0008BW-US for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:14:55 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40193) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnCU-0008BF-3X for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:14:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWnCL-0004Mv-5V for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:14:44 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34789) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWnCL-0004Mr-2D; Mon, 08 Aug 2016 12:14:41 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4765 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWnCJ-0002Ps-ME; Mon, 08 Aug 2016 12:14:40 -0400 Date: Mon, 08 Aug 2016 19:14:20 +0300 Message-Id: <83lh07i6g3.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-reply-to: (message from Lars Ingebrigtsen on Mon, 08 Aug 2016 17:54:33 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Lars Ingebrigtsen > Cc: Eli Zaretskii , stakemorii@gmail.com, 24117@debbugs.gnu.org > Date: Mon, 08 Aug 2016 17:54:33 +0200 > > It's be sad if > > (url-generic-parse-url "http://góogle.com/fóo") > > stopped working. It's already broken, because that function does nothing special for non-ASCII characters, although the corresponding RFC says they are not allowed. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:19:02 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:19:02 +0000 Received: from localhost ([127.0.0.1]:60242 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnGU-0008HW-Fd for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:19:02 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:36258) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnGT-0008HP-6R for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:18:57 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWnGQ-0004uL-9D; Mon, 08 Aug 2016 18:18:56 +0200 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAADFBMVEVGKD5URVFoXmGJjYLo 0o+RAAACe0lEQVQokQFwAo/9AP8PQNAJUGqupCrVuAD/g1CgJmBa5hkr/uQAe4G0EKtQGrVVaH/Q APfVeAKugGmlWqK/iACm8VaVlpValupGv3gA6TxlQara2uqpB/4UAPat+lKqqaGulC/oFgDuvRlR rWpWY1B2Zb4ADroKhFkqkpZIW4f+AMQKVllBhAFumFWv/gD/gYA6C1Fk+sEB9ZUA//we0CqhRSrq BUZVAP/5PsCrpAF9GhEEZAD9+CqApooCqjAQBdoAv9Ue4pHLgQedAKr/ABBWb+YerggZNBv//wCq lV55D7lcPlCfhv8AmYFW2auGrr5Je8ABAAkRVsFK1W/9Xp8EFQACVWWiB6rD/rkBwAEABAAVmgL/ w/+uvUJAAEFAAFAVv/++/+uVqgCpEAAAaPW9dKrrqlUAf6QAdpDwmBHp6pqkAP/+gY7L8qql/mpF RAD/qQWWuaELkNvq+W8A//RKlaewC1Hn775vAOxmmqATEAoVqr//WgD/+QJQVQAloAoqe0YA/5C4 QrQAnqZEXq9nAIVL+UKeoaZZCG8uAgBED/x/ommXnQCZ6fwAFQP2vrWipq4kVEp/AOUV0b+qWaa8 KWSaCwDVVplrqqtqnHlRkAIAqrr2afqvRbS4/JQXAP+rxSQKvz8Up9gsWQD/6vQEibIdAKcrT1UA /8AQKqvqAUClrodVAP+BAXZn/gKAqatAlgD/6RTmkvgCQbmrBAEA3rrS1eFQAoJUqVEDAPryAkFS pQqDURKFEQD3wQV6S7Sqw0FngSAAj1gWmWqgq6+CaeCpAFoIWqylom0/jlTh/gD4EGr4akpWv4uJ vT8AkIPq9ms4Av9K6G8f2OUjVxsZWssAAAAASUVORK5CYII= Date: Mon, 08 Aug 2016 18:18:18 +0200 In-Reply-To: <83lh07i6g3.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Aug 2016 19:14:20 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Eli Zaretskii writes: >> It's be sad if >>=20 >> (url-generic-parse-url "http://g=C3=B3ogle.com/f=C3=B3o") >>=20 >> stopped working. > > It's already broken, because that function does nothing special for > non-ASCII characters, although the corresponding RFC says they are not > allowed. No, it does what users expect it to. (url-generic-parse-url "http://g=C3=B3ogle.com/f=C3=B3o") =3D> [cl-struct-url "http" nil nil "g=C3=B3ogle.com" nil "/f=C3=B3o" nil ni= l t nil t] "Aha, so 'g=C3=B3ogle.com' is the domain name." That you have to encode the data returned before doing network stuff (which is what the RFC talks about) is a completely different matter. The function is used to decompose URLs found in nature. Those URLs contain non-ASCII characters. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:22:16 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:22:16 +0000 Received: from localhost ([127.0.0.1]:60256 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnJg-0008N6-CK for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:22:16 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:36273) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnJf-0008Mz-85 for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:22:15 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWnJc-0004w3-O1; Mon, 08 Aug 2016 18:22:14 +0200 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAADFBMVEVGKD5URVFoXmGJjYLo 0o+RAAACe0lEQVQokQFwAo/9AP8PQNAJUGqupCrVuAD/g1CgJmBa5hkr/uQAe4G0EKtQGrVVaH/Q APfVeAKugGmlWqK/iACm8VaVlpValupGv3gA6TxlQara2uqpB/4UAPat+lKqqaGulC/oFgDuvRlR rWpWY1B2Zb4ADroKhFkqkpZIW4f+AMQKVllBhAFumFWv/gD/gYA6C1Fk+sEB9ZUA//we0CqhRSrq BUZVAP/5PsCrpAF9GhEEZAD9+CqApooCqjAQBdoAv9Ue4pHLgQedAKr/ABBWb+YerggZNBv//wCq lV55D7lcPlCfhv8AmYFW2auGrr5Je8ABAAkRVsFK1W/9Xp8EFQACVWWiB6rD/rkBwAEABAAVmgL/ w/+uvUJAAEFAAFAVv/++/+uVqgCpEAAAaPW9dKrrqlUAf6QAdpDwmBHp6pqkAP/+gY7L8qql/mpF RAD/qQWWuaELkNvq+W8A//RKlaewC1Hn775vAOxmmqATEAoVqr//WgD/+QJQVQAloAoqe0YA/5C4 QrQAnqZEXq9nAIVL+UKeoaZZCG8uAgBED/x/ommXnQCZ6fwAFQP2vrWipq4kVEp/AOUV0b+qWaa8 KWSaCwDVVplrqqtqnHlRkAIAqrr2afqvRbS4/JQXAP+rxSQKvz8Up9gsWQD/6vQEibIdAKcrT1UA /8AQKqvqAUClrodVAP+BAXZn/gKAqatAlgD/6RTmkvgCQbmrBAEA3rrS1eFQAoJUqVEDAPryAkFS pQqDURKFEQD3wQV6S7Sqw0FngSAAj1gWmWqgq6+CaeCpAFoIWqylom0/jlTh/gD4EGr4akpWv4uJ vT8AkIPq9ms4Av9K6G8f2OUjVxsZWssAAAAASUVORK5CYII= Date: Mon, 08 Aug 2016 18:21:37 +0200 In-Reply-To: <83lh07i6g3.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Aug 2016 19:14:20 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) (And it is perfectly valid for the domain bits of URLs to contain non-ASCII characters after the IDNA changeover in the RFCs.) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:33:56 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:33:57 +0000 Received: from localhost ([127.0.0.1]:60267 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnUy-0000DM-Me for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:33:56 -0400 Received: from eggs.gnu.org ([208.118.235.92]:44100) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnUt-0000D4-Vk for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:33:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWnUk-0008BN-1A for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:33:46 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35058) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWnUj-0008BJ-U7; Mon, 08 Aug 2016 12:33:41 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4780 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWnUi-0007CZ-0E; Mon, 08 Aug 2016 12:33:40 -0400 Date: Mon, 08 Aug 2016 19:33:23 +0300 Message-Id: <83k2fri5kc.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-reply-to: (message from Lars Ingebrigtsen on Mon, 08 Aug 2016 18:18:18 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Lars Ingebrigtsen > Cc: dgutov@yandex.ru, stakemorii@gmail.com, 24117@debbugs.gnu.org > Date: Mon, 08 Aug 2016 18:18:18 +0200 > > That you have to encode the data returned before doing network stuff > (which is what the RFC talks about) is a completely different matter. But no one does. So evidently, leaving this to applications just makes us buggy. > The function is used to decompose URLs found in nature. Those URLs > contain non-ASCII characters. URLs found in nature are all unibyte strings. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:34:24 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:34:24 +0000 Received: from localhost ([127.0.0.1]:60271 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnVO-0000EO-Vf for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:34:24 -0400 Received: from eggs.gnu.org ([208.118.235.92]:44172) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWnVK-0000E8-4M for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:34:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWnVE-0008K3-4G for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:34:13 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35060) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWnVE-0008Jx-1A; Mon, 08 Aug 2016 12:34:12 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4782 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWnVC-0008Gl-4n; Mon, 08 Aug 2016 12:34:10 -0400 Date: Mon, 08 Aug 2016 19:33:54 +0300 Message-Id: <83invbi5jh.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-reply-to: (message from Lars Ingebrigtsen on Mon, 08 Aug 2016 18:21:37 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Lars Ingebrigtsen > Cc: dgutov@yandex.ru, stakemorii@gmail.com, 24117@debbugs.gnu.org > Date: Mon, 08 Aug 2016 18:21:37 +0200 > > (And it is perfectly valid for the domain bits of URLs to contain > non-ASCII characters after the IDNA changeover in the RFCs.) They must be unibyte. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 12:59:08 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 16:59:08 +0000 Received: from localhost ([127.0.0.1]:60283 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWntM-0000mD-Jp for submit@debbugs.gnu.org; Mon, 08 Aug 2016 12:59:08 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:36647) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWntK-0000ls-Qb for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 12:59:07 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWntD-0005I4-O9; Mon, 08 Aug 2016 18:59:04 +0200 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83invbi5jh.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEXd1st5cmxZUk+lnpYO CQg7NTMgGxn9+/V9J90pAAACT0lEQVQ4jV2TzW7rIBCFJ2LhbQcr3UNE1lB0+wRc70OFH8DCrC1l kde/Z3DcpneiJM755o8ZQn54/NgW/OOkAluraaQX8CAaaDwFay2Z+EI2tZFykK0jM3p6AcO9FMNn gGyiJzqYGrazeQvnVQDHGGlvYYsP/6e8KV0sWcOBWaIkU/CadWbWAJbFIhB0Fhm+2gnQeBqFqKCt K6iuO4ABBGE4AMCXiuEAdk+HIyMgk2J9ANMrseiljMFoCpb3ZKwFSImSw40RxdKviGhmB8WakVCR eURlqGKimzgyxeEe3hQmuSk+cklbMqbNY0bPKZ60s+4SmTF/mdOxrW1Q2sT4AbC9blA2RUpOGn/t rwOZtQrj/0DIKGG/QL8YEkPUd3eUOd3kYSPf9bB/Ebw+l0GKRxTHDrTvYgerH+T64JxupfDsgwWM ytM9G81uIfYChvsOxCXLGhbqdyEAxPjecuSoSrYlL+RuGOvot6j8e1tlxy6X0hYqJZwRLMO/tpYt h1Lm1iaA7ErfhMNvpNEdJCr7yvpna2s2kqilRk1sFa9VAFzgXycBU6uLK3P6W54eUFolPM8NoKXF tW+rlaAjB1KkddfWuaXasnSlEd24FIkFrSlNC9O6WglNazHW7jFpulkAKdWW/AHHNKVuUyszyRd8 Lp8pSQ55ZWNrou4Cj/RtmPF72kH7UZPNHD+g0DNtqlWeJpw8u4qqe6rUGxNdbm9azvUAvXJaLa4n 14XnVOkn+8TyN+TzdAnXoys4z/kSxfg6fV3Ts6u0jKQg4o0a8yTgH32kFKC6XxR1AAAAAElFTkSu QmCC Date: Mon, 08 Aug 2016 18:58:23 +0200 In-Reply-To: <83invbi5jh.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Aug 2016 19:33:54 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Eli Zaretskii writes: >> (And it is perfectly valid for the domain bits of URLs to contain >> non-ASCII characters after the IDNA changeover in the RFCs.) > > They must be unibyte. I have no idea what you mean. If we have an instance, we have to decompose the URL into the domain part (g=C3=B3ogle.com) and the local part (/foo), and then connect to the domain part (after IDNA encoding) and issue "GET /foo". If the decomposition function barfs on the URL, then we can't make the connection. Anyway, I won't have much time to carry on bickering here, so do whatever you want to break the way Emacs handles URLs. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 13:11:48 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 17:11:48 +0000 Received: from localhost ([127.0.0.1]:60292 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWo5c-00014v-3M for submit@debbugs.gnu.org; Mon, 08 Aug 2016 13:11:48 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:45285) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWo5a-00014k-8u for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 13:11:46 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3s7P8d1HYfz3hjkG; Mon, 8 Aug 2016 19:11:43 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.68]) by mail.m-online.net (Postfix) with ESMTP id 3s7P8b3x2QzvkDG; Mon, 8 Aug 2016 19:11:43 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.68]) (amavisd-new, port 10024) with ESMTP id SZu53GmZLpJ7; Mon, 8 Aug 2016 19:11:42 +0200 (CEST) X-Auth-Info: SdcMHIZvCoVS6dXqND4uvVPbFS1zmjYK18ycRXPd9T+qNDv2tc31UVQY+Iz5x6hX Received: from igel.home (ppp-88-217-18-74.dynamic.mnet-online.de [88.217.18.74]) by mail.mnet-online.de (Postfix) with ESMTPA; Mon, 8 Aug 2016 19:11:42 +0200 (CEST) Received: by igel.home (Postfix, from userid 1000) id 07D922C00D7; Mon, 8 Aug 2016 19:11:42 +0200 (CEST) From: Andreas Schwab To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> X-Yow: Why don't you ever enter any CONTESTS, Marvin?? Don't you know your own ZIPCODE? Date: Mon, 08 Aug 2016 19:11:41 +0200 In-Reply-To: <83k2fri5kc.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Aug 2016 19:33:23 +0300") Message-ID: <87oa53i3si.fsf@linux-m68k.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Lars Ingebrigtsen , 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Mo, Aug 08 2016, Eli Zaretskii wrote: > URLs found in nature are all unibyte strings. http://góogle.com/fóo isn't unibyte. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 13:12:14 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 17:12:14 +0000 Received: from localhost ([127.0.0.1]:60296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWo5x-00015r-CE for submit@debbugs.gnu.org; Mon, 08 Aug 2016 13:12:14 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54434) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWo5w-00015f-Fq for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 13:12:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWo5n-00009d-OT for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 13:12:03 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35523) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWo5n-00008v-Kz; Mon, 08 Aug 2016 13:11:59 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4819 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWo5l-00034M-OT; Mon, 08 Aug 2016 13:11:58 -0400 Date: Mon, 08 Aug 2016 20:11:42 +0300 Message-Id: <83eg5zi3sh.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-reply-to: (message from Lars Ingebrigtsen on Mon, 08 Aug 2016 18:58:23 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83invbi5jh.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Lars Ingebrigtsen > Cc: dgutov@yandex.ru, stakemorii@gmail.com, 24117@debbugs.gnu.org > Date: Mon, 08 Aug 2016 18:58:23 +0200 > > Eli Zaretskii writes: > > >> (And it is perfectly valid for the domain bits of URLs to contain > >> non-ASCII characters after the IDNA changeover in the RFCs.) > > > > They must be unibyte. > > I have no idea what you mean. What I said: the URL strings must be unibyte strings. Then they will still work in url-generic-parse-url, and the problems which started this bug report won't happen. > If we have an instance, we have to > decompose the URL into the domain part (góogle.com) and the local part > (/foo), and then connect to the domain part (after IDNA encoding) and > issue "GET /foo". You can do all that with unibyte strings. > If the decomposition function barfs on the URL, then we can't make > the connection. No one intends to make that function barf. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 13:31:29 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 17:31:29 +0000 Received: from localhost ([127.0.0.1]:60304 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWoOa-0001YU-6e for submit@debbugs.gnu.org; Mon, 08 Aug 2016 13:31:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:58728) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWoOV-0001YE-Ko for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 13:31:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWoOM-0003Y5-J7 for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 13:31:14 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35740) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWoOB-0003U2-QP; Mon, 08 Aug 2016 13:30:59 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4874 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWoO9-0003ri-2Y; Mon, 08 Aug 2016 13:30:59 -0400 Date: Mon, 08 Aug 2016 20:30:33 +0300 Message-Id: <83bn13i2x2.fsf@gnu.org> From: Eli Zaretskii To: Andreas Schwab In-reply-to: <87oa53i3si.fsf@linux-m68k.org> (message from Andreas Schwab on Mon, 08 Aug 2016 19:11:41 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Andreas Schwab > Cc: Lars Ingebrigtsen , stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru > Date: Mon, 08 Aug 2016 19:11:41 +0200 > > On Mo, Aug 08 2016, Eli Zaretskii wrote: > > > URLs found in nature are all unibyte strings. > > http://góogle.com/fóo isn't unibyte. Every string outside of Emacs is unibyte. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 15:16:36 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 19:16:36 +0000 Received: from localhost ([127.0.0.1]:60333 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWq2J-0003vv-GV for submit@debbugs.gnu.org; Mon, 08 Aug 2016 15:16:36 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:60818) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWq2E-0003vk-Ss for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 15:16:30 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3s7RwT3l4Pz3hjMt; Mon, 8 Aug 2016 21:16:25 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.68]) by mail.m-online.net (Postfix) with ESMTP id 3s7RwT2JXJzvkFS; Mon, 8 Aug 2016 21:16:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.68]) (amavisd-new, port 10024) with ESMTP id hUNQbrHr2ZL3; Mon, 8 Aug 2016 21:16:21 +0200 (CEST) X-Auth-Info: T9M+4/4wK/gXeeeGDj3yTVt5wx8x0WSRssVVBae9Zg3bRfIBHDsXjfrn5t+am+9k Received: from igel.home (ppp-88-217-18-74.dynamic.mnet-online.de [88.217.18.74]) by mail.mnet-online.de (Postfix) with ESMTPA; Mon, 8 Aug 2016 21:16:21 +0200 (CEST) Received: by igel.home (Postfix, from userid 1000) id 157482C25AB; Mon, 8 Aug 2016 21:16:21 +0200 (CEST) From: Andreas Schwab To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> X-Yow: Oh my GOD -- the SUN just fell into YANKEE STADIUM!! Date: Mon, 08 Aug 2016 21:16:21 +0200 In-Reply-To: <83bn13i2x2.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Aug 2016 20:30:33 +0300") Message-ID: <87fuqfhy0q.fsf@linux-m68k.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Mo, Aug 08 2016, Eli Zaretskii wrote: >> From: Andreas Schwab >> Cc: Lars Ingebrigtsen , stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru >> Date: Mon, 08 Aug 2016 19:11:41 +0200 >> >> On Mo, Aug 08 2016, Eli Zaretskii wrote: >> >> > URLs found in nature are all unibyte strings. >> >> http://góogle.com/fóo isn't unibyte. > > Every string outside of Emacs is unibyte. This string is inside Emacs. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 15:46:48 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 19:46:48 +0000 Received: from localhost ([127.0.0.1]:60344 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWqVc-0004dh-7d for submit@debbugs.gnu.org; Mon, 08 Aug 2016 15:46:48 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:38154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWqVa-0004dV-5L for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 15:46:46 -0400 Received: by mail-wm0-f43.google.com with SMTP id o80so161254805wme.1 for <24117@debbugs.gnu.org>; Mon, 08 Aug 2016 12:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=+gwErTMlYK9dy2bsfYS+r3Sx29PK3NVXoppf3CiCkRs=; b=La/RMezCHOAnByCZo9smyLRDv9+q6O1mPfION4KLN3/nvSMkUNu/g80ajF2Z9D7gmy 4tqu+snto6CNTxZPZLW3kpBhFvj2VvZSsTeOTDkJH+B3pJhCeCWOBjOPvzoSxpu+Yy8R K9rikhAI2xvS7QTBysxzfL2IKbCYxmgokxGUpvwcbiREqnkExA2uxfpHL0dHi0yBm0Dr q5IJILJ/7BtUa37a9MPcANwgJPcbOlOQipeeja1x++mLZou2Tmt5etEMGHZsWOhDl8El 5aBlVKda79qOs0dKVn4a+C+mf42J+sroORTqcuFbt6A8tTWLSdFeQOUVFkyLLvrdbpw2 rY0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=+gwErTMlYK9dy2bsfYS+r3Sx29PK3NVXoppf3CiCkRs=; b=SFfEPZAHQJCdtmU9/qgJWulxeDlXxk+BDpnTIiIziUyB5oS31bEie8M+b7jhu3mVvF NeRkYqnu9v4EoKNE0G/4MVjVsbmPNFOErlvzPjGv8cXcJElko3ZOVOkp+fqXwF4XyRb4 A8V9JB2uGqBQep6lzcmXDzszdgvGSDJCZ3AC/ULB+ZegPrnGlZAL44Baho19euvshnO8 8Sw0mA4nyQpQSwu+3anAUtUTRFINf2i8n5qE2888UzLxlLVIv9AW67dgVKE/munyvzWL tJMDLaHr9hNznTOeJj4qXfvcsd79y3cVSQz7bdxT4eIeM+88JIB08e2L9lsRobjT2B8H Jd5A== X-Gm-Message-State: AEkoouuaP2fnc1fTWtdhU+ihrv3VeUrad1Igr6tMq38JF8/dTYNWSTiCRers39zz3Rq9hQ== X-Received: by 10.25.214.40 with SMTP id n40mr29459711lfg.105.1470685600340; Mon, 08 Aug 2016 12:46:40 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id h62sm5951032lji.28.2016.08.08.12.46.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Aug 2016 12:46:39 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> From: Dmitry Gutov Message-ID: <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> Date: Mon, 8 Aug 2016 22:46:37 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/08/2016 06:54 PM, Lars Ingebrigtsen wrote: > It's be sad if > > (url-generic-parse-url "http://góogle.com/fóo") > > stopped working. Why don't you pass the url string through `url-encode-url' first? url-retrieve-internal does that. > This function is Emacs' workhorse for chopping up URLs, and it's a very > useful function. There's a bunch of hand-rolled URL parsing > implementations that don't quite work right. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 16:20:40 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 20:20:40 +0000 Received: from localhost ([127.0.0.1]:60362 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWr2L-0005Rf-6E for submit@debbugs.gnu.org; Mon, 08 Aug 2016 16:20:40 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:39501) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWr2J-0005RX-BG for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 16:20:35 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWr2D-0007cb-Tm; Mon, 08 Aug 2016 22:20:31 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAHlBMVEUnbV1GjsA0r/07sP0t mu8un/VJpvU4uP47s/03rfy8tEg2AAACL0lEQVQ4jW1UsY7iMBAdn2BrG0FqGIhi2qMIZTaCD7go 9KTxB0RaxXU4KS5PbAH87c3YCbC7PFFE8+bNm7HHwP7wEiUc8Iw/44iAm80N10iIn+MJfJ42n7i+ 3W5Pwrgg4nraXM/rG/0eEtbD1f3GgPhRq2CPAMr6bj4gfkUUXvSVuNSD5BuRWveaSKxztZ8wjv8I tb+3e7Fd04bRMZNKMYU8+cU623JU40ooVSkFM0+Qh60DkUXmzVRKzgoeMG1s04ZBhDEqMkoFj9R2 XehrHhlzNJWowpF0XMozyhgqpEwVzLfOuVbT54IERlK56SFmguJdrbFFzqVujZndFR0r4khyVBoz 4boN3QcBdTtXkk2OUeWJGpZMaH0Czif3qMqLQdERoxR7E94qunFM+lJumfVxo44YPLjdk9MLEUqZ oywec7humbfblW/2OPXXSetjXdM1Okc+MOLkuAil0oZg54CufSwMNj3h2om2+ATqKrUk0Eme9EsR Y99Vx5Vc8m9S35crDEjTkQJF8diUMhNsThJL9yrvS1cCAHm4ptEt6tV4WFKOy96jJsWuJ3YAI6/g dnmCnX8G5ZTyx7TLoV0mcnZQcgRiHI4k9aU8Ue4lCDHy7dKKeiLx3xMJMAtx9ObMJTUQkYXNHYh6 RaktQFHEQ9yvqFsoEJC/78KAhT8vuHyojFociV/x8HS8DLZ/QQpQdFZfXyfNoeaj/RAty+Glw9a+ 33N3HuH/CM4cLXc/8B/HkSq5n8VG8wAAAABJRU5ErkJggg== Date: Mon, 08 Aug 2016 22:19:53 +0200 In-Reply-To: <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> (Dmitry Gutov's message of "Mon, 8 Aug 2016 22:46:37 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.6 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Dmitry Gutov writes: >> (url-generic-parse-url "http://góogle.com/fóo") >> >> stopped working. > > Why don't you pass the url string through `url-encode-url' first? > > url-retrieve-internal does that. [...] Content analysis details: (1.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [80.91.224.195 listed in list.dnswl.org] 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.6 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Dmitry Gutov writes: >> (url-generic-parse-url "http://góogle.com/fóo") >> >> stopped working. > > Why don't you pass the url string through `url-encode-url' first? > > url-retrieve-internal does that. [...] Content analysis details: (1.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [80.91.224.195 listed in list.dnswl.org] 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname Dmitry Gutov writes: >> (url-generic-parse-url "http://g=C3=B3ogle.com/f=C3=B3o") >> >> stopped working. > > Why don't you pass the url string through `url-encode-url' first? > > url-retrieve-internal does that. (url-encode-url "http://g=C3=B3ogle.com/f=C3=B3o") =3D> "http://g=C3=B3ogle.com/f%C3%B3o" --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 16:35:41 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 20:35:41 +0000 Received: from localhost ([127.0.0.1]:60385 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWrGv-0005pn-LE for submit@debbugs.gnu.org; Mon, 08 Aug 2016 16:35:41 -0400 Received: from mail-wm0-f51.google.com ([74.125.82.51]:35940) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWrGu-0005pb-4x for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 16:35:40 -0400 Received: by mail-wm0-f51.google.com with SMTP id q128so140647068wma.1 for <24117@debbugs.gnu.org>; Mon, 08 Aug 2016 13:35:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=4GYFq7l1DBhqGEoHdU/VqEjyARRUYvoMNLlD6Ee/0gQ=; b=TsvsQ0aG4LlyN48kBcDEKYHwSJeL+cA81BO7K2icXp1qF9l497eZRW0ZqxNBvHJVkV ILbdXwTMxpT+VmfNJuawaa5swmeoqPwCCjiDwOWv0+Crc947H8nYaOx4Xo62Is1Z0AhS zRw56v7JcuMkAl0ieeuG+J1r1i8SWpH3QxQwAR6Fzkg0OfkdXE7dQoWOfl+dnlIkLkDr xGmy9CidzZGJI1vfXFrr1qT+ZHjWcYRUn3bO1tzyW74oDuBPCl4yOiPIygeqilyekwPn y5hFHi7mgaauQPk7xDIWSv5Noe7DwnjAqsnhT8iq79bc+ELXP7J94SUxSFJPRVeVzu5p sjXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=4GYFq7l1DBhqGEoHdU/VqEjyARRUYvoMNLlD6Ee/0gQ=; b=k0psx8tITimkwyOWrZMlWYVG4tsigY0rT5a9Uo1WIuj7B7v+uSxeKzvO4l3fumMecH wblSsv/KXxYAdTPs/5dgyg/++CdTNPwm4+oTDYRg/IEjvjlcoOXDgpVjmOyJNdILuie7 uKTRPP2Wy/S8V13WLnPIFp9HxhXMzjQH7qhIU6Js+5pgcUft4+j1r4uk2P8x+XDvviIm NOZZcLnmayeNkhJ+/nYVkSiORPi/7a+5GvLN+Q60e4/NDhROPQmtzGnSoO/XBf0GrTtl MaDF8Scx4kNH5zOSZLYKZkhJCLBaVlBkd5JBrzvJaSg0lr6Xe1IJ4m7QdHWdJm/Yevcc rgKw== X-Gm-Message-State: AEkoout3aQcEOniPtqyJPR8voL/6Oc8bUetSqnkh3HA4YueyK9tfJfDaTZAgLtrCa6iCCA== X-Received: by 10.25.16.162 with SMTP id 34mr30742141lfq.127.1470688534143; Mon, 08 Aug 2016 13:35:34 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id h88sm5993473ljh.23.2016.08.08.13.35.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Aug 2016 13:35:33 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> From: Dmitry Gutov Message-ID: <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> Date: Mon, 8 Aug 2016 23:35:32 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > (url-encode-url "http://góogle.com/fóo") > => "http://góogle.com/f%C3%B3o" That is true in master, but not in emacs-25, AFAICS. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.51 listed in list.dnswl.org] 0.6 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [178.252.127.239 listed in dnsbl.sorbs.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (dgutov[at]yandex.ru) -0.0 SPF_PASS SPF: sender matches SPF record 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.51 listed in wl.mailspike.net] 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > (url-encode-url "http://góogle.com/fóo") > => "http://góogle.com/f%C3%B3o" That is true in master, but not in emacs-25, AFAICS. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.51 listed in list.dnswl.org] 0.6 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [178.252.127.239 listed in dnsbl.sorbs.net] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.51 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (raaahh[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > (url-encode-url "http://góogle.com/fóo") > => "http://góogle.com/f%C3%B3o" That is true in master, but not in emacs-25, AFAICS. (Is that related to your work on punycode?) On emacs-25, it returns "http://g%C3%B3ogle.com/f%C3%B3o". From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 16:37:40 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 20:37:41 +0000 Received: from localhost ([127.0.0.1]:60393 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWrIn-0005sw-6o for submit@debbugs.gnu.org; Mon, 08 Aug 2016 16:37:40 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:39659) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWrIl-0005so-Bm for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 16:37:35 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bWrIi-0007l5-J7; Mon, 08 Aug 2016 22:37:34 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAHlBMVEUnbV1GjsA0r/07sP0t mu8un/VJpvU4uP47s/03rfy8tEg2AAACL0lEQVQ4jW1UsY7iMBAdn2BrG0FqGIhi2qMIZTaCD7go 9KTxB0RaxXU4KS5PbAH87c3YCbC7PFFE8+bNm7HHwP7wEiUc8Iw/44iAm80N10iIn+MJfJ42n7i+ 3W5Pwrgg4nraXM/rG/0eEtbD1f3GgPhRq2CPAMr6bj4gfkUUXvSVuNSD5BuRWveaSKxztZ8wjv8I tb+3e7Fd04bRMZNKMYU8+cU623JU40ooVSkFM0+Qh60DkUXmzVRKzgoeMG1s04ZBhDEqMkoFj9R2 XehrHhlzNJWowpF0XMozyhgqpEwVzLfOuVbT54IERlK56SFmguJdrbFFzqVujZndFR0r4khyVBoz 4boN3QcBdTtXkk2OUeWJGpZMaH0Czif3qMqLQdERoxR7E94qunFM+lJumfVxo44YPLjdk9MLEUqZ oywec7humbfblW/2OPXXSetjXdM1Okc+MOLkuAil0oZg54CufSwMNj3h2om2+ATqKrUk0Eme9EsR Y99Vx5Vc8m9S35crDEjTkQJF8diUMhNsThJL9yrvS1cCAHm4ptEt6tV4WFKOy96jJsWuJ3YAI6/g dnmCnX8G5ZTyx7TLoV0mcnZQcgRiHI4k9aU8Ue4lCDHy7dKKeiLx3xMJMAtx9ObMJTUQkYXNHYh6 RaktQFHEQ9yvqFsoEJC/78KAhT8vuHyojFociV/x8HS8DLZ/QQpQdFZfXyfNoeaj/RAty+Glw9a+ 33N3HuH/CM4cLXc/8B/HkSq5n8VG8wAAAABJRU5ErkJggg== Date: Mon, 08 Aug 2016 22:36:56 +0200 In-Reply-To: <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> (Dmitry Gutov's message of "Mon, 8 Aug 2016 23:35:32 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.6 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Dmitry Gutov writes: > On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > >> (url-encode-url "http://góogle.com/fóo") >> => "http://góogle.com/f%C3%B3o" > > That is true in master, but not in emacs-25, AFAICS. > > (Is that related to your work on punycode?) [...] Content analysis details: (1.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [80.91.224.195 listed in list.dnswl.org] 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.6 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Dmitry Gutov writes: > On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > >> (url-encode-url "http://góogle.com/fóo") >> => "http://góogle.com/f%C3%B3o" > > That is true in master, but not in emacs-25, AFAICS. > > (Is that related to your work on punycode?) [...] Content analysis details: (1.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [80.91.224.195 listed in list.dnswl.org] 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname Dmitry Gutov writes: > On 08/08/2016 11:19 PM, Lars Ingebrigtsen wrote: > >> (url-encode-url "http://g=C3=B3ogle.com/f=C3=B3o") >> =3D> "http://g=C3=B3ogle.com/f%C3%B3o" > > That is true in master, but not in emacs-25, AFAICS. > > (Is that related to your work on punycode?) Might be; I can't recall, though. > On emacs-25, it returns "http://g%C3%B3ogle.com/f%C3%B3o". Which is, of course, completely wrong. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 19:48:53 2016 Received: (at 24117) by debbugs.gnu.org; 8 Aug 2016 23:48:54 +0000 Received: from localhost ([127.0.0.1]:60460 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWuHt-0001tY-NI for submit@debbugs.gnu.org; Mon, 08 Aug 2016 19:48:53 -0400 Received: from mail-hampton.hostforweb.net ([205.234.186.191]:53988 helo=hampton.hostforweb.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWuHr-0001tK-6N for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 19:48:52 -0400 Received: from s70.gtokyofl21.vectant.ne.jp ([202.215.75.70]:62343 helo=localhost) by hampton.hostforweb.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87) (envelope-from ) id 1bWuHi-003pHh-VC; Mon, 08 Aug 2016 18:48:44 -0500 Date: Tue, 09 Aug 2016 08:48:44 +0900 Message-ID: From: Katsumi Yamaoka To: dgutov@yandex.ru Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <871t1zcrnn.fsf@lifelogs.com> Organization: Emacsen advocacy group X-Face: #kKnN,xUnmKia.'[pp`; Omh}odZK)?7wQSl"4o04=EixTF+V[""w~iNbM9ZL+.b*_CxUmFk B#Fu[*?MZZH@IkN:!"\w%I_zt>[$nm7nQosZ<3eu; B:$Q_:p!',P.c0-_Cy[dz4oIpw0ESA^D*1Lw= L&i*6&( User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (i686-pc-cygwin) Cancel-Lock: sha1:dFqt/FLgRdnfB3LsFtzsffFBGwc= MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hampton.hostforweb.net X-AntiAbuse: Original Domain - debbugs.gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - jpl.org X-Get-Message-Sender-Via: hampton.hostforweb.net: authenticated_id: yamaoka/from_h X-Authenticated-Sender: hampton.hostforweb.net: yamaoka@jpl.org X-Source: X-Source-Args: X-Source-Dir: X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Mon, 08 Aug 2016 09:32:44 -0400, Ted Zlatanov wrote: > CC-ed Katsumi Yamaoka since he may have some feedback as well. Well, I'm not familiar with url-*.el but there seems to be no function to encode a url containing non-ASCII characters such as: "http://=D0=B1=D0=B0=D0=BD=D0=BA=D0=B8.=D1=80=D1=84/" In emacs-w3m[1] there are some encoder functions: ;; An encoder used for a full url string: (w3m-url-transfer-encode-string "http://=D0=B1=D0=B0=D0=BD=D0=BA=D0=B8.=D1= =80=D1=84/") =3D> "http://xn--80abwho.xn--p1ai/" ;; An encoder used for a string that is a part of url: (concat "https://ja.wikipedia.org/wiki/" (w3m-url-encode-string "=E6=97=A5=E6=9C=AC=E8=AA=9E=E3=83=89=E3=83=A1=E3= =82=A4=E3=83=B3=E5=90=8D")) =3D> "https://ja.wikipedia.org/wiki/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%89%E= 3%83%A1%E3%82%A4%E3%83%B3%E5%90%8D" Those two encoded urls will work. [1] ,---- | % cvs -d :pserver:anonymous@cvs.namazu.org:/storage/cvsroot login | CVS password: # No password is set. Just hit Enter/Return key. | % cvs -d :pserver:anonymous@cvs.namazu.org:/storage/cvsroot co emacs-w3m `---- From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 22:13:15 2016 Received: (at 24117) by debbugs.gnu.org; 9 Aug 2016 02:13:15 +0000 Received: from localhost ([127.0.0.1]:60559 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWwXb-0000BL-0b for submit@debbugs.gnu.org; Mon, 08 Aug 2016 22:13:15 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:38398) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWwXZ-0000B5-TN for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 22:13:14 -0400 Received: by mail-wm0-f44.google.com with SMTP id o80so4016408wme.1 for <24117@debbugs.gnu.org>; Mon, 08 Aug 2016 19:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=LnGl+C8brFcLNwlQOKg82O34S/UOhZyBt/6ZRXWXms8=; b=jkBGi92TnkbzIioAHhS5WxNqo3Fn3xFX3xJUtY7wjJeUqmgzmJEidGIGYua9ynVYWg sbPGWFmn2DL2Ki5fa60FtNnpcKE5m+TS7euNdr/gK8abCWJj8A7GTuVf3RTc3C4xznjr d+yqAndyht7tvbV+tr7rW+fJnJQ/h9C9hpBkm5BuKHQifTb8liHC79MRrqp6mvQ0/YqH wVPRZHXLKQ8YO8HbCqEdZ9wTa7fevE2jXHO/1FwMhsMzncFogUyUylWfWshVqna9VQWn +YEfx+zlQzUz2hPnubTu8tDvqolRtsWkSLkryzHS/lTSc7WpAh2woJPu6CkmNXHOEL4E JOjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=LnGl+C8brFcLNwlQOKg82O34S/UOhZyBt/6ZRXWXms8=; b=FNkSaE4wFLLj/1wv7dUVTM1qlfFJakknxELH2TSd7mdaBEj1tgUM1uDUnIYgzWLE8c LTddbMRhySiJVisF6nVqZvXSxAm1udktFWXnnjdmjrGgsEj8PoSDK3v1eKjt0lhLwksZ l5V5SXXCFRluzJhY5GUbnndUtwCEb7DGBGIpBP8GGTLSu59VEbD3usbs9rbS90yGbJwx q5dXRAFgvZw1jmAnPg81gGfxFiw7J21SkNeksthfHaF7zs89FalOg/lZl1ZdyYMO5bhW 5FYV8oXp5SiBTqK/PNnidiMKq0t0Z4jB8hE3NeYKfwZbX/Bh22g68IM0QTnT2TA/F1OW y4pA== X-Gm-Message-State: AEkooutTBenGzmOQKzMJeGNvyJjT3hwEwIzqje13AfFG9xxFIkolIzxmoAmDBQSWmVOPHg== X-Received: by 10.25.89.65 with SMTP id n62mr30096682lfb.89.1470708788093; Mon, 08 Aug 2016 19:13:08 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id l10sm6005413lfd.19.2016.08.08.19.13.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Aug 2016 19:13:07 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> From: Dmitry Gutov Message-ID: Date: Tue, 9 Aug 2016 05:13:04 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/08/2016 11:36 PM, Lars Ingebrigtsen wrote: >>> (url-encode-url "http://góogle.com/fóo") >>> => "http://góogle.com/f%C3%B3o" >> >> That is true in master, but not in emacs-25, AFAICS. >> >> (Is that related to your work on punycode?) > > Might be; I can't recall, though. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.6 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [178.252.127.239 listed in dnsbl.sorbs.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (dgutov[at]yandex.ru) -0.0 SPF_PASS SPF: sender matches SPF record -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.44 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.44 listed in wl.mailspike.net] 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/08/2016 11:36 PM, Lars Ingebrigtsen wrote: >>> (url-encode-url "http://góogle.com/fóo") >>> => "http://góogle.com/f%C3%B3o" >> >> That is true in master, but not in emacs-25, AFAICS. >> >> (Is that related to your work on punycode?) > > Might be; I can't recall, though. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.44 listed in list.dnswl.org] 0.6 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [178.252.127.239 listed in dnsbl.sorbs.net] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.44 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (raaahh[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.6 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid On 08/08/2016 11:36 PM, Lars Ingebrigtsen wrote: >>> (url-encode-url "http://góogle.com/fóo") >>> => "http://góogle.com/f%C3%B3o" >> >> That is true in master, but not in emacs-25, AFAICS. >> >> (Is that related to your work on punycode?) > > Might be; I can't recall, though. Here's another question: why does url-encode-url pass the argument through encode-coding-string before passing it to url-generic-parse-url, if the latter is expected to be able to deal with non-ASCII characters? The only recent change in that function is your commit 8b61c22e dated last December, which very much looks like a band-aid in this context. >> On emacs-25, it returns "http://g%C3%B3ogle.com/f%C3%B3o". > > Which is, of course, completely wrong. I see. Since you're better versed in this area than me, can you propose a specific fix for the currently discussed bug? It is more serious than not being able to use unicode in URLs. On master, the domain part, which is untouched by url-encode-url, is converted to an ASCII unibyte string with puny-encode-domain, inside url-http-create-request. But real-fname remains a multibyte string, triggering the problem anyway. From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 08 22:33:17 2016 Received: (at 24117) by debbugs.gnu.org; 9 Aug 2016 02:33:17 +0000 Received: from localhost ([127.0.0.1]:60568 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWwqu-0002F7-NV for submit@debbugs.gnu.org; Mon, 08 Aug 2016 22:33:17 -0400 Received: from eggs.gnu.org ([208.118.235.92]:35552) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWwqq-0002Es-6J for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 22:33:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWwqh-00005U-Ql for 24117@debbugs.gnu.org; Mon, 08 Aug 2016 22:33:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:43171) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWwqY-0008Tq-6v; Mon, 08 Aug 2016 22:32:50 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1259 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bWwqW-0008Bu-7N; Mon, 08 Aug 2016 22:32:48 -0400 Date: Tue, 09 Aug 2016 05:32:33 +0300 Message-Id: <837fbqise6.fsf@gnu.org> From: Eli Zaretskii To: Andreas Schwab In-reply-to: <87fuqfhy0q.fsf@linux-m68k.org> (message from Andreas Schwab on Mon, 08 Aug 2016 21:16:21 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Andreas Schwab > Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru > Date: Mon, 08 Aug 2016 21:16:21 +0200 > > On Mo, Aug 08 2016, Eli Zaretskii wrote: > > >> From: Andreas Schwab > >> Cc: Lars Ingebrigtsen , stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru > >> Date: Mon, 08 Aug 2016 19:11:41 +0200 > >> > >> On Mo, Aug 08 2016, Eli Zaretskii wrote: > >> > >> > URLs found in nature are all unibyte strings. > >> > >> http://góogle.com/fóo isn't unibyte. > > > > Every string outside of Emacs is unibyte. > > This string is inside Emacs. Then it should be made unibyte before parsing it. From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 09 04:05:49 2016 Received: (at 24117) by debbugs.gnu.org; 9 Aug 2016 08:05:49 +0000 Received: from localhost ([127.0.0.1]:60639 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX22i-0001Rc-28 for submit@debbugs.gnu.org; Tue, 09 Aug 2016 04:05:49 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:57555) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX22c-0001RR-Sj for 24117@debbugs.gnu.org; Tue, 09 Aug 2016 04:05:42 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3s7n0140P8z3hl17; Tue, 9 Aug 2016 10:05:36 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.68]) by mail.m-online.net (Postfix) with ESMTP id 3s7n003Z69zvkQ6; Tue, 9 Aug 2016 10:05:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.68]) (amavisd-new, port 10024) with ESMTP id VAyaDf_JvCf1; Tue, 9 Aug 2016 10:05:32 +0200 (CEST) X-Auth-Info: /eQGj7HfsBNGTySPArfKKc/lZcuehcf81iwYAs0UpOt3TVR0BN4nye+4ywWNSDCW Received: from hawking.suse.de (charybdis-ext.suse.de [195.135.221.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Tue, 9 Aug 2016 10:05:32 +0200 (CEST) From: Andreas Schwab To: Eli Zaretskii Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> X-Yow: Bagels... Date: Tue, 09 Aug 2016 10:05:31 +0200 In-Reply-To: <837fbqise6.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 09 Aug 2016 05:32:33 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Di, Aug 09 2016, Eli Zaretskii wrote: >> From: Andreas Schwab >> Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru >> Date: Mon, 08 Aug 2016 21:16:21 +0200 >> >> On Mo, Aug 08 2016, Eli Zaretskii wrote: >> >> >> From: Andreas Schwab >> >> Cc: Lars Ingebrigtsen , stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru >> >> Date: Mon, 08 Aug 2016 19:11:41 +0200 >> >> >> >> On Mo, Aug 08 2016, Eli Zaretskii wrote: >> >> >> >> > URLs found in nature are all unibyte strings. >> >> >> >> http://góogle.com/fóo isn't unibyte. >> > >> > Every string outside of Emacs is unibyte. >> >> This string is inside Emacs. > > Then it should be made unibyte before parsing it. You can't encode it properly without parsing it first. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 09 05:40:04 2016 Received: (at 24117) by debbugs.gnu.org; 9 Aug 2016 09:40:05 +0000 Received: from localhost ([127.0.0.1]:60683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX3W0-0003hD-MY for submit@debbugs.gnu.org; Tue, 09 Aug 2016 05:40:04 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:50260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX3Vy-0003gi-82 for 24117@debbugs.gnu.org; Tue, 09 Aug 2016 05:40:02 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bX3Vt-0000yI-MA; Tue, 09 Aug 2016 11:39:59 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAElBMVEVHQUFlXlwSEBGxqaUr KCg5NTWwNAYgAAACeElEQVQ4jX2TQY8zJwyGTRbuuIT7Lp29h2/UOxnZd0+K//9f6custlpVbYkS gR/br20I7f++Ov0H2P8H9H9YyjfgH6byt30nLj+8895L/6HRv2F97n1lACRsGOsL/Pq976VchErp 3LlwWWHs5bF3+BcmBgXBwnnbrkDYmArSwGelLv0Ya3O5EXPgVcmVuX3sOCx7Jqacv7V3fuv7Y4GS KTslxJSl1P25kpa9B6ZHSiWgxrwqOD7wG8qeuVJG9BtaQdPc2ztDDMXnSumEcWXtjz23917K0h6R ql2lgeUO8CgcQmCDsobsjAISAXGGLJ1MB9Fgd8I6mifmFEIyIjnIkCR6rtzau3N2M1c1AGjXRBA6 2gaQzKLoHAeFO4DRn73qKVz9dBVPiHirqK+juSrW3qOIi0xXRFCngFpQyGiNEir0OcfEroeQMcxw vKCf4EmUKBIpO+oL1Y5ba58JZgx8AWJL1+doWLDiLtwnUrCR4zwvkC1TkrmhD1zUajzJ64rwCPMm rwUSeUguc4GUECmCPhCQ58hThWD/iNI2uAAkkCmGxvICN8XO1V4AIag+ZQt3ALRuajrniiBWyyK3 XwskhZgsEENksbjp/GOlwjBUFCq4KAqGwWmMrb0WgP1+NDI7TUaU+5I4PkZS+fTVkOmoI6xp4zSf Rojwhs7pnJPyOEfcWrNsJKrS0DlCpmZJ21ggOKnPly7gShjyow30fMqzqqMOj4R55yokc4i08TrZ YQh8ww0q1Vw/Ua9tesMd8/pGinhuyeYnhFTsN+KvdafheC2pYbj5Jekevv5P4U4q26ZySApVNNVw +VPpJO3AhTUZlqNebx92gL8AeVOzm8XarqMAAAAASUVORK5CYII= Date: Tue, 09 Aug 2016 11:39:20 +0200 In-Reply-To: (Dmitry Gutov's message of "Tue, 9 Aug 2016 05:13:04 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Dmitry Gutov writes: > Here's another question: why does url-encode-url pass the argument > through encode-coding-string before passing it to > url-generic-parse-url, if the latter is expected to be able to deal > with non-ASCII characters? I don't know. I don't think `url-encode-url' has ever really worked in any sensible way in the presence of non-ASCII. > The only recent change in that function is your commit 8b61c22e dated > last December, which very much looks like a band-aid in this context. It's debatable what that function should return in the presence of non-ASCII domain names, but it's a debatable function all around. > Since you're better versed in this area than me, can you propose a > specific fix for the currently discussed bug? It is more serious than > not being able to use unicode in URLs. I didn't understand the original bug report and there was no simple recipe to reproduce the bug. Why changing url-generic-parse-url was proposed as a solution is even less unclear. Perhaps you could write a test case and summarise what you think the problem is? > On master, the domain part, which is untouched by url-encode-url, is > converted to an ASCII unibyte string with puny-encode-domain, inside > url-http-create-request. But real-fname remains a multibyte string, > triggering the problem anyway. The domain is encoded according to IDNA, which is an ASCII string, yes. (Whether the function returns a unibyte string or not I can't recall.) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 09 10:51:24 2016 Received: (at 24117) by debbugs.gnu.org; 9 Aug 2016 14:51:24 +0000 Received: from localhost ([127.0.0.1]:33256 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX8NC-0000Xu-RQ for submit@debbugs.gnu.org; Tue, 09 Aug 2016 10:51:24 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39001) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bX8N7-0000Xe-Vc for 24117@debbugs.gnu.org; Tue, 09 Aug 2016 10:51:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bX8My-0002Ja-Id for 24117@debbugs.gnu.org; Tue, 09 Aug 2016 10:51:08 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:51739) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bX8Mn-0002Cp-NY; Tue, 09 Aug 2016 10:50:53 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1746 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bX8Mk-0003Ym-I8; Tue, 09 Aug 2016 10:50:52 -0400 Date: Tue, 09 Aug 2016 17:50:32 +0300 Message-Id: <834m6uhu87.fsf@gnu.org> From: Eli Zaretskii To: Andreas Schwab In-reply-to: (message from Andreas Schwab on Tue, 09 Aug 2016 10:05:31 +0200) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.4 (/) > From: Andreas Schwab > Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru > Date: Tue, 09 Aug 2016 10:05:31 +0200 > > On Di, Aug 09 2016, Eli Zaretskii wrote: > > >> From: Andreas Schwab > >> Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org, dgutov@yandex.ru > >> Date: Mon, 08 Aug 2016 21:16:21 +0200 > >> > >> On Mo, Aug 08 2016, Eli Zaretskii wrote: > >> > >> >> From: Andreas Schwab > >> >> Cc: Lars Ingebrigtsen , stakemorii@gmail.com, 24117@debbugs.gnu.org, dgutov@yandex.ru > >> >> Date: Mon, 08 Aug 2016 19:11:41 +0200 > >> >> > >> >> On Mo, Aug 08 2016, Eli Zaretskii wrote: > >> >> > >> >> > URLs found in nature are all unibyte strings. > >> >> > >> >> http://góogle.com/fóo isn't unibyte. > >> > > >> > Every string outside of Emacs is unibyte. > >> > >> This string is inside Emacs. > > > > Then it should be made unibyte before parsing it. > > You can't encode it properly without parsing it first. You don't say what you meant by "encode properly". It's just a string, and there are ways to make a string unibyte without any parsing. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 10 11:39:10 2016 Received: (at 24117) by debbugs.gnu.org; 10 Aug 2016 15:39:11 +0000 Received: from localhost ([127.0.0.1]:52409 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVb1-000138-FO for submit@debbugs.gnu.org; Wed, 10 Aug 2016 11:39:10 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVaw-00012M-KE for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 11:39:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bXUbi-0000ux-3T for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 10:35:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41326) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXUbh-0000ut-W2; Wed, 10 Aug 2016 10:35:46 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3733 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bXUbg-0007UG-CU; Wed, 10 Aug 2016 10:35:45 -0400 Date: Wed, 10 Aug 2016 17:35:28 +0300 Message-Id: <83bn10hetr.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> (message from Dmitry Gutov on Wed, 10 Aug 2016 10:12:40 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Wed, 10 Aug 2016 10:12:40 +0300 > > On 08/09/2016 05:50 PM, Eli Zaretskii wrote: > > >> You can't encode it properly without parsing it first. > > > > You don't say what you meant by "encode properly". It's just a > > string, and there are ways to make a string unibyte without any > > parsing. > > Different parts of an URL are supposed to be encoded in different ways. > > For instance, > > http://банки.рф/фыва/ > > turns into > > http://xn--80abwho.xn--p1ai/%D1%84%D1%8B%D0%B2%D0%B0/ Are you saying that url-generic-parse-url performs this encoding, and that using a unibyte buffer causes that to fail? > So I think the encoding of the URL parts should be performed inside > url-http-create-request. Fine with me, but when I suggested that, you didn't like the suggestion. If you changed your mind, let's do that. > On the master branch, host is passed through IDNA encoding, but > real-fname is untouched. On emacs-25, I think we should convert both > to unibyte. Not sure I understand why there should be a difference between the two branches. Encoding an ASCII string doesn't do any harm. > Not sure encode-coding-string is the way to go (why would we assume > UTF-8?). Because using UTF-8 doesn't lose anything, you basically get the same byte stream as stored internally (because 8-bit bytes are not supposed to happen in URLs). > (Why doesn't (encode-coding-string "aaaa" 'ascii) work?) It's 'us-ascii, not 'ascii. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 10 11:45:40 2016 Received: (at 24117) by debbugs.gnu.org; 10 Aug 2016 15:45:41 +0000 Received: from localhost ([127.0.0.1]:52464 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVhM-00020S-Eo for submit@debbugs.gnu.org; Wed, 10 Aug 2016 11:45:40 -0400 Received: from mail-wm0-f54.google.com ([74.125.82.54]:36320) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVhK-0001sU-Vn for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 11:45:39 -0400 Received: by mail-wm0-f54.google.com with SMTP id q128so101911400wma.1 for <24117@debbugs.gnu.org>; Wed, 10 Aug 2016 08:45:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=OJYiqsExLZ9K8ic4bqat+XArjLKbIOxcn1tXQjgQuXk=; b=j2CDuP5qo5Cln3NXPvby81odxRuX4qxwLn0c3UlYcr7NjnmbSNLInAkkLDyD4FGVhx FGQN/ydPZpeiwdXQKRIIatuI+UG1LloIk0PuAH9NtPuPXeUT5/mFpOGMvzCM/RkyvTvY +ACW3rglTaQBsoEzE4motFQ9S+2FH/UDKueoOdh4fciOyTHOaHqjSmwyNVhAS4L729WG BDl/VpCV7UbL+24nQANLMNZ8oNuko83wQRXgZh4j8edg6WbsStQeSen76bvbXUxE7Frv YM1BJnXiF50aDI2px2cthMTC37GpOJT+hFjtefVigCyiNi2TWI1EArnGRFf2oG05+0od L/eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=OJYiqsExLZ9K8ic4bqat+XArjLKbIOxcn1tXQjgQuXk=; b=cT6Gdm1L+wW776JLxyGaYEQ/gpi0zhAyFLg59fqG28xy3XsQAGRVEOHkpizp969+hV paS4Ptnequ5ZU36AYOxrJC40px7VZoY5M3wZKs1EX06sMf8GDmWy4XOsKp6eqC7C18JQ E2TlARViBzJQX10LnArM54KbeRbkpg4mUo6S0r7rKYUBzEIyaJlNPb5Fbu10Nl5Vd+D5 ecXKR2YUXzUtFefCfVF5fI5oNMRZcStdAuSyI3fo0K6wEHfwt7Rt1ss29XuI3WXuTvP7 D5EDJk+PhWpVWbQwcR71pq51lJOYZCa5o2E1N7XH2iiOFFJ2VVPHuvg/aZ6ev9CJKtiZ LHcA== X-Gm-Message-State: AEkoouvXYuHCd2Uousas8EXwnUUZ+qND/qSwCVWEDN3mToD44XIlrqYLvnEoADDcatZRdA== X-Received: by 10.25.1.133 with SMTP id 127mr410566lfb.54.1470811817988; Tue, 09 Aug 2016 23:50:17 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id e123sm7284060lfg.16.2016.08.09.23.50.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Aug 2016 23:50:17 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> From: Dmitry Gutov Message-ID: <762be2e9-c82e-fa36-7d3f-0755e3bcdbaa@yandex.ru> Date: Wed, 10 Aug 2016 09:50:16 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/09/2016 12:39 PM, Lars Ingebrigtsen wrote: > I don't know. I don't think `url-encode-url' has ever really worked in > any sensible way in the presence of non-ASCII. My point is, you're saying that url-generic-parse-url should accept (and handle properly) multibyte URLs. But url-encode-url still encodes the URL string before passing it to url-generic-parse-url. > It's debatable what that function should return in the presence of > non-ASCII domain names, but it's a debatable function all around. The way the version in master works makes quite a bit of sense to me. > I didn't understand the original bug report and there was no simple > recipe to reproduce the bug. Why changing url-generic-parse-url was > proposed as a solution is even less unclear. Perhaps you could write a > test case and summarise what you think the problem is? Please try this: (with-current-buffer (let ((url-request-data (encode-coding-string "фыва" 'utf-8))) (url-retrieve-synchronously "http://posttestserver.com/post.php")) (buffer-string)) You'll get the "Multibyte text in HTTP request" error, which was added in a98aa02a5dbf079f7b4f3be5487a2f2b741d103d, to validate request data and make sure that users don't have to spend too much time investigating problems in their own code like bug#23750. But the added validation relied on the assumption that the situation with multibyte/unibyte strings that url-http-create-request acts on is somewhat sane, which is not true, as the current discussion has demonstrated. So we either need to straighten it up, or change the validation logic. If everything fails, of course, we can revert the aforementioned commit, but that would be bad for users. >> On master, the domain part, which is untouched by url-encode-url, is >> converted to an ASCII unibyte string with puny-encode-domain, inside >> url-http-create-request. But real-fname remains a multibyte string, >> triggering the problem anyway. > > The domain is encoded according to IDNA, which is an ASCII string, yes. > (Whether the function returns a unibyte string or not I can't recall.) It does. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 10 12:04:00 2016 Received: (at 24117) by debbugs.gnu.org; 10 Aug 2016 16:04:00 +0000 Received: from localhost ([127.0.0.1]:52482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVz5-0003Mg-Rz for submit@debbugs.gnu.org; Wed, 10 Aug 2016 12:04:00 -0400 Received: from mail-wm0-f47.google.com ([74.125.82.47]:37702) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXVz4-0003MO-6T for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 12:03:58 -0400 Received: by mail-wm0-f47.google.com with SMTP id i5so114197424wmg.0 for <24117@debbugs.gnu.org>; Wed, 10 Aug 2016 09:03:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=k3kDPLjHTTRZie8F9BNqDJXLh/ef+SAgKdVdGFhaLhg=; b=ZOO4g/q9gS1DK/eqVj9wSZ0f0TpPsXFwEHBC9HziUk/3yA3byqeldgeu1Sb/wO+y3D Aqtvr2BPaU02TYix8zbqXji33gtZFWNRZTq9dCWSUsBvQHMSZv49WooPLqOU03C8+krE dobKzwiQ7mpeV+FuSHSvVmScSiJX7Zn9dRXVGhYdK2dchMkNRbDPSHwGMG1d/l5GdZod /ZBXLtnTymUnPeqc2iUjjFo121LS5tnajJwoOUrwJRAhWyI/8rBEedkj0l4MDM+Hfkhe teOG+2CuLeSm7XAaq67IOc0M20d8hMbxagAVVgHCxvLvvio0CT/VKLUuGeIQymMJffil PCKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=k3kDPLjHTTRZie8F9BNqDJXLh/ef+SAgKdVdGFhaLhg=; b=Xr5CtsveU41C9JHAWBOq2SgOW7ttvEbIwxwRuz9waFUtfMP87Hmo64f3MwmPwjHWpg MlhS518ZfMkwg5/1JwUP5IkfZDw38H71q3tDiY7iokoqx9mnLNbku80pwGsQ7iH0Nu+/ 431Z7Dp7XVCBpNvoNFEQQlqqJmUcZw/voR6J2exqjiqdNaZfFCDcIoECkObCuRVwn78Q K+KPM64I3WlvwMuPaHwnfdII+tuqkEiqGJtspWUTQ5NvvtXCTU/hUd/SG/IpqetAG7Ii 7zw0o25QMwEqC+DcKgFOnkobXgy4RrNNye3Ma1fUYGXAXgIWgSr9KFqcqtw6Cmh05vmo VuUg== X-Gm-Message-State: AEkoouuNBTH/QqcaPujUxlLJiNN3YX17607iX5m94UKlacWY7+YDEeCy1DB48ayPtmShAg== X-Received: by 10.25.91.148 with SMTP id p142mr366313lfb.161.1470813162808; Wed, 10 Aug 2016 00:12:42 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id 29sm7247592lfu.43.2016.08.10.00.12.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Aug 2016 00:12:42 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii , Andreas Schwab References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> From: Dmitry Gutov Message-ID: <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> Date: Wed, 10 Aug 2016 10:12:40 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <834m6uhu87.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/09/2016 05:50 PM, Eli Zaretskii wrote: >> You can't encode it properly without parsing it first. > > You don't say what you meant by "encode properly". It's just a > string, and there are ways to make a string unibyte without any > parsing. Different parts of an URL are supposed to be encoded in different ways. For instance, http://банки.рф/фыва/ turns into http://xn--80abwho.xn--p1ai/%D1%84%D1%8B%D0%B2%D0%B0/ The domain is encoded with IDNA, whereas the path uses percent-encoding. And they're also often encoded separately (e.g. when you copy-paste the above URL from Firefox to a text editor, the result is http://банки.рф/%D1%84%D1%8B%D0%B2%D0%B0/). So I think the encoding of the URL parts should be performed inside url-http-create-request. On the master branch, host is passed through IDNA encoding, but real-fname is untouched. On emacs-25, I think we should convert both to unibyte. Not sure encode-coding-string is the way to go (why would we assume UTF-8?). Personally, using string-as-unibyte makes more sense (neither string should contain any multibyte characters at that point), but I defer to the more qualified colleagues. (Why doesn't (encode-coding-string "aaaa" 'ascii) work?) From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 10 21:31:14 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 01:31:14 +0000 Received: from localhost ([127.0.0.1]:52778 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXeq2-0004vf-Iq for submit@debbugs.gnu.org; Wed, 10 Aug 2016 21:31:14 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:35909) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXeq0-0004vT-Sg for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 21:31:13 -0400 Received: by mail-wm0-f43.google.com with SMTP id q128so2298598wma.1 for <24117@debbugs.gnu.org>; Wed, 10 Aug 2016 18:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=CwIOJM2v7Sb3Icqv2LdYQsUMQoJ2FT/Jj3AYeWJD/5g=; b=zx2MYHms12UNSNU+ituLaYkVnUevti0xXwcih5kwvEE+Y055tzUhb2m4MWlCXVKCep iC16Zorh5bVcVDUHbEi6lLWdZ+m0LUWRrGINCkERA5H8nfuIZyGDM2qN+PJXkVG53OL+ AMYulcTmcVvW3YFthjm5LFaR2GgANX+PC4OR6goFUwxG8MyaSoT/xs3Mph/qlfR2M66+ 11iT8xwDyNWiRs138phNVfBX72KBHSk7gIRqQcm+8C4iawONk6Klojw9Lx1MN5vd7+O5 HEtsHAtjMKJ7YxitsOi1oA1L311mklPt+zYU7sjrRrKSZhTxcV1CigRTmlIG0FPSnA1x qQvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=CwIOJM2v7Sb3Icqv2LdYQsUMQoJ2FT/Jj3AYeWJD/5g=; b=eEVfzT/0XQA+dv9kUUqL/29dFoVenC7URBtzqQecl/0mR4QCfMl59zwp03iq1ayLVY exqeRd9tIMHlqA34YLvKKhOnqUD6YlDhXyx+EIOQiFw6y5xTCuYwc/eIi/GGToC2RKcZ jn7c0SrNZimiiORbZ5hCNJ8/ApKBh6cpTG0+rpwNGWcWF4/pPy95qkQ0vXkvCvZ04mJ1 qUKfgktXYpF1g2IM1QxifzX+NCMubAq76rvq5k+mqxJwOGyiB0H1HAU1w39RYAGDBEVr EVS6tCQocep/CprPOckvH/KzeqCiVQwK+xiF2edyh9YdNNiIgV1wjcoMrhuFk0JOxvVV +JVg== X-Gm-Message-State: AEkoouv5/bxKzUtevbfvAL0ZjGydoy/vzWEfODGMpj5jwxgYswaEJ7Deyr4TGclMw1cX4w== X-Received: by 10.25.77.15 with SMTP id a15mr1168257lfb.179.1470879067059; Wed, 10 Aug 2016 18:31:07 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id g3sm46628lfe.14.2016.08.10.18.31.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Aug 2016 18:31:06 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <30c5515e-57e3-9f6f-6c7e-8171e9300aeb@yandex.ru> <6da984b9-0c9c-e685-75b8-d21468068148@yandex.ru> <762be2e9-c82e-fa36-7d3f-0755e3bcdbaa@yandex.ru> From: Dmitry Gutov Message-ID: Date: Thu, 11 Aug 2016 04:31:04 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <762be2e9-c82e-fa36-7d3f-0755e3bcdbaa@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/10/2016 09:50 AM, Dmitry Gutov wrote: > On 08/09/2016 12:39 PM, Lars Ingebrigtsen wrote: > >> I don't know. I don't think `url-encode-url' has ever really worked in >> any sensible way in the presence of non-ASCII. > > My point is, you're saying that url-generic-parse-url should accept (and > handle properly) multibyte URLs. But url-encode-url still encodes the > URL string before passing it to url-generic-parse-url. By the way, one consequence of the current implementation is that the set-buffer-multibyte patch proposed earlier does not break url-retrieve on multibyte URLs, on master. So it has one benefit, at least. Does that assuage your concerns with the patch? From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 10 22:52:52 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 02:52:52 +0000 Received: from localhost ([127.0.0.1]:52800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXg72-0006kt-9c for submit@debbugs.gnu.org; Wed, 10 Aug 2016 22:52:52 -0400 Received: from mail-wm0-f50.google.com ([74.125.82.50]:38699) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXg71-0006ke-0J for 24117@debbugs.gnu.org; Wed, 10 Aug 2016 22:52:51 -0400 Received: by mail-wm0-f50.google.com with SMTP id o80so5050522wme.1 for <24117@debbugs.gnu.org>; Wed, 10 Aug 2016 19:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=M1xVYVDhf0/2kfeszCe3qVn5oCvsvGOCHbhI2lo1fs4=; b=h4BOr0VnrHsGIfO8WAlgAa5tVGi+VaWHeKWhPyoT2BSDgQqG6MfA2DzduKuWd4T637 gzem7Fx/MoEj42j6PyNOJkocW/B6eRQ2/4/1LZPYTm/5L3KFu85pCNth2vFS4VJXu5Xl 0m7A6ZTFb0pU8AKzOlPFQdAE2VcxA+a1e4OA51rzWcurr6rzku+mmZ0BUNM0HFR10G1A lYdMDNV0NnmcyaO20WF5leKqMReIRO07GNTpn6tF/KeiDAS4LPcDZA70zv01+WFI54+E g8ZDqLFY2/5xJeBVrjvxXl5D0O77fg9DBQTZYwfoivt2VZwqFBgkQhG54PGlXRqMqCQZ rJnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to; bh=M1xVYVDhf0/2kfeszCe3qVn5oCvsvGOCHbhI2lo1fs4=; b=LLNCTABZRYcJVzwHQX3vHycFynReNCffZW33w1742CNDXkgPTZiVztdxzLehlwechq NK//LXQ21Y16+A65F8GIrVbc0GGQgzyKXzluTjKO9JaOGnjWl9Ck7bjggYaY8sKOVROh TFur+3rekVOu8tZNUQYLV8vPsTM3UTzavXfwIxbQ78I3sj+oYtjGDYSeuhg4SmAPT/a+ iOG3n79A8lxD5e1K3xNgM6Tim/wbMUt0Slc8FxkOS/eve4iKTLn/0MeavhJdbKctwqBh h15C0lcflSX+g0CWqsWApkkpVK24MvObywIFU1WQ9aBqKOflbFiGTrDnsRke1EVhH/pS eM5w== X-Gm-Message-State: AEkoousvu9Qw3XFqLWkvveVgxwY8Hodgk3wlensL1CkfKRgN7r7bfiWQGM1BRkdLxEtEXA== X-Received: by 10.25.219.10 with SMTP id s10mr1218300lfg.101.1470883965052; Wed, 10 Aug 2016 19:52:45 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id 74sm97863ljb.36.2016.08.10.19.52.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Aug 2016 19:52:44 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Thu, 11 Aug 2016 05:52:42 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83bn10hetr.fsf@gnu.org> Content-Type: multipart/mixed; boundary="------------F9F4DF02E330CE03629D055E" X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) This is a multi-part message in MIME format. --------------F9F4DF02E330CE03629D055E Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 08/10/2016 05:35 PM, Eli Zaretskii wrote: > Are you saying that url-generic-parse-url performs this encoding, and > that using a unibyte buffer causes that to fail? No, url-generic-parse-url contains logic that allows to distinguish between the domain and the path parts of an URL. So apparently it might have to work on multibyte URLs. That's not strictly necessary, however, given how url-encode-url uses it currently (it performs encode-coding-string and decode-coding-string on the URL string). That approach seems flawed to me, but either way, someone will have to choose how url-encode-url should use url-generic-parse-url. If we intend to leave it as-is, then the proposed patch using set-buffer-multibyte actually works fine, even on master, with multibyte URLs. >> So I think the encoding of the URL parts should be performed inside >> url-http-create-request. > > Fine with me, but when I suggested that, you didn't like the > suggestion. If you changed your mind, let's do that. See below. But yes, I'm more inclined toward this approach now, after Lar's objection, and after looking at the code in master. >> On the master branch, host is passed through IDNA encoding, but >> real-fname is untouched. On emacs-25, I think we should convert both >> to unibyte. > > Not sure I understand why there should be a difference between the two > branches. Encoding an ASCII string doesn't do any harm. Since it's ASCII, using utf-8 there seems misleading to me. It's a question of readability. As a bonus, using us-ascii will validate that the strings indeed do not contain any unexpected characters. >> (Why doesn't (encode-coding-string "aaaa" 'ascii) work?) > > It's 'us-ascii, not 'ascii. Thanks. Attaching a patch, it seems to work well enough. I'd like to wait for Lar's response now, but someone will have to make an executive decision. Both patches (this and the set-multibyte-buffer-p one), work in the cases I've tested. This one seems more conservative, but it'll require a manual merge to master. The other one is very trivial, will merge automatically, but might cause problems for potential less-careful uses of url-generic-parse-url. --------------F9F4DF02E330CE03629D055E Content-Type: text/x-patch; name="url-http--encode-string.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="url-http--encode-string.diff" diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 7156e6f..860e652 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -235,7 +235,7 @@ url-http-create-request 'url-http-proxy-basic-auth-storage)) (url-get-authentication url-http-proxy nil 'any nil)))) (real-fname (url-filename url-http-target-url)) - (host (url-host url-http-target-url)) + (host (url-http--encode-string (url-host url-http-target-url))) (auth (if (cdr-safe (assoc "Authorization" url-http-extra-headers)) nil (url-get-authentication (or @@ -278,7 +278,8 @@ url-http-create-request (concat ;; The request (or url-http-method "GET") " " - (if using-proxy (url-recreate-url url-http-target-url) real-fname) + (url-http--encode-string + (if using-proxy (url-recreate-url url-http-target-url) real-fname)) " HTTP/" url-http-version "\r\n" ;; Version of MIME we speak "MIME-Version: 1.0\r\n" @@ -360,6 +361,11 @@ url-http-create-request (url-http-debug "Request is: \n%s" request) request)) +(defun url-http--encode-string (s) + (if (multibyte-string-p s) + (encode-coding-string s 'us-ascii) + s)) + ;; Parsing routines (defun url-http-clean-headers () "Remove trailing \r from header lines. --------------F9F4DF02E330CE03629D055E-- From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 04:53:34 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 08:53:34 +0000 Received: from localhost ([127.0.0.1]:52963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXlk6-0003iq-Az for submit@debbugs.gnu.org; Thu, 11 Aug 2016 04:53:34 -0400 Received: from mail-pa0-f45.google.com ([209.85.220.45]:34343) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXlk4-0003id-Fe for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 04:53:33 -0400 Received: by mail-pa0-f45.google.com with SMTP id fi15so24285536pac.1 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 01:53:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version; bh=i2n4/EIbenEN4FfMEfQ30xORgWfwFQQbdUYkXfaIkr0=; b=hhFZG7c3N0TVJ3er08mith6e0nswx1SSJYsotQPJyZrYrf7krNebPLVU3OFVRsTXfp 8q+Bj8R/9u6WGzGSYr8jwwDjxppu6PYG/H5DKadsxawEpS9l4wJI3WbRV9vycGxPwv2i Q8unBhmgXAw5+tgYFIoeuD+T/yjKwV1TkWdQw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version; bh=i2n4/EIbenEN4FfMEfQ30xORgWfwFQQbdUYkXfaIkr0=; b=clwMDYOC0kP40tIjwkWfiWSZ6yBwTL8qbpFu8X6LoJPeoY1qWcmyMovWs52kJwoGid q5/I+oWeYJ1vviJG06cmT0oWImYWmuwRC4w9ba3tlIGgZs/lErnrU0xUZIJYv8DO4rHU G68i4K0ljPMb3vwWKEofva71jGvgiqVBsjBTil8AXT8PvR2I6R/1kDhY0xDv5Nhaej6f 4pUMlOOU7PH2Ty6vDz6ngBGVer94CVHkcbcy2CZoqr6eSJjTxwO8NwQBbcziuZPO13Ju tlMAmJsrGKRbhWowXr0l4mx0nKm4goIPuu4SpEUx+TvdekSBfovNRePNQ/KQ1WLLWpNa Pnjw== X-Gm-Message-State: AEkoouvjzsB/xkhIR05C2WMpRltKyRb5IHaTWdjp1FzljVEZYXVIcNl/BqVvt2by7QYtJg== X-Received: by 10.66.49.40 with SMTP id r8mr14983564pan.69.1470905606648; Thu, 11 Aug 2016 01:53:26 -0700 (PDT) Received: from flea (c-98-229-60-157.hsd1.ma.comcast.net. [98.229.60.157]) by smtp.gmail.com with ESMTPSA id m24sm3255769pfi.34.2016.08.11.01.53.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 01:53:25 -0700 (PDT) From: Ted Zlatanov To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos References: <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Thu, 11 Aug 2016 04:53:19 -0400 In-Reply-To: (Dmitry Gutov's message of "Thu, 11 Aug 2016 05:52:42 +0300") Message-ID: <87inv7k7pc.fsf@lifelogs.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , schwab@linux-m68k.org, larsi@gnus.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Thu, 11 Aug 2016 05:52:42 +0300 Dmitry Gutov wrote: DG> I'd like to wait for Lar's response now, but someone will have to make an DG> executive decision. Both patches (this and the set-multibyte-buffer-p DG> one), work in the cases I've tested. DG> This one seems more conservative, but it'll require a manual merge to master. DG> The other one is very trivial, will merge automatically, but might cause DG> problems for potential less-careful uses of url-generic-parse-url. Could you add to your patch the cases you've tested? There's a specific place for URL parsing tests in test/lisp/url/url-parse-tests.el that would help everyone. Thanks Ted From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 07:05:56 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 11:05:56 +0000 Received: from localhost ([127.0.0.1]:53038 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXnoC-0008VC-0l for submit@debbugs.gnu.org; Thu, 11 Aug 2016 07:05:56 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:36101) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXnoA-0008V4-I3 for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 07:05:54 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bXno7-0008EM-J3; Thu, 11 Aug 2016 13:05:53 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAElBMVEWPosIvNFZZaJIPDBTm 7fQdHC5oSeU0AAACUklEQVQ4jV2Uy3rjIAxGZT66t4jZg5rum6rsicfdhxC9/6uMhHPpDFkk0bGk XxcMIVBGDBgbImbaHgcIQAGKDBDSsMbmIQf9r3Y9Sih7tZ9FFkj6yXGAaGBW8CZyhZky5d2+eymw wKChQ9zjoPRSxG8V1WCAREpDwnYQYVm2tXNpYFqkCyZVp45cbtsPcxEFFIWb2sHliJ2v27sCBAzU egkA5Ex5VHDgFgiQKJqDfmEjjD1sh4b6oHakF0wuS5NG2cBHQDLwYZ5ZtDFN3UQ2R4HQQogZrg7n mEzZcaXY2YA59Abkp7lOUZaakboCCWRNsfYncFEuW1qjhVIHClll5+Srgrb5msMd2NESNwPia8oE mbQSssJtEgMslVArz+1hNWB6LzpXBagi/WPS1WZ2HRPHrOC1Atrp0nQkYB7zb9CZFWwDhH9AL/P4 hTb4J6iaYn/OwOk30L4l0wK2GumlSrQHmO6gX55gFZ1hmf0dfL48ik6dAx036FJ6gV+hTG9QwIU7 p2f2GCbmE3n1YBY+HZ9JgvvgL6c5SmdNc31m2cA7DyO5rp0ucfKvVg5eBhg3J/j/AI/rpPAJKGl7 16lc3DnBiviQcNaFArc4Xs5wXukJqkUAvzomWN6OK7a9nPrOBhy7Xv2Sz9CaNbNOIylC5uTeLnOE ySTXMULTCZHj4XSlaPcwwaQ3dycaavq53chqlOR0H1NWwAPQfBvzkhnG6Q9wm773V0MadjpYLL2r 0+efb4gWyjx0883uQHRf+gnvoTy44XBKwOOUbu+MoGDX+wV/AZfg1K2OOBawAAAAAElFTkSuQmCC Date: Thu, 11 Aug 2016 13:05:12 +0200 In-Reply-To: (Dmitry Gutov's message of "Thu, 11 Aug 2016 05:52:42 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Eli Zaretskii , schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Dmitry Gutov writes: > This one seems more conservative, but it'll require a manual merge to > master. The other one is very trivial, will merge automatically, but > might cause problems for potential less-careful uses of > url-generic-parse-url. Yes, the fix here should be in url-http-create-request, not in the URL parsing functions. The main issue here is that the URL request buffer is a multibyte buffer and (as with all network connection buffers), it shouldn't be. (Or, rather, that function just creates a string instead of a buffer, but the same principle applies.) But I think this fix looks OK: > - (host (url-host url-http-target-url)) > + (host (url-http--encode-string (url-host url-http-target-url))) > (auth (if (cdr-safe (assoc "Authorization" url-http-extra-headers)) (etc) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 08:31:21 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 12:31:21 +0000 Received: from localhost ([127.0.0.1]:53092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXp8r-0003k2-La for submit@debbugs.gnu.org; Thu, 11 Aug 2016 08:31:21 -0400 Received: from mail-wm0-f52.google.com ([74.125.82.52]:36156) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXp8p-0003jn-VG for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 08:31:20 -0400 Received: by mail-wm0-f52.google.com with SMTP id q128so27018372wma.1 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 05:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=LfuSRMbwCk7i8JTeRT5uRdEA0FaOJudNKKpNYH/1VsE=; b=xXaEsFROxdRY8FCQ0YXDL3hNp312CijVtj+RPLcaPFI5zPGxfFaiBaT45C6nnPxffv 589NKtSZ1LB59rYfkdP64jsBqmXR8OD8/ZbgdtuWLUmYBmWawoQI9XueIwWzXh2UCi10 ooadkXKnPmspkrU4fB5kYN2fxM3NrL4lsqlBusVGgK80InZgvyD2QQEuAHd6r0lPJtc/ BDpfg9GtnHrcvEyfystZ+c0SFbD4yh3BR4ObCNm8uBeu2H0KjKnX4z8jTB4wbGcGz+ZQ 3d7EF7FUKfFfIObxjbnTu2P3dXK/pYFl5HjZ1qaOZ0aSBEdDbOMuWRrfYhpqGYr8eIdi T/mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=LfuSRMbwCk7i8JTeRT5uRdEA0FaOJudNKKpNYH/1VsE=; b=M+YkG6ujfhdFDDG+Bk7ymJ6XouGAQlKpeXPbTXSAVD2wXG0rjwAjTS0MZiWGAoY6DN Mvw8xx1eKiSVuTpRwpOeTuXMCV8jj7pX6lou2hQauycQpMDsNAV7VZgope7wSUI9UVTr dzYH/Axxq0K7VCFbrGfvPPuYBfJgXzUpvrBAwJd7WU3BD9R5Mg4Yb1wgUJwHMEgdzHrj uER+0dIyKoGopPe6Xm4fQm3LKQy+wlKMINoKoYZf0iQIaRAm7nDYTY44VH5tEwjUqwxs ogkliCyWrbywGHy/bVHEzWiiQVrxtPODHOxR5zb64RCh2+qa8uI/s3O7mOu8vdPiuT7c EsLg== X-Gm-Message-State: AEkoouvO0ZGArnDMaJultEwjOCh9qjLQDWOU+5IS7+dU4l1iD6VacClty7jSQAHXiIeAiA== X-Received: by 10.25.221.71 with SMTP id u68mr1467804lfg.146.1470918674035; Thu, 11 Aug 2016 05:31:14 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id o10sm77555lfo.47.2016.08.11.05.31.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 05:31:13 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Ted Zlatanov References: <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <87inv7k7pc.fsf@lifelogs.com> From: Dmitry Gutov Message-ID: <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> Date: Thu, 11 Aug 2016 15:31:11 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <87inv7k7pc.fsf@lifelogs.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/11/2016 11:53 AM, Ted Zlatanov wrote: > Could you add to your patch the cases you've tested? There's a specific > place for URL parsing tests in test/lisp/url/url-parse-tests.el that > would help everyone. Sure, but only one of the patches affects URL parsing (and Lars prefers the other one). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 08:58:11 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 12:58:11 +0000 Received: from localhost ([127.0.0.1]:53161 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpYm-0004QD-A7 for submit@debbugs.gnu.org; Thu, 11 Aug 2016 08:58:11 -0400 Received: from mail-pf0-f177.google.com ([209.85.192.177]:34403) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpYh-0004Pa-FL for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 08:58:07 -0400 Received: by mail-pf0-f177.google.com with SMTP id p64so882470pfb.1 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 05:58:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version; bh=IhnlPlNEU5XD6/UU59qg/0mb5iMhnDpqHc4NaHAW+QM=; b=tUmjgnM4njFbYMVmOtAnNny/w5yFG0EVI+/eZeaUXMvQoBCVtiQpG1xXjeWMknUqba gGVUiGcgWdgK4LVWHz/uIgVRaMKB+lQHNhc6WQpYGHIwKwLghr8DW4jBTQOrxCBQ+9Y4 /i6MdqK6p9hSMjVNWKzVdIqxzxGT63zJXVZX4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version; bh=IhnlPlNEU5XD6/UU59qg/0mb5iMhnDpqHc4NaHAW+QM=; b=UA6CiUzR4iP7pCYieQf9gG2BJvspgw2g95nYtTJTj6I7f1qydLp815oHZtEQea6GzV 64viAL69Ase1ioZHDC4+S5JcyeGi8wECKfVNB0aY7Cunz8lbcEHnVwjE8LnDjawEl7hw lPI0AdPAPohLnBWJ7SPtf5UZQo3knX7KqUWys/ou/US6nVU+I1no4Sct+RbuwXZRkC/y WPpGm+ao5lp33B4M3D+j+cUxpg+kwmeYhdVUs9RbiYmoXBanZphr3bvt8Twh/mAmP1os Z4CyfLCff0XaAIoUayNTyiLw3zfv86E+DR0i/BCVDPjB54q9QpmBrcMs6DssSEf2YsYc i81w== X-Gm-Message-State: AEkoouvvSdY6IcEDEiV1tYUKIZ5JGSe8VdDD1XVtfTFdocVAZxX2IlCLKz7O/ELJbT0tig== X-Received: by 10.98.74.201 with SMTP id c70mr16888150pfj.113.1470920277610; Thu, 11 Aug 2016 05:57:57 -0700 (PDT) Received: from flea (c-98-229-60-157.hsd1.ma.comcast.net. [98.229.60.157]) by smtp.gmail.com with ESMTPSA id d185sm5313786pfd.80.2016.08.11.05.57.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 05:57:56 -0700 (PDT) From: Ted Zlatanov To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos References: <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Thu, 11 Aug 2016 08:57:50 -0400 In-Reply-To: <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> (Dmitry Gutov's message of "Thu, 11 Aug 2016 15:31:11 +0300, Thu, 11 Aug 2016 13:05:12 +0200") Message-ID: <874m6rjwdt.fsf_-_@lifelogs.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Lars Ingebrigtsen , schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Thu, 11 Aug 2016 15:31:11 +0300 Dmitry Gutov wrote: DG> On 08/11/2016 11:53 AM, Ted Zlatanov wrote: >> Could you add to your patch the cases you've tested? There's a specific >> place for URL parsing tests in test/lisp/url/url-parse-tests.el that >> would help everyone. DG> Sure, but only one of the patches affects URL parsing (and Lars prefers the DG> other one). Maybe the tests should be in a separate patch then. Neither your Russian example nor Lars' example have a parallel in the tests AFAICS. I'd also add the example hostname that Katsumi Yamaoka gave from the w3m source. Somewhat related: it would be nice if the URL parser also listed the non-ASCII scripts used in the domain name. Then eww and other programs could do one of the typical defenses: either ensure only one script is used; or allow only scripts that match the user's locale; or catch any non-ASCII domain names. Typically they'd use Punycode to display such suspicious domain names: https://en.wikipedia.org/wiki/IDN_homograph_attack I bring it up since explicitly allowing non-ASCII domain names automatically opens up these security concerns, and it's a bit hard to collect the confusables externally: https://elpa.gnu.org/packages/uni-confusables.html On Thu, 11 Aug 2016 13:05:12 +0200 Lars Ingebrigtsen wrote: LI> Yes, the fix here should be in url-http-create-request, not in the URL LI> parsing functions. The main issue here is that the URL request buffer LI> is a multibyte buffer and (as with all network connection buffers), it LI> shouldn't be. (Or, rather, that function just creates a string instead LI> of a buffer, but the same principle applies.) I think this is correct: the URL parsing should not care about the provenance or potential use of that URL to make a HTTP request or otherwise. But maybe the URL parsing can be smart enough to return both the IDNA version and the original domain name, plus some parsing information like the list of scripts I suggested above, to save user agents from doing that extra work? Ted From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 09:01:44 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 13:01:44 +0000 Received: from localhost ([127.0.0.1]:53165 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpcF-00067i-Ud for submit@debbugs.gnu.org; Thu, 11 Aug 2016 09:01:44 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:39187) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpcC-00064Q-8S for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 09:01:42 -0400 Received: from cm-84.215.1.64.getinternet.no ([84.215.1.64] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1bXpc6-00023I-S5; Thu, 11 Aug 2016 15:01:38 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> <874m6rjwdt.fsf_-_@lifelogs.com> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAElBMVEX+0Nz+x9X+y9jcp7OH Ymv+xNK/3UY+AAACNUlEQVQ4jW1US7KbMBBUgd8+Ix57NC72NpIO8KxhHxl0/6ukRwLiVKXLaEHT 823LOMoFsI6dcwx0BsdkjCsV5CzRaJVRwhjTBJuP0Q8e2OLUqcDQWBXrz5qG8JOiLF2ngotIlfDP tEyfxLauKW2IlPxD4wCZlMiZqGSX2XJ3EWOGCMSYnWXH5iRsjaWKsuHk6dBYLvgYDwVJYUGDU1cV eDOSYksiq+fpJJgJXTseAxS7X44MIEgJsgMIxPIX4ahhWL0knxKfBFEm65wN3qtALiLXoVpOcRDE EmQ/qtLagRT9CkG4FNqadXZOfo+jeLF89aE57D34IAVZ3nwMJWuskYY9olyR3bLBSEDlVi+2V3ML a+9V4UipOwivGu7ayjFEq8QgUZA6yXs+6sU0GlFSbfHtzLFaqvOFYos+f2tVSMOqaEOXWErI+/7W LartyBXKqlhLkGeQugUlNAOeQX7GOJIk1mKsKmZ8wnTHzAWtow9uCowQC3Rzffst8j4Jq56B1dHF OOsSL4KrxTtJabljvX7h+ndQhTb6JSGFuMm+LlxhDtPfZI+wrtyr2+u4GtHL7WHMHr5jMo2Y2i77 2/wyxu/PF3ewnCqakfythyLcHt78i6m/P0zX96/eHC458dUtKn/Vr66jMr8N1v1JnEokNfw3RHWE GsOg7OkjvK5rqrtpF8IHwXPB9TC3C8Jmcu5XJeDCeiOkenq43j+qSxy1q8IfJ35LVcAJLcwJWJYw 8W4s/8Gb/wBwqJtm38VVOgAAAABJRU5ErkJggg== Date: Thu, 11 Aug 2016 15:00:55 +0200 In-Reply-To: <874m6rjwdt.fsf_-_@lifelogs.com> (Ted Zlatanov's message of "Thu, 11 Aug 2016 08:57:50 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Ted Zlatanov writes: > Somewhat related: it would be nice if the URL parser also listed the > non-ASCII scripts used in the domain name. Then eww and other programs > could do one of the typical defenses: either ensure only one script is > used; or allow only scripts that match the user's locale; or catch any > non-ASCII domain names. Typically they'd use Punycode to display such > suspicious domain names: > https://en.wikipedia.org/wiki/IDN_homograph_attack This is implemented in puny and eww: ;; Check whether the domain only uses "Highly Restricted" Unicode ;; IDNA characters. If not, transform to punycode to indicate that ;; there may be funny business going on. (let ((parsed (url-generic-parse-url url))) (unless (puny-highly-restrictive-domain-p (url-host parsed)) (setf (url-host parsed) (puny-encode-domain (url-host parsed))) (setq url (url-recreate-url parsed)))) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 09:18:52 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 13:18:52 +0000 Received: from localhost ([127.0.0.1]:53178 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpsm-0006ZM-Lv for submit@debbugs.gnu.org; Thu, 11 Aug 2016 09:18:52 -0400 Received: from mail-pa0-f42.google.com ([209.85.220.42]:34542) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXpsh-0006Z2-P2 for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 09:18:47 -0400 Received: by mail-pa0-f42.google.com with SMTP id fi15so974965pac.1 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 06:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version; bh=Cku0rzR5RJAWSap8540AhRHv5HzIjfYS5R/MZSQY1+k=; b=orzCxOYuDgIjEqt2gD8bqolLOS/aBublZLM/OdsZJkuR/IB/TRnQyGuSbZcap0UWMy +hDb1pIJM/Jxhxywscu9Xa8BvsRf8QSnKfYmU7qWcHgmQZJV+3WFXKxUg53ADGcVqxc2 bZS2YDzXeBkATp4S8XBdjY+K2u+jFX97V67Vo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version; bh=Cku0rzR5RJAWSap8540AhRHv5HzIjfYS5R/MZSQY1+k=; b=DM0FmOZR7dSbla+6jhrLuJWNebYyCv6FyQ5ObckxRWOTYzMcTA6LkQaWfowcvg4pCR z5fVHbKRuDe1SAU2ArR1qEpV1pctRtl25KQ7rYQrtDQfq06da4aH2YR1mDoGd7zxL2H9 9kCUgq0xvDXVJoxQ6Izz0OTtlbKnOsDDwhJRVdzzvSmj3QkUVTAa5ugw9bMUD4icSPbZ lEEVjFWz38TV1P3/Bz9Y91J4gYDLsI6EuRqtetqHnjTDbEL+f+TMym921kRmEzX0895b +vyBW31HrJo2SkjduT/t5m80jXtYmq473X04n5baKaFsS58w2i0FMcnSdjCvOeYpgw4a xXtw== X-Gm-Message-State: AEkoouuKON0hECd3EItyk1i18G1iv9fFfI2GCQXhQe1R6ODcFkTYaAOhCalJMtMkoRqbDA== X-Received: by 10.66.2.135 with SMTP id 7mr16789903pau.136.1470921518083; Thu, 11 Aug 2016 06:18:38 -0700 (PDT) Received: from flea (c-98-229-60-157.hsd1.ma.comcast.net. [98.229.60.157]) by smtp.gmail.com with ESMTPSA id vt10sm5451037pab.43.2016.08.11.06.18.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 06:18:37 -0700 (PDT) From: Ted Zlatanov To: Lars Ingebrigtsen Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos References: <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> <874m6rjwdt.fsf_-_@lifelogs.com> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Thu, 11 Aug 2016 09:18:31 -0400 In-Reply-To: (Lars Ingebrigtsen's message of "Thu, 11 Aug 2016 15:00:55 +0200") Message-ID: <87shubiguw.fsf@lifelogs.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, schwab@linux-m68k.org, 24117@debbugs.gnu.org, Dmitry Gutov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Thu, 11 Aug 2016 15:00:55 +0200 Lars Ingebrigtsen wrote: LI> Ted Zlatanov writes: >> Somewhat related: it would be nice if the URL parser also listed the >> non-ASCII scripts used in the domain name. Then eww and other programs >> could do one of the typical defenses: either ensure only one script is >> used; or allow only scripts that match the user's locale; or catch any >> non-ASCII domain names. Typically they'd use Punycode to display such >> suspicious domain names: >> https://en.wikipedia.org/wiki/IDN_homograph_attack LI> This is implemented in puny and eww: LI> ;; Check whether the domain only uses "Highly Restricted" Unicode LI> ;; IDNA characters. If not, transform to punycode to indicate that LI> ;; there may be funny business going on. LI> (let ((parsed (url-generic-parse-url url))) LI> (unless (puny-highly-restrictive-domain-p (url-host parsed)) LI> (setf (url-host parsed) (puny-encode-domain (url-host parsed))) LI> (setq url (url-recreate-url parsed)))) Awesome! Thanks for pointing this out, and sorry for digressing. Ted From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 10:48:09 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 14:48:09 +0000 Received: from localhost ([127.0.0.1]:54157 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrHB-0000it-AU for submit@debbugs.gnu.org; Thu, 11 Aug 2016 10:48:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56535) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrH5-0000iI-Ug for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 10:48:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bXrGv-0005Qj-Kh for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 10:47:54 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59791) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXrGv-0005Qd-HD; Thu, 11 Aug 2016 10:47:49 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1627 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bXrGt-0008Oa-NG; Thu, 11 Aug 2016 10:47:48 -0400 Date: Thu, 11 Aug 2016 17:47:37 +0300 Message-Id: <83lh03fjli.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Thu, 11 Aug 2016 05:52:42 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, > 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 11 Aug 2016 05:52:42 +0300 > > >> On the master branch, host is passed through IDNA encoding, but > >> real-fname is untouched. On emacs-25, I think we should convert both > >> to unibyte. > > > > Not sure I understand why there should be a difference between the two > > branches. Encoding an ASCII string doesn't do any harm. > > Since it's ASCII, using utf-8 there seems misleading to me. It's a > question of readability. But AFAIU it doesn't have to be ASCII, it could include non-ASCII characters, no? > As a bonus, using us-ascii will validate that the strings indeed do > not contain any unexpected characters. If we did allow non-ASCII characters until now, we will definitely hear from someone who'd say this is a regression. > Thanks. Attaching a patch, it seems to work well enough. LGTM, modulo the considerations about the encoding. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 10:59:37 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 14:59:37 +0000 Received: from localhost ([127.0.0.1]:54163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrSG-0000yC-3D for submit@debbugs.gnu.org; Thu, 11 Aug 2016 10:59:37 -0400 Received: from mail-qk0-f170.google.com ([209.85.220.170]:36317) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrS8-0000xu-Hu for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 10:59:30 -0400 Received: by mail-qk0-f170.google.com with SMTP id l2so6816995qkf.3 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 07:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=ecXK5YE27GV2sUIpb9aIkgRcfgoYGIAZLLV8iXcK1oA=; b=mQX4BwTojR7Em07WmzEuH2eLkLnoZXEH5sj4QpqGUMIIpgZhuwmnfC8CYtvSegz5PJ hdcVzadfC/5qoxSIlniHLvtaERhn11v9vONp8XjG2Z8hG1JkrSCPopfc1igTALelFsln 3rqdOUnLvEE2F0+Rz155Sk3OIC6Sh9MxmfDcAZm+1P+BR0ncI36ylDcxZi9iTRMaOOBr liniyFkh1cuzJnsnDBZFRN5QnDCvdGLCrrvZhLfJyrKCu3nHEF43a+VxCv+QH8IllaAQ bKCRmQGMRublozXP/RiudOiEOn+ty7VkGeQmicUb/s9RuIujLLXvcp7Szh2Jn2O1GNs8 +ccw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=ecXK5YE27GV2sUIpb9aIkgRcfgoYGIAZLLV8iXcK1oA=; b=TUXHqCYSoF0tqwoWn581x2oEIiwm/jX7xQa3OttfPpTw7trpbn95RX2eBlnJja9Yfq ubz32IX7yzMkGV6N54QR1Y8Fbb56b0qgFzNIF3YTVmOvonccLZigf+/oqZIc69XwZL4C 4UdamZNtt9iwvw0T2ZVKcl8VIVOjAFtrCkr8G2rb9gUAQRZJmwbxGKDikAJgU971B5uR tvX8fjer1YroDYmyuqNsvtIEYYI9eFSIJYE+RNe0y8Jt6zkPbGzWK/qZRXqusUvqv9YL cP9zk7VhplvLEYMfNpTKEFmKoi7sKGEn20p6VRKt6sgInDTfSWV/xFTLTLQxa05ev1u7 joZw== X-Gm-Message-State: AEkoouvSCGjUL0hfVOtUFN+D65CC5yIKGdTkYFvtZDpwB5E1rcXyPed7WQZx+k5r2VpN/g== X-Received: by 10.46.1.69 with SMTP id 66mr1734563ljb.27.1470927558736; Thu, 11 Aug 2016 07:59:18 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id h68sm620467ljh.38.2016.08.11.07.59.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 07:59:17 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83lh03fjli.fsf@gnu.org> From: Dmitry Gutov Message-ID: <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> Date: Thu, 11 Aug 2016 17:59:16 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83lh03fjli.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/11/2016 05:47 PM, Eli Zaretskii wrote: > But AFAIU it doesn't have to be ASCII, it could include non-ASCII > characters, no? Not the values we pass to url-http--encode-string in this patch, no. As you said, when the request hits the wire, it has to be unibyte. When the strings reach url-http-create-request, they either contain no multibyte characters, or url-http-create-request has to convert the strings in some meaningful fashion, and url-http--encode-string is not that. In master, it uses puny-encode-domain. We can safely assume that internationalized domain names don't work in emacs-25. >> As a bonus, using us-ascii will validate that the strings indeed do >> not contain any unexpected characters. > > If we did allow non-ASCII characters until now, we will definitely > hear from someone who'd say this is a regression. I find that highly unlikely. HTTP URLs with multibyte characters need to be encoded in a specific way. > LGTM, modulo the considerations about the encoding. Thanks. Can we designate this bug as a blocker? From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 11:31:51 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 15:31:51 +0000 Received: from localhost ([127.0.0.1]:54196 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrxW-0001kW-Vn for submit@debbugs.gnu.org; Thu, 11 Aug 2016 11:31:51 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39752) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXrxS-0001jv-E7 for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 11:31:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bXrxJ-0007zi-0F for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 11:31:41 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60813) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXrxI-0007yk-TC; Thu, 11 Aug 2016 11:31:36 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1701 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bXrxG-0001gd-U4; Thu, 11 Aug 2016 11:31:35 -0400 Date: Thu, 11 Aug 2016 18:31:25 +0300 Message-Id: <83eg5vfhki.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> (message from Dmitry Gutov on Thu, 11 Aug 2016 17:59:16 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83lh03fjli.fsf@gnu.org> <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, > 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 11 Aug 2016 17:59:16 +0300 > > On 08/11/2016 05:47 PM, Eli Zaretskii wrote: > > > But AFAIU it doesn't have to be ASCII, it could include non-ASCII > > characters, no? > > Not the values we pass to url-http--encode-string in this patch, no. As > you said, when the request hits the wire, it has to be unibyte. Unibyte != ASCII. > > LGTM, modulo the considerations about the encoding. > > Thanks. Can we designate this bug as a blocker? Why not just push the fix to emacs-25? From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 14:08:00 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 18:08:00 +0000 Received: from localhost ([127.0.0.1]:54307 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXuOe-0000OY-Gs for submit@debbugs.gnu.org; Thu, 11 Aug 2016 14:08:00 -0400 Received: from mail-wm0-f51.google.com ([74.125.82.51]:38730) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXuOc-0000OK-QK for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 14:07:59 -0400 Received: by mail-wm0-f51.google.com with SMTP id o80so7277060wme.1 for <24117@debbugs.gnu.org>; Thu, 11 Aug 2016 11:07:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=0+8PT0OBdmWD08X/wYimWd/WWDE62A49YLL83SUSRPM=; b=GbFKC2C+AqAZd1neiAysWW74ud3qhKEtSV1I6sBlZhftUV+T9RSTB7Fnvy59b7mYqR ArbCyc3HJckGe96WrDfJBBfHKQGVwQUtb3tJOIv4OBTDUFjuZq4IijffTSiN/lzCkmm5 7IauEU49r80Sb+VenEKELZjzIjWx8v3RLCSI+nGfj/Il2Dm3lunHqBRtK7ao5ADSjK8n d5nDSfoYfeckBNNz3n2frR9JH6nFSarrODLb2Bogct+HCj8E5KFUrPXRmTbR8uT4ssY/ 5AiORu9SSW2RZqmRmTtssyBVYuIhPbNmKDQOkmCrK79Nop70M2i1Jt7Fiomuo8NCda0m GLCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=0+8PT0OBdmWD08X/wYimWd/WWDE62A49YLL83SUSRPM=; b=KPuxusMxWKWzG+Hux1cciLrh7itbr+NLKcb2Bq3LWFbkiRrpkK1RSDDElW/5dcCbmb ICcgXqA5eKW3X9naUs0buREHvCmvQiaUGPZVkGOzdn5S4FpRwdPdmRYY5+2sMwuiXJxp aawGPDsqboYkQaTiA4hgWif0rgN3GQsn9YUbOfoK+bcc9SnVqzdCCYAOW8zlKDhVPARE DJ8yn5Rmw9GfGLrdruOarOLAt4ni8fnrwV3glfB0UMdOYnmt+w6hHUHqwOM9GXgsMg+H /IIcrVfM/mXWjEGmGDWxnZB0Kr1eWWHSW0eWvUqrd380B8jVEDlz63f2+wvk8uS3h7J8 bv7A== X-Gm-Message-State: AEkoouu9WPxP9GzkfsVGyCmrJtavQm1ODklDPiAZYG9h5tbu5r8kSnAXDWEQFsKJjUKnRA== X-Received: by 10.25.10.208 with SMTP id 199mr1909886lfk.124.1470938872948; Thu, 11 Aug 2016 11:07:52 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id o10sm273902lfo.47.2016.08.11.11.07.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Aug 2016 11:07:52 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83lh03fjli.fsf@gnu.org> <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> <83eg5vfhki.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Thu, 11 Aug 2016 21:07:50 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <83eg5vfhki.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/11/2016 06:31 PM, Eli Zaretskii wrote: >> Not the values we pass to url-http--encode-string in this patch, no. As >> you said, when the request hits the wire, it has to be unibyte. > > Unibyte != ASCII. Indeed. And the above function converts the latter to the former. >> Thanks. Can we designate this bug as a blocker? > > Why not just push the fix to emacs-25? Because John likes to question commits in emacs-25 that do not reference bugs marked as blocking. See also the recent message in the emacs-devel thread. Anyway, pushed. I'll add some tests to master a bit later. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 11 15:48:22 2016 Received: (at 24117) by debbugs.gnu.org; 11 Aug 2016 19:48:22 +0000 Received: from localhost ([127.0.0.1]:54371 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXvxj-0004Vx-3W for submit@debbugs.gnu.org; Thu, 11 Aug 2016 15:48:22 -0400 Received: from eggs.gnu.org ([208.118.235.92]:38820) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bXvxf-0004Vh-4J for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 15:48:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bXvxV-0007ow-P9 for 24117@debbugs.gnu.org; Thu, 11 Aug 2016 15:48:09 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37847) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXvxV-0007od-M5; Thu, 11 Aug 2016 15:48:05 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2166 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bXvxR-0005j7-OE; Thu, 11 Aug 2016 15:48:04 -0400 Date: Thu, 11 Aug 2016 22:47:37 +0300 Message-Id: <8337mbf5pi.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Thu, 11 Aug 2016 21:07:50 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83lh03fjli.fsf@gnu.org> <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> <83eg5vfhki.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, > 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 11 Aug 2016 21:07:50 +0300 > > > Why not just push the fix to emacs-25? > > Because John likes to question commits in emacs-25 that do not reference > bugs marked as blocking. See also the recent message in the emacs-devel > thread. > > Anyway, pushed. I'll add some tests to master a bit later. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 12 17:44:47 2016 Received: (at 24117) by debbugs.gnu.org; 12 Aug 2016 21:44:47 +0000 Received: from localhost ([127.0.0.1]:55508 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYKFz-0005XY-Au for submit@debbugs.gnu.org; Fri, 12 Aug 2016 17:44:47 -0400 Received: from mail-pf0-f178.google.com ([209.85.192.178]:33530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYKFx-0005XJ-Nd for 24117@debbugs.gnu.org; Fri, 12 Aug 2016 17:44:46 -0400 Received: by mail-pf0-f178.google.com with SMTP id y134so12505013pfg.0 for <24117@debbugs.gnu.org>; Fri, 12 Aug 2016 14:44:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:in-reply-to:date:message-id:references :user-agent:mime-version; bh=M4LqZq14Rdt3jxMuLW/UbmUdyzh6XfgAxVW/aixffkI=; b=nh4cD25fdTPH58JBayFaLH3mS8MgK9NClrSckEy+fGtj+22B5wY9A9KYHBus2xJ+31 FWoAh5i8xVp6E8IkDF6+8YInTE9a4IRaakIT5wnfd4LNOfAIXHL4rzHbdHV6aHyh+HK5 UtJMogirUI074wDU2ve7yn9OCgNaWn6B7S52qJUm8rpEgsloaeAjvi7W5r5gyf+JM7Fw K7+s81xLHCS4xR1Jk2uNzHoru6gtNkYS3paWdV/bvfL1VKROgZuIBGKBDBqZubntLbdB TBp8BIvxtcUhuAlMY9Wf0/raoUkM7Dl7l3fhQ+KOJKGnk9vnz2Pkwn+22stlM1loebFs OdIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:in-reply-to:date:message-id :references:user-agent:mime-version; bh=M4LqZq14Rdt3jxMuLW/UbmUdyzh6XfgAxVW/aixffkI=; b=W3HEeyQ6Fz5wDK3tnFHMPub9LcvKLmDKyiu/qlxT2J7lSXZF0tk2BNqpdCeAyNiVbW 84AfxvWnhA8bz/dIdnRCGZkaTJtJqsEqxxMfY0+ixQ5CPf3EJYBEskNtYgo1PM+W/zto T2/eQdo/OzFIPN9KqX9wcNxZH0hk25J4weJrdegDW59Ggm50diGVR1d5T9njaCGYAXSH G/NPuUdxXSo24xKQc5DUnNVWNo7RMuxuoBlwLG+uLCGV+8V4gcXwNKeSFyLJjoRcBvsC Q90DvYaglb45K6lTVMgePo8is17vlrARfA34iNX5t+w/lV3JOnpBocstCzPq/2C34+LD 2lDw== X-Gm-Message-State: AEkoouvsMmQ2F6Oj9XCjHS4MtNr/ipw4mxO0lS7xWyvAumK0OrsPkQJeg3ZwxsFB0DBVDw== X-Received: by 10.98.83.70 with SMTP id h67mr30763175pfb.115.1471038279857; Fri, 12 Aug 2016 14:44:39 -0700 (PDT) Received: from Vulcan.local (76-234-69-149.lightspeed.frokca.sbcglobal.net. [76.234.69.149]) by smtp.gmail.com with ESMTPSA id 9sm15241904pfo.74.2016.08.12.14.44.36 (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 12 Aug 2016 14:44:36 -0700 (PDT) From: John Wiegley X-Google-Original-From: "John Wiegley" Received: by Vulcan.local (Postfix, from userid 501) id D32731D79109; Fri, 12 Aug 2016 14:44:35 -0700 (PDT) To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request In-Reply-To: (Dmitry Gutov's message of "Thu, 11 Aug 2016 21:07:50 +0300") Date: Fri, 12 Aug 2016 14:44:32 -0700 Message-ID: References: <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83lh03fjli.fsf@gnu.org> <716de1ee-7f2c-786d-6ba2-86f82f0adadb@yandex.ru> <83eg5vfhki.fsf@gnu.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.0.95 (darwin) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: schwab@linux-m68k.org, stakemorii@gmail.com, Eli Zaretskii , 24117@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) >>>>> "DG" == Dmitry Gutov writes: >>> Thanks. Can we designate this bug as a blocker? >> >> Why not just push the fix to emacs-25? GD> Because John likes to question commits in emacs-25 that do not reference DG> bugs marked as blocking. See also the recent message in the emacs-devel DG> thread. Yes, you know me well. :) In fact, I came to this thread after seeing your commit for exactly this reason, so I appreciate this comment. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2 From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 12 20:30:43 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 00:30:43 +0000 Received: from localhost ([127.0.0.1]:55598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYMqZ-0004MT-31 for submit@debbugs.gnu.org; Fri, 12 Aug 2016 20:30:43 -0400 Received: from mail-oi0-f54.google.com ([209.85.218.54]:36434) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYMqY-0004MI-7N for 24117@debbugs.gnu.org; Fri, 12 Aug 2016 20:30:42 -0400 Received: by mail-oi0-f54.google.com with SMTP id f189so2429472oig.3 for <24117@debbugs.gnu.org>; Fri, 12 Aug 2016 17:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7ayuyYQAELqmqpWAKSPjPWEJhcn/ONDwU/OLD93Ab0A=; b=lBZM3aXwsncPHe+0sXV0HFP8NAV0b8a6PtONaQaIuabnsZF1ZEVwpY4f/AljfLrZbo G/Fg17dySWrXk/UST11tUl/uzUDpfYPXQXgPoD5ophyAg6qzN9+SIKJVG64/2ow3o69o ANypW838WtyYakU1mcTzuRwxLY2gdkKBYnBISNjFuEmr8/5javsowXanFZvDj2GIN2ox WPIsCrMBWujQ0OlfNRGOV8BOsXUPecdv84ynTaxjM+Qc/9XBHLv7JQtIeNxXsQizUuLY uExa7GtdTvOFlannz/y/ykTv1jLlB9ZAlOjXss+uNsher5p63USx84ezXSU3tqzzO3S3 wBRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7ayuyYQAELqmqpWAKSPjPWEJhcn/ONDwU/OLD93Ab0A=; b=aolwAKanSBiMK7xdH6qz9RoeAU158/tmfibqybCXVOw/SXLj6KDgA8ixH5QgZ0PMu9 Rl7Wm/YsUoY55gqBAGBAnykRGHl8RYr8sOcLD42rSCoioOjVOvyfFwVIovDaKrVOKVuo kLCS9Yt6gteWLFoIa7aOuKeNNnwLNmafC3Go+di7l5sdaQj35EwtgTC2p/5XUyYhCVTE BM9aYMrP0wG+RTplS1rmaWefVP5zjPwkreTIRY7ssOg0NyKSOMwm6DYpX3DG2gQob7/v iSVr0ZqSuPQW4FtdbJal9a3/yofzydxsV+jmsYtbmTHjgSCpP89cTuRkzEeKvTpxlp1V 4uog== X-Gm-Message-State: AEkooutqkjDvosH+Xk1GasFOF35pY/S1C938iSQCehusUaoszG/4UETCTHsaIIzJmO5py/7SJoXChZZEAsQmpw== X-Received: by 10.202.83.20 with SMTP id h20mr9984427oib.147.1471048236528; Fri, 12 Aug 2016 17:30:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.37.66 with HTTP; Fri, 12 Aug 2016 17:30:35 -0700 (PDT) In-Reply-To: References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> From: Sho Takemori Date: Sat, 13 Aug 2016 09:30:35 +0900 Message-ID: Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Dmitry Gutov Content-Type: multipart/alternative; boundary=001a113b004e95f9ff0539e91bc7 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, Eli Zaretskii , schwab@linux-m68k.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a113b004e95f9ff0539e91bc7 Content-Type: text/plain; charset=UTF-8 Is this bug fixed in emacs-25? With "emacs -Q", I do not see this error anymore. Thank you very much. But I set the language environment by (set-language-environment "Japanese") in ~/.emacs.d/init.el. With the configration, I still see the same error when using anaconda-mode. And url-mime-charset-string in url-http-create-request is a multibyte stirng and is bound to "euc-jp;q=1, iso-2022-jp;q=0.5, shift_jis;q=0.5, euc-jis-2004;q=0.5, iso-2022-jp-2004;q=0.5, utf-8;q=0.5, iso-8859-1;q=0.5, big5;q=0.5, iso-2022-jp-2;q=0.5, gb2312;q=0.5, euc-tw;q=0.5, euc-kr;q=0.5, us-ascii;q=0.5, utf-7;q=0.5, hz-gb-2312;q=0.5, big5-hkscs;q=0.5, gbk;q=0.5, gb18030;q=0.5, iso-8859-5;q=0.5, koi8-r;q=0.5, koi8-u;q=0.5, cp866;q=0.5, koi8-t;q=0.5, windows-1251;q=0.5, cp855;q=0.5, iso-8859-2;q=0.5, iso-8859-3;q=0.5, iso-8859-4;q=0.5, iso-8859-9;q=0.5, iso-8859-10;q=0.5, iso-8859-13;q=0.5, iso-8859-14;q=0.5, iso-8859-15;q=0.5, windows-1250;q=0.5, windows-1252;q=0.5, windows-1254;q=0.5, windows-1257;q=0.5, cp775;q=0.5, cp850;q=0.5, cp852;q=0.5, cp857;q=0.5, cp858;q=0.5, cp860;q=0.5, cp861;q=0.5, cp863;q=0.5, cp865;q=0.5, cp437;q=0.5, macintosh;q=0.5, next;q=0.5, hp-roman8;q=0.5, adobe-standard-encoding;q=0.5, iso-8859-16;q=0.5, iso-8859-7;q=0.5, windows-1253;q=0.5, cp737;q=0.5, cp851;q=0.5, cp869;q=0.5, iso-8859-8;q=0.5, windows-1255;q=0.5, cp862;q=0.5, cp874;q=0.5, iso-8859-11;q=0.5, viscii;q=0.5, windows-1258;q=0.5, iso-8859-6;q=0.5, windows-1256;q=0.5, iso-2022-cn;q=0.5, iso-2022-cn-ext;q=0.5, iso-2022-kr;q=0.5, utf-16le;q=0.5, utf-16be;q=0.5, utf-16;q=0.5, x-ctext;q=0.5". --001a113b004e95f9ff0539e91bc7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Is t= his bug fixed in emacs-25? With "emacs -Q", I do not see this err= or anymore. Thank you very much.

=
But I set the language environment by (set-langu= age-environment "Japanese") in ~/.emacs.d/init.el.
With the configration, I still see the same error when us= ing anaconda-mode.

And url-mime-charset-string in url-http-create-request is = a multibyte stirng
and is bound to "eu= c-jp;q=3D1, iso-2022-jp;q=3D0.5, shift_jis;q=3D0.5, euc-jis-2004;q=3D0.5, i= so-2022-jp-2004;q=3D0.5, utf-8;q=3D0.5, iso-8859-1;q=3D0.5, big5;q=3D0.5, i= so-2022-jp-2;q=3D0.5, gb2312;q=3D0.5, euc-tw;q=3D0.5, euc-kr;q=3D0.5, us-as= cii;q=3D0.5, utf-7;q=3D0.5, hz-gb-2312;q=3D0.5, big5-hkscs;q=3D0.5, gbk;q= =3D0.5, gb18030;q=3D0.5, iso-8859-5;q=3D0.5, koi8-r;q=3D0.5, koi8-u;q=3D0.5= , cp866;q=3D0.5, koi8-t;q=3D0.5, windows-1251;q=3D0.5, cp855;q=3D0.5, iso-8= 859-2;q=3D0.5, iso-8859-3;q=3D0.5, iso-8859-4;q=3D0.5, iso-8859-9;q=3D0.5, = iso-8859-10;q=3D0.5, iso-8859-13;q=3D0.5, iso-8859-14;q=3D0.5, iso-8859-15;= q=3D0.5, windows-1250;q=3D0.5, windows-1252;q=3D0.5, windows-1254;q=3D0.5, = windows-1257;q=3D0.5, cp775;q=3D0.5, cp850;q=3D0.5, cp852;q=3D0.5, cp857;q= =3D0.5, cp858;q=3D0.5, cp860;q=3D0.5, cp861;q=3D0.5, cp863;q=3D0.5, cp865;q= =3D0.5, cp437;q=3D0.5, macintosh;q=3D0.5, next;q=3D0.5, hp-roman8;q=3D0.5, = adobe-standard-encoding;q=3D0.5, iso-8859-16;q=3D0.5, iso-8859-7;q=3D0.5, w= indows-1253;q=3D0.5, cp737;q=3D0.5, cp851;q=3D0.5, cp869;q=3D0.5, iso-8859-= 8;q=3D0.5, windows-1255;q=3D0.5, cp862;q=3D0.5, cp874;q=3D0.5, iso-8859-11;= q=3D0.5, viscii;q=3D0.5, windows-1258;q=3D0.5, iso-8859-6;q=3D0.5, windows-= 1256;q=3D0.5, iso-2022-cn;q=3D0.5, iso-2022-cn-ext;q=3D0.5, iso-2022-kr;q= =3D0.5, utf-16le;q=3D0.5, utf-16be;q=3D0.5, utf-16;q=3D0.5, x-ctext;q=3D0.5= ".

--001a113b004e95f9ff0539e91bc7-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 03:02:49 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 07:02:49 +0000 Received: from localhost ([127.0.0.1]:55666 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYSy1-00073h-EV for submit@debbugs.gnu.org; Sat, 13 Aug 2016 03:02:49 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46588) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYSxx-00073R-8w for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 03:02:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bYSxo-0000cm-HI for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 03:02:40 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:58971) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bYSxo-0000cV-E1; Sat, 13 Aug 2016 03:02:36 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4404 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bYSxm-0004YB-B6; Sat, 13 Aug 2016 03:02:34 -0400 Date: Sat, 13 Aug 2016 10:02:28 +0300 Message-Id: <83d1lddud7.fsf@gnu.org> From: Eli Zaretskii To: Sho Takemori In-reply-to: (message from Sho Takemori on Sat, 13 Aug 2016 09:30:35 +0900) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, larsi@gnus.org, schwab@linux-m68k.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Sho Takemori > Date: Sat, 13 Aug 2016 09:30:35 +0900 > Cc: Eli Zaretskii , larsi@gnus.org, schwab@linux-m68k.org, > 24117@debbugs.gnu.org > > But I set the language environment by (set-language-environment "Japanese") in ~/.emacs.d/init.el. > With the configration, I still see the same error when using anaconda-mode. > > And url-mime-charset-string in url-http-create-request is a multibyte stirng I guess we need to run url-mime-charset-string through url-http--encode-string as well. Can you try that? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 03:32:12 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 07:32:12 +0000 Received: from localhost ([127.0.0.1]:55688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYTQP-0007lm-2g for submit@debbugs.gnu.org; Sat, 13 Aug 2016 03:32:12 -0400 Received: from mail-oi0-f53.google.com ([209.85.218.53]:33632) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYTQJ-0007lE-Ju for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 03:32:07 -0400 Received: by mail-oi0-f53.google.com with SMTP id c15so9149138oig.0 for <24117@debbugs.gnu.org>; Sat, 13 Aug 2016 00:32:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=NOnNPdH4samff4L9BySkrdagaiH/I4lfK5/PsvUZKhA=; b=O/yYi17o/zX/Akg5gmJYC/yRBuKJL5ADt2CEG6BN0PRvOxbBJw1MRf/QkovAZDv6Ej 2aPbfsmRJBhmHVVrNz5HXhZOZbwh/F8+LUMJw5OoKgcvP2R1wXEU2P37g6fOoKXN7RW7 EhRxX84wvwqaiQ3CwFdyMvNFYKhacGm4PFXFhTSRie/1a/FoVdR3E++2LK8ETLiIi9hH MYetFO7boBfH30LefE6LKjyEfnIzb9z+yl9EdlaoqRSvTavQKP/RzR8kjsPvjgb75XFq ab1QJrnJYEdvGvGzZjfPJrvef+lOtwvzpIi6ifRqMZjmEk2RadzJ86391tknqDNUS+g0 uE4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=NOnNPdH4samff4L9BySkrdagaiH/I4lfK5/PsvUZKhA=; b=JkXpfDfkj1ki6Pgw3ty6clmWeV/4Dtgfq7EfZD3C+u6X913aFBxTHq8A69BM1bttKy dVaer8xeeF1BnWurRi7LCxNozGTFVuylPA92+/8HqO8xcO/IZ59yAmwaeSBCmWYjs2ne WndPrwx/uvp0t3tQ4BRgz+RPemvn5n3E116zJT5CMeLRof2rjGxxXuwJ3XRzMLqlctvs Tyhu8Ld8zw2P5LxDIfcnJUYxzcfZBmJzZbLe0GbAdRO5SwoyuxBOn6bSVPQ+OuOD6x66 1mAqPY6QU4+seYpACu/RpVI+U1LYkDc1z+1F71fJ8nl3y5MfBGxHcbJ7RCjdE4NgmZ2t kuAA== X-Gm-Message-State: AEkoouuhDRwfUU7pwNyqM3BDCHKecdmV4s1z0RJwxlCQ0TQbJOz7lhH7ewm9aUZcaWt80yiaiX/cvRPDSmwr1w== X-Received: by 10.157.49.2 with SMTP id e2mr7504479otc.121.1471073517979; Sat, 13 Aug 2016 00:31:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.37.66 with HTTP; Sat, 13 Aug 2016 00:31:56 -0700 (PDT) In-Reply-To: <83d1lddud7.fsf@gnu.org> References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> From: Sho Takemori Date: Sat, 13 Aug 2016 16:31:56 +0900 Message-ID: Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii Content-Type: multipart/mixed; boundary=001a114178867a6d030539eefe53 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, Lars Ingebrigtsen , schwab@linux-m68k.org, Brief Busters X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a114178867a6d030539eefe53 Content-Type: multipart/alternative; boundary=001a114178867a6cfd0539eefe51 --001a114178867a6cfd0539eefe51 Content-Type: text/plain; charset=UTF-8 > I guess we need to run url-mime-charset-string through > url-http--encode-string as well. Can you try that? Yes. I attached a patch. 2016-08-13 16:02 GMT+09:00 Eli Zaretskii : > > From: Sho Takemori > > Date: Sat, 13 Aug 2016 09:30:35 +0900 > > Cc: Eli Zaretskii , larsi@gnus.org, schwab@linux-m68k.org, > > 24117@debbugs.gnu.org > > > > But I set the language environment by (set-language-environment > "Japanese") in ~/.emacs.d/init.el. > > With the configration, I still see the same error when using > anaconda-mode. > > > > And url-mime-charset-string in url-http-create-request is a multibyte > stirng > > I guess we need to run url-mime-charset-string through > url-http--encode-string as well. Can you try that? > > Thanks. > --001a114178867a6cfd0539eefe51 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
> I guess we need to run= url-mime-charset-string through
> url-http--encode-string as well.=C2=A0 Can you= try that?

Yes. I attached a patch.

2016-08-13 16:02 GM= T+09:00 Eli Zaretskii <eliz@gnu.org>:
> From: Sho Takemori <= stakemorii@gmail.com>
> Date: Sat, 13 Aug 2016 09:30:35 +0900
> Cc: Eli Zaretskii <eliz@gnu.org= >, larsi@gnus.org, schwab@linux-m68k.org,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0241= 17@debbugs.gnu.org
>
> But I set the language environment by (set-language-environment "= Japanese") in ~/.emacs.d/init.el.
> With the configration, I still see the same error when using anaconda-= mode.
>
> And url-mime-charset-string in url-http-create-request is a multibyte = stirng

I guess we need to run url-mime-charset-string through
url-http--encode-string as well.=C2=A0 Can you try that?

Thanks.

--001a114178867a6cfd0539eefe51-- --001a114178867a6d030539eefe53 Content-Type: text/x-patch; charset=US-ASCII; name="diff.patch" Content-Disposition: attachment; filename="diff.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_irsv39ix0 ZGlmZiAtLWdpdCBhL2xpc3AvdXJsL3VybC1odHRwLmVsIGIvbGlzcC91cmwvdXJsLWh0dHAuZWwK aW5kZXggODYwZTY1Mi4uNTg0MGEwOCAxMDA2NDQKLS0tIGEvbGlzcC91cmwvdXJsLWh0dHAuZWwK KysrIGIvbGlzcC91cmwvdXJsLWh0dHAuZWwKQEAgLTMxMyw5ICszMTMsMTAgQEAgdXJsLWh0dHAt Y3JlYXRlLXJlcXVlc3QKIAkJCSAgKHNldHEgdXJsLW1pbWUtZW5jb2Rpbmctc3RyaW5nICJnemlw IikpKQogICAgICAgICAgICAgICAgICAoY29uY2F0CiAgICAgICAgICAgICAgICAgICAiQWNjZXB0 LWVuY29kaW5nOiAiIHVybC1taW1lLWVuY29kaW5nLXN0cmluZyAiXHJcbiIpKQotICAgICAgICAg ICAgIChpZiB1cmwtbWltZS1jaGFyc2V0LXN0cmluZwotICAgICAgICAgICAgICAgICAoY29uY2F0 Ci0gICAgICAgICAgICAgICAgICAiQWNjZXB0LWNoYXJzZXQ6ICIgdXJsLW1pbWUtY2hhcnNldC1z dHJpbmcgIlxyXG4iKSkKKyAgICAgICAgICAgICAodXJsLWh0dHAtLWVuY29kZS1zdHJpbmcKKyAg ICAgICAgICAgICAgKGlmIHVybC1taW1lLWNoYXJzZXQtc3RyaW5nCisgICAgICAgICAgICAgICAg ICAoY29uY2F0CisgICAgICAgICAgICAgICAgICAgIkFjY2VwdC1jaGFyc2V0OiAiIHVybC1taW1l LWNoYXJzZXQtc3RyaW5nICJcclxuIikpKQogICAgICAgICAgICAgIDs7IExhbmd1YWdlcyB3ZSB1 bmRlcnN0YW5kCiAgICAgICAgICAgICAgKGlmIHVybC1taW1lLWxhbmd1YWdlLXN0cmluZwogICAg ICAgICAgICAgICAgICAoY29uY2F0Cg== --001a114178867a6d030539eefe53-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 04:31:32 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 08:31:32 +0000 Received: from localhost ([127.0.0.1]:55711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYULo-0000l0-Ss for submit@debbugs.gnu.org; Sat, 13 Aug 2016 04:31:32 -0400 Received: from eggs.gnu.org ([208.118.235.92]:57367) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYULk-0000kk-8w for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 04:31:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bYULb-0007gq-6c for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 04:31:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59786) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bYULb-0007gU-35; Sat, 13 Aug 2016 04:31:15 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4722 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bYULY-0003uD-Rc; Sat, 13 Aug 2016 04:31:13 -0400 Date: Sat, 13 Aug 2016 11:31:06 +0300 Message-Id: <837fbldq9h.fsf@gnu.org> From: Eli Zaretskii To: Sho Takemori In-reply-to: (message from Sho Takemori on Sat, 13 Aug 2016 16:31:56 +0900) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, larsi@gnus.org, schwab@linux-m68k.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) > From: Sho Takemori > Date: Sat, 13 Aug 2016 16:31:56 +0900 > Cc: Brief Busters , Lars Ingebrigtsen , schwab@linux-m68k.org, > 24117@debbugs.gnu.org > > > I guess we need to run url-mime-charset-string through > > url-http--encode-string as well. Can you try that? > > Yes. I attached a patch. Thanks. I pushed a slightly different fix, please test it. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 09:02:32 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 13:02:32 +0000 Received: from localhost ([127.0.0.1]:55804 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYYa4-0002GP-Ux for submit@debbugs.gnu.org; Sat, 13 Aug 2016 09:02:32 -0400 Received: from mail-oi0-f42.google.com ([209.85.218.42]:33821) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYYa0-0002G7-6U for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 09:02:27 -0400 Received: by mail-oi0-f42.google.com with SMTP id l203so14183703oib.1 for <24117@debbugs.gnu.org>; Sat, 13 Aug 2016 06:02:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tT3Qg1UdpGfiG+m8F83nVxV+COqR0ZV6QxMK5nqAON4=; b=JDs2PqRjbOJLwCkvWR4hy634uIxcfVgaxENFwPlzV6uI9BnvZvyt40Ng3E0/XJvFpb QkpWyXNo5sXKQkLch8BS3AhddMJtS6ztAaEiCLnCxqAk0b/TVhc6tJGjM99bbkNeSKZQ B0JwPqFtsEywZgHc1mfR9xA4Md7dLWBci03UTNH7eyVNyIa5yivmCxX7nZ6H+ffocutX MiZctrQpl/jrPeRwKfYEIX6F7Olz0Shw2R28zQ86N74SWh9vn+cnEIfw3NYMz7TQTOVk by3ylyZ5RIPz7/0fvFOBZuKpF1T94q5BGw8n3v41IHInqgAfbEuTNl0Ia2REieVfDW3m dWKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tT3Qg1UdpGfiG+m8F83nVxV+COqR0ZV6QxMK5nqAON4=; b=eCHKOoLNU5LvczadumYijBbY3PLrm3dtniL6kgo9c3mpulRcRsxJgmDCHfP3gTcZOd /zTR/avAyjZjwgBiePwYgwoF+RIpVGcKJeQJ5Xdzb7KDjH+6oRbBWKRs35g/putatO62 Vt5MgWSkChrXa3LsCoKFFLkiHfE1+bPJ4yls0gcjQWdyn1OeBH3yiWKX+gdLEDdMLLq4 L3qcne06hF2yBH55l4SAYB+JmOjY+8oW7YjHs5gRk6Vp1xpIqmMVXeUyntgtmNEr2xCH qitl67qfmfPykdcWOUSxVlO9bQJpFbt6sQsDw12cS/U9gDNwSwCPprUkx6oQAKe1fTEt r8SQ== X-Gm-Message-State: AEkooutliFfUBZEir/o5BrU7by1kz7C1JElChZUZElBEUindNAl1fKEzNFfWgPDisQEPTSXEjBZoq7zO77j4Hw== X-Received: by 10.157.17.134 with SMTP id v6mr7916992otf.141.1471093338650; Sat, 13 Aug 2016 06:02:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.37.66 with HTTP; Sat, 13 Aug 2016 06:02:17 -0700 (PDT) In-Reply-To: <837fbldq9h.fsf@gnu.org> References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> <837fbldq9h.fsf@gnu.org> From: Sho Takemori Date: Sat, 13 Aug 2016 22:02:17 +0900 Message-ID: Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii Content-Type: multipart/alternative; boundary=94eb2c191454e1c0830539f39bca X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, Lars Ingebrigtsen , schwab@linux-m68k.org, Brief Busters X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --94eb2c191454e1c0830539f39bca Content-Type: text/plain; charset=UTF-8 > > Thanks. I pushed a slightly different fix, please test it. > anaconda-mode works fine. Thanks! 2016-08-13 17:31 GMT+09:00 Eli Zaretskii : > > From: Sho Takemori > > Date: Sat, 13 Aug 2016 16:31:56 +0900 > > Cc: Brief Busters , Lars Ingebrigtsen , > schwab@linux-m68k.org, > > 24117@debbugs.gnu.org > > > > > I guess we need to run url-mime-charset-string through > > > url-http--encode-string as well. Can you try that? > > > > Yes. I attached a patch. > > Thanks. I pushed a slightly different fix, please test it. > --94eb2c191454e1c0830539f39bca Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Thanks. I pushed a slightly different fix, please test it.

anaconda-mode works fine. Thanks!


2016-08-13 = 17:31 GMT+09:00 Eli Zaretskii <eliz@gnu.org>:
> From: Sho Takemori <stakemorii@gmail.com>
> Date: Sat, 13 Aug 2016 16:31:56 +0900
> Cc: Brief Busters <dgutov@yande= x.ru>, Lars Ingebrigtsen <larsi= @gnus.org>, schwab@linux-m6= 8k.org,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0241= 17@debbugs.gnu.org
>
> > I guess we need to run url-mime-charset-string through
> > url-http--encode-string as well. Can you try that?
>
> Yes. I attached a patch.

Thanks.=C2=A0 I pushed a slightly different fix, please test it.

--94eb2c191454e1c0830539f39bca-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 09:12:09 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 13:12:09 +0000 Received: from localhost ([127.0.0.1]:55812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYYjO-0002Xj-3A for submit@debbugs.gnu.org; Sat, 13 Aug 2016 09:12:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40211) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYYjJ-0002XA-HE for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 09:12:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bYYjA-0007ku-NU for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 09:11:56 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33791) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bYYjA-0007kq-K9; Sat, 13 Aug 2016 09:11:52 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1531 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bYYj7-0005wv-Ik; Sat, 13 Aug 2016 09:11:51 -0400 Date: Sat, 13 Aug 2016 16:11:38 +0300 Message-Id: <83wpjkdd9x.fsf@gnu.org> From: Eli Zaretskii To: Sho Takemori In-reply-to: (message from Sho Takemori on Sat, 13 Aug 2016 22:02:17 +0900) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> <837fbldq9h.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.6 (-----) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, larsi@gnus.org, schwab@linux-m68k.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.6 (-----) > From: Sho Takemori > Date: Sat, 13 Aug 2016 22:02:17 +0900 > Cc: Brief Busters , Lars Ingebrigtsen , schwab@linux-m68k.org, > 24117@debbugs.gnu.org > > Thanks. I pushed a slightly different fix, please test it. > > anaconda-mode works fine. Thanks! Great, thanks for testing. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 11:32:47 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 15:32:47 +0000 Received: from localhost ([127.0.0.1]:56510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYavW-00065s-TE for submit@debbugs.gnu.org; Sat, 13 Aug 2016 11:32:47 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:35407) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYavV-00065g-01 for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 11:32:45 -0400 Received: by mail-wm0-f43.google.com with SMTP id f65so24305142wmi.0 for <24117@debbugs.gnu.org>; Sat, 13 Aug 2016 08:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=cqx5g3f2WOun7/v8cuZjt/AInbmez0gfHTSkDF8hhLE=; b=Bvfu5wIjgLNyIAkZCYvqgnWNqu48EO+zJSv4J1GhTuFSKMC/wLp10ViF5EAuK8Mjsz Kw23pAeuOlcUqbzKKWhoUZSicblAucdOfxb3ZgQmfp0DokNQKsVYnIoQTHXd028kXU2H lzKPhMb2/hgF14bixuRnDEavx05nPnuk/dgjGG/DPjxIkT+jR91IrdFqrSDtP2rWPAIr j0GVrj5NP4+W4JCk4FrZvmIrREJNanPPXUgH6AMAA3LA3PPuqaCdXLsg7fgTyx/BUduO czUAW+Ydk0t0q76oQ9T/eTpGOcHH3Edy5ZXUrDrPqen003gAZuTE6lMv0MXHKkO5vTNO mtlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=cqx5g3f2WOun7/v8cuZjt/AInbmez0gfHTSkDF8hhLE=; b=mdg3aQ4Jltp0QvAAluwdSTp0W+e+/lLPSVcd90Z7INXSPox1l2V1D9Qh/iWzcLF2Ho VR8xrT4j5kSpPzKihajhUluMPEGlfU0TZGb0TRXwuMB9UAhbRrExqT2fRM9Xn9cUSBij NJ5k5J6HwPFxaZMmPAiG4AsPiuiJCwiSytYxzB91QDlGyWpe+SYjX0hBRiRrBdskMh3U DZeOwDFuU8HgFZR120RGiDsmTWs2hd5tXSq6dL0ElP5P1whGFG4OzCdn55a3ezoBfv3N i6XMJMLmvOFHp0+YEIgvHfUV91YJo9KZdWVxO6o1elCRUL5KcdzhUe28SZJOs5MuaFMr 2sLw== X-Gm-Message-State: AEkoousjX6IuPRL++fYRatphEzY3/9kwN1V4k484v/KLyVHga2P4n5jpm7ll2E0nb9sHNQ== X-Received: by 10.25.18.154 with SMTP id 26mr3363166lfs.186.1471102359170; Sat, 13 Aug 2016 08:32:39 -0700 (PDT) Received: from [192.168.1.190] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id p82sm1797732lfd.15.2016.08.13.08.32.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Aug 2016 08:32:38 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Eli Zaretskii , Sho Takemori References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> <837fbldq9h.fsf@gnu.org> From: Dmitry Gutov Message-ID: <1d177e2b-f79f-a5f3-b30b-a07d216a8279@yandex.ru> Date: Sat, 13 Aug 2016 18:32:36 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Thunderbird/47.0 MIME-Version: 1.0 In-Reply-To: <837fbldq9h.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 24117 Cc: 24117@debbugs.gnu.org, larsi@gnus.org, schwab@linux-m68k.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 08/13/2016 11:31 AM, Eli Zaretskii wrote: > I pushed a slightly different fix, please test it. Thanks, Eli. Sorry I didn't get to it first. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 13 11:56:47 2016 Received: (at 24117) by debbugs.gnu.org; 13 Aug 2016 15:56:47 +0000 Received: from localhost ([127.0.0.1]:56516 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYbIh-0006dn-Qi for submit@debbugs.gnu.org; Sat, 13 Aug 2016 11:56:47 -0400 Received: from eggs.gnu.org ([208.118.235.92]:34775) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bYbIc-0006dY-Pe for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 11:56:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bYbIT-0003Zu-Cy for 24117@debbugs.gnu.org; Sat, 13 Aug 2016 11:56:33 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34917) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bYbIT-0003Zp-9j; Sat, 13 Aug 2016 11:56:29 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1711 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bYbIR-0006Wl-6L; Sat, 13 Aug 2016 11:56:27 -0400 Date: Sat, 13 Aug 2016 18:56:21 +0300 Message-Id: <83popcd5ne.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <1d177e2b-f79f-a5f3-b30b-a07d216a8279@yandex.ru> (message from Dmitry Gutov on Sat, 13 Aug 2016 18:32:36 +0300) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83d1ltq3p6.fsf@gnu.org> <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <83d1lddud7.fsf@gnu.org> <837fbldq9h.fsf@gnu.org> <1d177e2b-f79f-a5f3-b30b-a07d216a8279@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.6 (-----) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, larsi@gnus.org, schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.6 (-----) > Cc: schwab@linux-m68k.org, larsi@gnus.org, 24117@debbugs.gnu.org > From: Dmitry Gutov > Date: Sat, 13 Aug 2016 18:32:36 +0300 > > On 08/13/2016 11:31 AM, Eli Zaretskii wrote: > > > I pushed a slightly different fix, please test it. > > Thanks, Eli. Sorry I didn't get to it first. No sweat. You make up for it many times over, when I'm not available. From debbugs-submit-bounces@debbugs.gnu.org Mon May 08 09:36:46 2017 Received: (at 24117) by debbugs.gnu.org; 8 May 2017 13:36:46 +0000 Received: from localhost ([127.0.0.1]:33083 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d7iqE-0004dX-Bc for submit@debbugs.gnu.org; Mon, 08 May 2017 09:36:46 -0400 Received: from mail-wm0-f65.google.com ([74.125.82.65]:33890) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d7iqC-0004dK-9T for 24117@debbugs.gnu.org; Mon, 08 May 2017 09:36:44 -0400 Received: by mail-wm0-f65.google.com with SMTP id d127so7167407wmf.1 for <24117@debbugs.gnu.org>; Mon, 08 May 2017 06:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=X+mDAlHWCkUnEpHKdzSXYMpUtfi9NDX3nHVlbfBh3p0=; b=rBwiNF1vZPuILaZH0Ymm5l5NCuzXVcU+2dw2xRDFH0fMVCSzApOPsnm4qBozh4+uqy fT590LsL37p2NqV/xCykCIixJEMaHfuJg5a3vvueqDUZKq4QoLo6CHdEQ+dBrvjOmmXS XRciyxmHl9VYg4eYXvpwBub8kqkl85lrc44gjTSoqXwiXL7hIU5hVUYpcwLiqlcrhaVb 9BnDs45yeUWSqEb9hZ7tG9v06WN2m96mIiKhfhMrc2rNhEHwuyyAeml+Tg6DRlGCQlN2 Z6xSMB5jlw647xQYqntXUjuZ6NLitwEqF2fTPuFlW759rZeR9XPFEBYBeIqaOUunrlut Ngrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=X+mDAlHWCkUnEpHKdzSXYMpUtfi9NDX3nHVlbfBh3p0=; b=izuqLcjNbVk9LCvGtZHIoMFXfRQxoCpIULpPZrSX2U5I5smUNTsZMPQA0cuy67ww05 UqLuxmmoq4v64JzysjNpGY5oGSZ+LNwxuN6sAqBEnrAXhTp2/wrL0tI5JG1bODHIxcDV M+TmGvH+Y+TkNDPCTPKyGIhDrH9v/zjlWKuAIFnw5n2/gwfUU8i1+axDJ8qON1jiFtgV 51Pqbt9gCtSm4+Uqa4KUrqUiIhLCgKRNjwTtHIFv/lJBGZfTuHcx3PtYfcgXBp/ySVte LBcUqHSnQrOIrky1S4H/ihxzjUWW+fFpF3Ey8Lw6zlLbYhOWPFCa8g4jnT/ydhDOPCWb B2dQ== X-Gm-Message-State: AN3rC/4ukFk2iZKfuUWGpkBAWxRNJ247v+3/k+L/fUlfRqPw+N4ww+mo WHvVk9bsvx/Wm89Odcs= X-Received: by 10.28.31.135 with SMTP id f129mr12649806wmf.78.1494250598566; Mon, 08 May 2017 06:36:38 -0700 (PDT) Received: from [192.168.0.133] ([212.50.99.193]) by smtp.googlemail.com with ESMTPSA id q140sm11857660wmb.14.2017.05.08.06.36.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 May 2017 06:36:37 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Ted Zlatanov References: <83popsocg8.fsf@gnu.org> <7fb3540a-7b74-68cf-2c63-66474de26640@yandex.ru> <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> <874m6rjwdt.fsf_-_@lifelogs.com> From: Dmitry Gutov Message-ID: <605199d2-551d-07c8-71b4-ca73c008246a@yandex.ru> Date: Mon, 8 May 2017 16:36:36 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Thunderbird/53.0 MIME-Version: 1.0 In-Reply-To: <874m6rjwdt.fsf_-_@lifelogs.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Lars Magne Ingebrigtsen , schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.7 (/) On 11.08.2016 15:57, Ted Zlatanov wrote: > DG> On 08/11/2016 11:53 AM, Ted Zlatanov wrote: >>> Could you add to your patch the cases you've tested? There's a specific >>> place for URL parsing tests in test/lisp/url/url-parse-tests.el that >>> would help everyone. > > DG> Sure, but only one of the patches affects URL parsing (and Lars prefers the > DG> other one). > > Maybe the tests should be in a separate patch then. Neither your Russian > example nor Lars' example have a parallel in the tests AFAICS. I'd also > add the example hostname that Katsumi Yamaoka gave from the w3m source. Just got around to this. The test I came up with looks like this: (ert-deftest url-generic-parse-url/multibyte-host-and-path () (should (equal (url-generic-parse-url "http://банки.рф/фыва/") (url-parse-make-urlobj "http" nil nil "банки.рф" nil "/фыва/" nil nil t)))) But! What behavior would this test? If we're making sure here that url-generic-parse-url can cope with multibyte characters anywhere in the URL, the encode-coding-string/decode-coding-string logic in url-encode-url is extraneous. I'm not sure that it is, or is there are some edge cases (are they fixable? should we add tests for them?). So if this test goes in, it should be accompanied with the simplification of url-encode-url. Lars, what do you think? From debbugs-submit-bounces@debbugs.gnu.org Mon May 08 16:57:43 2017 Received: (at 24117) by debbugs.gnu.org; 8 May 2017 20:57:43 +0000 Received: from localhost ([127.0.0.1]:60290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d7piw-00008z-Ph for submit@debbugs.gnu.org; Mon, 08 May 2017 16:57:42 -0400 Received: from hermes.netfonds.no ([80.91.224.195]:37316) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d7piv-00008q-3n for 24117@debbugs.gnu.org; Mon, 08 May 2017 16:57:41 -0400 Received: from cm-84.209.243.26.getinternet.no ([84.209.243.26] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1d7pio-00056M-S4; Mon, 08 May 2017 22:57:36 +0200 From: Lars Ingebrigtsen To: Dmitry Gutov Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request References: <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> <874m6rjwdt.fsf_-_@lifelogs.com> <605199d2-551d-07c8-71b4-ca73c008246a@yandex.ru> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEU/QFQXCR8dEiccDiYf HjUcDCQNBhsgFyevZmOwAAACSUlEQVQ4jXWSTXObMBCG1Rmmd6U1OXsF1rlSydlDd6JzrNnmGjuu 7qEQ/f2+wsaJM80aPLAP735KEbPxnTci0RtH3hNR77sfquHeex+9gBhnOvA8kDeKCL5YgBfvbRu7 aIeBSHnni7PcnYjzro1kGKDzDvG7M5ASkkoe1VFDiNzJystKSsjoOvnJSprG4K3TWkTrCO7Jx6YH oC4i2ApAHqRExZcNKSGavmVEwq/4pdsR+jEqDqGNYytaigRZDDlDALsxU7SIslqJ/BUZ2oFmYEca KMtsRTO0DsUarywEZFuROhX7Yy1xT+KUHVCspYd0tv2Omt5EpzhM8piHlLaJZvK8AM6w34l5dvcp 3WAUnlQoYEpYC3zkNpA0hl2vMoecX9OBm19UyoHKedr1asyQICdTwxlbI5cOhiIXEF7SHjNumHEx bZ7RPKkM8IqEYQyBi5HRE3akeOTq65PiixFtyqbUxd7A2oKo7QKqCxgsUVB5zDgu7hKpZxswP1al bw7NWw4ObIfd+B/AADQiZ8jjBxBkR6WYPPZXfqaHTwA2tCsAU3zn3ar7pPUMMHb8V4u/wki1m5Pn 6cg891g6vcfkNXLAj81+v4ymKou8sScw1el4nk11NwMo8gmkdAtUBX86K9jHrNDL6UmHE1jPI8n5 pU5XpvUZaEjqdarvDmlDaZNuaVHo+km9ty9nMEz6qLZbru6xynK9LIpJX2Wpa1WO1WOJtRS2xyd6 XwCfkpzBvtZzJerkxPN1waXcQetZft0JwLoIPgH1Pn20fz1YNgkbGTfPAAAAAElFTkSuQmCC Date: Mon, 08 May 2017 22:57:34 +0200 In-Reply-To: <605199d2-551d-07c8-71b4-ca73c008246a@yandex.ru> (Dmitry Gutov's message of "Mon, 8 May 2017 16:36:36 +0300") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 24117 Cc: stakemorii@gmail.com, Ted Zlatanov , schwab@linux-m68k.org, 24117@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Dmitry Gutov writes: > Just got around to this. The test I came up with looks like this: > > (ert-deftest url-generic-parse-url/multibyte-host-and-path () > (should (equal (url-generic-parse-url "http://=D0=B1=D0=B0=D0=BD=D0=BA= =D0=B8.=D1=80=D1=84/=D1=84=D1=8B=D0=B2=D0=B0/") > (url-parse-make-urlobj "http" nil nil "=D0=B1=D0=B0=D0= =BD=D0=BA=D0=B8.=D1=80=D1=84" nil > "/=D1=84=D1=8B=D0=B2=D0=B0/" nil = nil t)))) That looks like the correct decomposition of this URL, so url-generic-parse-url does the right thing. > But! What behavior would this test? If we're making sure here that > url-generic-parse-url can cope with multibyte characters anywhere in > the URL, the encode-coding-string/decode-coding-string logic in > url-encode-url is extraneous. I'm not sure that it is, or is there are > some edge cases (are they fixable? should we add tests for them?). (url-encode-url "http://=D0=B1=D0=B0=D0=BD=D0=BA=D0=B8.=D1=80=D1=84/=D1=84= =D1=8B=D0=B2=D0=B0/") =3D> "http://=D0=B1=D0=B0=D0=BD=D0=BA=D0=B8.=D1=80=D1=84/%D1%84%D1%8B%D0%B2= %D0%B0/" It is perhaps debatable whether the host name should be encoded (with punycode) here, but this is otherwise correct. > So if this test goes in, it should be accompanied with the > simplification of url-encode-url. > > Lars, what do you think? The utf-8 encoding does seem superfluous, especially since url-hexify-string also does the encoding... (url-hexify-string "=D1=84=D1=8B=D0=B2=D0=B0") =3D> "%D1%84%D1%8B%D0%B2%D0%B0" --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Tue May 09 20:40:28 2017 Received: (at 24117-done) by debbugs.gnu.org; 10 May 2017 00:40:28 +0000 Received: from localhost ([127.0.0.1]:34776 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d8Fg4-0006RT-2w for submit@debbugs.gnu.org; Tue, 09 May 2017 20:40:28 -0400 Received: from mail-wr0-f172.google.com ([209.85.128.172]:34243) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d8Fg2-0006RE-2a for 24117-done@debbugs.gnu.org; Tue, 09 May 2017 20:40:26 -0400 Received: by mail-wr0-f172.google.com with SMTP id l9so21683707wre.1 for <24117-done@debbugs.gnu.org>; Tue, 09 May 2017 17:40:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=6UiK6rNFMuHT/vTWmsW3Bgy6zNN75NmIst0mFu5qFJk=; b=iLI2aikbWcSg7m+jg5yN6EZCHJDA9XHqjNOnGJ+MyrtS1NCGgKR5KBWIeoxz54eK4R MpIUjXCdvI5MC44tvPjH8fYdhM5ybDonfvMZ3vKTkOzt0Gbu/9LLisN4okjeJGsmMwOe TEvWmFFECAH4FlpcXs1V87vm0+n854G+7OwaZR5p3ci4cavQQL+iVLWs7lYnAKwKsytY mejq+RqeD5pEe9yjfvuDDLuLRVwMj4eULXE3R8HTOADFnxsimEWFrheD4Ne8Nm8colqq TFV6PiQ4s2jpYScqwSUCWlIV8XTtOnWAJRccLCcrgoorfwWe36xc3mUsjfNwGE/89O2c Fchg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=6UiK6rNFMuHT/vTWmsW3Bgy6zNN75NmIst0mFu5qFJk=; b=WXocBRhbxXridWWhTg4UQj/o06BbucNEihCTmrXFub2N2yRLoMGyCY1H1/WbBqb+oi BOuxOTeGXZERPVoGwXbWcxa7pPYLRha28kOGWGdBBWd7wxNFidbnIkv2DsTvuCka/z7y JVTIYl2vetDDLkdE4qvXwvoLql9wKlMxxqJnVc5OScU2jRURyUMVX1BTrYyw/Kx1KB5p az6VmHxBLMeY9ZvM4iw3+5X8GW42A9nL3I90qygCP35N5PPVSg109P7sh0rwEEVA5sq3 7BiUdwm3upB9OfdbGRNQi3kpEskDMTCOk4/gNSupud7rxtw50o1wacmKNFHNk2t6K2X6 OM0Q== X-Gm-Message-State: AODbwcAylcduHPMQBLQK02oO9xcVoAqZJwveiOQHQsOu+eFvz2ysdemL e+/4JFMwX3sTTA== X-Received: by 10.28.103.3 with SMTP id b3mr1577241wmc.5.1494376820294; Tue, 09 May 2017 17:40:20 -0700 (PDT) Received: from [192.168.1.3] ([185.105.173.156]) by smtp.googlemail.com with ESMTPSA id i11sm1268343wrb.44.2017.05.09.17.40.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 May 2017 17:40:19 -0700 (PDT) Subject: Re: bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request To: Lars Ingebrigtsen References: <83mvkvmbv2.fsf@gnu.org> <27168f12-32d2-cb38-45c0-27d3339c75aa@yandex.ru> <83twf0lb5s.fsf@gnu.org> <83lh07i6g3.fsf@gnu.org> <83k2fri5kc.fsf@gnu.org> <87oa53i3si.fsf@linux-m68k.org> <83bn13i2x2.fsf@gnu.org> <87fuqfhy0q.fsf@linux-m68k.org> <837fbqise6.fsf@gnu.org> <834m6uhu87.fsf@gnu.org> <65f6508f-a464-7f66-fd14-1372dce86aa7@yandex.ru> <83bn10hetr.fsf@gnu.org> <50426141-3483-e5e4-a252-20b1198cde30@yandex.ru> <874m6rjwdt.fsf_-_@lifelogs.com> <605199d2-551d-07c8-71b4-ca73c008246a@yandex.ru> From: Dmitry Gutov Message-ID: Date: Wed, 10 May 2017 03:40:07 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Thunderbird/53.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.1 (--) X-Debbugs-Envelope-To: 24117-done Cc: stakemorii@gmail.com, Ted Zlatanov , schwab@linux-m68k.org, 24117-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) On 08.05.2017 23:57, Lars Ingebrigtsen wrote: > The utf-8 encoding does seem superfluous, especially since > url-hexify-string also does the encoding... Simplified and pushed. With that, I am closing the bug. Thanks all. From unknown Wed Jun 18 23:07:15 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 07 Jun 2017 11:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator