From unknown Fri Jun 20 07:14:48 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#22667 <22667@debbugs.gnu.org> To: bug#22667 <22667@debbugs.gnu.org> Subject: Status: string->bytevector encoding error Reply-To: bug#22667 <22667@debbugs.gnu.org> Date: Fri, 20 Jun 2025 14:14:48 +0000 retitle 22667 string->bytevector encoding error reassign 22667 guile submitter 22667 Josep Portella Florit severity 22667 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Feb 14 17:18:37 2016 Received: (at submit) by debbugs.gnu.org; 14 Feb 2016 22:18:37 +0000 Received: from localhost ([127.0.0.1]:39126 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aV501-000303-89 for submit@debbugs.gnu.org; Sun, 14 Feb 2016 17:18:37 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34860) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aV4zz-0002zr-8a for submit@debbugs.gnu.org; Sun, 14 Feb 2016 17:18:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aV4zs-0001Oq-Ua for submit@debbugs.gnu.org; Sun, 14 Feb 2016 17:18:29 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:51422) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aV4zs-0001Om-R3 for submit@debbugs.gnu.org; Sun, 14 Feb 2016 17:18:28 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54064) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aV4zs-0008W5-0v for bug-guile@gnu.org; Sun, 14 Feb 2016 17:18:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aV4zo-0001OF-0N for bug-guile@gnu.org; Sun, 14 Feb 2016 17:18:27 -0500 Received: from primfilat.com ([71.19.154.166]:59950) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aV4zn-0001O7-R4 for bug-guile@gnu.org; Sun, 14 Feb 2016 17:18:23 -0500 Received: from [192.168.0.4] (unknown [46.18.40.216]) by primfilat.com (Postfix) with ESMTPSA id A79E47BB52 for ; Sun, 14 Feb 2016 23:18:21 +0100 (CET) From: Josep Portella Florit Subject: string->bytevector encoding error X-Enigmail-Draft-Status: N1110 To: bug-guile@gnu.org Message-ID: <56C0FD28.9050709@primfilat.com> Date: Sun, 14 Feb 2016 23:18:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) 'string->bytevector' throws 'encoding-error' when the encoding is "UTF-16" and the length of the string is greater than 128. The same for the encoding "UTF-32" and a string of length greater than 64. Tested on Guile 2.0.11 and 2.1.2 with the same result. How to reproduce: (use-modules (ice-9 iconv)) ;; These expressions evaluate as expected: (string->bytevector (make-string 128 #\A) "UTF-16") ;; =3D> #vu8(254 255 0 65 0 65 ...) (string->bytevector (make-string 64 #\A) "UTF-32") ;; =3D> #vu8(0 0 254 255 0 0 0 65 0 0 0 65 ...) ;; These expressions throw 'encoding-error': (string->bytevector (make-string 129 #\A) "UTF-16") (string->bytevector (make-string 65 #\A) "UTF-32") From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 03 12:56:15 2016 Received: (at 22667) by debbugs.gnu.org; 3 Mar 2016 17:56:15 +0000 Received: from localhost ([127.0.0.1]:60431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1abXTz-0005o5-Lr for submit@debbugs.gnu.org; Thu, 03 Mar 2016 12:56:15 -0500 Received: from world.peace.net ([50.252.239.5]:38296 ident=hope5) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1abXTx-0005nx-HS for 22667@debbugs.gnu.org; Thu, 03 Mar 2016 12:56:13 -0500 Received: from pool-71-174-32-69.bstnma.east.verizon.net ([71.174.32.69] helo=jojen) by world.peace.net with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1abXTr-0005Kr-12; Thu, 03 Mar 2016 12:56:07 -0500 From: Mark H Weaver To: Josep Portella Florit Subject: Re: bug#22667: string->bytevector encoding error References: <56C0FD28.9050709@primfilat.com> Date: Thu, 03 Mar 2016 12:55:55 -0500 In-Reply-To: <56C0FD28.9050709@primfilat.com> (Josep Portella Florit's message of "Sun, 14 Feb 2016 23:18:16 +0100") Message-ID: <87oaavsaxg.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 22667 Cc: 22667@debbugs.gnu.org, Ludovic =?utf-8?Q?Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Josep Portella Florit writes: > 'string->bytevector' throws 'encoding-error' when the encoding is > "UTF-16" and the length of the string is greater than 128. The same > for the encoding "UTF-32" and a string of length greater than 64. > > Tested on Guile 2.0.11 and 2.1.2 with the same result. > > How to reproduce: > > (use-modules (ice-9 iconv)) > > ;; These expressions evaluate as expected: > (string->bytevector (make-string 128 #\A) "UTF-16") > ;; => #vu8(254 255 0 65 0 65 ...) > (string->bytevector (make-string 64 #\A) "UTF-32") > ;; => #vu8(0 0 254 255 0 0 0 65 0 0 0 65 ...) > > ;; These expressions throw 'encoding-error': > (string->bytevector (make-string 129 #\A) "UTF-16") > (string->bytevector (make-string 65 #\A) "UTF-32") Thee bug is in 'display_string_using_iconv', introduced in commit f4bc4e5934, which improperly assumes that a 256-byte buffer (encoded_output) will be large enough to hold the results of converting up to 256 bytes of UTF-8 (utf8_buf) to an arbitrary encoding using 'iconv'. When this assumption fails, the call to 'iconv' on line 997 done = iconv (id->output_cd, &input, &input_left, &output, &output_left); reports a failure due to the output buffer being too small, and the code fails to handle this case. To be continued... Mark From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 20 12:07:32 2016 Received: (at 22667) by debbugs.gnu.org; 20 Jun 2016 16:07:32 +0000 Received: from localhost ([127.0.0.1]:47915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bF1jY-0003Ch-Ec for submit@debbugs.gnu.org; Mon, 20 Jun 2016 12:07:32 -0400 Received: from pb-sasl1.pobox.com ([64.147.108.66]:60963 helo=sasl.smtp.pobox.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bF1jW-0003CZ-NX for 22667@debbugs.gnu.org; Mon, 20 Jun 2016 12:07:31 -0400 Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id 5C68223444; Mon, 20 Jun 2016 12:07:30 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=1eQeCq5SArlayrxUl7iMMqrxHOY=; b=wokP1S jkptVX3iN4bFfA7LbUkWhJyZ2osqzDudaNmN3bYcTv1w37jHRiXb7e02tL2GpCfp C6X+Yp5dZj64YgmsLTNzlrt+BsL996soOZSmVaJbpECwUzVqsOZwzlTcx8YuOttT 8J1WbOVR6j7jYgpZtCR8zVEuBQoCt2UgL2kqc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=VEX3ZHsoB4WgzNZQIInw6Uqj340dff7N 9w4jIbKxnzkMuvcUDuEnPYgVgJFn+cFhotfjLkx77zk4B6v8JUdTKEQEAzXqA59b aOYpmtZzJhbwivUi23iqIXer0lA+2PrEC5Rkv7DP+YrS+rP+vIUUSEVV3cKxYTAi GYkgh4x/W2o= Received: from pb-sasl1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id 564ED23443; Mon, 20 Jun 2016 12:07:30 -0400 (EDT) Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl1.pobox.com (Postfix) with ESMTPSA id 6D10C23442; Mon, 20 Jun 2016 12:07:29 -0400 (EDT) From: Andy Wingo To: Josep Portella Florit Subject: Re: bug#22667: string->bytevector encoding error References: <56C0FD28.9050709@primfilat.com> Date: Mon, 20 Jun 2016 18:07:21 +0200 In-Reply-To: <56C0FD28.9050709@primfilat.com> (Josep Portella Florit's message of "Sun, 14 Feb 2016 23:18:16 +0100") Message-ID: <87lh1zg7ye.fsf@pobox.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: 16C392C8-3701-11E6-914A-C1836462E9F6-02397024!pb-sasl1.pobox.com X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 22667 Cc: 22667@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) On Sun 14 Feb 2016 23:18, Josep Portella Florit writes: > 'string->bytevector' throws 'encoding-error' when the encoding is > "UTF-16" and the length of the string is greater than 128. The same > for the encoding "UTF-32" and a string of length greater than 64. > > Tested on Guile 2.0.11 and 2.1.2 with the same result. > > How to reproduce: > > (use-modules (ice-9 iconv)) > > ;; These expressions evaluate as expected: > (string->bytevector (make-string 128 #\A) "UTF-16") > ;; => #vu8(254 255 0 65 0 65 ...) > (string->bytevector (make-string 64 #\A) "UTF-32") > ;; => #vu8(0 0 254 255 0 0 0 65 0 0 0 65 ...) > > ;; These expressions throw 'encoding-error': > (string->bytevector (make-string 129 #\A) "UTF-16") > (string->bytevector (make-string 65 #\A) "UTF-32") For what it's worth, this appears to be fixed in 2.1.3. However we should fix 2.0 as well. Andy From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 07 17:25:40 2016 Received: (at 22667-done) by debbugs.gnu.org; 7 Aug 2016 21:25:40 +0000 Received: from localhost ([127.0.0.1]:59263 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWVZk-00029t-Lg for submit@debbugs.gnu.org; Sun, 07 Aug 2016 17:25:40 -0400 Received: from pb-sasl2.pobox.com ([64.147.108.67]:54957 helo=sasl.smtp.pobox.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bWVZj-00029m-GI for 22667-done@debbugs.gnu.org; Sun, 07 Aug 2016 17:25:39 -0400 Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id E7AD3308D5; Sun, 7 Aug 2016 17:25:37 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=x1CZThAIV14rzje57wBurGEiaKc=; b=yGDHgw DYv1z0BYGBz3j9SB+JLfKE6G3y8uSiuLC5tHespiBz0DdRa4Z74OIHqjnu10d4J1 HrUdH1zilGTIB0lZl/u3C8ALXMm02Z/b0ML+iRlQmt469Rw1QFf/J6wqciXozdGH sgvvrAujUAu8gLAmCtfuagvivEYLFMRsLNlns= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=GX1nEkLcJiIg4u0nm7kQurSynlGiZFIg +8rAZBoszUAz1icV8FHHlJXw1esrviPIKxNbuzIhv5A/F6ltcqyzHgod04zxnhY4 hqgmBaM5jCPwAI4GtBPadPgbsr0O75n6sRFDQ54pzIPILMweNFvioo7b4X8rpoxI t5O2DM96HL0= Received: from pb-sasl2.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id D5A58308D4; Sun, 7 Aug 2016 17:25:37 -0400 (EDT) Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl2.pobox.com (Postfix) with ESMTPSA id 00E3D308D2; Sun, 7 Aug 2016 17:25:36 -0400 (EDT) From: Andy Wingo To: Josep Portella Florit Subject: Re: bug#22667: string->bytevector encoding error References: <56C0FD28.9050709@primfilat.com> <87lh1zg7ye.fsf@pobox.com> Date: Sun, 07 Aug 2016 23:25:29 +0200 In-Reply-To: <87lh1zg7ye.fsf@pobox.com> (Andy Wingo's message of "Mon, 20 Jun 2016 18:07:21 +0200") Message-ID: <87y448nueu.fsf@pobox.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: 7BA2E306-5CE5-11E6-B6DA-28A6F1301B6D-02397024!pb-sasl2.pobox.com X-Spam-Score: -0.4 (/) X-Debbugs-Envelope-To: 22667-done Cc: 22667-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.4 (/) Hi, On Mon 20 Jun 2016 18:07, Andy Wingo writes: > On Sun 14 Feb 2016 23:18, Josep Portella Florit writes: > >> (use-modules (ice-9 iconv)) >> ;; These expressions throw 'encoding-error': >> (string->bytevector (make-string 129 #\A) "UTF-16") >> (string->bytevector (make-string 65 #\A) "UTF-32") > > For what it's worth, this appears to be fixed in 2.1.3. However we > should fix 2.0 as well. Fixed. Thanks for the report, and thanks to Mark for diagnosing the problem. Andy. From unknown Fri Jun 20 07:14:48 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 05 Sep 2016 11:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator