GNU bug report logs - #11197
problems with string ports and unicode

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: tracker <at> debbugs.gnu.org
Subject: bug#11197: closed (problems with string ports and unicode)
Date: Wed, 20 Jun 2012 21:07:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Wed, 20 Jun 2012 23:03:02 +0200
with message-id <87obodn1bt.fsf <at> gnu.org>
and subject line Re: bug#11197: problems with string ports and unicode
has caused the debbugs.gnu.org bug report #11197,
regarding problems with string ports and unicode
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
11197: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11197
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Klaus Stehle <klaus.stehle <at> uni-tuebingen.de>
To: bug-guile <at> gnu.org
Subject: problems with string ports and unicode
Date: Sat, 7 Apr 2012 22:07:01 +0200 (CEST)

[Message part 3 (text/plain, inline)]

Hi,

;;;; a very very short example script to describe the problem:

;; open a string port with unicode characters >= 0x0100
(define p (open-input-string "čtyří"))


Put the line into a script and start guile. You will see the output:
=> Backtrace:

That's all, and guile will hang in an eternal loop.

If you enter the line interactively into the REPL, everything works
properly and you can read all characters with (read-char p).



;;;; another very short script, which is possibly the same problem:

;; open a string port and unread a unicode character >= 0x0100
(define p (open-input-string "ibenik"))
(unread-char #\Š p)


Running these two lines as a script generates an error message:
=> ERROR: In procedure unread-char:
=> ERROR: Throw to key `encoding-error' with args
          `("scm_ungetc" "conversion to port encoding failed" 84 #f #\540)'.

If you enter the lines interactively into the REPL, everything works
properly and you can read all characters with (read-char p).


Cheers,
Klaus Stehle


----------------------------
guile --version
guile (GNU Guile) 2.0.5

uname -srm
Linux 2.6.32-5-amd64 x86_64

echo $LANG
de_DE.UTF-8

[Message part 4 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: 11197-done <at> debbugs.gnu.org, Klaus Stehle <klaus.stehle <at> uni-tuebingen.de>
Subject: Re: bug#11197: problems with string ports and unicode
Date: Wed, 20 Jun 2012 23:03:02 +0200

Hi,

ludo <at> gnu.org (Ludovic Courtès) skribis:

> Indeed, it’s stuck in a deadlock:
>
> (gdb) bt
> #0  0x00007ffff75e1204 in __lll_lock_wait () from /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #1  0x00007ffff75dc4d4 in _L_lock_999 () from /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #2  0x00007ffff75dc2ea in pthread_mutex_lock () from /nix/store/vxycd107wjbhcj720hzkw2px7s7kr724-glibc-2.12.2/lib/libpthread.so.0
> #3  0x00007ffff7b30499 in scm_dynwind_pthread_mutex_lock (mutex=0x7ffff7dd28c0) at threads.c:1962
> #4  0x00007ffff7b2bb0e in scm_mkstrport (pos=0x2, str=0x4, modes=327680, caller=<value optimized out>) at strports.c:287
> #5  0x00007ffff7aac20b in display_backtrace_body (a=0x7fffffffc1a0) at backtrace.c:487
> #6  0x00007ffff7b46c7b in vm_regular_engine (vm=0x6f61f0, program=0x7f5d50, argv=0x6fa3b0, nargs=-1) at vm-i-system.c:895
> #7  0x00007ffff7ac039e in scm_call_3 (proc=0x7f5d50, arg1=<value optimized out>, arg2=<value optimized out>, arg3=<value optimized out>) at eval.c:500
> #8  0x00007ffff7b32504 in scm_internal_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>) at throw.c:222
> #9  0x00007ffff7aabbba in scm_display_backtrace_with_highlights (stack=<value optimized out>, port=<value optimized out>, first=<value optimized out>, depth=<value optimized out>, highlights=<value optimized out>)
>     at backtrace.c:558
> #10 0x00007ffff7ab725e in print_exception_and_backtrace (error_port=0x6f6170, tag=0x66d4c0, args=0x8e6ea0) at continuations.c:490
> #11 pre_unwind_handler (error_port=0x6f6170, tag=0x66d4c0, args=0x8e6ea0) at continuations.c:534
> #12 0x00007ffff7b46c7b in vm_regular_engine (vm=0x6f61f0, program=0x7f3ce0, argv=0x6fa300, nargs=-1) at vm-i-system.c:895
> #13 0x00007ffff7b4846e in scm_call_with_vm (vm=0x6f61f0, proc=0x7f3ce0, args=<value optimized out>) at vm.c:878
> #14 0x00007ffff7b296db in scm_to_stringn (str=0x8dba80, lenp=0x7fffffffc4e8, encoding=<value optimized out>, handler=SCM_FAILED_CONVERSION_ERROR) at strings.c:2102
> #15 0x00007ffff7b2bb73 in scm_mkstrport (pos=0x2, str=0x8dba80, modes=196608, caller=<value optimized out>) at strports.c:312
>
> This could be fixed by calling ‘scm_new_port_table_entry’ after having
> prepared the backing buffer, but the problem is that ‘pt->encoding’ is
> needed before.

Fixed in 03fcf93bff9f02a3d12ab86be4e67b996310aad4 (not particularly
elegant, but I couldn’t think of a better way.)  The test in that commit
captures the initial problem.

I’m marking this bug as “done”.  If you would like to discuss string
port encodings, separate binary/textual ports, or any other significant
change, you’re welcome to do so on guile-devel <at> gnu.org, of course.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.