From unknown Sat Jun 14 03:49:00 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#39610 <39610@debbugs.gnu.org> To: bug#39610 <39610@debbugs.gnu.org> Subject: Status: R6RS `flush-output-port` not playing along with `transcoded-port` Reply-To: bug#39610 <39610@debbugs.gnu.org> Date: Sat, 14 Jun 2025 10:49:00 +0000 retitle 39610 R6RS `flush-output-port` not playing along with `transcoded-p= ort` reassign 39610 guile submitter 39610 Andreas Rottmann severity 39610 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 14 19:34:19 2020 Received: (at submit) by debbugs.gnu.org; 15 Feb 2020 00:34:19 +0000 Received: from localhost ([127.0.0.1]:34893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j2lPX-0005GI-85 for submit@debbugs.gnu.org; Fri, 14 Feb 2020 19:34:19 -0500 Received: from lists.gnu.org ([209.51.188.17]:46429) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j2l6w-0004np-FG for submit@debbugs.gnu.org; Fri, 14 Feb 2020 19:15:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53861) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2l6v-0003u3-FK for bug-guile@gnu.org; Fri, 14 Feb 2020 19:15:06 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2l6u-000809-I8 for bug-guile@gnu.org; Fri, 14 Feb 2020 19:15:05 -0500 Received: from ciao.gmane.io ([159.69.161.202]:46382) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j2l6u-0007ub-Bp for bug-guile@gnu.org; Fri, 14 Feb 2020 19:15:04 -0500 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1j2l6s-000DO1-Bg for bug-guile@gnu.org; Sat, 15 Feb 2020 01:15:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: bug-guile@gnu.org From: Andreas Rottmann Subject: R6RS `flush-output-port` not playing along with `transcoded-port` Date: Sat, 15 Feb 2020 01:08:42 +0100 Message-ID: <87d0agu2yd.fsf@londo.h.r0tty.org> Mime-Version: 1.0 Content-Type: text/plain User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Cancel-Lock: sha1:BFWW/XSRbSSGg7g4+WylGpBUvJY= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 159.69.161.202 X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Fri, 14 Feb 2020 19:34:18 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Hi fellow Schemers! I was gently nudged on IRC into having a look into my Scheme code, after many years of abandon, and found that `dorodango` (a reasonably large body of R6RS code) hangs with Guile 2.2 and 3.0, while it worked on 2.0 (IIRC). I isolated the cause; the following snippet hangs on Guile 2.2 and 3.0, while it worked as expected on 2.0: ;; ------------------ (import (rnrs)) (let* ((p (pipe)) (in (car p)) (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) (put-datum out "foo") (flush-output-port out) (display "Should have written to pipe by now, attempting reading a byte\n") (display "Got") (display (get-u8 in)) (newline)) ;; ------------------- It seems the underlying port is no longer flushed to the OS, so the `get-u8` now hangs waiting for input, starting with Guile 2.2. Regards, Rotty From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 15 11:24:51 2020 Received: (at submit) by debbugs.gnu.org; 15 Feb 2020 16:24:51 +0000 Received: from localhost ([127.0.0.1]:36322 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j30FO-000559-MZ for submit@debbugs.gnu.org; Sat, 15 Feb 2020 11:24:51 -0500 Received: from lists.gnu.org ([209.51.188.17]:36598) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j2ufM-0002nk-Oj for submit@debbugs.gnu.org; Sat, 15 Feb 2020 05:27:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43931) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2ufL-0001CG-Mg for bug-guile@gnu.org; Sat, 15 Feb 2020 05:27:16 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_20,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2ufK-0005hk-Lv for bug-guile@gnu.org; Sat, 15 Feb 2020 05:27:15 -0500 Received: from ciao.gmane.io ([159.69.161.202]:54458) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j2ufK-0005hI-GR for bug-guile@gnu.org; Sat, 15 Feb 2020 05:27:14 -0500 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1j2ufI-000Kjo-Qo for bug-guile@gnu.org; Sat, 15 Feb 2020 11:27:12 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: bug-guile@gnu.org From: Andreas Rottmann Subject: Re: bug#39610: R6RS `flush-output-port` not playing along with `transcoded-port` Date: Sat, 15 Feb 2020 11:27:06 +0100 Message-ID: <87wo8orvr9.fsf@londo.h.r0tty.org> References: <87d0agu2yd.fsf@londo.h.r0tty.org> Mime-Version: 1.0 Content-Type: text/plain User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Cancel-Lock: sha1:FIHkptShXHpUXeYRyp6NBQqSql8= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 159.69.161.202 X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Sat, 15 Feb 2020 11:24:50 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Andreas Rottmann writes: > [...] I isolated the cause; the following snippet hangs on Guile 2.2 > and 3.0, while it worked as expected on 2.0: > > ;; ------------------ > (import (rnrs)) > > (let* ((p (pipe)) > (in (car p)) > (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) > (put-datum out "foo") > (flush-output-port out) > (display "Should have written to pipe by now, attempting reading a byte\n") > (display "Got") > (display (get-u8 in)) > (newline)) > ;; ------------------- > > It seems the underlying port is no longer flushed to the OS, so the > `get-u8` now hangs waiting for input, starting with Guile 2.2. > I'd like to add that this is indeed not passing data to the OS, as verified by strace. Also, I have now figured out the commit introducing the regression, namely 8399e7af5 ("Generic port facility provides buffering uniformly"); the commit before (e8eeeeb1d) still runs the above code to completion. I'd be somewhat motivated to try coming up with a fix, and turn the above snippet into a testcase, but I guess I could use some hinting in the right direction. Regards, Rotty From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 15 11:24:52 2020 Received: (at 39610) by debbugs.gnu.org; 15 Feb 2020 16:24:52 +0000 Received: from localhost ([127.0.0.1]:36324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j30FP-00055E-NT for submit@debbugs.gnu.org; Sat, 15 Feb 2020 11:24:52 -0500 Received: from ghost.xx.vu ([185.33.11.101]:4883) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j2xdC-0000us-0n for 39610@debbugs.gnu.org; Sat, 15 Feb 2020 08:37:14 -0500 Received: from mail0.xx.vu (localhost [::1]) by ghost.xx.vu (OpenSMTPD) with ESMTP id 66b0c26e for <39610@debbugs.gnu.org>; Sat, 15 Feb 2020 13:37:05 +0000 (UTC) Received: from londo ([213.162.73.60]) by mail0.xx.vu with ESMTPSA id 3ED0NQD0R15jgQEAg/rtjg (envelope-from ) for <39610@debbugs.gnu.org>; Sat, 15 Feb 2020 13:37:04 +0000 User-agent: mu4e 1.3.6; emacs 26.3 From: Andreas Rottmann To: 39610@debbugs.gnu.org Subject: R6RS `flush-output-port` not playing along with `transcoded-port` Message-ID: <87pnegrmyn.fsf@londo.h.r0tty.org> Date: Sat, 15 Feb 2020 14:37:04 +0100 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39610 X-Mailman-Approved-At: Sat, 15 Feb 2020 11:24:50 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) [ re-send, used gmane to post a follow-up on the initial bug report, and realized that this likely cannot work. Apologies if I'm wrong and this ends up as a duplicate. ] Andreas Rottmann writes: > [...] I isolated the cause; the following snippet hangs on Guile 2.2 > and 3.0, while it worked as expected on 2.0: > > ;; ------------------ > (import (rnrs)) > > (let* ((p (pipe)) > (in (car p)) > (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) > (put-datum out "foo") > (flush-output-port out) > (display "Should have written to pipe by now, attempting reading a byte\n") > (display "Got") > (display (get-u8 in)) > (newline)) > ;; ------------------- > > It seems the underlying port is no longer flushed to the OS, so the > `get-u8` now hangs waiting for input, starting with Guile 2.2. > I'd like to add that this is indeed not passing data to the OS, as verified by strace. Also, I have now figured out the commit introducing the regression, namely 8399e7af5 ("Generic port facility provides buffering uniformly"); the commit before (e8eeeeb1d) still runs the above code to completion. I'd be somewhat motivated to try coming up with a fix, and turn the above snippet into a testcase, but I guess I could use some hinting in the right direction. Regards, Rotty From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 21 13:55:56 2020 Received: (at 39610) by debbugs.gnu.org; 21 Mar 2020 17:55:56 +0000 Received: from localhost ([127.0.0.1]:47956 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFiLk-0002kE-1b for submit@debbugs.gnu.org; Sat, 21 Mar 2020 13:55:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43529) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFiLi-0002k2-RL for 39610@debbugs.gnu.org; Sat, 21 Mar 2020 13:55:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48659) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jFiLd-0007gx-6c; Sat, 21 Mar 2020 13:55:49 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56136 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jFiLc-00057j-Kz; Sat, 21 Mar 2020 13:55:49 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andreas Rottmann Subject: Re: bug#39610: R6RS `flush-output-port` not playing along with `transcoded-port` References: <87d0agu2yd.fsf@londo.h.r0tty.org> <87wo8orvr9.fsf@londo.h.r0tty.org> Date: Sat, 21 Mar 2020 18:55:46 +0100 In-Reply-To: <87wo8orvr9.fsf@londo.h.r0tty.org> (Andreas Rottmann's message of "Sat, 15 Feb 2020 11:27:06 +0100") Message-ID: <874kuhzj6l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39610 Cc: 39610@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Andreas, And welcome back! :-) Andreas Rottmann skribis: > Andreas Rottmann writes: > >> [...] I isolated the cause; the following snippet hangs on Guile 2.2 >> and 3.0, while it worked as expected on 2.0: >> >> ;; ------------------ >> (import (rnrs)) >> >> (let* ((p (pipe)) >> (in (car p)) >> (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) >> (put-datum out "foo") >> (flush-output-port out) >> (display "Should have written to pipe by now, attempting reading a byt= e\n") >> (display "Got") >> (display (get-u8 in)) >> (newline)) >> ;; ------------------- >> >> It seems the underlying port is no longer flushed to the OS, so the >> `get-u8` now hangs waiting for input, starting with Guile 2.2. >> > I'd like to add that this is indeed not passing data to the OS, as > verified by strace. Also, I have now figured out the commit introducing > the regression, namely 8399e7af5 ("Generic port facility provides > buffering uniformly"); the commit before (e8eeeeb1d) still runs the > above code to completion. Actually I think the code above behaves as expected. =E2=80=98pipe=E2=80= =99 returns buffered ports by default. When flushing the transcoded port, =E2=80=98transcoded_port_write=E2=80=99 is called, but then bytes written t= o the pipe are buffered. The fix is to add: (setvbuf (cdr p) 'none) Does that make sense? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 22 18:51:01 2020 Received: (at 39610) by debbugs.gnu.org; 22 Mar 2020 22:51:01 +0000 Received: from localhost ([127.0.0.1]:51130 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jG9Qr-0000pn-Bt for submit@debbugs.gnu.org; Sun, 22 Mar 2020 18:51:01 -0400 Received: from ghost.xx.vu ([185.33.11.101]:47895) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jG9Qp-0000pe-Aj for 39610@debbugs.gnu.org; Sun, 22 Mar 2020 18:51:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=r0tty.org; s=20200309; t=1584917449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1tFiajl+UxYm5ZrgI+r6FkpnOQ8Klp2Z6noaQqyJx8s=; b=tlltqwMoVUFv5GXlHHmtzX8x5yusbro0LYKvLmm06DIhI1azn0iCjCzasvn9g/wBkTr4+L yOLGPeO7hYZ/v+HR8vw6XJO1KDbz9duRtk/taWmo9wVwUDRJpMimLw4bQRs9gJ6ExcS8L7 +aoyx3gd75/IwYFggvwA3jNF3pYdE4w= Received: from mail0.xx.vu (localhost [::1]) by ghost.xx.vu (OpenSMTPD) with ESMTP id 162a9ab4; Sun, 22 Mar 2020 22:50:49 +0000 (UTC) Received: from londo ([46.125.250.111]) by mail0.xx.vu with ESMTPSA id yiBsFsnrd15lUQEAg/rtjg (envelope-from ); Sun, 22 Mar 2020 22:50:49 +0000 References: <87d0agu2yd.fsf@londo.h.r0tty.org> <87wo8orvr9.fsf@londo.h.r0tty.org> <874kuhzj6l.fsf@gnu.org> User-agent: mu4e 1.3.6; emacs 27.0.50 From: Andreas Rottmann To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#39610: R6RS `flush-output-port` not playing along with `transcoded-port` In-reply-to: <874kuhzj6l.fsf@gnu.org> Date: Sun, 22 Mar 2020 23:50:47 +0100 Message-ID: <87r1xkj96g.fsf@r0tty.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39610 Cc: 39610@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hi Andreas, > > And welcome back! :-) > Thanks -- and thanks in return for looking into this! > Andreas Rottmann skribis: > >> Andreas Rottmann writes: >> >>> [...] I isolated the cause; the following snippet hangs on Guile 2.2 >>> and 3.0, while it worked as expected on 2.0: >>> >>> ;; ------------------ >>> (import (rnrs)) >>> >>> (let* ((p (pipe)) >>> (in (car p)) >>> (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) >>> (put-datum out "foo") >>> (flush-output-port out) >>> (display "Should have written to pipe by now, attempting reading a by= te\n") >>> (display "Got") >>> (display (get-u8 in)) >>> (newline)) >>> ;; ------------------- >>> >>> It seems the underlying port is no longer flushed to the OS, so the >>> `get-u8` now hangs waiting for input, starting with Guile 2.2. >>> >> I'd like to add that this is indeed not passing data to the OS, as >> verified by strace. Also, I have now figured out the commit introducing >> the regression, namely 8399e7af5 ("Generic port facility provides >> buffering uniformly"); the commit before (e8eeeeb1d) still runs the >> above code to completion. > > Actually I think the code above behaves as expected. =E2=80=98pipe=E2=80= =99 returns > buffered ports by default. When flushing the transcoded port, > =E2=80=98transcoded_port_write=E2=80=99 is called, but then bytes written= to the pipe > are buffered. > > The fix is to add: > > (setvbuf (cdr p) 'none) > > Does that make sense? > It makes sense, and I can confirm that it makes the boiled-down example I posted work. However, I'm not sure it is the expected behavior. Regardless of buffering modes used, I would expect a `flush-output-port` in a "port stack" (as produced by `transcoded-output-port`) to propagate all the way to the OS. It seems that was the case in Guile 2.0, as I'm pretty sure I observed the "breaking" behavior change with 8399e7af5 applied, and not with the commit preceding it. If the current behavior is indeed the intended one, we should make sure the docs somehow reflect this caveat, as I imagine it may surprise future Guile users which happen to use its R6RS support and pipes in combination. Regards, Rotty From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 23 05:22:19 2020 Received: (at 39610) by debbugs.gnu.org; 23 Mar 2020 09:22:19 +0000 Received: from localhost ([127.0.0.1]:51446 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGJHn-0000T7-ES for submit@debbugs.gnu.org; Mon, 23 Mar 2020 05:22:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37637) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jGJHl-0000Sn-KH for 39610@debbugs.gnu.org; Mon, 23 Mar 2020 05:22:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49468) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jGJHg-0006vs-26; Mon, 23 Mar 2020 05:22:12 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41244 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jGJHf-0007Um-Ja; Mon, 23 Mar 2020 05:22:11 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andreas Rottmann Subject: Re: bug#39610: R6RS `flush-output-port` not playing along with `transcoded-port` References: <87d0agu2yd.fsf@londo.h.r0tty.org> <87wo8orvr9.fsf@londo.h.r0tty.org> <874kuhzj6l.fsf@gnu.org> <87r1xkj96g.fsf@r0tty.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Germinal an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 23 Mar 2020 10:22:09 +0100 In-Reply-To: <87r1xkj96g.fsf@r0tty.org> (Andreas Rottmann's message of "Sun, 22 Mar 2020 23:50:47 +0100") Message-ID: <874kufs9xa.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39610 Cc: 39610@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi, Andreas Rottmann skribis: >> Andreas Rottmann skribis: >> >>> Andreas Rottmann writes: >>> >>>> [...] I isolated the cause; the following snippet hangs on Guile 2.2 >>>> and 3.0, while it worked as expected on 2.0: >>>> >>>> ;; ------------------ >>>> (import (rnrs)) >>>> >>>> (let* ((p (pipe)) >>>> (in (car p)) >>>> (out (transcoded-port (cdr p) (make-transcoder (utf-8-codec))))) >>>> (put-datum out "foo") >>>> (flush-output-port out) >>>> (display "Should have written to pipe by now, attempting reading a b= yte\n") >>>> (display "Got") >>>> (display (get-u8 in)) >>>> (newline)) >>>> ;; ------------------- >>>> >>>> It seems the underlying port is no longer flushed to the OS, so the >>>> `get-u8` now hangs waiting for input, starting with Guile 2.2. >>>> >>> I'd like to add that this is indeed not passing data to the OS, as >>> verified by strace. Also, I have now figured out the commit introducing >>> the regression, namely 8399e7af5 ("Generic port facility provides >>> buffering uniformly"); the commit before (e8eeeeb1d) still runs the >>> above code to completion. >> >> Actually I think the code above behaves as expected. =E2=80=98pipe=E2= =80=99 returns >> buffered ports by default. When flushing the transcoded port, >> =E2=80=98transcoded_port_write=E2=80=99 is called, but then bytes writte= n to the pipe >> are buffered. >> >> The fix is to add: >> >> (setvbuf (cdr p) 'none) >> >> Does that make sense? >> > It makes sense, and I can confirm that it makes the boiled-down example > I posted work. > > However, I'm not sure it is the expected behavior. Regardless of > buffering modes used, I would expect a `flush-output-port` in a "port > stack" (as produced by `transcoded-output-port`) to propagate all the > way to the OS. It seems that was the case in Guile 2.0, as I'm pretty > sure I observed the "breaking" behavior change with 8399e7af5 applied, > and not with the commit preceding it. Port types don=E2=80=99t have a =E2=80=9Cflush=E2=80=9D operation, only =E2= =80=9Cwrite=E2=80=9D. Thus, it=E2=80=99s impossible to define a port type that would propagate flushes. There are pros and cons I guess, but it seems like a reasonable choice to me. When implementing a =E2=80=9Cproxying=E2=80=9D port type like =E2=80=98tran= scoded-port=E2=80=99 that does its own buffering, it=E2=80=99s probably OK to say that it=E2=80=99s t= he proxy=E2=80=99s responsibility to ensure there=E2=80=99s no double-buffering taking place. > If the current behavior is indeed the intended one, we should make sure > the docs somehow reflect this caveat, as I imagine it may surprise > future Guile users which happen to use its R6RS support and pipes in > combination. Maybe we should not document this specific combination but rather the more general issue? I=E2=80=99m not sure how to do that. Thoughts? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 29 17:16:02 2020 Received: (at 39610) by debbugs.gnu.org; 29 Mar 2020 21:16:02 +0000 Received: from localhost ([127.0.0.1]:60020 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jIfHl-0004O2-NZ for submit@debbugs.gnu.org; Sun, 29 Mar 2020 17:16:02 -0400 Received: from ghost.xx.vu ([185.33.11.101]:45941) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jIfHi-0004Np-If for 39610@debbugs.gnu.org; Sun, 29 Mar 2020 17:15:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=r0tty.org; s=20200309; t=1585516550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UxCZA/EBuhKtr2JDdrUa8F7baS6U3TqfpAsVB6IoEBA=; b=t/c/CF1c1q43heJAysvzWeWhmZNDhl6dy1lOY8El+U2zcuDMEZmdkAyisvyVUb/zIf+m/1 Yi01EFMjTSmKsDRsMTCJPt5WHuqWZwNI/5V2DJDbsRSLW2O6+7KmNCXlHLFDI/QdKI6vx/ U/E6adUXaZlNr0B5nG5LAf4d5cmO4Mo= Received: from mail0.xx.vu (localhost [::1]) by ghost.xx.vu (OpenSMTPD) with ESMTP id 56a1445f; Sun, 29 Mar 2020 21:15:50 +0000 (UTC) Received: from londo ([213.162.73.46]) by mail0.xx.vu with ESMTPSA id o3J3GQYQgV55cAEAg/rtjg (envelope-from ); Sun, 29 Mar 2020 21:15:50 +0000 References: <87d0agu2yd.fsf@londo.h.r0tty.org> <87wo8orvr9.fsf@londo.h.r0tty.org> <874kuhzj6l.fsf@gnu.org> <87r1xkj96g.fsf@r0tty.org> <874kufs9xa.fsf@gnu.org> User-agent: mu4e 1.3.6; emacs 27.0.50 From: Andreas Rottmann To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#39610: R6RS `flush-output-port` not playing along with `transcoded-port` In-reply-to: <874kufs9xa.fsf@gnu.org> Message-ID: <87bloe981m.fsf@r0tty.org> Date: Sun, 29 Mar 2020 23:15:49 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39610 Cc: 39610@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hi, > > Andreas Rottmann skribis: > >>> Andreas Rottmann skribis: >>> >>>> Andreas Rottmann writes: >>>> >>>>> [...] I isolated the cause; the following snippet hangs on Guile 2.2 >>>>> and 3.0, while it worked as expected on 2.0: >>>>> >>>>> [...] >>>>> >>>>> It seems the underlying port is no longer flushed to the OS, so the >>>>> `get-u8` now hangs waiting for input, starting with Guile 2.2. >>>>> >>>> [...] >>> >>> Actually I think the code above behaves as expected. =E2=80=98pipe=E2= =80=99 returns >>> buffered ports by default. When flushing the transcoded port, >>> =E2=80=98transcoded_port_write=E2=80=99 is called, but then bytes writt= en to the pipe >>> are buffered. >>> >>> The fix is to add: >>> >>> (setvbuf (cdr p) 'none) >>> >>> Does that make sense? >>> >> It makes sense, and I can confirm that it makes the boiled-down example >> I posted work. >> >> However, I'm not sure it is the expected behavior. Regardless of >> buffering modes used, I would expect a `flush-output-port` in a "port >> stack" (as produced by `transcoded-output-port`) to propagate all the >> way to the OS. It seems that was the case in Guile 2.0, as I'm pretty >> sure I observed the "breaking" behavior change with 8399e7af5 applied, >> and not with the commit preceding it. > > Port types don=E2=80=99t have a =E2=80=9Cflush=E2=80=9D operation, only = =E2=80=9Cwrite=E2=80=9D. Thus, it=E2=80=99s > impossible to define a port type that would propagate flushes. > There are pros and cons I guess, but it seems like a reasonable choice > to me. > > When implementing a =E2=80=9Cproxying=E2=80=9D port type like =E2=80=98tr= anscoded-port=E2=80=99 that > does its own buffering, it=E2=80=99s probably OK to say that it=E2=80=99s= the proxy=E2=80=99s > responsibility to ensure there=E2=80=99s no double-buffering taking place. > In my understanding, the "proxy type" is the R6RS transcoded port in this case, which has no control over the underlying port, but provides a flush operation (`flush-output-port`), which R6RS specifies as: Flushes any buffered output from the buffer of output-port to the underlying file, device, or object. The flush-output-port procedure returns unspecified values. So if I obtain a transcoded port from a pipe port, and call `flush-output-port` on the transcoded port, I'd expect the bytes to end up at _least_ at the pipe port. That probably happens currently; however, the thing is that R6RS also says about `transcoded-port`: As a side effect, however, transcoded-port closes binary-port in a special way that allows the new textual port to continue to use the byte source or sink represented by binary-port, even though binary-port itself is closed and cannot be used by the input and output operations described in this chapter. So, I conclude: when you use `transcoded-port` with any underlying Guile port, and you care about buffering behavior, you _need_ to set the underlying port's buffer mode to 'none`, _before_ constructing the transcoded port. I have not tried if Guile enforces the "specially closed mode", but I am thinking in the context of an abstraction over `pipe` and `primitive-fork`, intended to provide a basis for "pure" R6RS code [1], so the client code cannot just call Guile's `setvbuf`, without losing portability. [1] https://github.com/rotty/spells/blob/master/spells/process/compat.guile= .sls Do you agree with that conclusion? [ After writing the above, I realize that might be what you meant, after all: do you propose that, as a fix, `transcoded-port` sets the buffer mode of the underlying port to `none`? ] I must admit that I am confused -- how can a port type have no flush operation (which is evidently true from looking at scm_t_port_type), and Guile's `force-output` procedure, generic over port types, still exist? >> If the current behavior is indeed the intended one, we should make sure >> the docs somehow reflect this caveat, as I imagine it may surprise >> future Guile users which happen to use its R6RS support and pipes in >> combination. > > Maybe we should not document this specific combination but rather the > more general issue? I=E2=80=99m not sure how to do that. Thoughts? > I now realized I need to wrap my head around the changed port implementation, and how it relates to R6RS in general, and `transcoded-port` specifically, before starting to think about documentation. I think I have identified another related issue, this time for reading from `trancoded-port`, but I'd rather make sure we have a shared understanding of the expected flushing and buffering behavior, before bringing that up. Regards, Rotty