GNU bug report logs - #43364
Possible bug with output redirection

Previous Next

To reply to this bug, email your comments to 43364 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: pinoaffe <pinoaffe <at> airmail.cc>
To: bug-guile <at> gnu.org
Subject: Possible bug with output redirection
Date: Sat, 12 Sep 2020 22:59:23 +0200

Dear guilers,

When using with-output-to-string, the output of external processes
started using system* and the like is not redirected to the temporary
port. As far as I can tell, it just redirects things written/displayed
from within guile.
This seems to be a bug, or if this is intended behaviour it might be
beneficial to document this somewhere.

As an example:
I'd expect snippet [0] to behave like snippet [1] and return "bar",
instead it just returns the empty string.

This probably affects other similar functions.

Pandemically,
pinoaffe

[0]:
(with-output-to-string
 (lambda ()
   (system* "echo" "bar")))

[1]:
(with-output-to-string
 (lambda ()
   (display "bar")))

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: <tomas <at> tuxteam.de>
To: bug-guile <at> gnu.org
Subject: Re: bug#43364: Possible bug with output redirection
Date: Sun, 13 Sep 2020 09:13:02 +0200

[Message part 1 (text/plain, inline)]

On Sat, Sep 12, 2020 at 10:59:23PM +0200, pinoaffe wrote:
> Dear guilers,
> 
> When using with-output-to-string, the output of external processes
> started using system* and the like is not redirected to the temporary
> port. As far as I can tell, it just redirects things written/displayed
> from within guile.
> This seems to be a bug, or if this is intended behaviour it might be
> beneficial to document this somewhere.

I think this is intentional (or rather: out of Guile's scope). While the
Guile process's output functions are the scope of `with-output-to-string',
the subprocess started with `system' just happens to inherit the standard
output file descriptor, which is an operating system mechanism.

No idea, for example, about what would happen under Windows or other
(more or less) operating systems.

But you are right that it can be a bit confusing. I'm not sure whether a
hint in the doc would be in place.

What do oters think?

Cheers
-- t

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 43364 <at> debbugs.gnu.org (full text, mbox):

From: Felix Lechner <felix.lechner <at> lease-up.com>
To: 43364 <at> debbugs.gnu.org
Cc: pinoaffe <pinoaffe <at> airmail.cc>
Subject: with-output-to-port works with file ports
Date: Fri, 8 Sep 2023 10:53:00 -0700

Hi,

In an interesting (or perhaps maddening) inconsistency,
'with-output-to-port' captures stdout from system* here

(call-with-output-file "/tmp/test.log"
  (lambda (port)
    (with-output-to-port
      port
      (lambda ()
        (system* "mktemp" "-d")))))

but 'with-output-to-string' does not do so here

(with-output-to-string
  (lambda ()
    (system* "mktemp" "-d")))

According to lloda on #guile, system* handles the redirection only
when the current ports are file ports. Thanks for that pointer!

Kind regards
Felix

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Felix Lechner <felix.lechner <at> lease-up.com>
Cc: bug-guile <at> gnu.org, pinoaffe <pinoaffe <at> airmail.cc>, 43364 <at> debbugs.gnu.org
Subject: Re: bug#43364: with-output-to-port works with file ports
Date: Fri, 08 Sep 2023 21:06:11 +0200

Felix Lechner via "Bug reports for GUILE, GNU's Ubiquitous Extension Language" <bug-guile <at> gnu.org> writes:

> Hi,
>
> In an interesting (or perhaps maddening) inconsistency,
> 'with-output-to-port' captures stdout from system* here
>
> (call-with-output-file "/tmp/test.log"
>   (lambda (port)
>     (with-output-to-port
>       port
>       (lambda ()
>         (system* "mktemp" "-d")))))
>
> but 'with-output-to-string' does not do so here
>
> (with-output-to-string
>   (lambda ()
>     (system* "mktemp" "-d")))
>
> According to lloda on #guile, system* handles the redirection only
> when the current ports are file ports. Thanks for that pointer!

That’s correct.  I’ve been using the following monstrosity to capture
and process output.  Perhaps someone finds a prettier way?

--8<---------------cut here---------------start------------->8---
(define* (call-with-output-processor command proc #:optional capture-stderr?)
  "Silently execute COMMAND, a list of strings representing an
executable with its arguments, and apply PROC to every line printed to
standard output and, optionally when CAPTURE-STDERR? is #T, standard
error.  Return the exit status of COMMAND."
  ;; We can only capture a program's standard error by parameterizing
  ;; current-error-port to a *file* port before using system* or
  ;; open-pipe*.  The process will write its standard error stream to
  ;; the provided file descriptor.  Meanwhile we read from the file
  ;; descriptor (blocking) for new lines until the process exits.
  (match (socketpair PF_UNIX SOCK_STREAM 0)
    ((in . out)
     (let ((err (if capture-stderr?
                    (dup out)
                    (%make-void-port "w"))))
       (catch #true
         (lambda ()
           (let ((thread
                  (parameterize ((current-error-port err)
                                 (current-output-port out))
                    (call-with-new-thread
                     (lambda ()
                       (let ((status
                              (status:exit-val
                               (apply system* command))))
                         (close-port err)
                         (close-port out)
                         status))))))
             (let loop ()
               (match (read-line in 'concat)
                 ((? eof-object?)
                  (for-each
                   (lambda (port)
                     (false-if-exception (close-port port)))
                   (list err out in))
                  (join-thread thread))
                 (line
                  (proc line)
                  (loop))))))
         (lambda (key . args)
           (for-each
            (lambda (port)
              (false-if-exception (close-port port)))
            (list err out in))
           (apply throw key args)))))))
--8<---------------cut here---------------end--------------->8---


-- 
Ricardo

Message #20 received at 43364 <at> debbugs.gnu.org (full text, mbox):

From: Fabio Natali <me <at> fabionatali.com>
To: 43364 <at> debbugs.gnu.org
Subject: bug#43364: with-output-to-port works with file ports
Date: Tue, 23 Apr 2024 17:32:52 +0100

Dear All 👋,

I've been encountering a few times the need for a procedure like
'system' or 'system*' but that might be capable of capturing the output
of the program that's being called.

While this is something that can be solved with a simple 'open-pipe*'
wrapper, the pattern seems relatively frequent in Guix, which is where I
noticed it. I suggested that some variant of such a wrapper could be
added to '(guix build utils)'⁰.

In reply to my post, it was brought to my attention that the problem
might be addressed at the Guile level and pointed me to this bug report.

I thought of sending a quick follow-up here to see if there's any rough
consensus on a possible way of addressing this - or why it might be
difficult or potentially not worth the effort.

Thanks, all best, Fabio.

⁰ https://lists.gnu.org/archive/html/guix-devel/2024-04/msg00199.html


-- 
Fabio Natali
https://fabionatali.com

Message #23 received at 43364 <at> debbugs.gnu.org (full text, mbox):

From: Tony Garnock-Jones <tonyg <at> leastfixedpoint.com>
To: Fabio Natali <me <at> fabionatali.com>, 43364 <at> debbugs.gnu.org
Subject: Re: bug#43364: with-output-to-port works with file ports
Date: Wed, 24 Apr 2024 09:00:42 +0200

This is a nice idea. The way Racket does it is to make 
with-output-to-string cooperate with system:

    (with-output-to-string (lambda () (system "date")))

but a quick check shows that Guile does not yet have such cooperation in 
place. Would that be a good place to start looking?

Tony


On 23/04/2024 18:32, Fabio Natali wrote:
> Dear All 👋,
> 
> I've been encountering a few times the need for a procedure like
> 'system' or 'system*' but that might be capable of capturing the output
> of the program that's being called.
> 
> While this is something that can be solved with a simple 'open-pipe*'
> wrapper, the pattern seems relatively frequent in Guix, which is where I
> noticed it. I suggested that some variant of such a wrapper could be
> added to '(guix build utils)'⁰.
> 
> In reply to my post, it was brought to my attention that the problem
> might be addressed at the Guile level and pointed me to this bug report.
> 
> I thought of sending a quick follow-up here to see if there's any rough
> consensus on a possible way of addressing this - or why it might be
> difficult or potentially not worth the effort.
> 
> Thanks, all best, Fabio.
> 
> ⁰ https://lists.gnu.org/archive/html/guix-devel/2024-04/msg00199.html
> 
> 

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.