GNU bug report logs - #76103
[PATCH] New function 'process-output'

Previous Next

To reply to this bug, email your comments to 76103 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Álvaro Ramírez <alvaro <at> xenodium.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] New function 'process-output'
Date: Thu, 06 Feb 2025 19:06:25 +0000

[Message part 1 (text/plain, inline)]

Hi folks,

I find process-lines super convenient for glueing command line 
utilities.

Often wished I had a similar function that neither split into 
lines (process-lines) nor relied on shell 
(shell-command-to-string).

Proposing process-output as a close cousin to process-lines.

Happy to drop process-lines in favour of an existing function I 
may have missed.

Álvaro

In GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin24.0.0, NS
appkit-2566.00 Version 15.0.1 (Build 24A348)) of 2025-02-06 built 
on
MacBookPro
Repository revision: e333985b36e6efd06eb288256d82e0eaeb4d2bb4
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2566
System Description:  macOS 15.0.1

Configured using:
'configure --with-ns
--prefix=/Users/alvaro/stuff/active/code/third_party/emacs/nextstep/Emacs.app/Contents/MacOS
--enable-locallisppath=/Users/alvaro/stuff/active/code/third_party/emacs/nextstep/Emacs.app/Contents/MacOS'

[0001-New-function-process-output.patch (text/patch, attachment)]

Message #8 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Álvaro Ramírez <alvaro <at> xenodium.com>,
 Stefan Kangas <stefankangas <at> gmail.com>, Andrea Corallo <acorallo <at> gnu.org>
Cc: 76103 <at> debbugs.gnu.org
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 09:10:57 +0200

> From: Álvaro Ramírez <alvaro <at> xenodium.com>
> Date: Thu, 06 Feb 2025 19:06:25 +0000
> 
> I find process-lines super convenient for glueing command line 
> utilities.
> 
> Often wished I had a similar function that neither split into 
> lines (process-lines) nor relied on shell 
> (shell-command-to-string).
> 
> Proposing process-output as a close cousin to process-lines.

Thanks, but can you explain why shell-command-to-string is not what
you want?

And in any case, given that the implementation is basically 4 lines of
Lisp, why do we need this in Emacs, and in subr.el on top of that
(which means it will be preloaded)?  Sounds like not justified for
such a simple function.  If some package needs it, and cannot use
shell-command-to-string (for reasons I still don't understand), it can
easily implement it as a utility function, no?

Stefan and Andrea, WDYT?

Message #11 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Álvaro Ramírez <alvaro <at> xenodium.com>
Cc: 76103 <at> debbugs.gnu.org
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 09:29:20 +0200

> I find process-lines super convenient for glueing command line utilities.
>
> Often wished I had a similar function that neither split into lines
> (process-lines) nor relied on shell (shell-command-to-string).
>
> Proposing process-output as a close cousin to process-lines.
>
> Happy to drop process-lines in favour of an existing function I may have
> missed.

Unfortunately, there is no such function for process output.
What is worse there is no similar function even for files.
So everyone have to reimplement the same idiom dozens of times:

  (defun read-file (file)
    "Return FILE content as string."
    (with-temp-buffer
      (insert-file-contents file)
      (buffer-string)))

  (defun read-lines (file)
    "Return a list of lines of FILE."
    (with-temp-buffer
      (insert-file-contents file)
      (split-string (buffer-string) "\n" t)))

For example, there is a complete implementation of read-file
in org-babel-eval-read-file, uudecode-tests-read-file,
eshell-test-file-string.

Also there is a complete implementation of read-lines
in vc--read-lines.

Also this idiom is widely used in comint-exec, eieio-persistent-read,
ert-write-junit-test-summary-report, faceup-test-font-lock-file,
package--get-description, epa-file-insert-file-contents,
erc-load-irc-script, gnus-splash-svg-color-symbols, eww-submit,
tramp-get-lock-file, org-babel-R-evaluate-external-process,
org-babel-R-evaluate-session, ob-session-async-R-value-callback,
org-babel-julia-evaluate-external-process,
org-babel-julia-evaluate-session, org-babel-execute:plantuml,
org-babel-execute:sass, org-babel-screen-test,
org-texinfo-supports-math-p, ps-prologue-file, shell,
vc-cvs-file-to-string, vc-hg--active-bookmark-internal,
erc-tests-common-snapshot-compare em-extpipe-tests--deftest,
em-hist-test/check-history-file, eshell-test-file-string,
files-tests-save-buffer-read-only-file, dbus--test-introspect, url-file,

> +(defun process-output (program &rest args)
> +  "Execute PROGRAM with ARGS, returning its output as a string.
> +Signal an error if the program returns with a non-zero exit status.
> +Also see `process-lines'."
> +  (declare (important-return-value t))
> +  (with-temp-buffer
> +    (let ((status (apply #'call-process program nil (current-buffer) nil args)))
> +      (unless (eq status 0)
> +	(error "%s exited with status %s" program status))
> +      (buffer-string))))

Although process-lines than can be implemented with this as

  (split-string (process-output program args ...) "\n" t)

probably better to not change the existing process-lines-handling-status
since it's not clear whether it's enough to use 'split-string', or to use
'string-lines' like in css-mode-test-selectors and scss-mode-test-selectors.

Message #14 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: alvaro <at> xenodium.com, 76103 <at> debbugs.gnu.org
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 10:53:26 +0200

> Cc: 76103 <at> debbugs.gnu.org
> From: Juri Linkov <juri <at> linkov.net>
> Date: Fri, 07 Feb 2025 09:29:20 +0200
> 
> > I find process-lines super convenient for glueing command line utilities.
> >
> > Often wished I had a similar function that neither split into lines
> > (process-lines) nor relied on shell (shell-command-to-string).
> >
> > Proposing process-output as a close cousin to process-lines.
> >
> > Happy to drop process-lines in favour of an existing function I may have
> > missed.
> 
> Unfortunately, there is no such function for process output.
> What is worse there is no similar function even for files.
> So everyone have to reimplement the same idiom dozens of times:

Why do so many Lisp programs use strings instead of processing the
text in the buffer?  I think it's a bad idea, both in terms of
performance and in terms of GC pressure.

My impression is that this is due to people having habits from other
programming languages.

Also please note the with-output-to-string macro, which could help in
at least some of these situations.

Message #17 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Alvaro Ramirez <alvaro <at> xenodium.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Andrea Corallo <acorallo <at> gnu.org>, 76103 <at> debbugs.gnu.org,
 Stefan Kangas <stefankangas <at> gmail.com>, juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 09:17:34 +0000

Hi Eli,

Thanks for the questions and feedback.

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Álvaro Ramírez <alvaro <at> xenodium.com>
>> Date: Thu, 06 Feb 2025 19:06:25 +0000
>>
>> I find process-lines super convenient for glueing command line
>> utilities.
>>
>> Often wished I had a similar function that neither split into
>> lines (process-lines) nor relied on shell
>> (shell-command-to-string).
>>
>> Proposing process-output as a close cousin to process-lines.
>
> Thanks, but can you explain why shell-command-to-string is not 
> what
> you want?

1. Performance: Shell overhead isn't needed to get a process 
output.

(benchmark-run 1000
   (process-output "seq" "200")) => (1.239617 0 0.0)

(benchmark-run 1000
   (shell-command-to-string "seq 200")) => (4.049149999999999 0 
   0.0)

2. Convenience API: The existing process-lines API is simple and 
easy to reach out to when glueing command line utilities.

Would be handy to have the same convenience available via 
process-output.

 - Predictably and concisely named: easy to find, starts with 
 "process-".
 - Structure + versatility: IMO while list of args is partially 
 structured, it remains fairly versatile (benefits from other 
 list-operating functions) vs unstructured string.
 - Avoids all issues with unescaped params (compared to 
 shell-command-to-string).

> And in any case, given that the implementation is basically 4 
> lines of

Users would need to either re-implement this for each package they 
build or duplicate the 4 lines every time they want a short-lived 
process output.

> Lisp, why do we need this in Emacs,

Seems like a good core basic function to offer (same reasons as 
above) which package authors can rely on without duplicating 
across every packages.

>  and in subr.el on top of that

subr.el was chosen since that's where the existing process-lines 
lives. Happy to move to a more appropriate location. Suggestions?

> (which means it will be preloaded)?  Sounds like not justified 
> for
> such a simple function.  If some package needs it, and cannot 
> use
> shell-command-to-string (for reasons I still don't understand),
> it can easily implement it as a utility function, no?

I hope the above rationale helps. While in its entirety, it could 
be open to opinion, perhaps we can justify purely on performance 
alone?

>
> Stefan and Andrea, WDYT?

Message #20 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Álvaro Ramírez <alvaro <at> xenodium.com>,
 76103 <at> debbugs.gnu.org
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 10:54:48 +0100

Juri Linkov <juri <at> linkov.net> writes:

>> I find process-lines super convenient for glueing command line utilities.
>>
>> Often wished I had a similar function that neither split into lines
>> (process-lines) nor relied on shell (shell-command-to-string).
>>
>> Proposing process-output as a close cousin to process-lines.
>>
>> Happy to drop process-lines in favour of an existing function I may have
>> missed.
>
> Unfortunately, there is no such function for process output.
> What is worse there is no similar function even for files.
> So everyone have to reimplement the same idiom dozens of times:
>
>   (defun read-file (file)
>     "Return FILE content as string."
>     (with-temp-buffer
>       (insert-file-contents file)
>       (buffer-string)))
>
>   (defun read-lines (file)
>     "Return a list of lines of FILE."
>     (with-temp-buffer
>       (insert-file-contents file)
>       (split-string (buffer-string) "\n" t)))
>
> For example, there is a complete implementation of read-file
> in org-babel-eval-read-file, uudecode-tests-read-file,
> eshell-test-file-string.
>
> Also there is a complete implementation of read-lines
> in vc--read-lines.
>
> Also this idiom is widely used in comint-exec, eieio-persistent-read,
> ert-write-junit-test-summary-report, faceup-test-font-lock-file,
> package--get-description, epa-file-insert-file-contents,
> erc-load-irc-script, gnus-splash-svg-color-symbols, eww-submit,
> tramp-get-lock-file, org-babel-R-evaluate-external-process,
> org-babel-R-evaluate-session, ob-session-async-R-value-callback,
> org-babel-julia-evaluate-external-process,
> org-babel-julia-evaluate-session, org-babel-execute:plantuml,
> org-babel-execute:sass, org-babel-screen-test,
> org-texinfo-supports-math-p, ps-prologue-file, shell,
> vc-cvs-file-to-string, vc-hg--active-bookmark-internal,
> erc-tests-common-snapshot-compare em-extpipe-tests--deftest,
> em-hist-test/check-history-file, eshell-test-file-string,
> files-tests-save-buffer-read-only-file, dbus--test-introspect, url-file,

+1 I am in favor of adding such small utility functions to read small
files or handle limited process output. I think many of us have
implemented such functions, and Juri's code search in the Emacs code
base demonstrates that. I agree with Eli that for large files or more
extensive processing working with a buffer is better. But in other cases
these functions are still useful.

If a set of such functions will be added to Emacs subr.el or subr-x.el I
am happy to port them back via Compat, such that they are available to
all ELPA packages, including the ones targeting older Emacs versions.

Daniel

Message #23 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Alvaro Ramirez <alvaro <at> xenodium.com>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 76103 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 10:52:45 +0000

Daniel Mendler <mail <at> daniel-mendler.de> writes:

> Juri Linkov <juri <at> linkov.net> writes:
>
>>> I find process-lines super convenient for glueing command line 
>>> utilities.
>>>
>>> Often wished I had a similar function that neither split into 
>>> lines
>>> (process-lines) nor relied on shell (shell-command-to-string).
>>>
>>> Proposing process-output as a close cousin to process-lines.
>>>
>>> Happy to drop process-lines in favour of an existing function 
>>> I may have
>>> missed.
>>
>> Unfortunately, there is no such function for process output.
>> What is worse there is no similar function even for files.
>> So everyone have to reimplement the same idiom dozens of times:
>>
>>   (defun read-file (file)
>>     "Return FILE content as string."
>>     (with-temp-buffer
>>       (insert-file-contents file)
>>       (buffer-string)))
>>
>>   (defun read-lines (file)
>>     "Return a list of lines of FILE."
>>     (with-temp-buffer
>>       (insert-file-contents file)
>>       (split-string (buffer-string) "\n" t)))

These two also resonate dearly. Specially read-file available.

I know now to go via with-temp-buffer, but as a newcomer, 
read-file would have been way more discoverable.

>>
>> For example, there is a complete implementation of read-file
>> in org-babel-eval-read-file, uudecode-tests-read-file,
>> eshell-test-file-string.
>>
>> Also there is a complete implementation of read-lines
>> in vc--read-lines.
>>
>> Also this idiom is widely used in comint-exec, 
>> eieio-persistent-read,
>> ert-write-junit-test-summary-report, 
>> faceup-test-font-lock-file,
>> package--get-description, epa-file-insert-file-contents,
>> erc-load-irc-script, gnus-splash-svg-color-symbols, eww-submit,
>> tramp-get-lock-file, org-babel-R-evaluate-external-process,
>> org-babel-R-evaluate-session, 
>> ob-session-async-R-value-callback,
>> org-babel-julia-evaluate-external-process,
>> org-babel-julia-evaluate-session, org-babel-execute:plantuml,
>> org-babel-execute:sass, org-babel-screen-test,
>> org-texinfo-supports-math-p, ps-prologue-file, shell,
>> vc-cvs-file-to-string, vc-hg--active-bookmark-internal,
>> erc-tests-common-snapshot-compare em-extpipe-tests--deftest,
>> em-hist-test/check-history-file, eshell-test-file-string,
>> files-tests-save-buffer-read-only-file, dbus--test-introspect, 
>> url-file,
>
> +1 I am in favor of adding such small utility functions to read 
> small
> files or handle limited process output. I think many of us have
> implemented such functions, and Juri's code search in the Emacs 
> code
> base demonstrates that. I agree with Eli that for large files or 
> more
> extensive processing working with a buffer is better. But in 
> other cases
> these functions are still useful.
>
> If a set of such functions will be added to Emacs subr.el or 
> subr-x.el I
> am happy to port them back via Compat, such that they are 
> available to
> all ELPA packages, including the ones targeting older Emacs 
> versions.
>
> Daniel

Message #26 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alvaro Ramirez <alvaro <at> xenodium.com>
Cc: acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org, stefankangas <at> gmail.com,
 juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 13:42:55 +0200

> From: Alvaro Ramirez <alvaro <at> xenodium.com>
> Cc: Stefan Kangas <stefankangas <at> gmail.com>,  Andrea Corallo
>  <acorallo <at> gnu.org>,  76103 <at> debbugs.gnu.org, juri <at> linkov.net
> Date: Fri, 07 Feb 2025 09:17:34 +0000
> 
> Hi Eli,
> 
> Thanks for the questions and feedback.

Thanks for answering them.

> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Álvaro Ramírez <alvaro <at> xenodium.com>
> >> Date: Thu, 06 Feb 2025 19:06:25 +0000
> >>
> >> I find process-lines super convenient for glueing command line
> >> utilities.
> >>
> >> Often wished I had a similar function that neither split into
> >> lines (process-lines) nor relied on shell
> >> (shell-command-to-string).
> >>
> >> Proposing process-output as a close cousin to process-lines.
> >
> > Thanks, but can you explain why shell-command-to-string is not 
> > what
> > you want?
> 
> 1. Performance: Shell overhead isn't needed to get a process 
> output.
> 
> (benchmark-run 1000
>     (process-output "seq" "200")) => (1.239617 0 0.0)
> 
> (benchmark-run 1000
>     (shell-command-to-string "seq 200")) => (4.049149999999999 0 
>     0.0)
> 
> 2. Convenience API: The existing process-lines API is simple and 
> easy to reach out to when glueing command line utilities.
> 
> Would be handy to have the same convenience available via 
> process-output.
> 
>   - Predictably and concisely named: easy to find, starts with 
>   "process-".
>   - Structure + versatility: IMO while list of args is partially 
>   structured, it remains fairly versatile (benefits from other 
>   list-operating functions) vs unstructured string.
>   - Avoids all issues with unescaped params (compared to 
>   shell-command-to-string).
> 
> > And in any case, given that the implementation is basically 4 
> > lines of
> 
> Users would need to either re-implement this for each package they 
> build or duplicate the 4 lines every time they want a short-lived 
> process output.
> 
> > Lisp, why do we need this in Emacs,
> 
> Seems like a good core basic function to offer (same reasons as 
> above) which package authors can rely on without duplicating 
> across every packages.
> 
> >  and in subr.el on top of that
> 
> subr.el was chosen since that's where the existing process-lines 
> lives. Happy to move to a more appropriate location. Suggestions?
> 
> > (which means it will be preloaded)?  Sounds like not justified 
> > for
> > such a simple function.  If some package needs it, and cannot 
> > use
> > shell-command-to-string (for reasons I still don't understand),
> > it can easily implement it as a utility function, no?
> 
> I hope the above rationale helps. While in its entirety, it could 
> be open to opinion, perhaps we can justify purely on performance 
> alone?

I'm not sure a 2.5 boost in performance (on which platform, btw?)
justifies yet another API.  We need to think about the downsides of
enlarging Emacs: the need to document each new API, the need to
maintain it, the need to somehow explain which one of the alternatives
to use in each case, etc.

But let's see what other maintainers think.

Message #29 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Alvaro Ramirez <alvaro <at> xenodium.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org, stefankangas <at> gmail.com,
 juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 14:01:45 +0000

Thanks Eli.

Eli Zaretskii <eliz <at> gnu.org> writes:

> I'm not sure a 2.5 boost in performance

(let ((before (benchmark-run 1000
               (shell-command-to-string "seq 200")))
     (after (benchmark-run 1000
              (process-output "seq" "200"))))
 (/ (car before)
    (car after))) => 3.27

Roughly a 3.27 gain, or should I calculate differently?

> (on which platform, btw?) justifies yet another API.

macOS. Need to grab a hold of a linux box to get equivalent 
figure.

>   We need to think about the downsides of
> enlarging Emacs: the need to document each new API, the need to
> maintain it, the need to somehow explain which one of the 
> alternatives
> to use in each case, etc.

Makes sense.

>
> But let's see what other maintainers think.

Sounds good.

Message #32 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> posteo.net>
To: Alvaro Ramirez <alvaro <at> xenodium.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org,
 stefankangas <at> gmail.com, juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 17:15:44 +0000

Alvaro Ramirez <alvaro <at> xenodium.com> writes:

> (let ((before (benchmark-run 1000
>                 (shell-command-to-string "seq 200")))
>       (after (benchmark-run 1000
>                (process-output "seq" "200"))))
>   (/ (car before)
>      (car after))) => 3.27
>
> Roughly a 3.27 gain, or should I calculate differently?

Linux. Not the same every times, but always larger than 2x improvement.

; 5.333148970679263
; 3.204657728630843
; 2.0283408689679625
; 4.1317904673085035
; 5.0238578857970575

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Message #35 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Alvaro Ramirez <alvaro <at> xenodium.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 7 Feb 2025 13:21:57 -0600

Alvaro Ramirez <alvaro <at> xenodium.com> writes:

> Thanks Eli.
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>> I'm not sure a 2.5 boost in performance
>
> (let ((before (benchmark-run 1000
>                 (shell-command-to-string "seq 200")))
>       (after (benchmark-run 1000
>                (process-output "seq" "200"))))
>   (/ (car before)
>      (car after))) => 3.27
>
> Roughly a 3.27 gain, or should I calculate differently?

I would benchmark this against `process-lines` instead, because it
doesn't spawn a shell.  On this macOS machine, I get 1.15 for that
benchmark.

>> (on which platform, btw?) justifies yet another API.
>
> macOS. Need to grab a hold of a linux box to get equivalent
> figure.

AFAIU, the body of the proposed `process-output` could be reduced to

    (string-join (process-lines cmd) "\n")

This makes me doubt if it's worth adding this new API.

To find arguments that would justify the addition, I'd take a look at
emacs.git to see which code we have that it would simplify.  I'd also
take a look at packages on GNU ELPA, NonGNU ELPA, and MELPA, to find
even more supporting evidence.

Message #38 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Alvaro Ramirez <alvaro <at> xenodium.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org,
 juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Fri, 07 Feb 2025 23:19:17 +0000

Thanks Stefan

Stefan Kangas <stefankangas <at> gmail.com> writes:

> Alvaro Ramirez <alvaro <at> xenodium.com> writes:
>
>> Thanks Eli.
>>
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>
>>> I'm not sure a 2.5 boost in performance
>>
>> (let ((before (benchmark-run 1000
>>                 (shell-command-to-string "seq 200")))
>>       (after (benchmark-run 1000
>>                (process-output "seq" "200"))))
>>   (/ (car before)
>>      (car after))) => 3.27
>>
>> Roughly a 3.27 gain, or should I calculate differently?
>
> I would benchmark this against `process-lines` instead, because 
> it
> doesn't spawn a shell.  On this macOS machine, I get 1.15 for 
> that
> benchmark.

Eli can correct me here, but he may have suggested 
shell-command-to-string as it provided a closer-looking interface.

When benchmarking against process-lines, the performance 
difference is indeed negligible.

>
>>> (on which platform, btw?) justifies yet another API.
>>
>> macOS. Need to grab a hold of a linux box to get equivalent
>> figure.
>
> AFAIU, the body of the proposed `process-output` could be 
> reduced to
>
>     (string-join (process-lines cmd) "\n")
>
> This makes me doubt if it's worth adding this new API.

Is there a slight chance re-joining lines with a "\n" does not 
preserve process output integrity?

If so, wouldn't users need to know process-lines internal details, 
thus inspect process-lines (elisp), process-lines-handling-status 
(elisp), line-beginning-position/line-end-position (.c), etc to 
ensure correct usage?

> To find arguments that would justify the addition, I'd take a 
> look at
> emacs.git to see which code we have that it would simplify.  I'd 
> also
> take a look at packages on GNU ELPA, NonGNU ELPA, and MELPA, to 
> find
> even more supporting evidence.

I had a quick look at emacs.git and GitHub (maybe where most of 
MELPA lives) and process-lines + string-join did not seem like a 
common pattern. Funnily, I found one in my code ;-) 
with-temp-buffer + call-process seems more prevalent.

While with-temp-buffer or process-lines options do the job just 
fine, this route may not be as discoverable as process-output for 
an elisp newcomer.

Message #41 received at 76103 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Alvaro Ramirez <alvaro <at> xenodium.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, acorallo <at> gnu.org, 76103 <at> debbugs.gnu.org,
 juri <at> linkov.net
Subject: Re: bug#76103: [PATCH] New function 'process-output'
Date: Sun, 9 Feb 2025 07:24:09 -0800

Alvaro Ramirez <alvaro <at> xenodium.com> writes:

>> AFAIU, the body of the proposed `process-output` could be
>> reduced to
>>
>>     (string-join (process-lines cmd) "\n")
>>
>> This makes me doubt if it's worth adding this new API.
>
> Is there a slight chance re-joining lines with a "\n" does not
> preserve process output integrity?

I don't know, sorry.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.