GNU bug report logs - #79442
RFE: process-lines equivalent for NUL-separated output

Previous Next

To reply to this bug, email your comments to 79442 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tim Landscheidt <tim <at> tim-landscheidt.de>
To: bug-gnu-emacs <at> gnu.org
Subject: RFE: process-lines equivalent for NUL-separated output
Date: Sat, 13 Sep 2025 11:27:34 +0000

Severity: wishlist

The function process-lines executes a program and returns
the lines of its output as a list.  However using newlines
as output separators is prone to errors, and therefore many
programs support using NUL (?\C-@) to separate their output.

Currently, with Emacs one needs to use constructs like:

| (split-string
|  (shell-command-to-string
|   (concat "foo "
|           (shell-quote-argument "bar baz")))
|  "\0"
|  t)

It would be nice if Emacs shipped a function so that the
above could be rewritten as:

| (process-output "foo" "bar baz")

Message #8 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Tim Landscheidt <tim <at> tim-landscheidt.de>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sat, 13 Sep 2025 14:51:30 +0300

> From: Tim Landscheidt <tim <at> tim-landscheidt.de>
> Date: Sat, 13 Sep 2025 11:27:34 +0000
> 
> Severity: wishlist
> 
> The function process-lines executes a program and returns
> the lines of its output as a list.  However using newlines
> as output separators is prone to errors, and therefore many
> programs support using NUL (?\C-@) to separate their output.
> 
> Currently, with Emacs one needs to use constructs like:
> 
> | (split-string
> |  (shell-command-to-string
> |   (concat "foo "
> |           (shell-quote-argument "bar baz")))
> |  "\0"
> |  t)
> 
> It would be nice if Emacs shipped a function so that the
> above could be rewritten as:
> 
> | (process-output "foo" "bar baz")

So you want process-lines to call split-string for you (given some
optional argument)?

Message #11 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Tim Landscheidt <tim <at> tim-landscheidt.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sat, 13 Sep 2025 12:00:37 +0000

Eli Zaretskii <eliz <at> gnu.org> wrote:

>> The function process-lines executes a program and returns
>> the lines of its output as a list.  However using newlines
>> as output separators is prone to errors, and therefore many
>> programs support using NUL (?\C-@) to separate their output.

>> Currently, with Emacs one needs to use constructs like:

>> | (split-string
>> |  (shell-command-to-string
>> |   (concat "foo "
>> |           (shell-quote-argument "bar baz")))
>> |  "\0"
>> |  t)

>> It would be nice if Emacs shipped a function so that the
>> above could be rewritten as:

>> | (process-output "foo" "bar baz")

> So you want process-lines to call split-string for you (given some
> optional argument)?

I'm not sure whether it is possible to add an optional
argument to process-lines as it passes the remaining
arguments to the executed program.  How would such a call
look like?

Message #14 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Tim Landscheidt <tim <at> tim-landscheidt.de>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sat, 13 Sep 2025 15:57:30 +0300

> From: Tim Landscheidt <tim <at> tim-landscheidt.de>
> Cc: 79442 <at> debbugs.gnu.org
> Date: Sat, 13 Sep 2025 12:00:37 +0000
> 
> Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> >> The function process-lines executes a program and returns
> >> the lines of its output as a list.  However using newlines
> >> as output separators is prone to errors, and therefore many
> >> programs support using NUL (?\C-@) to separate their output.
> 
> >> Currently, with Emacs one needs to use constructs like:
> 
> >> | (split-string
> >> |  (shell-command-to-string
> >> |   (concat "foo "
> >> |           (shell-quote-argument "bar baz")))
> >> |  "\0"
> >> |  t)
> 
> >> It would be nice if Emacs shipped a function so that the
> >> above could be rewritten as:
> 
> >> | (process-output "foo" "bar baz")
> 
> > So you want process-lines to call split-string for you (given some
> > optional argument)?
> 
> I'm not sure whether it is possible to add an optional
> argument to process-lines as it passes the remaining
> arguments to the executed program.  How would such a call
> look like?

That's not the important part of my question.  I'm asking why is there
a need for a new function whose only job is to call split-string.

process-lines exists because several places in Emacs call it.  By
contrast, there was no reason until now to have a function that deals
with lines of output separated by nulls.  So adding such a function
would mean we have a function that no one calls, and why would we do
that, when the way to handle such output in a Lisp program is as
simple as a single call of an existing function?

Message #17 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Tim Landscheidt <tim <at> tim-landscheidt.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sun, 14 Sep 2025 01:59:58 +0000

Eli Zaretskii <eliz <at> gnu.org> wrote:

>> >> The function process-lines executes a program and returns
>> >> the lines of its output as a list.  However using newlines
>> >> as output separators is prone to errors, and therefore many
>> >> programs support using NUL (?\C-@) to separate their output.

>> >> Currently, with Emacs one needs to use constructs like:

>> >> | (split-string
>> >> |  (shell-command-to-string
>> >> |   (concat "foo "
>> >> |           (shell-quote-argument "bar baz")))
>> >> |  "\0"
>> >> |  t)

>> >> It would be nice if Emacs shipped a function so that the
>> >> above could be rewritten as:

>> >> | (process-output "foo" "bar baz")

>> > So you want process-lines to call split-string for you (given some
>> > optional argument)?

>> I'm not sure whether it is possible to add an optional
>> argument to process-lines as it passes the remaining
>> arguments to the executed program.  How would such a call
>> look like?

> That's not the important part of my question.  I'm asking why is there
> a need for a new function whose only job is to call split-string.

> process-lines exists because several places in Emacs call it.  By
> contrast, there was no reason until now to have a function that deals
> with lines of output separated by nulls.  So adding such a function
> would mean we have a function that no one calls, and why would we do
> that, when the way to handle such output in a Lisp program is as
> simple as a single call of an existing function?

Well, there are four calls in:

| (split-string
|  (shell-command-to-string
|   (concat "foo "
|           (shell-quote-argument "bar baz")))
|  "\0"
|  t)

I want to reduce this to one.

I don't know about the needs of Emacs core, but I use this
pattern regularly to process the output of external programs
where data may include newlines.  As Emacs does not provide
a single function for that purpose yet, any user who wants
to process data that may include newlines must use this
pattern as well (or a similar one).  Providing a function
that makes it easy to pass data properly encapsulated to an
external program and retrieve its separated output would
make it harder for users to write insecure code.

Message #20 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Tim Landscheidt <tim <at> tim-landscheidt.de>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sun, 14 Sep 2025 07:04:27 +0300

> From: Tim Landscheidt <tim <at> tim-landscheidt.de>
> Cc: 79442 <at> debbugs.gnu.org
> Date: Sun, 14 Sep 2025 01:59:58 +0000
> 
> Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > That's not the important part of my question.  I'm asking why is there
> > a need for a new function whose only job is to call split-string.
> 
> > process-lines exists because several places in Emacs call it.  By
> > contrast, there was no reason until now to have a function that deals
> > with lines of output separated by nulls.  So adding such a function
> > would mean we have a function that no one calls, and why would we do
> > that, when the way to handle such output in a Lisp program is as
> > simple as a single call of an existing function?
> 
> Well, there are four calls in:
> 
> | (split-string
> |  (shell-command-to-string
> |   (concat "foo "
> |           (shell-quote-argument "bar baz")))
> |  "\0"
> |  t)
> 
> I want to reduce this to one.

You could start by writing your own function for that if you use this
frequently, no?

> I don't know about the needs of Emacs core, but I use this
> pattern regularly to process the output of external programs
> where data may include newlines.  As Emacs does not provide
> a single function for that purpose yet, any user who wants
> to process data that may include newlines must use this
> pattern as well (or a similar one).  Providing a function
> that makes it easy to pass data properly encapsulated to an
> external program and retrieve its separated output would
> make it harder for users to write insecure code.

I see your point, but I'm not convinced we should add something like
that.  I welcome other opinions, with rationale.

Message #23 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Tim Landscheidt <tim <at> tim-landscheidt.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sun, 14 Sep 2025 11:11:02 +0000

Eli Zaretskii <eliz <at> gnu.org> wrote:

>> > That's not the important part of my question.  I'm asking why is there
>> > a need for a new function whose only job is to call split-string.

>> > process-lines exists because several places in Emacs call it.  By
>> > contrast, there was no reason until now to have a function that deals
>> > with lines of output separated by nulls.  So adding such a function
>> > would mean we have a function that no one calls, and why would we do
>> > that, when the way to handle such output in a Lisp program is as
>> > simple as a single call of an existing function?

>> Well, there are four calls in:

>> | (split-string
>> |  (shell-command-to-string
>> |   (concat "foo "
>> |           (shell-quote-argument "bar baz")))
>> |  "\0"
>> |  t)

>> I want to reduce this to one.

> You could start by writing your own function for that if you use this
> frequently, no?

Yes.

>> I don't know about the needs of Emacs core, but I use this
>> pattern regularly to process the output of external programs
>> where data may include newlines.  As Emacs does not provide
>> a single function for that purpose yet, any user who wants
>> to process data that may include newlines must use this
>> pattern as well (or a similar one).  Providing a function
>> that makes it easy to pass data properly encapsulated to an
>> external program and retrieve its separated output would
>> make it harder for users to write insecure code.

> I see your point, but I'm not convinced we should add something like
> that.  I welcome other opinions, with rationale.

Then it would be prudent to add the criteria that this
rationale has to meet to (emacs) Bugs.  Usually, the first
or second adjective in any (self-)description of Emacs is
"extensible".  If feature requests are ineligible if users
can implement them on their own, then documenting this could
save some time for all parties involved.

Message #26 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Tim Landscheidt <tim <at> tim-landscheidt.de>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sun, 14 Sep 2025 15:12:01 +0300

> From: Tim Landscheidt <tim <at> tim-landscheidt.de>
> Cc: 79442 <at> debbugs.gnu.org
> Date: Sun, 14 Sep 2025 11:11:02 +0000
> 
> Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> >> > That's not the important part of my question.  I'm asking why is there
> >> > a need for a new function whose only job is to call split-string.
> 
> >> > process-lines exists because several places in Emacs call it.  By
> >> > contrast, there was no reason until now to have a function that deals
> >> > with lines of output separated by nulls.  So adding such a function
> >> > would mean we have a function that no one calls, and why would we do
> >> > that, when the way to handle such output in a Lisp program is as
> >> > simple as a single call of an existing function?
> 
> >> Well, there are four calls in:
> 
> >> | (split-string
> >> |  (shell-command-to-string
> >> |   (concat "foo "
> >> |           (shell-quote-argument "bar baz")))
> >> |  "\0"
> >> |  t)
> 
> >> I want to reduce this to one.
> 
> > You could start by writing your own function for that if you use this
> > frequently, no?
> 
> Yes.
> 
> >> I don't know about the needs of Emacs core, but I use this
> >> pattern regularly to process the output of external programs
> >> where data may include newlines.  As Emacs does not provide
> >> a single function for that purpose yet, any user who wants
> >> to process data that may include newlines must use this
> >> pattern as well (or a similar one).  Providing a function
> >> that makes it easy to pass data properly encapsulated to an
> >> external program and retrieve its separated output would
> >> make it harder for users to write insecure code.
> 
> > I see your point, but I'm not convinced we should add something like
> > that.  I welcome other opinions, with rationale.
> 
> Then it would be prudent to add the criteria that this
> rationale has to meet to (emacs) Bugs.

If someone can come up with such criteria up front, I'm all ears.
Usually, it's a judgment call of the Emacs maintainers, to be made on
a per-case basis.

> Usually, the first or second adjective in any (self-)description of
> Emacs is "extensible".

Emacs is extensible, but not every possible extension must be in Emacs
OOTB.  Many people have quite a lot of local code in their init files
and local packages, without ever submitting that for inclusion in
Emacs, and that is how it should be, IMO.  Each one of us has his/her
local needs and requirements which are not necessarily shared by or
important to others.  Emacs being extensible by users is important
precisely for this reason.

> If feature requests are ineligible if users can implement them on
> their own, then documenting this could save some time for all
> parties involved.

It goes without saying that not every extension should be part of
Emacs, I'm surprised that this is not self-evident.

Message #29 received at 79442 <at> debbugs.gnu.org (full text, mbox):

From: Tim Landscheidt <tim <at> tim-landscheidt.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79442 <at> debbugs.gnu.org
Subject: Re: bug#79442: RFE: process-lines equivalent for NUL-separated output
Date: Sun, 14 Sep 2025 12:29:52 +0000

Eli Zaretskii <eliz <at> gnu.org> wrote:

> […]

>> If feature requests are ineligible if users can implement them on
>> their own, then documenting this could save some time for all
>> parties involved.

> It goes without saying that not every extension should be part of
> Emacs, I'm surprised that this is not self-evident.

It should also have been self-evident that the feature
requested here can be implemented by users themselves, yet
there was a question about it, and I answered it.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.