GNU bug report logs -
#66020
[PATCH] Reduce GC churn in read_process_output
Previous Next
Reported by: Dmitry Gutov <dmitry <at> gutov.dev>
Date: Sat, 16 Sep 2023 01:27:02 UTC
Severity: wishlist
Tags: patch
Done: Dmitry Gutov <dmitry <at> gutov.dev>
Bug is archived. No further changes may be made.
Full log
View this message in rfc822 format
On 20/09/2023 14:20, Eli Zaretskii wrote:
>> Date: Tue, 19 Sep 2023 22:59:43 +0300
>> Cc: 66020 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry <at> gutov.dev>
>>
>> - Maybe we declare the difference insignificant and bump the value of
>> read-process-output-max, given that it helps in other cases,
>> - Or try to find out the cause for degradation,
>> - Or keep the default the same, but make it easier to use different
>> value for different processes (meaning, we resurrect the discussion in
>> bug#38561).
>
> I'd try the same experiment on other use cases, say "M-x grep" and
> "M-x compile" with large outputs, and if you see the same situation
> there (i.e. larger buffers are no worse), try increasing the default
> value on master.
I've run one particular rgrep search a few times (24340 hits, ~44s when
the variable's value is either 4096 or 409600). And it makes sense that
there is no difference: compilation modes do a lot more work than just
capturing the process output or splitting it into strings.
That leaves the question of what new value to use. 409600 is optimal for
a large-output process but seems too much as default anyway (even if I
have very little experimental proof for that hesitance: any help with
that would be very welcome).
I did some more experimenting, though. At a superficial glance,
allocating the 'chars' buffer at the beginning of read_process_output is
problematic because we could instead reuse a buffer for the whole
duration of the process. I tried that (adding a new field to
Lisp_Process and setting it in make_process), although I had to use a
value produced by make_uninit_string: apparently simply storing a char*
field inside a managed structure creates problems for the GC and early
segfaults. Anyway, the result was slightly _slower_ than the status quo.
So I read what 'alloca' does, and it looks hard to beat. But it's only
used (as you of course know) when the value is <= MAX_ALLOCA, which is
currently 16384. Perhaps an optimal default value shouldn't exceed this,
even if it's hard to create a benchmark that shows a difference. With
read-process-output-max set to 16384, my original benchmark gets about
halfway to the optimal number.
And I think we should make the process "remember" the value at its
creation either way (something touched on in bug#38561): in bug#55737 we
added an fcntl call to make the larger values take effect. But this call
is in create_process: so any subsequent increase to a large value of
this var won't have effect. Might as well remember it there (in a new
field), then it'll be easier to use different values of it for different
processes (set using let-binding at the time of the process' creation).
This bug report was last modified 1 year and 31 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.