GNU bug report logs -
#72830
Big rectangular selections are slow
Previous Next
To reply to this bug, email your comments to 72830 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 12:41:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Mattias Engdegård <mattias.engdegard <at> gmail.com>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Tue, 27 Aug 2024 12:41:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Selecting rectangles that stretch across many lines is very slow. Recipe:
C-x b example RET
C-u 60 x C-u 200000 M-x duplicate-dwim RET C-a
C-x SPC
M-> C-f C-f C-f C-f
Each keystroke at the end lags noticeably, to the point that this isn't practical.
Why this matters: rectangular selections are very useful for editing large amount of data in a homogeneous way, and such a file (12 MB) isn't huge.
Possible causes:
- Slow `region-beginning` and `region-end`.
Fixed on master (3a8222e7).
- `select-active-regions` makes each selection change very expensive.
Can be set to nil by the user, but I really suggest that we change the default to nil, at least for non-X11 since the default assumes old-school X11 PRIMARY selection which is alien on most other platforms.
- Slow `rectangle--highlight-for-redisplay`. This is the remaining cause.
Profiling shows that most of the time is spent in the function passed to `apply-on-rectangle`.
Here is an excerpt of that profile:
| Weight Self
| 31.93 Gc 57.4% 103.06 Mc Fmove_to_column
| 31.52 Gc 56.7% 884.80 Mc scan_for_column
| 14.45 Gc 26.0% 27.82 Mc check_display_width
| 14.30 Gc 25.7% 648.23 Mc get_char_property_and_overlay
| 9.79 Gc 17.6% 1.07 Gc itree_iterator_next
| 2.00 Gc 3.6% 42.68 Mc Fget_text_property
| 743.88 Mc 1.3% 154.63 Mc itree_iterator_start
| 678.00 Mc 1.2% 183.02 Mc lookup_char_property
| 7.28 Gc 13.0% 155.15 Mc skip_invisible
| 4.22 Gc 7.6% 701.27 Mc composition_compute_stop_pos
| 3.00 Gc 5.3% 153.86 Mc window_loop
| 1.28 Gc 2.3% 216.97 Mc find_newline
| 2.28 Gc 4.1% 80.59 Mc Fmove_overlay
| 1.46 Gc 2.6% 40.18 Mc lookup_char_property
This excludes GC costs (about 18 % I think), which in turn is dominated by scanning overlays.
It's a bit surprising (to me) how expensive move-to-column is. The text is all-ASCII without any complications (no long lines, properties, etc).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 13:51:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
> From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
> Date: Tue, 27 Aug 2024 14:39:37 +0200
>
> It's a bit surprising (to me) how expensive move-to-column is. The text is all-ASCII without any complications (no long lines, properties, etc).
Does turning off auto-composition-mode help in any way?
Also, since highlighting is AFAIR implemented as overlays, it slows
down check_display_width, which I think would otherwise have moved
very fast if you say there are no properties and no overlays.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 14:06:02 GMT)
Full text and
rfc822 format available.
Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):
> Selecting rectangles that stretch across many lines is very slow.
Yup.
> - Slow `region-beginning` and `region-end`.
> Fixed on master (3a8222e7).
Seems unlikely.
> - `select-active-regions` makes each selection change very expensive.
> Can be set to nil by the user, but I really suggest that we change the
> default to nil, at least for non-X11 since the default assumes old-school
> X11 PRIMARY selection which is alien on most other platforms.
Didn't think of that. Do we re-set the PRIMARY every time the (active)
rectangular region is changed (i.e. after each key press)? That would
make it non-scalable to large rectangular regions, indeed.
I thought the extraction of the region's content to put it into PRIMARY
and friends happened more lazily (e.g. when another application
requests the PRIMARY/CLIPBOARD, or when we deactivate the region).
> - Slow `rectangle--highlight-for-redisplay`. This is the remaining cause.
This is the one I'm aware of.
> Profiling shows that most of the time is spent in the function passed
> to `apply-on-rectangle`. Here is an excerpt of that profile:
>
> | Weight Self
> | 31.93 Gc 57.4% 103.06 Mc Fmove_to_column
> | 31.52 Gc 56.7% 884.80 Mc scan_for_column
> | 14.45 Gc 26.0% 27.82 Mc check_display_width
> | 14.30 Gc 25.7% 648.23 Mc get_char_property_and_overlay
> | 9.79 Gc 17.6% 1.07 Gc itree_iterator_next
> | 2.00 Gc 3.6% 42.68 Mc Fget_text_property
> | 743.88 Mc 1.3% 154.63 Mc itree_iterator_start
> | 678.00 Mc 1.2% 183.02 Mc lookup_char_property
> | 7.28 Gc 13.0% 155.15 Mc skip_invisible
> | 4.22 Gc 7.6% 701.27 Mc composition_compute_stop_pos
> | 3.00 Gc 5.3% 153.86 Mc window_loop
> | 1.28 Gc 2.3% 216.97 Mc find_newline
> | 2.28 Gc 4.1% 80.59 Mc Fmove_overlay
> | 1.46 Gc 2.6% 40.18 Mc lookup_char_property
I don't think we want to try and optimize the operations here because
that will only push the limit of what is "too large" a bit further.
More promising to me would be to try and skip any part of the buffer
that's not displayed.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 16:44:02 GMT)
Full text and
rfc822 format available.
Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):
27 aug. 2024 kl. 15.48 skrev Eli Zaretskii <eliz <at> gnu.org>:
> Does turning off auto-composition-mode help in any way?
I don't think so. Were you able to reproduce the problem?
> Also, since highlighting is AFAIR implemented as overlays, it slows
> down check_display_width, which I think would otherwise have moved
> very fast if you say there are no properties and no overlays.
Yes, overlays indeed seem to be at the heart of the problem.
27 aug. 2024 kl. 16.04 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
>> - Slow `region-beginning` and `region-end`.
>> Fixed on master (3a8222e7).
>
> Seems unlikely.
It did show up on the profile, something had to be done. Before that change, (region-beginning) with the active region as in the example took 0.96 s on this machine. Now, maybe 30 µs.
>> - `select-active-regions` makes each selection change very expensive.
>> Can be set to nil by the user, but I really suggest that we change the
>> default to nil, at least for non-X11 since the default assumes old-school
>> X11 PRIMARY selection which is alien on most other platforms.
>
> Didn't think of that. Do we re-set the PRIMARY every time the (active)
> rectangular region is changed (i.e. after each key press)? That would
> make it non-scalable to large rectangular regions, indeed.
It's pretty bonkers, yes. There is even a noticeable slow-down with a single contiguous selection.
Fortunately the cure is easy.
> I thought the extraction of the region's content to put it into PRIMARY
> and friends happened more lazily (e.g. when another application
> requests the PRIMARY/CLIPBOARD, or when we deactivate the region).
That would indeed have made some kind of sense. If confined to X11, that is. And opt-in, even then.
> I don't think we want to try and optimize the operations here because
> that will only push the limit of what is "too large" a bit further.
Yes, but it is educational to see what the costs are
> More promising to me would be to try and skip any part of the buffer
> that's not displayed.
Yes, I see no reason for having the highlight overlay on something that can't be seen.
But how do we know what the viewport will be in `rectangle--highlight-for-redisplay`? Since it is redisplay that determines window-start and window-end, and it hasn't run yet, this puts us in some kind of chicken-egg situation. Maybe it has to be done inside redisplay.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 17:51:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
> Date: Tue, 27 Aug 2024 18:42:19 +0200
> Cc: Emacs Bug Report <bug-gnu-emacs <at> gnu.org>,
> Juri Linkov <juri <at> linkov.net>
>
> 27 aug. 2024 kl. 15.48 skrev Eli Zaretskii <eliz <at> gnu.org>:
>
> > Does turning off auto-composition-mode help in any way?
>
> I don't think so. Were you able to reproduce the problem?
I didn't have time to try, sorry. Too much stuff on my plate, and
still only 24 hours a day, even with Emacs 30...
> Yes, I see no reason for having the highlight overlay on something that can't be seen.
>
> But how do we know what the viewport will be in `rectangle--highlight-for-redisplay`? Since it is redisplay that determines window-start and window-end, and it hasn't run yet, this puts us in some kind of chicken-egg situation. Maybe it has to be done inside redisplay.
It has to be done with the display code, but does not have to be as
part of redisplay. We can use functions like vertical-motion,
window-text-pixel-size, posn-at-point, and other similar APIs, which
all employ display code that performs layout-related calculations
without actually displaying anything.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 18:29:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 72830 <at> debbugs.gnu.org (full text, mbox):
>> 27 aug. 2024 kl. 15.48 skrev Eli Zaretskii <eliz <at> gnu.org>:
>>
>> > Does turning off auto-composition-mode help in any way?
>>
>> I don't think so. Were you able to reproduce the problem?
>
> I didn't have time to try, sorry. Too much stuff on my plate, and
> still only 24 hours a day, even with Emacs 30...
For Emacs 30 it's time to increase the default value of hours-per-day to 30.
>> 27 aug. 2024 kl. 16.04 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
>>
>> More promising to me would be to try and skip any part of the buffer
>> that's not displayed.
>
> Yes, I see no reason for having the highlight overlay on something that can't be seen.
>
> But how do we know what the viewport will be in
> `rectangle--highlight-for-redisplay`? Since it is redisplay that determines
> window-start and window-end, and it hasn't run yet, this puts us in some
> kind of chicken-egg situation. Maybe it has to be done inside redisplay.
I'm sure it should be possible to use `pre-redisplay-functions'
that by default updates region highlighting, and I was able to use it
to make region highlighting window-local even when the same buffer
is displayed in separate windows.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 18:57:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> For Emacs 30 it's time to increase the default value of hours-per-day to 30.
+1.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 19:05:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> From: Juri Linkov <juri <at> linkov.net>
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Eli Zaretskii
> <eliz <at> gnu.org>, 72830 <at> debbugs.gnu.org
> Date: Tue, 27 Aug 2024 21:23:05 +0300
>
> >> 27 aug. 2024 kl. 15.48 skrev Eli Zaretskii <eliz <at> gnu.org>:
> >>
> > I didn't have time to try, sorry. Too much stuff on my plate, and
> > still only 24 hours a day, even with Emacs 30...
>
> For Emacs 30 it's time to increase the default value of hours-per-day to 30.
That'd be backward-incompatible, so we'd need a user option to get
back the old behavior. And it's too late for new features in Emacs 30
anyway.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 19:19:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> Cc: 72830 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
> Date: Tue, 27 Aug 2024 20:47:18 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
>
> > But how do we know what the viewport will be in `rectangle--highlight-for-redisplay`? Since it is redisplay that determines window-start and window-end, and it hasn't run yet, this puts us in some kind of chicken-egg situation. Maybe it has to be done inside redisplay.
>
> It has to be done with the display code, but does not have to be as
> part of redisplay. We can use functions like vertical-motion,
> window-text-pixel-size, posn-at-point, and other similar APIs, which
> all employ display code that performs layout-related calculations
> without actually displaying anything.
Btw, the horizontal dimensions of the window are known in advance even
without calling the display code, and we could use posn-at-x-y to find
the buffer position that corresponds to the right edge of the window.
Isn't that what's needed to avoid highlighting the portions of text
that are outside of the viewport?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Tue, 27 Aug 2024 19:46:01 GMT)
Full text and
rfc822 format available.
Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):
>>> - `select-active-regions` makes each selection change very expensive.
>>> Can be set to nil by the user, but I really suggest that we change the
>>> default to nil, at least for non-X11 since the default assumes old-school
>>> X11 PRIMARY selection which is alien on most other platforms.
>>
>> Didn't think of that. Do we re-set the PRIMARY every time the (active)
>> rectangular region is changed (i.e. after each key press)? That would
>> make it non-scalable to large rectangular regions, indeed.
>
> It's pretty bonkers, yes. There is even a noticeable slow-down with
> a single contiguous selection. Fortunately the cure is easy.
I rely on it on a regular basis, and I'm probably not the only one, so
even if we disable it by default, we should make it work well enough for
large rectangular regions.
> But how do we know what the viewport will be in
> `rectangle--highlight-for-redisplay`? Since it is redisplay that determines
> window-start and window-end, and it hasn't run yet, this puts us in some
> kind of chicken-egg situation. Maybe it has to be done inside redisplay.
The code should use `window-start` and `window-end` for that.
There might be some remaining issues with (re)running the
`pre-redisplay-function(s)` when redisplay forces a change in
`window-start` (or in the unlikely case where the highlighting moves the
`window-end` further), but these are things we need to fix anyway.
There are further issues when several windows display the buffer. 🙂
We could also move some of the work to jit-lock, which would have the
advantage that it would additionally be able to skip over (large) chunks
that are marked as `invisible`.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 00:42:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 72830 <at> debbugs.gnu.org (full text, mbox):
Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:
> - `select-active-regions` makes each selection change very expensive.
> Can be set to nil by the user, but I really suggest that we change
> the default to nil, at least for non-X11 since the default assumes
> old-school X11 PRIMARY selection which is alien on most other
> platforms.
No!!! Primary selections are an integral part of X, and users (many
trained on X) have come to expect it of Emacs on other systems.
Emacs supports saving a region in a buffer as a selection rather than an
entire string, which could be exercised in the case of rectangular
selections.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 00:47:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 72830 <at> debbugs.gnu.org (full text, mbox):
Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:
> It's pretty bonkers, yes. There is even a noticeable slow-down with a
> single contiguous selection. Fortunately the cure is easy.
>
>> I thought the extraction of the region's content to put it into PRIMARY
>> and friends happened more lazily (e.g. when another application
>> requests the PRIMARY/CLIPBOARD, or when we deactivate the region).
>
> That would indeed have made some kind of sense. If confined to X11,
> that is. And opt-in, even then.
No well-written X program of the past 30 years does not export its
selected text as the primary selection, and users of free desktops are
long accustomed to its existence, with the result that Wayland desktops
implement primary selections by a number of protocols. This is simply
not an option.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 03:41:01 GMT)
Full text and
rfc822 format available.
Message #41 received at submit <at> debbugs.gnu.org (full text, mbox):
Po Lu via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs <at> gnu.org> writes:
> No well-written X program of the past 30 years does not export its
> selected text as the primary selection, and users of free desktops are
> long accustomed to its existence, with the result that Wayland desktops
> implement primary selections by a number of protocols. This is simply
> not an option.
One data point from me: When I foll the recipe and then insert the
current primary selection into a different Emacs instance, I get a
column of 2500 "x"s inserted. The recipe had 200,000 lines of "x"s.
That means, at least using my settings, what Emacs does in this extreme
case seems to be a waste of time because 99% of the selection is thrown
away by X or whatever.
Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 03:41:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 03:57:01 GMT)
Full text and
rfc822 format available.
Message #47 received at submit <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:
> There are further issues when several windows display the buffer. 🙂
Is it that bad? Currently the overlays used for rectangle visualizing
specify a window overlay property. Only one window gets updated now.
My idea would be: highlight from or up to "rectangle point" maximally
(window-height) lines. Then the visual appearance is the same as now
(right?), and when the user moves point or selects a different window,
AFAIU the rectangle needs to be redrawn anyway.
Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 03:57:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 04:46:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 72830 <at> debbugs.gnu.org (full text, mbox):
Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> One data point from me: When I foll the recipe and then insert the
> current primary selection into a different Emacs instance, I get a
> column of 2500 "x"s inserted. The recipe had 200,000 lines of "x"s.
> That means, at least using my settings, what Emacs does in this extreme
> case seems to be a waste of time because 99% of the selection is thrown
> away by X or whatever.
This must be isolated to rectangular selections.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 08:11:01 GMT)
Full text and
rfc822 format available.
Message #56 received at submit <at> debbugs.gnu.org (full text, mbox):
27 aug. 2024 kl. 21.16 skrev Eli Zaretskii <eliz <at> gnu.org>:
>> It has to be done with the display code, but does not have to be as
>> part of redisplay. We can use functions like vertical-motion,
>> window-text-pixel-size, posn-at-point, and other similar APIs, which
>> all employ display code that performs layout-related calculations
>> without actually displaying anything.
>
> Btw, the horizontal dimensions of the window are known in advance even
> without calling the display code, and we could use posn-at-x-y to find
> the buffer position that corresponds to the right edge of the window.
> Isn't that what's needed to avoid highlighting the portions of text
> that are outside of the viewport?
Unfortunately I don't think that helps. The way it works now is that a pre-redisplay function puts a highlight overlay on every line in the rectangular selection. All this work and memory completely tanks performance.
Only putting overlays on lines where they can be seen would solve the problem, but we have to know the first and last line that will be displayed. I tried using `posn-at-x-y` but it doesn't seem to recompute `window-start` so it has the same problem.
27 aug. 2024 kl. 21.44 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
> I rely on it on a regular basis, and I'm probably not the only one, so
> even if we disable it by default, we should make it work well enough for
> large rectangular regions.
Yes, that's probably true. Let's deal with that concern later; it's orthogonal to the redisplay problem.
> There might be some remaining issues with (re)running the
> `pre-redisplay-function(s)` when redisplay forces a change in
> `window-start` (or in the unlikely case where the highlighting moves the
> `window-end` further), but these are things we need to fix anyway.
That's what I wondered how to deal with. I agree we can probably assume that highlighting won't affect the viewport in any substantial way.
To be clear, just using the (inaccurate) window-start and window-end to clip the setting of highlight overlays works very well and with excellent speed until the viewport moves, in which case it no longer does.
> There are further issues when several windows display the buffer.
Are there? The highlight overlay is set with a `window` parameter that keeps it visible in one window only.
> We could also move some of the work to jit-lock, which would have the
> advantage that it would additionally be able to skip over (large) chunks
> that are marked as `invisible`.
Not sure how this would work. Isn't it guided by (comparatively persistent) `fontified` properties?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 10:25:01 GMT)
Full text and
rfc822 format available.
Message #59 received at 72830 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
29 aug. 2024 kl. 05.56 skrev Michael Heerdegen <michael_heerdegen <at> web.de>:
> My idea would be: highlight from or up to "rectangle point" maximally
> (window-height) lines.
Here I was digging into the redisplay code in the hope of finding an obscure hook to subvert, and you turn up and give us the obvious, simple and correct solution right away. Thank you, Michael!
The patch below is crude but seems to work. Please give it a try.
If the original recipe is still stuttering, try setting `select-active-regions` to nil.
[rect-highlight.diff (application/octet-stream, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 11:21:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 72830 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
29 aug. 2024 kl. 12.22 skrev Mattias Engdegård <mattias.engdegard <at> gmail.com>:
> The patch below is crude but seems to work.
That phrase is highly effective for forcing bugs into daylight. Here is an improved version.
Previously, `exchange-point-and-mark` didn't regenerate the overlays because it wasn't necessary, but now it is, so we save point in our rectangle overlay tuple. (Should use a struct at this point.)
[rect-highlight.diff (application/octet-stream, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Thu, 29 Aug 2024 20:07:02 GMT)
Full text and
rfc822 format available.
Message #65 received at 72830 <at> debbugs.gnu.org (full text, mbox):
>> There might be some remaining issues with (re)running the
>> `pre-redisplay-function(s)` when redisplay forces a change in
>> `window-start` (or in the unlikely case where the highlighting moves the
>> `window-end` further), but these are things we need to fix anyway.
> That's what I wondered how to deal with. I agree we can probably assume that
> highlighting won't affect the viewport in any substantial way.
> To be clear, just using the (inaccurate) window-start and window-end to clip
> the setting of highlight overlays works very well and with excellent speed
> until the viewport moves, in which case it no longer does.
"The viewport moves" is what I referred to as "when redisplay forces
a change in `window-start`" (there are actually many cases where the
viewport moves before redisplay is invoked, in which case "just using
the (inaccurate) window-start and window-end to clip the setting of
highlight overlays works very well").
We should investigate the case(s) where it doesn't work well and fix
them (not by changing your `pre-redisplay-function` but by re-running
that hook), because these cases likely affect other users of this hook
(such as the normal region highlighting).
>> There are further issues when several windows display the buffer.
> Are there? The highlight overlay is set with a `window` parameter that keeps
> it visible in one window only.
Ah, great, then. I didn't remember this.
>> We could also move some of the work to jit-lock, which would have the
>> advantage that it would additionally be able to skip over (large) chunks
>> that are marked as `invisible`.
> Not sure how this would work. Isn't it guided by (comparatively
> persistent) `fontified` properties?
We'd set that property to nil.
[ See the recentish discussion about how to change jit-lock so that it
can be told to flush only some of its backends. ]
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Fri, 30 Aug 2024 15:23:02 GMT)
Full text and
rfc822 format available.
Message #68 received at 72830 <at> debbugs.gnu.org (full text, mbox):
29 aug. 2024 kl. 22.04 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>
> We should investigate the case(s) where it doesn't work well and fix
> them (not by changing your `pre-redisplay-function` but by re-running
> that hook), because these cases likely affect other users of this hook
> (such as the normal region highlighting).
What would the problem be for normal (contiguous) region highlighting? There is a single overlay, so there should be no particular cost for large selections. Is there?
The latest patch posted seems to work well enough and all but eliminates the problem (when `select-active-regions` is nil, anyway), so that it outweighs some rare cosmetic issues.
(Those cosmetic issues may occur when a rectangular selection straddles invisible text, because this invalidates the assumption that the other end of the selection is at most (window-height) away. But the selection still operate on the invisible test, so I don't think it's a problem in practice.)
I think the patch is good enough for master. If you have an even better solution, perhaps it could be made as a subsequent improvement?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Fri, 20 Sep 2024 12:55:02 GMT)
Full text and
rfc822 format available.
Message #71 received at 72830 <at> debbugs.gnu.org (full text, mbox):
30 aug. 2024 kl. 17.20 skrev Mattias Engdegård <mattias.engdegard <at> gmail.com>:
> I think the patch is good enough for master.
And it has now been pushed. To recap, remaining concerns:
- `select-active-regions`: as mentioned, it slows down rectangle selection massively and is alien to non-X11 platforms so I'd suggest it be set to nil by default on macOS and Windows at least. Not sure what to do on TTY frames.
For that matter, on X11 it seems strange to copy the entire selection each time it is made or modified even slightly; PRIMARY selection data isn't sent to the server until actually requested.
- Rectangle operations other than selection are still very slow. This isn't quite as big a problem but still annoying. Both `move-to-column` and `line-end-position` are unexpectedly expensive here. (And shouldn't rect.el use `pos-eol` instead? It's not much faster, though.)
- There are some mostly theoretical cases when the current rectangular selection code produces ugly results. I don't think they are likely to occur in practice but would be happy to work with someone who thinks he or she can do better.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sat, 21 Sep 2024 02:09:02 GMT)
Full text and
rfc822 format available.
Message #74 received at 72830 <at> debbugs.gnu.org (full text, mbox):
Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:
> - `select-active-regions`: as mentioned, it slows down rectangle
> selection massively and is alien to non-X11 platforms so I'd suggest
> it be set to nil by default on macOS and Windows at least. Not sure
> what to do on TTY frames.
I see no problem with making this change on macOS. I have no idea about
anything related to MS-Windows, so I can't comment on that.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sat, 21 Sep 2024 03:06:01 GMT)
Full text and
rfc822 format available.
Message #77 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> - `select-active-regions`: as mentioned, it slows down rectangle selection
> massively and is alien to non-X11 platforms so I'd suggest it be set to nil
> by default on macOS and Windows at least.
I think we should first try and make it not-slow, by making it lazy.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sat, 21 Sep 2024 08:28:01 GMT)
Full text and
rfc822 format available.
Message #80 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Kangas <stefankangas <at> gmail.com>
> Date: Fri, 20 Sep 2024 19:07:29 -0700
> Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, Eli Zaretskii <eliz <at> gnu.org>, 72830 <at> debbugs.gnu.org,
> Juri Linkov <juri <at> linkov.net>
>
> Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:
>
> > - `select-active-regions`: as mentioned, it slows down rectangle
> > selection massively and is alien to non-X11 platforms so I'd suggest
> > it be set to nil by default on macOS and Windows at least. Not sure
> > what to do on TTY frames.
>
> I see no problem with making this change on macOS. I have no idea about
> anything related to MS-Windows, so I can't comment on that.
I can test on MS-Windows if someone tells what to try and what to pay
attention to.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 13:30:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 72830 <at> debbugs.gnu.org (full text, mbox):
21 sep. 2024 kl. 05.05 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
>> - `select-active-regions`: as mentioned, it slows down rectangle selection
>> massively and is alien to non-X11 platforms so I'd suggest it be set to nil
>> by default on macOS and Windows at least.
>
> I think we should first try and make it not-slow, by making it lazy.
No objections there but we would need to rework some plumbing. Emacs's behaviour is odd.
When a selection is made, that text is extracted and squirrelled away just in case, so the user can actually select some text, deselect and move around, and even make some changes the the buffer, and then paste PRIMARY from Emacs or any other X11 client and still get the originally selected text.
This isn't how typical old-fashioned X11 clients work and is more likely to be by accident than design but then again I'm not using X11 much these days.
Untangling this mess would take some time.
21 sep. 2024 kl. 10.26 skrev Eli Zaretskii <eliz <at> gnu.org>:
> I can test on MS-Windows if someone tells what to try and what to pay
> attention to.
Thank you Eli, I'll make sure to ask you when I have something to show.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 14:14:02 GMT)
Full text and
rfc822 format available.
Message #86 received at 72830 <at> debbugs.gnu.org (full text, mbox):
>>> - `select-active-regions`: as mentioned, it slows down rectangle selection
>>> massively and is alien to non-X11 platforms so I'd suggest it be set to nil
>>> by default on macOS and Windows at least.
>> I think we should first try and make it not-slow, by making it lazy.
> No objections there but we would need to rework some plumbing. Emacs's behaviour is odd.
> When a selection is made, that text is extracted and squirrelled away just
> in case, so the user can actually select some text, deselect and move
> around, and even make some changes the the buffer, and then paste PRIMARY
> from Emacs or any other X11 client and still get the originally
> selected text.
IIRC from the last time I looked at that code, I got the impression that
the design was made [pun ahead!] primarily for the CLIPBOARD selection
and *should* work something like this:
- when we make a new selection, we tell X11 that we own the CLIPBOARD.
This should be an O(1) operation.
- when the selection changes because we move point or mark, we don't
need to do anything.
- we get the content of that selection (an O(N) operation) only if/when
X11 asks for it.
- in order to still be able to send the CLIPBOARD's content after the
selection has disappeared, we pay the O(N) cost when the region is
deactivated and "squirrelled away" (like you say) that content.
So the big rectangular selections should slow down only steps 3 and
4 but not step 1 or 2. And as you point out, maybe step 4 could/should
be skipped for PRIMARY, tho I'm not sure in which cases it would
be beneficial (beside those cases where the region is so large that the
O(N) cost is a problem).
Stefan
PS: Of course, if needed, maybe the step 4 could be made faster by
squirreling away not the exact rectangular region, but something that
can be computed more quickly and from which we can later still extract
the exact rectangular region if/when needed.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 15:18:01 GMT)
Full text and
rfc822 format available.
Message #89 received at 72830 <at> debbugs.gnu.org (full text, mbox):
22 sep. 2024 kl. 16.12 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
> IIRC from the last time I looked at that code, I got the impression that
> the design was made [pun ahead!] primarily for the CLIPBOARD selection
> and *should* work something like this:
> - when we make a new selection, we tell X11 that we own the CLIPBOARD.
> This should be an O(1) operation.
> - when the selection changes because we move point or mark, we don't
> need to do anything.
> - we get the content of that selection (an O(N) operation) only if/when
> X11 asks for it.
> - in order to still be able to send the CLIPBOARD's content after the
> selection has disappeared, we pay the O(N) cost when the region is
> deactivated and "squirrelled away" (like you say) that content.
I don't think an X11 client should ever claim CLIPBOARD ownership merely for marking a selection, as opposed to issuing a 'Copy[ to clipboard]' operation, but seem to recall that it did happen with some broken clients. PRIMARY is indeed different and that's probably what confused programmers back in the day, especially since old clients like Xterm emphasised the use of PRIMARY.
> So the big rectangular selections should slow down only steps 3 and
> 4 but not step 1 or 2. And as you point out, maybe step 4 could/should
> be skipped for PRIMARY, tho I'm not sure in which cases it would
> be beneficial (beside those cases where the region is so large that the
> O(N) cost is a problem).
The O(N) cost in time and space is a problem. X11 convention is that PRIMARY is available for as long as the selection is visibly marked in the client.
[ What happens if another client claims PRIMARY? The first (losing) client typically has two choices: either remove the selection so that the user sees that the selection is no more (the classic Xterm way), or repaint it in a 'local-only' colour to show that it can still be used for other purposes. ]
> Of course, if needed, maybe the step 4 could be made faster by
> squirreling away not the exact rectangular region, but something that
> can be computed more quickly and from which we can later still extract
> the exact rectangular region if/when needed.
Yes, we'd need some sort of current-selection-extent object. The question is, what should its lifetime be?
A. For as long as the selection is active
B. Until buffer is modified
C. Indefinitely, using absolute buffer offsets
D. Indefinitely, using markers
I'd suggest A because that's closest to X11 practice. B would be doable. C and D are taking it too far and recovering a rectangle would be a mess.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 15:33:02 GMT)
Full text and
rfc822 format available.
Message #92 received at 72830 <at> debbugs.gnu.org (full text, mbox):
Mattias Engdegård [2024-09-22 17:16:20] wrote:
[ ... Various questions about what behavior we want to provide ... ]
Hmm... I thought your complaint about the cost of
`select-active-regions` was due to the cost being too high in step
2 (which would indicate that the code does not behave quite like the
design I described, and would thus be a simple bug we should fix).
Are you saying that when you said:
- `select-active-regions`: as mentioned, it slows down rectangle
selection massively ...
you were talking about the slowdown imposed in step 4 rather than in
step 2?
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 16:24:02 GMT)
Full text and
rfc822 format available.
Message #95 received at 72830 <at> debbugs.gnu.org (full text, mbox):
22 sep. 2024 kl. 17.32 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
> Hmm... I thought your complaint about the cost of
> `select-active-regions` was due to the cost being too high in step
> 2 (which would indicate that the code does not behave quite like the
> design I described, and would thus be a simple bug we should fix).
>
> Are you saying that when you said:
>
> - `select-active-regions`: as mentioned, it slows down rectangle
> selection massively ...
>
> you were talking about the slowdown imposed in step 4 rather than in
> step 2?
Sorry if I've lost track with your step numbering, but the problem is still that we extract and retain the selected text both at selection and each incremental modification of it.
So the extraction needs to be delayed until either requested by the X server or locally by mouse-yank-primary. This means that we need to decide whether it would be a problem to do so, as the buffer may have changed between the selection and text extraction. (I think we shouldn't bother preserving PRIMARY data across buffer modification.)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Sun, 22 Sep 2024 17:38:02 GMT)
Full text and
rfc822 format available.
Message #98 received at 72830 <at> debbugs.gnu.org (full text, mbox):
> Sorry if I've lost track with your step numbering, but the problem is still
> that we extract and retain the selected text both at selection and each
> incremental modification of it.
"incremental modification" is my step 2. And in the design, step
2 should not actually compute the content of the selection.
IOW, what you describe sounds like a plain bug.
> So the extraction needs to be delayed until either requested by the X server
> or locally by mouse-yank-primary. This means that we need to decide whether
> it would be a problem to do so, as the buffer may have changed between the
> selection and text extraction. (I think we shouldn't bother preserving
> PRIMARY data across buffer modification.)
Modifications deactivate the selection, so if we want to preserve the
selection until after modifications, that's when step 4 comes into play.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#72830; Package
emacs.
(Mon, 23 Sep 2024 10:44:02 GMT)
Full text and
rfc822 format available.
Message #101 received at 72830 <at> debbugs.gnu.org (full text, mbox):
22 sep. 2024 kl. 19.37 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
>
>> Sorry if I've lost track with your step numbering, but the problem is still
>> that we extract and retain the selected text both at selection and each
>> incremental modification of it.
>
> "incremental modification" is my step 2. And in the design, step
> 2 should not actually compute the content of the selection.
> IOW, what you describe sounds like a plain bug.
No argument there!
> Modifications deactivate the selection, so if we want to preserve the
> selection until after modifications, that's when step 4 comes into play.
Yes, let's not do that then.
(I'm not sure if it's even possible to delay the text extraction of PRIMARY until it is actually requested on macOS, where PRIMARY and SECONDARY are just alternative pasteboards. I think the idea is to cooperate with XQuartz, but it should be safe to have that feature disabled by default.)
This bug report was last modified 266 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.