GNU bug report logs - #21468
24.5; When next-line (visual) crosses overlay with before-string="\n" point goes to column 0

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21468 in the body.
You can then email your comments to 21468 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.5; When next-line (visual) crosses overlay with before-string="\n"
 point goes to column 0
Date: Sun, 13 Sep 2015 01:17:39 -0400

Run

    emacs -Q next-line-fail.el -l next-line-fail.el

where next-line-fail.el contains

(let ((pos 17))
  (goto-char pos)
  (overlay-put (make-overlay pos (1+ pos)) 'before-string
               (propertize "\n" 'face 'highlight)))

The overlay will create a highlighted blank line below line 1.
Move point forward so it's on line 1, column X (where X != 0), then do
C-n (next-line).

Expected: point should be on line 2 column X.
Actual: point goes to line 2, but in column 0.  Note that hitting C-n
a second time goes to line 3 column X.

The problem does not happen with (setq line-move-visual nil).

I've reproduced this in GUI and terminal.

This is a simplification from a magit bug
https://github.com/magit/magit/issues/2094

Message #8 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n"
 point	goes to column 0
Date: Sun, 13 Sep 2015 13:30:52 +0300

> Date: Sun, 13 Sep 2015 01:17:39 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> 
> Run
> 
>     emacs -Q next-line-fail.el -l next-line-fail.el
> 
> where next-line-fail.el contains
> 
> (let ((pos 17))
>   (goto-char pos)
>   (overlay-put (make-overlay pos (1+ pos)) 'before-string
>                (propertize "\n" 'face 'highlight)))
> 
> The overlay will create a highlighted blank line below line 1.
> Move point forward so it's on line 1, column X (where X != 0), then do
> C-n (next-line).
> 
> Expected: point should be on line 2 column X.
> Actual: point goes to line 2, but in column 0.  Note that hitting C-n
> a second time goes to line 3 column X.

Fixed in commit 6514b30 on master.

Once again, I must respectfully request that Magit maintainers refrain
from these practices of (ab)using Emacs display features where simpler
alternatives are readily available.  The buffer displayed by Magit is
entirely ephemeral, i.e. does not come from any file, so a much
simpler way would be to arrange the text in that buffer as Magit
needs, instead of using overlays and display strings to show something
that is not in the buffer.

E.g., in this fragment, cited by the Magit issue:

  (ov rend (1+ rend) 'after-string
      (propertize (concat (propertize "\s" 'display '(space :height (1)))
      		          (propertize "\n" 'line-height t))
                  'face 'magit-diff-lines-boundary)))

why can't Magit simply change the buffer text to be what it wants to
display? why does it need to use an overlay string?  All those text
properties are supported on buffer text as well.  Overlay strings and
display strings with embedded newlines are especially painful wrt
vertical cursor motion; changing the buffer text instead to present
the same display makes vertical motion's job much simpler.

The Emacs display engine was not designed for too heavy use of these
features.  By over-using them, you make Magit much less stable than
necessary, certainly much less future-proof and consistent across
different Emacs releases.  Please don't.

Message #13 received at 21468-done <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468-done <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Sun, 13 Sep 2015 13:20:04 -0400

On Sun, Sep 13, 2015 at 6:30 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Fixed in commit 6514b30 on master.

Thanks, it works.

> why can't Magit simply change the buffer text to be what it wants to
> display? why does it need to use an overlay string?

In this particular case, the overlays are delimiting a "magit region"
(consisting of whole logical lines touched by Emacs' region), having
them as actual text in the buffer would interfere with point movement
even more.

Or at least, that's what I saw when I tried just now to do something
similar with text properties:

(with-current-buffer (get-buffer-create "*text property test*")
  (erase-buffer)
  (insert-file "next-line-fail.el")
  (let ((pos 65))
    (goto-char pos)
    (insert (propertize (concat (propertize "\s" 'display '(space :height (1)))
                        (propertize "\n" 'line-height t))
                        'face 'highlight))
    (pop-to-buffer (current-buffer))))

Message #16 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sun, 13 Sep 2015 22:50:01 +0300

> Date: Sun, 13 Sep 2015 13:20:04 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468-done <at> debbugs.gnu.org
> 
> > why can't Magit simply change the buffer text to be what it wants to
> > display? why does it need to use an overlay string?
> 
> In this particular case, the overlays are delimiting a "magit region"
> (consisting of whole logical lines touched by Emacs' region), having
> them as actual text in the buffer would interfere with point movement
> even more.

Sorry, I don't understand: how can buffer contents interfere with
cursor motion?

> Or at least, that's what I saw when I tried just now to do something
> similar with text properties:
> 
> (with-current-buffer (get-buffer-create "*text property test*")
>   (erase-buffer)
>   (insert-file "next-line-fail.el")
>   (let ((pos 65))
>     (goto-char pos)
>     (insert (propertize (concat (propertize "\s" 'display '(space :height (1)))
>                         (propertize "\n" 'line-height t))
>                         'face 'highlight))
>     (pop-to-buffer (current-buffer))))

Please elaborate: what problems do you wee in this example?

Message #19 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Sun, 13 Sep 2015 16:22:10 -0400

On Sun, Sep 13, 2015 at 3:50 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>
>> (with-current-buffer (get-buffer-create "*text property test*")
>>   (erase-buffer)
>>   (insert-file "next-line-fail.el")
>>   (let ((pos 65))
>>     (goto-char pos)
>>     (insert (propertize (concat (propertize "\s" 'display '(space :height (1)))
>>                         (propertize "\n" 'line-height t))
>>                         'face 'highlight))
>>     (pop-to-buffer (current-buffer))))
>
> Please elaborate: what problems do you see in this example?

With point on line 1 (preceding the thin line), doing next-line ends
up inside the thin line, rather than after it.

Message #22 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Mon, 14 Sep 2015 09:17:47 +0300

> Date: Sun, 13 Sep 2015 16:22:10 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Sun, Sep 13, 2015 at 3:50 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>
> >> (with-current-buffer (get-buffer-create "*text property test*")
> >>   (erase-buffer)
> >>   (insert-file "next-line-fail.el")
> >>   (let ((pos 65))
> >>     (goto-char pos)
> >>     (insert (propertize (concat (propertize "\s" 'display '(space :height (1)))
> >>                         (propertize "\n" 'line-height t))
> >>                         'face 'highlight))
> >>     (pop-to-buffer (current-buffer))))
> >
> > Please elaborate: what problems do you see in this example?
> 
> With point on line 1 (preceding the thin line), doing next-line ends
> up inside the thin line, rather than after it.

Of course: that's a line of text.  Why is that a problem?  (I have no
idea what role is that thin line serving in Magit.)

Do you see any other issues with cursor motion?  I don't.

Message #25 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Mon, 14 Sep 2015 08:46:57 -0400

On Mon, Sep 14, 2015 at 2:17 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> Of course: that's a line of text.  Why is that a problem?  (I have no
> idea what role is that thin line serving in Magit.)

In this particular case, the overlays are delimiting a "magit region"
(consisting of whole logical lines touched by Emacs' region). This
means the thin line has to move with point, staying just ahead of it.

Here is a demo, eval this in a buffer, then do set-mark-command and
move around a bit:

(require 'cl-lib)

(defvar-local magit-region-overlays nil)

(defun 21468-update-hunk-region (start end window rol)
  (mapc #'delete-overlay magit-region-overlays)
  (let ((rbeg (save-excursion (goto-char (region-beginning))
                              (line-beginning-position)))
        (rend (save-excursion (goto-char (region-end))
                              (line-end-position))))
    (cl-flet ((ov (start end &rest args)
                  (let ((ov (make-overlay start end nil t)))
                    (overlay-put ov 'evaporate t)
                    (while args (overlay-put ov (pop args) (pop args)))
                    (push ov magit-region-overlays)
                    ov)))
      (ov rbeg (1+ rbeg) 'before-string
          (propertize (concat (propertize "\s" 'display '(space :height (1)))
                              (propertize "\n" 'line-height t))
                      'face 'highlight))
      (ov rend (1+ rend) 'after-string
          (propertize (concat (propertize "\s" 'display '(space :height (1)))
                              (propertize "\n" 'line-height t))
                      'face 'highlight)))))

(setq-local redisplay-highlight-region-function '21468-update-hunk-region)

(setq-local redisplay-unhighlight-region-function
            (lambda (rol) (mapc #'delete-overlay magit-region-overlays)))

Message #28 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Mon, 14 Sep 2015 09:22:44 -0400

> In this particular case, the overlays are delimiting a "magit region"
> (consisting of whole logical lines touched by Emacs' region). This
> means the thin line has to move with point, staying just ahead of it.

While I think I can see why you'd do it that way in earlier Emacsen,
I think the better way to do it nowadays is to simply arrange for the
region highlighting to extend to to whole lines.

You can do that with redisplay-(un)highlight-region-function, which was
introduced to allow rectangular-region-highlighting.


        Stefan

Message #31 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Mon, 14 Sep 2015 16:33:30 +0300

> Date: Mon, 14 Sep 2015 08:46:57 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Mon, Sep 14, 2015 at 2:17 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > Of course: that's a line of text.  Why is that a problem?  (I have no
> > idea what role is that thin line serving in Magit.)
> 
> In this particular case, the overlays are delimiting a "magit region"
> (consisting of whole logical lines touched by Emacs' region). This
> means the thin line has to move with point, staying just ahead of it.

What's wrong with marking the region with some special background
color?

Anyway, if you must use overlays here, by all means do, just try to
avoid newlines in its strings, if you want reliable cursor movement
across that overlay.

Please understand: what you originally perceived as a "bug" was
actually normal Emacs routine of placing point immediately after the
end of the overlay string.  It ended up in column zero because the
overlay string ended in a newline.  If there were no newline, the
problem would not have happened.  We now have special code in
vertical-motion that caters to this specific scenario.  That's why
using such overlay strings should be discouraged: they tend to force
us add similar special-case code all over the place.

> Here is a demo, eval this in a buffer, then do set-mark-command and
> move around a bit:

What am I supposed to see?  I see nothing that I don't see without
this.

Message #34 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Mon, 14 Sep 2015 09:55:57 -0400

On Mon, Sep 14, 2015 at 9:33 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> What's wrong with marking the region with some special background
> color?

I saw the fix you applied, and I certainly understand your distaste at
adding these special cases, but I don't want to start debating the
aesthetics now.

>
> What am I supposed to see?  I see nothing that I don't see without
> this.

In that buffer, after evaluating its code, when region is active you
should see 2 highlighted thin lines delimiting the region, instead of
the normal region highlight.

Message #37 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Mon, 14 Sep 2015 18:14:27 +0300

> Date: Mon, 14 Sep 2015 09:55:57 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Mon, Sep 14, 2015 at 9:33 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > What's wrong with marking the region with some special background
> > color?
> 
> I saw the fix you applied, and I certainly understand your distaste at
> adding these special cases, but I don't want to start debating the
> aesthetics now.

It's not the aesthetics I'm worried about, it's maintainability.  I
had the dubious pleasure of writing most of those special-case parts,
and I'm already beginning to have difficulty remembering what use case
does each one cater to, and how to make sure I don't break it by some
change.  How can we expect others to make changes in such a mess?

> > What am I supposed to see?  I see nothing that I don't see without
> > this.
> 
> In that buffer, after evaluating its code, when region is active you
> should see 2 highlighted thin lines delimiting the region, instead of
> the normal region highlight.

I think Stefan suggested a way to achieve this without all this
complexity.  I hope it will be possible.

Message #40 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Mon, 14 Sep 2015 16:50:10 -0400

On Mon, Sep 14, 2015 at 9:22 AM, Stefan Monnier
<monnier <at> iro.umontreal.ca> wrote:
> I think the better way to do it nowadays is to simply arrange for the
> region highlighting to extend to to whole lines.

On Mon, Sep 14, 2015 at 11:14 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Mon, 14 Sep 2015 09:55:57 -0400
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> In that buffer, after evaluating its code, when region is active you
>> should see 2 highlighted thin lines delimiting the region, instead of
>> the normal region highlight.
>
> I think Stefan suggested a way to achieve this without all this
> complexity.  I hope it will be possible.

If you can suggest a way to get horizontal bars that are ignored by
point movement without using newlines in overlays, then great. I
almost got there using :overline and :underline, but as far as I can
tell, there's no way to extend the line to the edge of the screen
without using an overlay with a newline.

If you just want to say that the region should be shown by
highlighting instead of horizontal bars, then *shrug*.

Message #43 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Tue, 15 Sep 2015 09:53:54 +0300

> Date: Mon, 14 Sep 2015 16:50:10 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> If you can suggest a way to get horizontal bars that are ignored by
> point movement without using newlines in overlays, then great. I
> almost got there using :overline and :underline, but as far as I can
> tell, there's no way to extend the line to the edge of the screen
> without using an overlay with a newline.

I think you should be able to extend the face by using a display
property with :align-to value.  The value should be computed to go all
the way to the edge of the window.

> If you just want to say that the region should be shown by
> highlighting instead of horizontal bars, then *shrug*.

I cannot do anything about these issues besides explaining how they
harm Emacs maintenance, and asking people to look for alternative
solutions.  And yes, highlighting sounds like a good alternative to
me.  Feel free to ignore the requests, if you don't consider them
important enough.

Message #46 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Thu, 1 Oct 2015 16:43:11 -0400

[Message part 1 (text/plain, inline)]

On Tue, Sep 15, 2015 at 2:53 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> I think you should be able to extend the face by using a display
> property with :align-to value.  The value should be computed to go all
> the way to the edge of the window.

We tried something like this:

(let ((align `(space :align-to (+ (,(window-body-width nil t))
                                  ,(window-hscroll)))))
  (ov rbeg eol 'face (list :overline color)
      'after-string (propertize "\s" 'face face 'display align)))

But when moving point to end of line this causes the cursor to appear
at the edge of the window instead of at the "real" end of line. I
guess it's because the aligned space pushes the newline character to
the edge of the window.

For a self-contained example do "emacs -Q overlay.el -f eval-buffer -f
set-mark-command -f move-end-of-line" with attached overlay.el.


> I cannot do anything about these issues besides explaining how they
> harm Emacs maintenance, and asking people to look for alternative
> solutions.

Well you *could* refuse to fix bugs like this, i.e. declare that
creating overlays with newlines invokes undefined behaviour. As it
stands, the best way to get the effect we want is to use an overlayed
newline, especially now that you've fixed this bug.

[overlay.el (application/octet-stream, attachment)]

Message #49 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Fri, 02 Oct 2015 13:01:06 +0300

> Date: Thu, 1 Oct 2015 16:43:11 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> > I think you should be able to extend the face by using a display
> > property with :align-to value.  The value should be computed to go all
> > the way to the edge of the window.
> 
> We tried something like this:
> 
> (let ((align `(space :align-to (+ (,(window-body-width nil t))
>                                   ,(window-hscroll)))))
>   (ov rbeg eol 'face (list :overline color)
>       'after-string (propertize "\s" 'face face 'display align)))
> 
> But when moving point to end of line this causes the cursor to appear
> at the edge of the window instead of at the "real" end of line. I
> guess it's because the aligned space pushes the newline character to
> the edge of the window.

Yes.  But why is that a problem?

If you dislike that, you could make the stretch one column shorter,
like this:

  (let* ((align (list 'space :align-to `(+ (,(- (window-body-width nil t)
						(default-font-width)))
					   ,(window-hscroll))))

(Btw, I think adding window-hscroll is incorrect here, as its value is
in columns, while window-body-width returns the width in pixels.)

> > I cannot do anything about these issues besides explaining how they
> > harm Emacs maintenance, and asking people to look for alternative
> > solutions.
> 
> Well you *could* refuse to fix bugs like this, i.e. declare that
> creating overlays with newlines invokes undefined behaviour.

I could, but I consider that a "doomsday weapon", hopefully never to
be used in Emacs development, certainly not by me.  I don't think it
was ever done.

Message #52 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Fri, 2 Oct 2015 15:58:08 -0400

On Fri, Oct 2, 2015 at 6:01 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> But when moving point to end of line this causes the cursor to appear
>> at the edge of the window instead of at the "real" end of line. I
>> guess it's because the aligned space pushes the newline character to
>> the edge of the window.
>
> Yes.  But why is that a problem?

Because we want the lines to be just visual effects that don't affect
cursor movement, i.e. the cursor should act the same whether or not
the lines are visible.

>
> If you dislike that, you could make the stretch one column shorter,

Which of course just means that the cursor appears at one column
before the edge of the window instead of at the "real" end of line.

>   (let* ((align (list 'space :align-to `(+ (,(- (window-body-width nil t)
>                                                 (default-font-width)))
>                                            ,(window-hscroll))))
>
> (Btw, I think adding window-hscroll is incorrect here, as its value is
> in columns, while window-body-width returns the width in pixels.)

window-body-width is inside a list indicating pixel value, and
window-hscroll is a plain integer indicating a multiple of the font
width, so it should work, no? That was my reading of "37.16.3 Pixel
Specification for Spaces" in the elisp manual.

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> For a self-contained example do "emacs -Q overlay.el -f eval-buffer -f
> set-mark-command -f move-end-of-line" with attached overlay.el.

By the way, I noticed this is wrong (for some reason it works in 24.5
even though functions are being called with the wrong number of
arguments), it should be called like

    emacs -Q overlay.el --eval "(progn (switch-to-buffer
\"overlay.el\") (eval-buffer) (set-mark-command nil) (move-end-of-line
nil))"

Message #55 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sat, 03 Oct 2015 00:00:30 +0300

> Date: Fri, 2 Oct 2015 15:58:08 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Fri, Oct 2, 2015 at 6:01 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> But when moving point to end of line this causes the cursor to appear
> >> at the edge of the window instead of at the "real" end of line. I
> >> guess it's because the aligned space pushes the newline character to
> >> the edge of the window.
> >
> > Yes.  But why is that a problem?
> 
> Because we want the lines to be just visual effects that don't affect
> cursor movement, i.e. the cursor should act the same whether or not
> the lines are visible.
> 
> >
> > If you dislike that, you could make the stretch one column shorter,
> 
> Which of course just means that the cursor appears at one column
> before the edge of the window instead of at the "real" end of line.

If you want to put the cursor there, you need an overlay with a
'cursor' property.

Or we could have a new feature whereby the stretch of white space
would allow positioning the cursor on it.  Right now, the flag that
disallows positioning the cursor is hard-coded in the display engine.

Message #58 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Fri, 9 Oct 2015 19:34:43 -0400

On 10/2/15, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> If you want to put the cursor there, you need an overlay with a
> 'cursor' property.

This works nicely. But there are some other glitches.

- On a truncated line with the overlay, move-end-of-line skips to the
next line. (I tried to avoid this by only adding the stretched space
when it's going to be visible. But that just made things worse
(looping in the display engine))

- When at the end of line, next-line goes to the end-of-line instead
of staying on the same column.

> Or we could have a new feature whereby the stretch of white space
> would allow positioning the cursor on it.  Right now, the flag that
> disallows positioning the cursor is hard-coded in the display engine.

Could be interesting, but I don't know of any need for it at present.

Message #61 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sat, 10 Oct 2015 10:14:39 +0300

> Date: Fri, 9 Oct 2015 19:34:43 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On 10/2/15, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > If you want to put the cursor there, you need an overlay with a
> > 'cursor' property.
> 
> This works nicely. But there are some other glitches.
> 
> - On a truncated line with the overlay, move-end-of-line skips to the
> next line. (I tried to avoid this by only adding the stretched space
> when it's going to be visible. But that just made things worse
> (looping in the display engine))
> 
> - When at the end of line, next-line goes to the end-of-line instead
> of staying on the same column.

Are these caused by the 'cursor' property, i.e. do not happen if that
property is not used?

In any case, I'd appreciate a short self-contained test case for each
of these two issues, as I'm not sure I understand what's going on.

Message #64 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Sat, 10 Oct 2015 09:37:43 -0400

[Message part 1 (text/plain, inline)]

On 10/10/15, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Fri, 9 Oct 2015 19:34:43 -0400
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> - On a truncated line with the overlay, move-end-of-line skips to the
>> next line. (I tried to avoid this by only adding the stretched space
>> when it's going to be visible. But that just made things worse
>> (looping in the display engine))
>>
>> - When at the end of line, next-line goes to the end-of-line instead
>> of staying on the same column.
>
> Are these caused by the 'cursor' property, i.e. do not happen if that
> property is not used?

They're not caused by the 'cursor' property, although it makes the 2nd
case more visually surprising.

>
> In any case, I'd appreciate a short self-contained test case for each
> of these two issues, as I'm not sure I understand what's going on.
>

Using attached 21468-overlay.el

The 1st case:

   emacs -Q -l 21468-overlay.el -f 21468-test-eol-skip

then hit C-e. Point lands at the end of line 19 instead of the end of
line 18 where it started.

The 2nd case

   emacs -Q -l 21468-overlay.el -f 21468-test-eol-col-movement

then hit C-e C-n. Point lands at the end of line 8 instead of line 8,
column 33 which is what happens when there is no overlay.

[21468-overlay.el (application/octet-stream, attachment)]

Message #67 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sat, 10 Oct 2015 17:20:39 +0300

> Date: Sat, 10 Oct 2015 09:37:43 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> The 2nd case
> 
>    emacs -Q -l 21468-overlay.el -f 21468-test-eol-col-movement
> 
> then hit C-e C-n. Point lands at the end of line 8 instead of line 8,
> column 33 which is what happens when there is no overlay.

This is not a bug: vertical-motion doesn't know about the 'cursor'
property, so it works as if the property didn't exist.

The 'cursor' property is only considered when actually putting the
cursor on screen.

I will look into the first case, sigh.

Message #70 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Sat, 10 Oct 2015 12:51:06 -0400

On Sat, Oct 10, 2015 at 10:20 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Sat, 10 Oct 2015 09:37:43 -0400
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> Cc: 21468 <at> debbugs.gnu.org
>>
>> The 2nd case
>>
>>    emacs -Q -l 21468-overlay.el -f 21468-test-eol-col-movement
>>
>> then hit C-e C-n. Point lands at the end of line 8 instead of line 8,
>> column 33 which is what happens when there is no overlay.
>
> This is not a bug: vertical-motion doesn't know about the 'cursor'
> property, so it works as if the property didn't exist.

Okay, that makes sense, but it means this approach is worse than
overlayed newlines for magit.

Message #73 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sat, 10 Oct 2015 20:03:03 +0300

> Date: Sat, 10 Oct 2015 09:37:43 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> >> - On a truncated line with the overlay, move-end-of-line skips to the
> >> next line. (I tried to avoid this by only adding the stretched space
> >> when it's going to be visible. But that just made things worse
> >> (looping in the display engine))
> >>
> >> - When at the end of line, next-line goes to the end-of-line instead
> >> of staying on the same column.
> >
> > Are these caused by the 'cursor' property, i.e. do not happen if that
> > property is not used?
> 
> They're not caused by the 'cursor' property, although it makes the 2nd
> case more visually surprising.
> 
> >
> > In any case, I'd appreciate a short self-contained test case for each
> > of these two issues, as I'm not sure I understand what's going on.
> >
> 
> Using attached 21468-overlay.el
> 
> The 1st case:
> 
>    emacs -Q -l 21468-overlay.el -f 21468-test-eol-skip
> 
> then hit C-e. Point lands at the end of line 19 instead of the end of
> line 18 where it started.

Fixed.

Message #76 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sat, 10 Oct 2015 20:09:33 +0300

> Date: Sat, 10 Oct 2015 12:51:06 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Sat, Oct 10, 2015 at 10:20 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> Date: Sat, 10 Oct 2015 09:37:43 -0400
> >> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> >> Cc: 21468 <at> debbugs.gnu.org
> >>
> >> The 2nd case
> >>
> >>    emacs -Q -l 21468-overlay.el -f 21468-test-eol-col-movement
> >>
> >> then hit C-e C-n. Point lands at the end of line 8 instead of line 8,
> >> column 33 which is what happens when there is no overlay.
> >
> > This is not a bug: vertical-motion doesn't know about the 'cursor'
> > property, so it works as if the property didn't exist.
> 
> Okay, that makes sense, but it means this approach is worse than
> overlayed newlines for magit.

I don't think so.  I think a casual mismatch in a column while moving
through a buffer, and in marginal cases at that, is not a big deal.

Message #79 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5; When next-line (visual) crosses overlay with
 before-string="\n" point goes to column 0
Date: Sat, 10 Oct 2015 17:06:59 -0400

On Sat, Oct 10, 2015 at 1:09 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Sat, 10 Oct 2015 12:51:06 -0400
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>>
>> Okay, that makes sense, but it means this approach is worse than
>> overlayed newlines for magit.
>
> I don't think so.  I think a casual mismatch in a column while moving
> through a buffer, and in marginal cases at that, is not a big deal.

It's not a *big* deal, but with Emacs 25 using overlayed newlines
gives no problems at all.

Message #82 received at 21468 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 21468 <at> debbugs.gnu.org
Subject: Re: bug#21468: 24.5;
 When next-line (visual) crosses overlay with before-string="\n" point
 goes to column 0
Date: Sun, 11 Oct 2015 05:38:17 +0300

> Date: Sat, 10 Oct 2015 17:06:59 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 21468 <at> debbugs.gnu.org
> 
> On Sat, Oct 10, 2015 at 1:09 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> Date: Sat, 10 Oct 2015 12:51:06 -0400
> >> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> >>
> >> Okay, that makes sense, but it means this approach is worse than
> >> overlayed newlines for magit.
> >
> > I don't think so.  I think a casual mismatch in a column while moving
> > through a buffer, and in marginal cases at that, is not a big deal.
> 
> It's not a *big* deal, but with Emacs 25 using overlayed newlines
> gives no problems at all.

I'm quite sure problems will surface sooner or later.  Overlays with
newlines hit on fragile parts of the display engine code, which was
never really designed for that kind of stuff.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.