Subject: Re: bug#67604: Motion problems with inline images

From: JD Smith <[email-masked]>
Date: Tue, 6 May 2025 23:30:02 -0400

> On Dec 11, 2023, at 2:44 PM, Eli Zaretskii <[email-masked]> wrote:
> 
>> From: JD Smith <[email-masked]>
>> Date: Mon, 11 Dec 2023 14:06:38 -0500
>> Cc: [email-masked]
>> 
>> 
>> 
>>> On Dec 5, 2023, at 10:31 PM, Eli Zaretskii <[email-masked]> wrote:
>>>> OK, so this was likely a false positive then.  I’m out of ideas.  Maybe Windows is magically immune.
>>> 
>>> Unlikely.  More likely is that we need some specific metrics of the
>>> displayed stuff to see the problem (which is therefore very rare).
>> 
>> I agree with this sentiment: it does seem to require special tuning to reveal itself.  I wanted to check to see if you had received my news that I reproduced the line-skip motion bug on a GNU/Linux system.  That would seem to confirm it’s a general issue with the display/motion code dealing with inline images.  Let me know if I can produce anything else to help with this one.  Thanks.
> 
> I've eventually succeeded in reproducing it.  I will get to it when I
> have time; however, with the current tempest on emacs-devel and other
> urgent issues, I don't know when will that be.

This bug remains present in Emacs v30.  I have managed some progress on it and feel I am quite close to narrowing to the culprit.

I've managed to get lldb working with a debug build of Emacs.  I can rapidly reproduce the bug, and am able to set breakpoints, step through the code, and print variable values, etc.

I've debugged through Fvertical_motion into the various move_it functions in xdisp.c.  They are very complicated functions, but here is what I've determined, referencing xdisp.c:

1. In `move_it_to`, when moving one vertical position down (op=MOVE_TO_VPOS), this is attempted:

	  /* If no TO_CHARPOS and no TO_X specified, stop at the
	     start of the line TO_VPOS.  */
            ...
		skip = move_it_in_display_line_to (it, -1, -1, 0);

I gather `move_it_in_display_line_to' here is supposed to be moving to the end of the display line, with op=0.

2. In that function, there is a bit of logic that looks like:

	  int single_glyph_width = it->pixel_width / it->nglyphs;
	  int new_x;
	  for (i = 0; i < it->nglyphs; ++i, x = new_x)
	    {
	      new_x = x + single_glyph_width;

In this example, we are moving glyph-by-glyph, accumulating each glpyh's pixel widths into to the new_x position.  This is continued, until a glyph is found which no longer fits on the visible line:

	      if (/* Lines are continued.  */
		  it->line_wrap != TRUNCATE
		  && (/* And glyph doesn't fit on the line.  */
		      new_x > it->last_visible_x

Here is the problem: if the first glyph on the line is wide (e.g. the green SVG rectangle in the example we were playing with), its width is incorrectly encoded in it->pixel_width as 7 pixels (the width of my normal font).  

As a result, for this very first glyph, `move_it_in_display_line_to' increments the x pixel position by too small a value (7 instead of 108 pixels).  This causes IT to eventually move past the end of the line, and on to the next wrapped line.  And if that next wrapped line is short enough, it even moves past that on to the following line, thus causing the full line "skip" we observed.

It's important to note that even when the line skip bug isn't present, if the first glyph is wide, vertical motion does unexpected things because of this mis-reporting of the first glyph's width.  If a wide glyph wraps to column 0, this motion bug is present.  It does not require an image, only a wide glyph. For example, I removed the green rectangle and inserted in its place:

  (insert (propertize "x" 'display '(space :width (108)) 'face '(:background "gray")))

and precisely the same issues occur.

I further verified this by placing a single character on the visible line just before the green rectangle.  With this change, on the 2nd time it->pixel_width is consulted above, it has the correct value of 108.  Only the *first* wrapped column position if affected.  Nor does this happen at the "real" start of a line, prior to its first wrap.  Only *wrapped* wide glyphs at visual column 0 have misreported pixel widths.

What I'm looking for is a reason why a wide glyph (only) at the very start of a wrapped visual display line would have its pixel width incorrectly stored, as if it were just a regular width character.  

I tried to get a sense of where this IT pixel_width is getting set, but it's referenced all over xdisp.c, and I have only a rudimentary understanding of the iterator structure (and its glyph rows, etc.).

If you have any hunches where this mistaken it->pixel_width might be sneaking in for wrapped wide glyphs, I'd be very happy to investigate further using my debug setup here.

Thanks,

JD