Subject: bug#73563: [Ben Simms] Performance bottleneck in ns_draw_fringe_bitmap

From: Jordan Ellis Coppard <[email-masked]>
Date: Tue, 13 May 2025 20:37:55 +0900

Howdy,

I've been using the following patch (bottom of this email) until my most 
recent rebuild of Emacs from master. Since the recent commit fixing 
stipple drawing the patch can be reduced to just: 
https://github.com/emacs-mirror/emacs/commit/7f2efe6503fdce2a4e552c14802644a05b581bc7 
(link to a diff authored by Ben Simms).

I've now noticed again extreme slowdown when fringe bitmaps are used. It 
appears to be proportional to the number of bits set in the bitmap. An 
'empty' bitmap (all zeroes) doesn't yield any noticeable slowdown, a 
bitmap with 1 bit set yields a little, with 2 its noticeable and with 
all 8 bits set Emacs takes up to 1 second to respond to any input at all.

This has been the case since at least August of last year. I can get a 
minimal reproduction but it has happened on completely different 
configurations with the only common denominator being the NS build (on 
ARM-based macOS), and the use of fringe bitmaps.

Below is trace with update_frame as the root context. Captured with 
Instruments.app (which uses dtrace and more under the hood IIRC).

ns_draw_fringe_bitmap takes up 94% of the time for the short 
reproduction this trace records. The deepest the trace goes is to 
CG::Path::recalculate_bounding_box() which eats 80% of time. So instead 
of a cached value the bounding box is being recalculated every time 
which appears very expensive, but beyond that I have close to zero 
experience with macOS' graphics stack.

I've just recompiled Emacs against the same commit of my recent master 
build (648453c04d9b91d96452b930c0c948b0b39b5dc0) except now with the 
patch applied (since the stipple changes are merged it's just the 
smallest subset: 
https://github.com/emacs-mirror/emacs/commit/7f2efe6503fdce2a4e552c14802644a05b581bc7) 
and once again fringe performance is back to being buttery smooth.

(trace also here in-case formatting gets borked:
trace:

3.07 s  100.0%	0 s	               update_frame
3.06 s  100.0%	0 s	                update_window_tree
3.06 s  100.0%	1.00 ms	                 update_window
2.89 s  94.2%	0 s	                  gui_update_window_end
2.88 s  94.1%	0 s	                   draw_window_fringes
2.88 s  93.9%	0 s	                    draw_fringe_bitmap
2.88 s  93.9%	0 s	                     draw_fringe_bitmap_1
2.88 s  93.9%	0 s	                      ns_draw_fringe_bitmap
2.72 s  88.7%	0 s	                       -[NSBezierPath copyWithZone:]
2.72 s  88.7%	0 s	                        -[NSBezierPath _appendToPath:]
2.72 s  88.7%	0 s	                         -[NSBezierPath 
_enumeratePathElementsUsingBlock:]
2.72 s  88.7%	0 s	                          CGPathApplyWithBlock2
2.72 s  88.7%	5.00 ms	                           CG::Path::apply(void 
(CGPathElementType, CGPoint const*, bool*) block_pointer) const
2.71 s  88.5%	30.00 ms	 
__CGPathApplyWithBlock2_block_invoke
2.68 s  87.4%	11.00 ms	                             __49-[NSBezierPath 
_enumeratePathElementsUsingBlock:]_block_invoke
2.48 s  80.8%	1.00 ms	 
-[NSBezierPath(NSBezierPathDevicePrimitives) _deviceMoveToPoint:]
2.47 s  80.6%	2.00 ms	                               CGPathMoveToPoint
2.46 s  80.3%	2.46 s	 
CG::Path::recalculate_bounding_box()
5.00 ms   0.2%	4.00 ms	 
CG::Path::move_to_point(CGPoint const&, CGAffineTransform const*)
1.00 ms   0.0%	1.00 ms	 
CG::Path::convert_to_huge()
2.00 ms   0.1%	2.00 ms	                                (anonymous 
namespace)::transform_is_valid(CGAffineTransform const*)
1.00 ms   0.0%	1.00 ms	 
CG::Path::convert_to_huge()
1.00 ms   0.0%	1.00 ms	 
CGFloatValidateWithLog
1.00 ms   0.0%	1.00 ms	 
DYLD-STUB$$CGPathMoveToPoint
1.00 ms   0.0%	1.00 ms	                               -[NSBezierPath 
_cgPath]
1.00 ms   0.0%	1.00 ms	                               objc_msgSend
1.00 ms   0.0%	1.00 ms	                               (anonymous 
namespace)::transform_is_valid(CGAffineTransform const*)
67.00 ms   2.2%	7.00 ms	 
-[NSBezierPath(NSBezierPathDevicePrimitives) _deviceLineToPoint:]
40.00 ms   1.3%	5.00 ms	 
-[NSBezierPath(NSBezierPathDevicePrimitives) _deviceClosePath]
39.00 ms   1.3%	2.00 ms	                              -[NSBezierPath 
lineToPoint:]
14.00 ms   0.5%	14.00 ms	                              objc_msgSend
6.00 ms   0.2%	6.00 ms	                              -[NSBezierPath _cgPath]
5.00 ms   0.2%	5.00 ms	                              CGPathAddLineToPoint
4.00 ms   0.1%	4.00 ms	                              __30-[NSBezierPath 
_appendToPath:]_block_invoke
4.00 ms   0.1%	4.00 ms	 
objc_msgSend$lineToPoint:
2.00 ms   0.1%	2.00 ms	                              CGPathIsEmpty
2.00 ms   0.1%	2.00 ms	 
objc_msgSend$moveToPoint:
2.00 ms   0.1%	2.00 ms	 
CG::Path::close_subpath()
1.00 ms   0.0%	1.00 ms	 
objc_msgSend$_deviceLineToPoint:
1.00 ms   0.0%	1.00 ms	                              CGPathGetCurrentPoint
1.00 ms   0.0%	1.00 ms	                              -[NSBezierPath 
moveToPoint:]
1.00 ms   0.0%	1.00 ms	 
objc_msgSend$_deviceClosePath
1.00 ms   0.0%	1.00 ms	                              objc_msgSend$closePath
1.00 ms   0.0%	1.00 ms	 
objc_msgSend$_deviceMoveToPoint:
3.00 ms   0.1%	3.00 ms	                             -[NSBezierPath 
lineToPoint:]
1.00 ms   0.0%	1.00 ms	 
-[NSBezierPath(NSBezierPathDevicePrimitives) _deviceLineToPoint:]
2.00 ms   0.1%	2.00 ms	                            __49-[NSBezierPath 
_enumeratePathElementsUsingBlock:]_block_invoke
1.00 ms   0.0%	1.00 ms	                        objc_msgSend$setFlatness:
104.00 ms   3.4%	0 s	                       -[NSBezierPath fill]
42.00 ms   1.4%	0 s	                       -[NSBezierPath 
transformUsingAffineTransform:]
7.00 ms   0.2%	0 s	                       NSRectFill
2.00 ms   0.1%	0 s	                       NSColorSetWithFillAndStroke
1.00 ms   0.0%	1.00 ms	                       CGContextClipToRect
1.00 ms   0.0%	0 s	                       -[NSBezierPath dealloc]
1.00 ms   0.0%	1.00 ms	                       CGGStateSetCompositeOperation
1.00 ms   0.0%	1.00 ms	                      lookup_named_face
5.00 ms   0.2%	4.00 ms	                    set_buffer_internal_2
1.00 ms   0.0%	0 s	                   unblock_input
1.00 ms   0.0%	0 s	                   ns_draw_window_cursor
121.00 ms   3.9%	0 s	                  ns_scroll_run
51.00 ms   1.7%	1.00 ms	                  update_window_line
4.00 ms   0.1%	4.00 ms	                  row_equal_p
1.00 ms   0.0%	1.00 ms	                  xwidget_end_redisplay
1.00 ms   0.0%	0 s	                ns_update_end

Patch:

--- src/nsimage.m.orig
+++ src/nsimage.m
@@ -28,6 +28,7 @@ Updated by Christian Limpach ([email-masked])
  /* This should be the first include, as it may set up #defines affecting
     interpretation of even the system includes.  */
  #include <config.h>
+#include <CoreGraphics/CoreGraphics.h>

  #include "lisp.h"
  #include "dispextern.h"
@@ -510,10 +511,20 @@ - (void) setAlphaAtX: (int) x Y: (int) y to: 
(unsigned char) a
  }

  /* Returns a pattern color, which is cached here.  */
-- (NSColor *)stippleMask
+- (CGImageRef)stippleMask
  {
-  if (stippleMask == nil)
-      stippleMask = [[NSColor colorWithPatternImage: self] retain];
+  if (stippleMask == nil) {
+    CGDataProviderRef provider = CGDataProviderCreateWithData (NULL, 
[bmRep bitmapData],
+                                                             [self 
sizeInBytes], NULL);
+    id mask = (id)CGImageMaskCreate(
+                                          [self size].width,
+                                          [self size].height,
+                                          8, 8, [self size].width,
+                                          provider, NULL, 0);
+
+    CGDataProviderRelease(provider);
+    stippleMask = (CGImageRef)[mask retain];
+  }
    return stippleMask;
  }

--- src/nsterm.h.orig
+++ src/nsterm.h
@@ -670,7 +670,7 @@ enum ns_return_frame_mode
  {
    NSBitmapImageRep *bmRep; /* used for accessing pixel data */
    unsigned char *pixmapData[5]; /* shortcut to access pixel data */
-  NSColor *stippleMask;
+  CGImageRef stippleMask;
  @public
    NSAffineTransform *transform;
    BOOL smoothing;
@@ -687,8 +687,8 @@ enum ns_return_frame_mode
                 green: (unsigned char)g blue: (unsigned char)b
                alpha:(unsigned char)a;
  - (void)setAlphaAtX: (int)x Y: (int)y to: (unsigned char)a;
-- (NSColor *)stippleMask;
+- (CGImageRef)stippleMask;
  - (Lisp_Object)getMetadata;
  - (BOOL)setFrame: (unsigned int) index;
  - (void)setTransform: (double[3][3]) m;

--- src/nsterm.m.orig
+++ src/nsterm.m
@@ -2903,22 +2903,24 @@ Hide the window (X11 semantics)
  static void
  ns_define_fringe_bitmap (int which, unsigned short *bits, int h, int w)
  {
-  NSBezierPath *p = [NSBezierPath bezierPath];
-
    if (!fringe_bmp)
      fringe_bmp = [[NSMutableDictionary alloc] initWithCapacity:25];

-  [p moveToPoint:NSMakePoint (0, 0)];

-  for (int y = 0 ; y < h ; y++)
-    for (int x = 0 ; x < w ; x++)
-      {
-        bool bit = bits[y] & (1 << (w - x - 1));
-        if (bit)
-          [p appendBezierPathWithRect:NSMakeRect (x, y, 1, 1)];
-      }
+  for (int i = 0; i < h; i++)
+    bits[i] = ~bits[i];
+
+  CGDataProviderRef provider = CGDataProviderCreateWithData (NULL, bits,
+					   sizeof (unsigned short) * h, NULL);
+  if (provider) {
+    id p = (id)CGImageMaskCreate (w, h, 1, 1,
+                 sizeof (unsigned short),
+                 provider, NULL, 0);
+    CGDataProviderRelease (provider);
+
+    [fringe_bmp setObject:p forKey:[NSNumber numberWithInt:which]];
+  }

-  [fringe_bmp setObject:p forKey:[NSNumber numberWithInt:which]];
  }

@@ -2981,26 +2983,29 @@ Hide the window (X11 semantics)
        NSRectFill (clearRect);
      }

-  NSBezierPath *bmp = [fringe_bmp objectForKey:[NSNumber 
numberWithInt:p->which]];
+  CGImageRef bmp = (CGImageRef)[fringe_bmp objectForKey:[NSNumber 
numberWithInt:p->which]];

    if (bmp == nil
        && p->which < max_used_fringe_bitmap)
      {
        gui_define_fringe_bitmap (f, p->which);
-      bmp = [fringe_bmp objectForKey: [NSNumber numberWithInt: p->which]];
+      bmp = (CGImageRef)[fringe_bmp objectForKey: [NSNumber 
numberWithInt: p->which]];
      }

    if (bmp)
      {
-      NSAffineTransform *transform = [NSAffineTransform transform];
-      NSColor *bm_color;
+      CGRect bounds = CGRectMake (p->x, p->y - p->dh,
+			   CGImageGetWidth (bmp), CGImageGetHeight (bmp));
+
+      NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
+      [ctx saveGraphicsState];
+      CGContextRef context = [ctx CGContext];

-      /* Because the image is defined at (0, 0) we need to take a copy
-         and then transform that copy to the new origin.  */
-      bmp = [bmp copy];
-      [transform translateXBy:p->x yBy:p->y - p->dh];
-      [bmp transformUsingAffineTransform:transform];
+      CGContextTranslateCTM (context,
+			     CGRectGetMinX (bounds), CGRectGetMaxY (bounds));
+      CGContextScaleCTM (context, 1, -1);

+      NSColor *bm_color;
        if (!p->cursor_p)
          bm_color = [NSColor colorWithUnsignedLong:face->foreground];
        else if (p->overlay_p)
@@ -3009,9 +3014,10 @@ Hide the window (X11 semantics)
          bm_color = f->output_data.ns->cursor_color;

        [bm_color set];
-      [bmp fill];
+      bounds.origin = CGPointZero;
+      CGContextDrawImage (context, bounds, bmp);

-      [bmp release];
+      [[NSGraphicsContext currentContext] restoreGraphicsState];
      }
    ns_unfocus (f);
  }
@@ -3273,11 +3279,10 @@ Note that CURSOR_WIDTH is meaningful only for 
(h)bar cursors.
  ns_draw_underwave (struct glyph_string *s, EmacsCGFloat width, 
EmacsCGFloat x)
  {
    int wave_height = 3, wave_length = 2;
-  int y, dx, dy, odd, xmax;
-  NSPoint a, b;
+  int y, dx, dy, xmax;
    NSRect waveClip;

-  dx = wave_length;
+  dx = wave_length * 2;
    dy = wave_height - 1;
    y =  s->ybase - wave_height + 3;
    xmax = x + width;
@@ -3287,25 +3292,24 @@ Note that CURSOR_WIDTH is meaningful only for 
(h)bar cursors.
    [[NSGraphicsContext currentContext] saveGraphicsState];
    NSRectClip (waveClip);

-  /* Draw the waves */
-  a.x = x - ((int)(x) % dx) + (EmacsCGFloat) 0.5;
-  b.x = a.x + dx;
-  odd = (int)(a.x/dx) % 2;
-  a.y = b.y = y + 0.5;
+  float ax = x - ((int)(x) % dx);
+  float ay = y + wave_height / 2.0;

-  if (odd)
-    a.y += dy;
-  else
-    b.y += dy;
+  NSBezierPath *path = [[NSBezierPath alloc] init];
+  [path moveToPoint: (NSPoint){ ax, ay }];
+
+  NSPoint stepOne = { dx, 0 };
+  NSPoint controlOne = { 0.5 * dx, dy };
+  NSPoint controlTwo = { 0.5 * dx, -dy };

-  while (a.x <= xmax)
+  while (ax <= xmax)
      {
-      [NSBezierPath strokeLineFromPoint:a toPoint:b];
-      a.x = b.x, a.y = b.y;
-      b.x += dx, b.y = y + 0.5 + odd*dy;
-      odd = !odd;
+      [path relativeCurveToPoint:stepOne controlPoint1:controlOne 
controlPoint2:controlTwo];
+      ax += dx;
      }

+  [path stroke];
+
    /* Restore previous clipping rectangle(s) */
    [[NSGraphicsContext currentContext] restoreGraphicsState];
  }
@@ -3825,10 +3829,35 @@ Function modeled after x_draw_glyph_string_box ().
        int box_line_width = max (s->face->box_horizontal_line_width, 0);

        if (s->stippled_p)
-	{
-	  struct ns_display_info *dpyinfo = FRAME_DISPLAY_INFO (s->f);
-	  [[dpyinfo->bitmaps[face->stipple-1].img stippleMask] set];
-	  goto fill;
+        {
+          [[NSColor colorWithUnsignedLong:face->background] set];
+          r = NSMakeRect (s->x, s->y + box_line_width,
+                          s->background_width,
+                          s->height - 2 * box_line_width);
+          NSRectFill (r);
+          s->background_filled_p = 1;
+
+          struct ns_display_info *dpyinfo = FRAME_DISPLAY_INFO (s->f);
+          CGImageRef mask =
+            [dpyinfo->bitmaps[face->stipple - 1].img stippleMask];
+
+          CGRect bounds = CGRectMake (s->x, s->y + box_line_width,
+                                      s->background_width,
+                                      s->height - 2 * box_line_width);
+          NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
+          [ctx saveGraphicsState];
+          CGContextRef context = [ctx CGContext];
+
+          CGContextClipToRect (context, bounds);
+
+          CGContextScaleCTM (context, 1, -1);
+          [[NSColor colorWithUnsignedLong:face->foreground] set];
+
+          CGRect imageSize = CGRectMake (0, 0, CGImageGetWidth (mask),
+                                         CGImageGetHeight (mask));
+
+          CGContextDrawTiledImage (context, imageSize, mask);
+          [[NSGraphicsContext currentContext] restoreGraphicsState];
  	}
        else if (FONT_HEIGHT (s->font) < s->height - 2 * box_line_width
  	       /* When xdisp.c ignores FONT_HEIGHT, we cannot trust font
@@ -3851,7 +3880,6 @@ Function modeled after x_draw_glyph_string_box ().
  	  else
  	    [FRAME_CURSOR_COLOR (s->f) set];

-	fill:
  	  r = NSMakeRect (s->x, s->y + box_line_width,
  			  s->background_width,
  			  s->height - 2 * box_line_width);
@@ -4175,12 +4203,42 @@ Function modeled after x_draw_glyph_string_box ().
  	  dpyinfo = FRAME_DISPLAY_INFO (s->f);
  	  if (s->hl == DRAW_CURSOR)
  	    [FRAME_CURSOR_COLOR (s->f) set];
-	  else if (s->stippled_p)
-	    [[dpyinfo->bitmaps[s->face->stipple - 1].img stippleMask] set];
-	  else
+	  else if (s->stippled_p) {
+              [[NSColor colorWithUnsignedLong:s->face->background]
+                set];
+              NSRectFill (
+                NSMakeRect (x, s->y, background_width, s->height));
+
+              CGImageRef mask =
+                [dpyinfo->bitmaps[s->face->stipple - 1]
+                    .img stippleMask];
+
+              CGRect bounds
+                = CGRectMake (s->x, s->y, s->background_width,
+                              s->height);
+
+              NSGraphicsContext *ctx =
+                [NSGraphicsContext currentContext];
+              [ctx saveGraphicsState];
+              CGContextRef context = [ctx CGContext];
+              CGContextClipToRect(context, bounds);
+              CGContextScaleCTM (context, 1, -1);
+              [[NSColor colorWithUnsignedLong:s->face->foreground]
+                set];
+
+              CGRect imageSize
+                = CGRectMake (0, 0, CGImageGetWidth (mask),
+                              CGImageGetHeight (mask));
+
+              CGContextDrawTiledImage (context, imageSize, mask);
+
+              [[NSGraphicsContext currentContext]
+		restoreGraphicsState];
+	    }
+	  else {
  	    [[NSColor colorWithUnsignedLong: s->face->background] set];
-
-	  NSRectFill (NSMakeRect (x, s->y, background_width, s->height));
+            NSRectFill (NSMakeRect (x, s->y, background_width, s->height));
+          }
  	}
      }