Subject: bug#48921: 28.0.50; Speed up native-compilation

From: Eli Zaretskii <[email-masked]>
Date: Tue, 08 Jun 2021 16:56:04 +0300

This bug report is intended to serve as a clearinghouse for
performance issues with the native-compilation feature.  It is
currently too slow, IMO, and we should try speeding it up, whether by
rewriting some of the comp.el code in C or by other code
optimizations.

As a starter, I profiled native-compilation of 2 Lisp files: files.el
and comp.el.  The profiles tell the same story, so I will present only
the profiles for compiling files.el here.

I present below 2 profiles: one collected when comp.eln and
comp-cstr.eln were used to compile files. el, and the other where
comp.el and comp-cstr.el (interpreted Lisp files) were used.  (I did
the second run because it produces a more detailed profile, so it is
easier to understand which parts of the code take most of the time.)

The conclusions from these runs are as follows, AFAIU:

 . The comp-fwprop pass takes the lion's share of the compilation
   time.
 . comp-final also takes a significant time, but most of its code is
   already in C, so I guess it's libgccjit that is the most
   significant player there.
 . The other passes are more or less insignificant, compared to those
   two.
 . Within comp-fwprop, most of the time is taken by comp-ssa-rname and
   its subroutines, in particular comp-ssa-rename-insn.  Note that
   comp-fwprop is called twice during the compilation.
 . The maphash call of comp-fwprop also takes significant percentage
   of the CPU time.
 
So: are there any reasonable ways we could speed up comp-fwprop and
its most expensive subroutines?

Here are the 2 profiles:

----------------------------------------------------------------------
       30687  65% - command-execute
       30648  65%  - funcall-interactively
       30648  65%   - execute-extended-command
       29857  63%    - command-execute
       29857  63%     - funcall-interactively
       29849  63%      - my-native-compile
       29849  63%       - let
       29776  63%        - native-compile
       29625  63%         - comp--native-compile
       18068  38%          - comp-fwprop
       12858  27%           - comp-ssa
       12854  27%            - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_111>
       11218  23%             - comp-ssa-rename
       11198  23%              - #<compiled 0x1fd0b23984a2df72>
       10911  23%               - #<compiled 0x1fd5cc9644a2df72>
       10307  21%                + #<compiled 0x1fd5cc9644a2df72>
         448   0%                + comp-ssa-rename-insn
         175   0%               + comp-ssa-rename-insn
          16   0%              + comp-new-frame
         469   0%             + comp-place-phis
         404   0%             + comp-log-block-info
         188   0%             + comp-compute-edges
         156   0%             + comp-compute-dominator-tree
          88   0%             + comp-finalize-phis
          32   0%             + comp-compute-dominator-frontiers
          28   0%               comp-clean-ssa
          15   0%             + comp-remove-unreachable-blocks
        3876   8%           + #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_120>
        1334   2%           + comp-dead-code
        9083  19%          - comp-final
        9023  19%           - comp-final1
        9015  19%            - comp-compile-ctxt-to-file
         196   0%             - comp--compile-ctxt-to-file
          88   0%              + comp-c-func-name
           4   0%                comp-delete-or-replace-file
          20   0%             + comp-finalize-relocs
           8   0%              #<compiled -0x2e2c9385efa1f31>
          52   0%           + comp-compute-function-type
        1371   2%          + comp-spill-lap
         787   1%          + comp-limplify
         280   0%          + comp-add-cstrs
          24   0%          + comp-call-optim
           8   0%          + comp-remove-type-hints
         139   0%         + byte-code
          12   0%         + defconst
         773   1%    + execute-extended-command--shorter
           8   0%    + sit-for
          39   0%  + byte-code
       16244  34% - ...
       16244  34%    Automatic GC
----------------------------------------------------------------------
      134091  75% - command-execute
      134067  75%  - funcall-interactively
      134020  75%   - my-native-compile
      134020  75%    - let
      134020  75%     - native-compile
      134020  75%      - comp--native-compile
      134020  75%       - catch
      134020  75%        - let*
      134020  75%         - condition-case
      134020  75%          - let*
      134020  75%           - while
      134020  75%            - if
      134020  75%             - setq
      134003  75%              - funcall
      110084  62%               - comp-fwprop
       71453  40%                - comp-ssa
       71453  40%                 - maphash
       71453  40%                  - #<lambda 0x5dd2ecc8d384d40>
       71453  40%                   - let*
       71449  40%                    - if
       56153  31%                     - comp-ssa-rename
       56145  31%                      - let
       56145  31%                       - let
       56145  31%                        - funcall
       55830  31%                         - #<lambda 0x1ed3cb5de3623c25>
       55830  31%                          - if
       55822  31%                           - let*
       42249  23%                            - while
       42245  23%                             - comp-ssa-rename-insn
       42241  23%                              - let*
       42213  23%                               - while
       42157  23%                                - let*
       42141  23%                                 - progn
       42081  23%                                  - let*
       42057  23%                                   - if
       42025  23%                                    - let*
       42005  23%                                     - cond
       32095  18%                                      - let*
       30783  17%                                       - cond
       30068  16%                                        - let*
       29492  16%                                         - cond
       25218  14%                                          - funcall
       21696  12%                                           - #<lambda -0x12308a278265eb50>
       21009  11%                                            - let
       15357   8%                                             - setcdr
       14992   8%                                              - cl-nsubst-if
       13980   7%                                               + cl-nsublis
        4728   2%                                             + comp-vec-aref
        2788   1%                                           + #<lambda -0xff571895f1be2d7>
          44   0%                                           + #<lambda -0xd3c8536673c35a5>
        3391   1%                                          + let
         156   0%                                            eq
        1292   0%                                       + if
        8846   4%                                      + funcall
        1012   0%                                      + comp-assign-op-p
           4   0%                                  setq
           8   0%                               + progn
           8   0%                               + -
       13565   7%                            + if
           8   0%                            + and
           4   0%                           + progn
         307   0%                         + comp-new-frame
           8   0%                         + gethash
           4   0%                      + comp-log
        9678   5%                     + comp-place-phis
        3396   1%                     + progn
        1263   0%                     + comp-finalize-phis
         645   0%                     + comp-log-block-info
         299   0%                     + comp-compute-dominator-frontiers
           4   0%                     + comp-log-func
           4   0%                    + progn
       30154  16%                - maphash
       30150  16%                 - #<lambda -0x1eebead9aaf4d49>
       30150  16%                  - if
       30138  16%                   - progn
       30138  16%                    - let
       29270  16%                     - let*
       29246  16%                      - while
       29246  16%                       - and
       29246  16%                        - comp-fwprop*
       29246  16%                         - catch
       29246  16%                          + let*
          20   0%                        comp-log
         779   0%                     + comp-fwprop-prologue
          81   0%                     + comp-rewrite-non-locals
           8   0%                   + and
        8477   4%                + comp-dead-code
       10262   5%               + comp-final
        6452   3%               + comp-limplify
        3415   1%               + comp-add-cstrs
        2666   1%               + comp-spill-lap
         740   0%               + comp-remove-type-hints
         376   0%               + comp-call-optim
           4   0%               + comp-ipa-pure
           4   0%               + comp-tco
          17   0%              + cons
          28   0%   + execute-extended-command
          19   0%   + minibuffer-complete
          24   0%  + byte-code
       43354  24% - ...
       41398  23%    Automatic GC
        1892   1%  + #<lambda -0x2d0629db72dff04>
          32   0%  + #<lambda -0x1ac765b72d989315>
          28   0%  + comp-emit-switch
           4   0%  + byte-compile-save-restriction

In GNU Emacs 28.0.50 (build 162, i686-pc-mingw32)
 of 2021-06-07 built on HOME-C4E4A596F7
Repository revision: 17aced49a80e0b9df1eef9624928854f07ac5b7d
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int --with-native-compilation
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
XPM ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp
comp-cstr warnings subr-x rx cl-seq cl-macs cl-extra help-mode seq
byte-opt gv cl-loaddefs cl-lib bytecomp byte-compile cconv iso-transl
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 84793 11655)
 (symbols 48 8964 1)
 (strings 16 25180 3737)
 (string-bytes 1 771548)
 (vectors 16 16976)
 (vector-slots 8 293776 13969)
 (floats 8 28 125)
 (intervals 40 267 89)
 (buffers 888 11))