GNU bug report logs - #51572
[PATCH] tree-sitter-lock-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 51572 in the body.
You can then email your comments to 51572 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: dick <dick.r.chiang <at> gmail.com>
To: bug-gnu-emacs <bug-gnu-emacs <at> gnu.org>
Subject: [PATCH] tree-sitter-lock-mode
Date: Tue, 02 Nov 2021 15:03:59 -0400

[Message part 1 (text/plain, inline)]

FWIW, fontification of xdisp.c under tree-sitter is under a
second versus 17s for jit-lock (on my 10yo Thinkpad).

Install rust backend:
git clone https://github.com/commercial-emacs/tree-sitter.git
make -C tree-sitter install
pkg-config --exact-version=0.6.3alpha tree-sitter || echo not found

Then build emacs:
./autogen.sh
LDFLAGS="-L$HOME/.local/lib" CFLAGS="-g3 -O2 -I$HOME/.local/include/" ./configure --prefix=$HOME/.local --with-tree-sitter
make -j4 bootstrap
ldd src/emacs | grep -q tree-sitter || echo not found
make test/src/tree-sitter-tests.log
src/emacs -Q --eval '(custom-set-variables (quote (font-lock-support-mode (quote tree-sitter-lock-mode))) (quote (font-lock-maximum-size nil)) (quote (package-archives (quote (("melpa" . "https://melpa.org/packages/"))))))' --eval '(package-refresh-contents)' --eval '(package-install (cadr (assq (quote tree-sitter-langs) package-archive-contents)))' --visit src/xdisp.c

[0001-tree-sitter-lock-mode.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

In Commercial Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.10)
 of 2021-11-02 built on dick
Repository revision: 79a446d9792b36e7d84b75757f88953023868421
Repository branch: dev
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.4 LTS

Configured using:
 'configure --prefix=/home/dick/.local --with-tree-sitter
 --enable-dumping-overwrite CC=gcc-10 'CFLAGS=-g3 -O2
 -I/home/dick/.local/include/' LDFLAGS=-L/home/dick/.local/lib
 PKG_CONFIG_PATH=/home/dick/.local/lib/pkgconfig'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
TREE-SITTER LCMS2 LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Diff

Minor modes in effect:
  whitespace-mode: t
  async-bytecomp-package-mode: t
  global-git-commit-mode: t
  shell-dirtrack-mode: t
  projectile-mode: t
  flx-ido-mode: t
  override-global-mode: t
  global-hl-line-mode: t
  winner-mode: t
  tooltip-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  tree-sitter-lock-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/dick/gomacro-mode/gomacro-mode hides /home/dick/.emacs.d/elpa/gomacro-mode-20200326.1103/gomacro-mode
/home/dick/.emacs.d/elpa/hydra-20170924.2259/lv hides /home/dick/.emacs.d/elpa/lv-20191106.1238/lv
/home/dick/.emacs.d/elpa/magit-3.3.0/magit-section-pkg hides /home/dick/.emacs.d/elpa/magit-section-3.3.0/magit-section-pkg
/home/dick/org-gcal.el/org-gcal hides /home/dick/.emacs.d/elpa/org-gcal-0.3/org-gcal
/home/dick/.emacs.d/elpa/tree-sitter-20210912.1211/tree-sitter hides /home/dick/.local/share/emacs/28.0.50/lisp/tree-sitter
/home/dick/.emacs.d/lisp/json hides /home/dick/.local/share/emacs/28.0.50/lisp/json
/home/dick/.emacs.d/elpa/transient-0.3.6/transient hides /home/dick/.local/share/emacs/28.0.50/lisp/transient
/home/dick/.emacs.d/elpa/hierarchy-20171221.1151/hierarchy hides /home/dick/.local/share/emacs/28.0.50/lisp/emacs-lisp/hierarchy

Features:
(shadow sort footnote mail-extr gnus-msg emacsbug sendmail whitespace
magit-extras shortdoc help-fns radix-tree rect org-element avl-tree ol-eww eww
xdg url-queue ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-search
eieio-opt speedbar ezimage dframe gnus-art mm-uu mml2015 mm-view mml-smime
smime dig gnus-sum gnus-group mm-url gnus-undo gnus-start gnus-dbus gnus-cloud
nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range
gnus-win ol-docview doc-view jka-compr image-mode exif ol-bibtex ol-bbdb
ol-w3m ol-doi org-link-doi org-tempo tempo org org-macro org-footnote
org-pcomplete org-list org-faces org-entities org-version ob-R ob-emacs-lisp
ob-ein ein-cell ein-output-area shr kinsoku svg dom ein-kernel ein-ipdb
ein-query ein-events ein-websocket websocket bindat ein-node ewoc ein-log
ein-classes ein-core request anaphora ein ein-utils deferred ob ob-tangle
org-src ob-ref ob-lob ob-table ob-exp ob-comint ob-core ob-eval org-table
oc-basic bibtex ol org-keys oc org-compat org-macs org-loaddefs find-func
cal-menu calendar cal-loaddefs misearch multi-isearch tramp-archive tramp-gvfs
tramp-cache zeroconf dbus xml tramp tramp-loaddefs trampver tramp-integration
files-x tramp-compat parse-time iso8601 ls-lisp scheme make-mode texinfo
texinfo-loaddefs benchmark autoconf autoconf-mode vc-git vc vc-dispatcher
bug-reference cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs face-remap magit-patch-changelog magit-patch
magit-submodule magit-obsolete magit-popup async-bytecomp async magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes
magit-worktree magit-tag magit-merge magit-branch magit-reset magit-files
magit-refs magit-status magit magit-repos magit-apply magit-wip magit-log
which-func imenu magit-diff smerge-mode diff diff-mode git-commit log-edit
message rmc puny dired-x dired dired-loaddefs rfc822 mml mml-sec epa epg
rfc6068 epg-config mm-decode mm-bodies mm-encode mailabbrev gmm-utils
mailheader pcvs-util add-log magit-core magit-margin magit-transient
magit-process with-editor shell pcomplete server magit-mode transient
format-spec paredit-ext paredit subed subed-vtt subed-srt subed-common
subed-mpv subed-debug subed-config inf-ruby ruby-mode smie company pcase
haskell-interactive-mode haskell-presentation-mode haskell-process
haskell-session haskell-compile haskell-mode haskell-cabal haskell-utils
haskell-font-lock haskell-indentation haskell-string haskell-sort-imports
haskell-lexeme haskell-align-imports haskell-complete-module
haskell-ghc-support noutline outline flymake-proc flymake warnings etags
fileloop generator xref project dabbrev haskell-customize hydra lv
use-package-ensure solarized-theme solarized-definitions projectile lisp-mnt
mail-parse rfc2231 ibuf-ext ibuffer ibuffer-loaddefs thingatpt
magit-autorevert autorevert filenotify magit-git magit-section magit-utils crm
dash rx grep compile comint ansi-color gnus nnheader gnus-util rmail
rmail-loaddefs rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
text-property-search time-date flx-ido flx google-translate-default-ui
google-translate-core-ui facemenu color ido google-translate-core
google-translate-tk google-translate-backend use-package-bind-key bind-key
auto-complete easy-mmode advice edmacro kmacro popup cus-edit pp cus-load
wid-edit emms-player-mplayer emms-player-simple emms emms-compat cl-extra
use-package-core derived hl-line winner ring help-mode finder-inf
json-reformat-autoloads json-snatcher-autoloads sml-mode-autoloads
tornado-template-mode-autoloads info package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tree-sitter tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer
select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese composite
emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help simple
abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind inotify
lcms2 dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 759773 97091)
 (symbols 48 42652 4)
 (strings 32 160168 11293)
 (string-bytes 1 5531795)
 (vectors 16 93438)
 (vector-slots 8 1715124 101435)
 (floats 8 587 2181)
 (intervals 56 9737 1218)
 (buffers 992 23))

Message #8 received at 51572 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: dick <dick.r.chiang <at> gmail.com>
Cc: 51572 <at> debbugs.gnu.org
Subject: Re: bug#51572: [PATCH] tree-sitter-lock-mode
Date: Thu, 04 Nov 2021 19:47:16 +0100

dick <dick.r.chiang <at> gmail.com> writes:

> FWIW, fontification of xdisp.c under tree-sitter is under a
> second versus 17s for jit-lock (on my 10yo Thinkpad).

That's really cool.

Skimming the patch is rather difficult, though, because it contains so
many changes that seem irrelevant.  Like:

> -	(when (null (cdr elt))
> +	(unless (cdr elt)

And just snarky bits:

> +  "This was better embedded in a minor mode, obviously."

And a bunch of commentary that's been removed:

> -;; Rather than the function, e.g., `font-lock-fontify-region' containing the
> -;; code to fontify a region, the function runs the function whose name is the
> -;; value of the variable, e.g., `font-lock-fontify-region-function'.  Normally,

So I'm not sure where we're going with this.  Anybody got an opinion?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #11 received at 51572 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>, Yuan Fu <casouri <at> gmail.com>
Cc: 51572 <at> debbugs.gnu.org, dick.r.chiang <at> gmail.com
Subject: Re: bug#51572: [PATCH] tree-sitter-lock-mode
Date: Thu, 04 Nov 2021 21:31:00 +0200

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Thu, 04 Nov 2021 19:47:16 +0100
> Cc: 51572 <at> debbugs.gnu.org
> 
> So I'm not sure where we're going with this.  Anybody got an opinion?

What's the status of the tree-sitter support that Yuan Fu (CC'ed) was
working on?  I thought he was very close to completion.

Message #16 received at 51572 <at> debbugs.gnu.org (full text, mbox):

From: Fu Yuan <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, 51572 <at> debbugs.gnu.org,
 dick.r.chiang <at> gmail.com
Subject: Re: bug#51572: [PATCH] tree-sitter-lock-mode
Date: Sun, 7 Nov 2021 09:21:08 -0800

> 在 2021年11月4日，下午12:31，Eli Zaretskii <eliz <at> gnu.org> 写道：
> 
> 
>> 
>> From: Lars Ingebrigtsen <larsi <at> gnus.org>
>> Date: Thu, 04 Nov 2021 19:47:16 +0100
>> Cc: 51572 <at> debbugs.gnu.org
>> 
>> So I'm not sure where we're going with this.  Anybody got an opinion?
> 
> What's the status of the tree-sitter support that Yuan Fu (CC'ed) was
> working on?  I thought he was very close to completion.

Sorry that I missed this. The code only lacks manual for indentation and font-lock integration. I was waiting for someone to suggest some improvement in that two area before writing the manual but so far no one has provided any. I’ll probably just write the manual and call it done. I’m also waiting for the author of tree-sitter to 1) release 1.0 and 2) allow changing malloc mechanism on run-time.

Yuan 

Message #19 received at 51572 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, 51572 <at> debbugs.gnu.org,
 dick.r.chiang <at> gmail.com
Subject: Re: bug#51572: [PATCH] tree-sitter-lock-mode
Date: Fri, 17 Dec 2021 22:26:58 -0800

>>> 
>>> From: Lars Ingebrigtsen <larsi <at> gnus.org>
>>> Date: Thu, 04 Nov 2021 19:47:16 +0100
>>> Cc: 51572 <at> debbugs.gnu.org
>>> 
>>> So I'm not sure where we're going with this.  Anybody got an opinion?
>> 
>> What's the status of the tree-sitter support that Yuan Fu (CC'ed) was
>> working on?  I thought he was very close to completion.
> 
> Sorry that I missed this. The code only lacks manual for indentation and font-lock integration. I was waiting for someone to suggest some improvement in that two area before writing the manual but so far no one has provided any. I’ll probably just write the manual and call it done. I’m also waiting for the author of tree-sitter to 1) release 1.0 and 2) allow changing malloc mechanism on run-time.

Someone notified me of this commercial-emacs thing made by dickmao [1]. Looking at his README, I suspect that this report is just an excuse for using the patch on his commercial-emacs. So we can probably close this report. I’m happy to be corrected, of course.

[1] https://github.com/commercial-emacs/commercial-emacs

Yuan

Message #22 received at 51572 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: larsi <at> gnus.org, 51572 <at> debbugs.gnu.org, dick.r.chiang <at> gmail.com
Subject: Re: bug#51572: [PATCH] tree-sitter-lock-mode
Date: Sat, 18 Dec 2021 09:38:05 +0200

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Fri, 17 Dec 2021 22:26:58 -0800
> Cc: Lars Ingebrigtsen <larsi <at> gnus.org>,
>  dick.r.chiang <at> gmail.com,
>  51572 <at> debbugs.gnu.org
> 
> Someone notified me of this commercial-emacs thing made by dickmao [1]. Looking at his README, I suspect that this report is just an excuse for using the patch on his commercial-emacs. So we can probably close this report. I’m happy to be corrected, of course.

If there's nothing in this proposal that can be useful for us, feel
free to close.

Message #27 received at 51572-done <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: 51572-done <at> debbugs.gnu.org
Date: Sat, 18 Dec 2021 22:56:36 -0800

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.