GNU bug report logs - #46713
27.1; Variable binding depth exceeds max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode nil on new line

Previous Next

Package: emacs;

Reported by: Ryan Olson <ryan.olson.x <at> gmail.com>

Date: Mon, 22 Feb 2021 22:23:01 UTC

Severity: normal

Tags: confirmed

Found in version 27.1

To reply to this bug, email your comments to 46713 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Mon, 22 Feb 2021 22:23:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ryan Olson <ryan.olson.x <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Mon, 22 Feb 2021 22:23:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ryan Olson <ryan.olson.x <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.1; Variable binding depth exceeds max-specpdl-size in js-mode with
 js-indent-level 2 and indent-tabs-mode nil on new line
Date: Mon, 22 Feb 2021 15:22:14 -0700
In GNU Emacs 27.1 (build 1, x86_64-apple-darwin18.7.0, NS
appkit-1671.60 Version 10.14.6 (Build 18G95))
 of 2020-08-11 built on builder10-14.porkrind.org
Windowing system distributor 'Apple', version 10.3.1894
System Description:  Mac OS X 10.15.7

Recent messages:
Wrote /Users/CENSORED/CENSORED/src/CENSORED/CENSORED/CENSORED.js
Quit
2 (#o2, #x2, ?\C-b)
Saving file /Users/CENSORED/CENSORED/src/CENSORED/CENSORED/CENSORED.js...
Wrote /Users/CENSORED/CENSORED/src/CENSORED/CENSORED/CENSORED.js
Error during redisplay: (internal--syntax-propertize 2120) signaled
(error "Variable binding depth exceeds max-specpdl-size")
back-to-indentation: internal--syntax-propertize did not move
syntax-propertize--doneError during redisplay:
(internal--syntax-propertize 2645) signaled (error "Variable binding
depth exceeds max-specpdl-size")
Error during redisplay: (jit-lock-function 2121) signaled (error
"internal--syntax-propertize did not move syntax-propertize--done")
Error during redisplay: (jit-lock-function 3138) signaled (error
"Variable binding depth exceeds max-specpdl-size")
Error during redisplay: (jit-lock-function 3638) signaled (error
"Variable binding depth exceeds max-specpdl-size")

Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
THREADS JSON PDUMPER

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: JavaScript[JSX]

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs text-property-search mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date
vc-git diff-mode easy-mmode js json subr-x map seq byte-opt gv bytecomp
byte-compile cconv imenu cc-mode cc-fonts easymenu cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
threads kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 77539 14591)
 (symbols 48 9435 1)
 (strings 32 25046 1242)
 (string-bytes 1 941698)
 (vectors 16 13840)
 (vector-slots 8 178575 12426)
 (floats 8 30 274)
 (intervals 56 717 4)
 (buffers 1000 13))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Mon, 22 Feb 2021 22:37:02 GMT) Full text and rfc822 format available.

Message #8 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Ryan Olson <ryan.olson.x <at> gmail.com>
Cc: 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Mon, 22 Feb 2021 23:36:04 +0100
Ryan Olson <ryan.olson.x <at> gmail.com> writes:

> (error "Variable binding depth exceeds max-specpdl-size")
> back-to-indentation: internal--syntax-propertize did not move
> syntax-propertize--doneError during redisplay:
> (internal--syntax-propertize 2645) signaled (error "Variable binding
> depth exceeds max-specpdl-size")

Do you have a recipe to reproduce this error, starting from "emacs -Q"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) moreinfo. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Mon, 22 Feb 2021 22:37:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Tue, 23 Feb 2021 02:04:01 GMT) Full text and rfc822 format available.

Message #13 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Ryan Olson <ryan.olson.x <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds max-specpdl-size
 in js-mode with js-indent-level 2 and indent-tabs-mode nil on new line
Date: Mon, 22 Feb 2021 19:03:39 -0700
Hi Lars,

I do. I'm not sure why it takes this, but for my specific js file
(with a few things replaced -- because it's company code) it has
issues.

Repro steps:
1. Open foo.js with emacs -Q --eval '(setq-default js-indent-level 2
indent-tabs-mode nil)' foo.js
2. Go to line 61
3. Delete brace at end of that line
4. Save
5. Add brace back
6. Press RET
7. Errors happen

foo.js:
import React, { useEffect } from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useHistory } from 'react-router';
import { Link } from 'react-router-dom';
import { Grid } from '@rmwc/grid';

import {
  getFoos,
  selectFoos,
  selectIsLoadingFoos,
  selectTotalTimesFoosFetched,
} from 'store/slices/foos.slice';
import { clearFoo } from 'store/slices/foo.slice';
import {
  filterDefinitions,
  getUniqueFooKey,
  searchProperties,
  getFeaturesForFoo,
} from './foosHelpers';
import { useFilters } from 'components/Filtering/useFilters';
import Fab from 'components/Fab';
import SortableDataTable from 'components/SortableDataTable';
import FilterBar from 'components/Filtering/FilterBar';
import useUpdateEffect from 'hooks/useUpdateEffect';

export default function FoosList() {
  const dispatch = useDispatch();
  const history = useHistory();
  const isLoading = useSelector(selectIsLoadingFoos);
  const foos = useSelector(selectFoos);
  const totalTimesFoosFetched = useSelector(
    selectTotalTimesFoosFetched
  );
  const filterBag = useFilters(
    'foos',
    foos,
    filterDefinitions,
    searchProperties
  );
  const { getFilterByKey, clearFilter } = filterBag;

  const canAddFoo = useSelector(state => state.permissions.canAddFoo);

  function requestFoos() {
    dispatch(getFoos({ dispatch }));
  }

  // Get foos from the server (if needed).
  // It attempts to skip reloading the foos, when it can.
  useEffect(() => {
    const isFirstLoad = totalTimesFoosFetched === 0;
    // Since we're only checking this on mount, I think
    // it's safe to say that if we don't have foos and it's
    // not our first load (while mounting), then this would be a
    // place where we've invalidated foos by clearing them out.
    // (to force a load)
    const foosInvalidated = !isFirstLoad && foos.length === 0;

    const shouldFetch = isFirstLoad || foosInvalidated;

    if (shouldFetch) {
      requestFoos();
    }
  }, []);

  // When show foos filter changes,
  // get foos from the server.
  useUpdateEffect(() => {
    requestFoos();
  }, []);

  // Clear foo data from redux.
  useEffect(() => {
    // Clear foo itself.
    dispatch(clearFoo());
  }, []);

  // If we initially load foos and none come back that follow
  // our initial filter (only show centralized foos == true)
  // Remove that filter and fetch foos again.
  useEffect(() => {
    if (totalTimesFoosFetched === 1 && foos.length === 0) {
      // NOTE: this triggers a different effect to get foos that
      // match this filter.
      clearFilter();
    }
  }, [foos, totalTimesFoosFetched]);

  function getFooLink(foo) {
    if (hasValue(foo.fooCode)) {
      return `/foos/foo/fooCode/${foo.fooCode}`;
    }

    if (foo(foo)) {
      return `/foos/foo//${foo.id}`;
    }

    // Shouldn't get here. If all else fails, it shouldn't route anywhere.
    return '#';
  }

  return (
    <Grid>
      <FilterBar filterBag={filterBag}>
        < />
      </FilterBar>
      <SortableDataTable
        data={filterBag.filteredData}
        isLoading={isLoading}
        noRowsMessage="No foos to display."
        schema={{
          getRowKey: getUniqueFooKey,
          columns: [
            {
              headerName: 'Foo Name',
              getColumnValue: foo => (
                <Link
                  to={getFooLink(foo)}
                  className={sharedCss.colorPrimary}
                >
                  {foo.fooName}
                </Link>
              ),
              getSortValue: foo => foo.fooName,
            },
            {
              headerName: 'Foo Code',
              getColumnValue: foo => foo.fooCode,
              className: globalCss.width100,
            },
            {
              headerName: 'Site ID',
              // If there's no site id, it comes down as 0.
              getColumnValue: foo => foo.siteId || null,
              className: globalCss.width100,
            },
            {
              headerName: 'Master Foo ID',
              getColumnValue: foo => foo.id,
              className: globalCss.width100,
            },
          ].filter(Boolean),
        }}
      />
      {canAddFoo && (
        <Fab icon="add" onClick={() => history.push('/foos/new')} />
      )}
    </Grid>
  );
}

On Mon, Feb 22, 2021 at 3:36 PM Lars Ingebrigtsen <larsi <at> gnus.org> wrote:
>
> Ryan Olson <ryan.olson.x <at> gmail.com> writes:
>
> > (error "Variable binding depth exceeds max-specpdl-size")
> > back-to-indentation: internal--syntax-propertize did not move
> > syntax-propertize--doneError during redisplay:
> > (internal--syntax-propertize 2645) signaled (error "Variable binding
> > depth exceeds max-specpdl-size")
>
> Do you have a recipe to reproduce this error, starting from "emacs -Q"?
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Tue, 23 Feb 2021 15:37:01 GMT) Full text and rfc822 format available.

Message #16 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Ryan Olson <ryan.olson.x <at> gmail.com>
Cc: 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Tue, 23 Feb 2021 16:36:21 +0100
Ryan Olson <ryan.olson.x <at> gmail.com> writes:

> Repro steps:
> 1. Open foo.js with emacs -Q --eval '(setq-default js-indent-level 2
> indent-tabs-mode nil)' foo.js
> 2. Go to line 61
> 3. Delete brace at end of that line
> 4. Save
> 5. Add brace back
> 6. Press RET
> 7. Errors happen

Thanks for the clear recipe.  With it, I'm also seeing errors (in both
Emacs 27 and 28):

Debugger entered--Lisp error: (error "internal--syntax-propertize did not move syntax-pr...")
  backward-prefix-chars()
  back-to-indentation()
  current-left-margin()
  move-to-left-margin(nil t)
  #f(compiled-function () #<bytecode 0x155c01f42651>)()
  self-insert-command(1)
  newline(nil 1)
  funcall-interactively(newline nil 1)
  call-interactively(newline nil nil)

But I'm not seeing the "binding depth" one.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) confirmed. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Tue, 23 Feb 2021 15:37:02 GMT) Full text and rfc822 format available.

Removed tag(s) moreinfo. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Tue, 23 Feb 2021 15:37:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Tue, 23 Feb 2021 16:33:02 GMT) Full text and rfc822 format available.

Message #23 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Ryan Olson <ryan.olson.x <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds max-specpdl-size
 in js-mode with js-indent-level 2 and indent-tabs-mode nil on new line
Date: Tue, 23 Feb 2021 09:31:53 -0700
[Message part 1 (text/plain, inline)]
It was with something like this that I saw the error I reported. I’m not
sure how to reproduce that.

I guess we can instead fix this one (that _is_ reproducible)?

On Tue, Feb 23, 2021 at 8:36 AM Lars Ingebrigtsen <larsi <at> gnus.org> wrote:

> Ryan Olson <ryan.olson.x <at> gmail.com> writes:
>
> > Repro steps:
> > 1. Open foo.js with emacs -Q --eval '(setq-default js-indent-level 2
> > indent-tabs-mode nil)' foo.js
> > 2. Go to line 61
> > 3. Delete brace at end of that line
> > 4. Save
> > 5. Add brace back
> > 6. Press RET
> > 7. Errors happen
>
> Thanks for the clear recipe.  With it, I'm also seeing errors (in both
> Emacs 27 and 28):
>
> Debugger entered--Lisp error: (error "internal--syntax-propertize did not
> move syntax-pr...")
>   backward-prefix-chars()
>   back-to-indentation()
>   current-left-margin()
>   move-to-left-margin(nil t)
>   #f(compiled-function () #<bytecode 0x155c01f42651>)()
>   self-insert-command(1)
>   newline(nil 1)
>   funcall-interactively(newline nil 1)
>   call-interactively(newline nil nil)
>
> But I'm not seeing the "binding depth" one.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>
-- 
Sent from my iPhone
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Wed, 24 Feb 2021 16:48:01 GMT) Full text and rfc822 format available.

Message #26 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Ryan Olson <ryan.olson.x <at> gmail.com>
Cc: 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Wed, 24 Feb 2021 17:47:42 +0100
Ryan Olson <ryan.olson.x <at> gmail.com> writes:

> I guess we can instead fix this one (that _is_ reproducible)?

Yup.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Sun, 19 Jun 2022 13:54:02 GMT) Full text and rfc822 format available.

Message #29 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Ryan Olson <ryan.olson.x <at> gmail.com>
Cc: 46713 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Sun, 19 Jun 2022 15:53:14 +0200
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Thanks for the clear recipe.  With it, I'm also seeing errors (in both
> Emacs 27 and 28):
>
> Debugger entered--Lisp error: (error "internal--syntax-propertize did
> not move syntax-pr...")
>   backward-prefix-chars()
>   back-to-indentation()
>   current-left-margin()
>   move-to-left-margin(nil t)
>   #f(compiled-function () #<bytecode 0x155c01f42651>)()
>   self-insert-command(1)
>   newline(nil 1)
>   funcall-interactively(newline nil 1)
>   call-interactively(newline nil nil)

Perhaps Stefan knows what's going on here; added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Sat, 25 Jun 2022 12:04:02 GMT) Full text and rfc822 format available.

Message #32 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Ryan Olson <ryan.olson.x <at> gmail.com>, 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Sat, 25 Jun 2022 08:03:04 -0400
> Perhaps Stefan knows what's going on here; added to the CCs.

[ The recipe works without the js-indent-level and
  indent-tabs-mode settings.  ]

I think the crux of the matter is the following:

                 (run-hook-wrapped
                  'syntax-propertize-extend-region-functions
                  (lambda (f)
                    (let ((new (funcall f start end))
                          ;; Avoid recursion!
                          (syntax-propertize--done most-positive-fixnum))
                      (if (or (null new)
                              (and (>= (car new) start) (<= (cdr new) end)))
                          nil

Where the binding of `syntax-propertize--done` should be around the call
to `f` (apparently I introduced this bug in commit
3928ef2dd5b8febf3b1d9c1bfb22af3698d16bea).

But the bug also shows that it's difficult for
syntax-propertize-functions to know when they use `syntax-ppss` since it
can happen as a side-effect of forward-sexp and hence I'm beginning to
think it's not practical to require them to call
`syntax-ppss-flush-cache` manually.

I'm thinking of installing the patch below on `master`.  We could
install a simpler patch on `emacs-28` which just swaps the two vars in
the above `let` (and changes it into a `let*`).

WDYT?


        Stefan


    * lisp/emacs-lisp/syntax.el: Rework the handling of nested calls.

    Nested calls to `syntax-ppss` and `syntax-propertize` can easily
    happen unexpectedly via ondemand propertizing or `forward-sexp`.
    Refine the handling of nested calls so we detect them more reliably
    (e.g. also within `syntax-propertize-extend-region-functions`)
    and so that the `syntax-ppss` cache is automatically flushed in case
    it might have been filled with data that's become obsolete since.

    (syntax-propertize--inhibit-flush): Delete var.
    (syntax-propertize--in-process-p): New function to replace it.
    (syntax-ppss-flush-cache): Use it.
    (syntax-ppss--updated-cache): New var.
    (syntax-propertize): Make `syntax-propertize--done` binding apply to
    `syntax-propertize-extend-region-functions` as well, as intended (fixes
    bug#46713).  Use `syntax-ppss--updated-cache` to flush
    syntax-ppss cache at the end when needed.
    Don't bind `syntax-propertize--inhibit-flush` any more.
    (syntax-ppss): Set `syntax-ppss--updated-cache` when applicable.


diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
index a4d7beade13..36b0c56e953 100644
--- a/lisp/emacs-lisp/syntax.el
+++ b/lisp/emacs-lisp/syntax.el
@@ -345,10 +345,16 @@ syntax-propertize-via-font-lock
 (defvar-local syntax-ppss-table nil
   "Syntax-table to use during `syntax-ppss', if any.")
 
-(defvar-local syntax-propertize--inhibit-flush nil
-  "If non-nil, `syntax-ppss-flush-cache' only flushes the ppss cache.
-Otherwise it flushes both the ppss cache and the properties
-set by `syntax-propertize'")
+(defun syntax-propertize--in-process-p ()
+  "Non-nil if we're inside `syntax-propertize'.
+This is used to avoid infinite recursion as well as to handle cases where
+`syntax-ppss' is called when the final `syntax-table' properties have not
+yet been setup, in which case we may end up putting invalid info into the cache.
+It's also used so that `syntax-ppss-flush-cache' can be used from within
+`syntax-propertize' without ruining the `syntax-table' already set."
+  (eq syntax-propertize--done most-positive-fixnum))
+
+(defvar-local syntax-ppss--updated-cache nil)
 
 (defun syntax-propertize (pos)
   "Ensure that syntax-table properties are set until POS (a buffer point)."
@@ -370,21 +376,24 @@ syntax-propertize
         (with-silent-modifications
           (with-syntax-table (or syntax-ppss-table (syntax-table))
             (make-local-variable 'syntax-propertize--done) ;Just in case!
+            ;; Make sure we let-bind it only buffer-locally.
+            (make-local-variable 'syntax-ppss--updated-cache)
             (let* ((start (max (min syntax-propertize--done (point-max))
                                (point-min)))
                    (end (max pos
                              (min (point-max)
                                   (+ start syntax-propertize-chunk-size))))
                    (first t)
-                   (repeat t))
+                   (repeat t)
+                   (syntax-ppss--updated-cache nil))
               (while repeat
                 (setq repeat nil)
                 (run-hook-wrapped
                  'syntax-propertize-extend-region-functions
                  (lambda (f)
-                   (let ((new (funcall f start end))
-                         ;; Avoid recursion!
-                         (syntax-propertize--done most-positive-fixnum))
+                   ;; Bind `syntax-propertize--done' to avoid recursion!
+                   (let* ((syntax-propertize--done most-positive-fixnum)
+                          (new (funcall f start end)))
                      (if (or (null new)
                              (and (>= (car new) start) (<= (cdr new) end)))
                          nil
@@ -399,20 +408,26 @@ syntax-propertize
               ;; Flush ppss cache between the original value of `start' and that
               ;; set above by syntax-propertize-extend-region-functions.
               (syntax-ppss-flush-cache start)
-              ;; Move the limit before calling the function, so the function
-              ;; can use syntax-ppss.
+              ;; Move the limit before calling the function, so it's
+              ;; done in case of errors.
               (setq syntax-propertize--done end)
               ;; (message "syntax-propertizing from %s to %s" start end)
               (remove-text-properties start end
                                       '(syntax-table nil syntax-multiline nil))
-              ;; Make sure we only let-bind it buffer-locally.
-              (make-local-variable 'syntax-propertize--inhibit-flush)
-              ;; Let-bind `syntax-propertize--done' to avoid infinite recursion!
-              (let ((syntax-propertize--done most-positive-fixnum)
-                    ;; Let `syntax-propertize-function' call
-                    ;; `syntax-ppss-flush-cache' without worries.
-                    (syntax-propertize--inhibit-flush t))
-                (funcall syntax-propertize-function start end)))))))))
+              ;; Bind `syntax-propertize--done' to avoid recursion!
+              (let ((syntax-propertize--done most-positive-fixnum))
+                (funcall syntax-propertize-function start end)
+                (when syntax-ppss--updated-cache
+                  ;; `syntax-ppss' was called and updated the cache while we
+                  ;; were propertizing so we need to flush the part of the
+                  ;; cache that may have been rendered out-of-date by the new
+                  ;; properties.
+                  ;; We used to require syntax-propertize-functions to do that
+                  ;; manually when applicable, but nowadays the `syntax-ppss'
+                  ;; cache can be updated by too many functions, so the author
+                  ;; of the syntax-propertize-function may not be aware it
+                  ;; can happen.
+                  (syntax-ppss-flush-cache start))))))))))
 
 ;;; Link syntax-propertize with syntax.c.
 
@@ -487,10 +502,10 @@ syntax-ppss-narrow-start
 
 (define-obsolete-function-alias 'syntax-ppss-after-change-function
   #'syntax-ppss-flush-cache "27.1")
-(defun syntax-ppss-flush-cache (beg &rest ignored)
+(defun syntax-ppss-flush-cache (beg &rest _ignored)
   "Flush the cache of `syntax-ppss' starting at position BEG."
   ;; Set syntax-propertize to refontify anything past beg.
-  (unless syntax-propertize--inhibit-flush
+  (unless (syntax-propertize--in-process-p)
     (setq syntax-propertize--done (min beg syntax-propertize--done)))
   ;; Flush invalid cache entries.
   (dolist (cell (list syntax-ppss-wide syntax-ppss-narrow))
@@ -517,10 +532,16 @@ syntax-ppss-flush-cache
        (setcdr cell cache)))
     ))
 
-;;; FIXME: Explain this variable.  Currently only its last (5th) slot is used.
-;;; Perhaps the other slots should be removed?
+;; FIXME: Explain this variable.  Currently only its last (5th) slot is used.
+;; Perhaps the other slots should be removed?
+;; This variable is only used when `syntax-begin-function' is used and
+;; will hence be removed together with `syntax-begin-function'.
 (defvar syntax-ppss-stats
-  [(0 . 0) (0 . 0) (0 . 0) (0 . 0) (0 . 0) (2 . 2500)])
+  [(0 . 0) (0 . 0) (0 . 0) (0 . 0) (0 . 0) (2 . 2500)]
+  "Statistics about which case is more/less frequent in `syntax-ppss'.
+The 5th slot drives the heuristic to use `syntax-begin-function'.
+The rest is only useful if you're interested in tweaking the algorithm.")
+
 (defun syntax-ppss-stats ()
   (mapcar (lambda (x)
 	    (condition-case nil
@@ -658,6 +679,7 @@ syntax-ppss
 	       ;; populate the cache so we won't need to do it again soon.
 	       (t
 		(syntax-ppss--update-stats 3 pt-min pos)
+                (setq syntax-ppss--updated-cache t)
 
 		;; If `pt-min' is too far, add a few intermediate entries.
 		(while (> (- pos pt-min) (* 2 syntax-ppss-max-span))
@@ -692,6 +714,7 @@ syntax-ppss
 			(push pair ppss-cache)
 		      (setcar ppss-cache pair)))))))))
 
+          (setq syntax-ppss--updated-cache t)
 	  (setq ppss-last (cons pos ppss))
           (setcar cell ppss-last)
           (setcdr cell ppss-cache)





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Sat, 25 Jun 2022 12:44:02 GMT) Full text and rfc822 format available.

Message #35 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Ryan Olson <ryan.olson.x <at> gmail.com>, 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Sat, 25 Jun 2022 14:43:46 +0200
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> I'm thinking of installing the patch below on `master`.  We could
> install a simpler patch on `emacs-28` which just swaps the two vars in
> the above `let` (and changes it into a `let*`).
>
> WDYT?

The fix on master sounds good, but we're just gearing up for 28.2 this
weekend, so while the change looks innocuous, I'd rather hold off until
after 28.2.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#46713; Package emacs. (Sun, 26 Jun 2022 13:01:02 GMT) Full text and rfc822 format available.

Message #38 received at 46713 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Ryan Olson <ryan.olson.x <at> gmail.com>, 46713 <at> debbugs.gnu.org
Subject: Re: bug#46713: 27.1; Variable binding depth exceeds
 max-specpdl-size in js-mode with js-indent-level 2 and indent-tabs-mode
 nil on new line
Date: Sun, 26 Jun 2022 09:00:09 -0400
> The fix on master sounds good, but we're just gearing up for 28.2 this
> weekend, so while the change looks innocuous, I'd rather hold off until
> after 28.2.

OK, pushed to `master`,


        Stefan





This bug report was last modified 3 years and 50 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.