From unknown Sat Jun 21 10:40:29 2025 X-Loop: help-debbugs@gnu.org Subject: bug#60656: 30.0.50; tree-sitter: editing a buffer invalidates visited node instances Resent-From: Mickey Petersen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 08 Jan 2023 11:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 60656 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 60656@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.167317611517020 (code B ref -1); Sun, 08 Jan 2023 11:09:02 +0000 Received: (at submit) by debbugs.gnu.org; 8 Jan 2023 11:08:35 +0000 Received: from localhost ([127.0.0.1]:60019 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pETXW-0004QS-Ko for submit@debbugs.gnu.org; Sun, 08 Jan 2023 06:08:34 -0500 Received: from lists.gnu.org ([209.51.188.17]:56006) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pETXV-0004QK-1F for submit@debbugs.gnu.org; Sun, 08 Jan 2023 06:08:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pETXU-0007p9-D6 for bug-gnu-emacs@gnu.org; Sun, 08 Jan 2023 06:08:32 -0500 Received: from mail-lo2gbr01on2099.outbound.protection.outlook.com ([40.107.10.99] helo=GBR01-LO2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pETXR-0004oT-0w for bug-gnu-emacs@gnu.org; Sun, 08 Jan 2023 06:08:32 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WxN0WL+5XCOy46qhgieQZD0gc+BeeIHTrl6tvKyshjfG/2tiTg/kDEQZSkuQJrM9yBVcmey7KgihSFuKhTzaFSdwpcgDS9+Lkfnp+z38Y7Q+GCrRlruLBwK4Q7qrKzCluH97E9/U4nFBEsZyLgAxSHZgI5nuQxNE2nhF1wDciJU5krJ6qfVVgGVmgL8jisDzIurZmqQjPdtm65exn3IKxqh/EC7lgUv0GtXmc9jU8VAku+r0d37aSUqs4QRMpXYMz6X9FuwXPEvOAPcM5aYmXcLFyv33Q+Ms6qRARKJ0U9v9We5CRycu04MhZcKcUj8qc1v9n0Mz+2EeneXbzOucAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8h5v0YPg76JbaRdysk2IUBsIcFbMw590ROHRYcVKQWs=; b=VAwaSbj81JN0e1qbg5X5CI1QrvdetzRXu8cbglLrI65UA/x6RjHOs0BpjZ26LWeqFmcAbaDeweJmlMP8YMQ+S2i+d4Knyt4BfxFHJIrzIe9D9Ql5OU6b746EyOYOj057qLhGbRENrUte6pUwayvOqHG68Kk69EZh5VB6TwlxPtLZH4Rnhztm6+npf4A2Q8lo0qrtLowAjWNm3ptpGnaYDTjIiKdkREhYLwdaIJtAXUBNubo2i70nd7xdYbU4rp1A1/no6/gzlH6Svu7SPgzaZWEDF6DX0jux1zJK/R0YQq2bH4j1AwemaucD+1gcnjymTpZXOCu+P/OaoQmTTw9EhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=pass (signature was verified) header.d=masteringemacs.org; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8h5v0YPg76JbaRdysk2IUBsIcFbMw590ROHRYcVKQWs=; b=eo3qMKiaLTFBTjxWwGptG+CEDfCxirKruoh1D1pmBPXHTH6Z8zn86yesDprqp3rrVj8SMidcX2WIiiVRUtAhEbgSQANeaiI/Mg3ooOHX6ztPTkUPGk+WAqIF8Wb8Uj4NJ1jMzlXziqu7AhgucCikaQ89aadjcUZCmgTNwC1m//Q= Received: from CWLP265CA0422.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1d7::22) by LO6P265MB6670.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:305::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Sun, 8 Jan 2023 11:08:21 +0000 Received: from CWLGBR01FT034.eop-gbr01.prod.protection.outlook.com (2603:10a6:400:1d7:cafe::4) by CWLP265CA0422.outlook.office365.com (2603:10a6:400:1d7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Sun, 8 Jan 2023 11:08:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=pass (signature was verified) header.d=masteringemacs.org;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by CWLGBR01FT034.mail.protection.outlook.com (10.152.40.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5966.17 via Frontend Transport; Sun, 8 Jan 2023 11:08:20 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 405EA114002; Sun, 8 Jan 2023 11:08:20 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=masteringemacs.org; s=masteringemacs.org; t=1673176100; bh=8h5v0YPg76JbaRdysk2IUBsIcFbMw590ROHRYcVKQWs=; h=From:To:Subject:Date:From; b=KtxADXOOBcBk4riTlAynUxQUJZ69URkSvhJLXGA/AGi/xYj/eSu4DgEzVtJVoOigw jBzu8D/1ZlmnNAbMzX7m68Aof6LugZcFKIInd5695hHLfFqelqVazYd2zLOrm7e7c1 eUYb9Dfb5MjD0CszWjlyJcoXkdqAYSL5pEo4MwTc= X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 From: Mickey Petersen Date: Sun, 08 Jan 2023 11:08:16 +0000 Message-ID: <87o7r9jnbz.fsf@masteringemacs.org> Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLGBR01FT034:EE_|LO6P265MB6670:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: b564786b-0122-46fb-95c7-08daf168a706 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b2IqJVG53VXFVJgQfNNa6j556XU34p7VnAFNbbQ37nyPUZKG+j1Cb0g4hJBrqO0IiKMZ1Q20hW2ppiYgILv/buA7t1OIbo9VFUVSckRBpsKermrm+42q9a9oBHdus7mMI5EYPFxnDNvgfRGaqo6iwxkNdlQ+PGQhrgdTIUbSMbZbxPVcRCe2z2N8/mC/3t8RBROKDqSF9HB0alk8NyRDBQF+8ZmirTZItkfEphe/zxWPu1XiYQCe3Xmb+V2YNQ+lgpXFAseIQYc863X5zmoJ1+LA5MsAVRiFY9+JC6G9KnxqB/WFuKUVGfmXuaM3OmV9f1Z8ktdL4/sAGqyWhcOnCgpXa6l5TsyJETPW0GPZfWOL+Rsz5qIGfhVnRUCblTjdiPCFeU7P3DyTUIscnIPMnR9BNKw2byeJ+QwXjyIQ7Ifm8MBqux4lxgfmkmlQHOGt6fMKWMOc4zOXW4q8MoPgwuhycn9ajlXrUi8Ac6iR6l4vF9IBTIaORGvACjKkLBzdNmNeZNLtPVWjL5X56DjuT5j95cnZIQuEcIzmiP4KF3xhpIpEJWn1dml2ojJvUxDnBJXvLh1fXx3LY79KPNjg8NXqAJhZm/zwJYu9kmUB4o6kUylZE1u6XLpH9n5Eq166gRli/Bw8amZGRVrM9Y4wyNUBcA4lYABcJG82YcJD7+R21UHx7fB4Q7xWkHoxJKuM0EO6e+5gzY2OIVYcATFP8V+57h4wMWx5O7WvazI5KC+cECNzZ4LtxNVxJ87ogZWSXezpMTo6By7J4thKHF7o12CNdyqCorLsfJirhcFEYTk= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230022)(396003)(346002)(39830400003)(136003)(376002)(451199015)(46966006)(36840700001)(36860700001)(6666004)(7596003)(7636003)(356005)(2906002)(478600001)(966005)(2616005)(186003)(6266002)(26005)(5660300002)(316002)(40480700001)(82310400005)(8936002)(36756003)(42186006)(86362001)(41300700001)(47076005)(70586007)(8676002)(70206006)(336012)(6916009)(38230200001)(14776008)(79816003); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2023 11:08:20.6997 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b564786b-0122-46fb-95c7-08daf168a706 X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: CWLGBR01FT034.eop-gbr01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO6P265MB6670 Received-SPF: pass client-ip=40.107.10.99; envelope-from=mickey@masteringemacs.org; helo=GBR01-LO2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) If you parse some text, retrieve a node -- using `treesit-node-at', for example -- and then edit the buffer, then the node you retrieved is marked outdated. However, tree-sitter is capable of handling that, to a greater or lesser extent: https://tree-sitter.github.io/tree-sitter/using-parsers#editing It is therefore possible to refresh node instances that were created _before_ the edit. I suppose it could remain an explicit step that you must enter a special form and then Emacs will track node instances issued inside that form and refresh them when edits take place inside of it. As it stands, it is very hard to edit and maintain a node registry at the same time. (I'm using markers and overlays as a crude hack to work around it.) In GNU Emacs 30.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0) of 2023-01-02 built on mickey-work Repository revision: c209802f7b3721a1b95113290934a23fee88f678 Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12013000 System Description: Ubuntu 20.04.3 LTS From unknown Sat Jun 21 10:40:29 2025 X-Loop: help-debbugs@gnu.org Subject: bug#60656: 30.0.50; tree-sitter: editing a buffer invalidates visited node instances References: <87o7r9jnbz.fsf@masteringemacs.org> In-Reply-To: <87o7r9jnbz.fsf@masteringemacs.org> Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 09 Jan 2023 03:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60656 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Mickey Petersen Cc: 60656@debbugs.gnu.org Received: via spool by 60656-submit@debbugs.gnu.org id=B60656.16732366636977 (code B ref 60656); Mon, 09 Jan 2023 03:58:01 +0000 Received: (at 60656) by debbugs.gnu.org; 9 Jan 2023 03:57:43 +0000 Received: from localhost ([127.0.0.1]:35379 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEjI7-0001oS-03 for submit@debbugs.gnu.org; Sun, 08 Jan 2023 22:57:43 -0500 Received: from mail-pg1-f172.google.com ([209.85.215.172]:33309) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEjI4-0001o6-AK for 60656@debbugs.gnu.org; Sun, 08 Jan 2023 22:57:41 -0500 Received: by mail-pg1-f172.google.com with SMTP id 141so5074019pgc.0 for <60656@debbugs.gnu.org>; Sun, 08 Jan 2023 19:57:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=zTdxRPg2VR5ZBkZmrmre5ubZtL63sf8Rq0N4/aF4+o0=; b=lKDJnlm3P/2Tj4bREAsWoS9kgxs6PSc2C+qX3o+tBhr5JC8bpSX7ox8LAacra9cFCM BCGVyd7lpi87Xt1JA+ggvdVxa+xCwIUxs++WOnTGx2qiKn4QGL6w+7sH5HHNWbBlovaG 94n2LXEmJh4YP5ZKRTairezeT+74CkTvJKPLw34A0AnAJUcqaPsbP6epIZpOdnKuOfXW 5KE/SirgfqIRcOttxgq2c8E8gxbaERfGnGqhKpDXvkC3UNFSAd7Ux13lmZJqB5wmmT4v PvK0rK+ViG9qNAAzbqMIya3mJkKwGvhk3cVJZW3Gz4JHVcnyO/g4wRcZOdhs62Eizp23 4JqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zTdxRPg2VR5ZBkZmrmre5ubZtL63sf8Rq0N4/aF4+o0=; b=beDZrLJYCvCNgS8QsN33SO6pVo6d3EHBOaQBWbUgQsKlwWS5SZi87KUpIx7IK+vAc9 URXBdPhGuKJ3v9E1R0GyixQtah46Fs0bqbEsL1mFCEsw3Azb0Oo7ZHQRAskJoVS4jX+y sMaozwwtoQnMYkfB0nY/6j37FWAO5pjCpzs9v5A+2ZBMmsi81bYdwHh/36KA1Ssj6qfV +2Zr2KHoHVO67VJUnLnYILXZ31gGZpFSSQBcZbgFEo9X0UNATm5ggOHwu2bbuK6Inkqg WS8FpurRD+WmhbC1kRiGBYVV3fEL/ty6gN1rgPSZcPOamYe3RE8TCKc6e1Ubx3/i7S3c ktwA== X-Gm-Message-State: AFqh2kpZ2mBL0a3LqDM8WDtatOahCiEUP3LRHaoaGB26oV/rDpgjs0JB X6ouNY+Ummlyd2fwzxbkVCeBVJUfBWo= X-Google-Smtp-Source: AMrXdXve3EhogAYsXXqV3kmx4BFEeXRf2AIegzyOth7EWbZDa1XG0tzzscQ6gzn8fmzBym+ijNhw+A== X-Received: by 2002:a05:6a00:1747:b0:582:7d41:c8a4 with SMTP id j7-20020a056a00174700b005827d41c8a4mr37291686pfc.15.1673236654312; Sun, 08 Jan 2023 19:57:34 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id t11-20020a62d14b000000b00572c12a1e91sm4905287pfl.48.2023.01.08.19.57.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Jan 2023 19:57:34 -0800 (PST) From: Yuan Fu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Message-Id: Date: Sun, 8 Jan 2023 19:57:32 -0800 X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Mickey Petersen writes: > If you parse some text, retrieve a node -- using `treesit-node-at', > for example -- and then edit the buffer, then the node you retrieved > is marked outdated. > > However, tree-sitter is capable of handling that, to a greater or = lesser extent: > > https://tree-sitter.github.io/tree-sitter/using-parsers#editing > > It is therefore possible to refresh node instances that were created > _before_ the edit. I suppose it could remain an explicit step that you > must enter a special form and then Emacs will track node instances > issued inside that form and refresh them when edits take place inside > of it. > > As it stands, it is very hard to edit and maintain a node registry at > the same time. (I'm using markers and overlays as a crude hack to work > around it.) This is kind of a limitation of tree-sitter. The "node editing" isn=E2=80=99= t like what you thought (it fooled me too when I first read it). Tree-sitter=E2=80=99s incremental parsing works roughly like this: 1. You have a parsed tree, TREE, corresponding to some TEXT 2. You make some edit to the TEXT, eg, TEXT=E2=80=99 =3D insert(TEXT, 1, = "abc") 3. Now you need to "edit" the old tree with _positions_ of your edit: edit(TREE, Insert(pos=3D1, len=3D3)) (Notice that this modifies the tree = in-place.) 4. You reparse the edited tree and gets a new tree: TREE=E2=80=99 =3D parse(TREE, TEXT=E2=80=99) (Notice that this returns a = new tree.) If you have a NODE from TREE, editing that node only updates position information. That corresponds to the eidt(TREE, ...) step. There is no equivalent of the parse(TREE, TEXT=E2=80=99) step for nodes: once the = tree is reparsed and a new tree is returned, none of the nodes in the old tree gets carried to the new tree. In practice, tree-sitter reuses old = tree=E2=80=99s data, but conceptually the old and new tree don=E2=80=99t share any = node. IOW, the editing feature for nodes is for very specific situations, where you edit the parse tree but didn=E2=80=99t reparse yet. In this = case, if you want to make your node=E2=80=99s positions to be correct, you edit = the node. But once you reparse, there is no way to somehow "update" this old node into its "equivalent" in the new tree. I=E2=80=99m not sure whether tree-sitter is capable to do what you want = (after all the old and new tree are sharing data). But currently it doesn=E2=80=99= t expose the feature to do that. Yuan From unknown Sat Jun 21 10:40:29 2025 X-Loop: help-debbugs@gnu.org Subject: bug#60656: 30.0.50; tree-sitter: editing a buffer invalidates visited node instances Resent-From: Mickey Petersen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 09 Jan 2023 08:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60656 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Yuan Fu Cc: 60656@debbugs.gnu.org Received: via spool by 60656-submit@debbugs.gnu.org id=B60656.167325473823248 (code B ref 60656); Mon, 09 Jan 2023 08:59:01 +0000 Received: (at 60656) by debbugs.gnu.org; 9 Jan 2023 08:58:58 +0000 Received: from localhost ([127.0.0.1]:35742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEnze-00062u-BX for submit@debbugs.gnu.org; Mon, 09 Jan 2023 03:58:58 -0500 Received: from mail-lo2gbr01on2097.outbound.protection.outlook.com ([40.107.10.97]:22389 helo=GBR01-LO2-obe.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEnza-00062c-FP for 60656@debbugs.gnu.org; Mon, 09 Jan 2023 03:58:57 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vb1jNjquSdbS5/zhJzuH5EfLvhWDAkYENO+cv/eyc6/fujzU69+BLeJwepdSABklAvfscPHC60bNQT4eGHBePhA5m/mTu0bG6apt2L5n2m8LQaIQYYgFwStg8izDm7SBUaBk49WOLGbt5vtaOciipfwc3vOBlid1UXWrRmCPcPrwpKall7yO/6oSOWEPHbXtvj9sYzhMmIFhqg9pIPxjd+dv0xM9n0315nNfrjn1GSuVNyKbOuNSl95kQI2Y00rCSknYsngQLEWZFJQ8dCED/BfowWVwX4GYi/ItWASL/FNqGYdWLDb2edP/XTmFZPuhDhmaFA+UIGNnaezJbVC8ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qp1Qs82L0MXZvvvH5pAarGFHF7koAwQX5rFLyOoFfNo=; b=Os1IxP2ZS533rS969kGBJCQrsKXrbORmPGFgoJbJcFbSguNqVxRS957NITp2hmSEKNMisxJp1zhjX5pFO8VaCaggFSynK6u4vLRnMOOGhFKJkH66Tqifrzvi9ARghwsHNRoWJ+d2SM0Y9R1br+DfdSIGVVNpQhZrLkuqoqSi2yA2K9wg/pOjuj5isAry18leCu+t4KZXjpn64hbcSlNWdZVTqlVcqT9i6f2YUM7Y0Bt/Gmz4YqnuXmse0Jat3u10LQTzOtacIS+TV1oBrOGY0RqYuQmqUZ5NNlvRhxUEMkM47Ons2E1L4cxMeRMHsMcEf7gnji7ai2sPpEvyCEs6rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=debbugs.gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=pass (signature was verified) header.d=masteringemacs.org; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qp1Qs82L0MXZvvvH5pAarGFHF7koAwQX5rFLyOoFfNo=; b=FsCKM04TsakDuWDGyZslrp6COxhMrhjFWkYk60OBP9ci3m7AdUpFvW5Kj+tj31vLoozrOXmgP4Jvt6xfshS+sjTayhPqmCmdhrNhiMVPGgQdxXjStlzNd0GpsoeR2UsdVdyoBbz77fAlSDhQfzktZ1VQDhXvqQJkOh64fM+6XAg= Received: from CWLP123CA0114.GBRP123.PROD.OUTLOOK.COM (2603:10a6:401:5f::30) by LO2P265MB5743.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:26d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 08:58:47 +0000 Received: from CWLGBR01FT021.eop-gbr01.prod.protection.outlook.com (2603:10a6:401:5f:cafe::58) by CWLP123CA0114.outlook.office365.com (2603:10a6:401:5f::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 08:58:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=pass (signature was verified) header.d=masteringemacs.org;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by CWLGBR01FT021.mail.protection.outlook.com (10.152.40.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 08:58:46 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 53825114002; Mon, 9 Jan 2023 08:58:46 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=masteringemacs.org; s=masteringemacs.org; t=1673254726; bh=qp1Qs82L0MXZvvvH5pAarGFHF7koAwQX5rFLyOoFfNo=; h=References:From:To:Cc:Subject:Date:In-reply-to:From; b=sC0/IIuPftbkZFf+jT+D+uqCir/ifFsYiu0/GKJYdZkenLfV8Bh4skXWKzbc6N5VI IfDGZAzDpq4+YCA35/dyY6o8sq8mGk9H56+hsuRwD0hvJD+clBCAfvPU9xxBlK8DLs Te+yna65yodZurJN4iZKOdg/32z8RAMkN9cVJ+g0= X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 References: User-agent: mu4e @VERSION@; emacs 30.0.50 From: Mickey Petersen Date: Mon, 09 Jan 2023 08:56:57 +0000 Organization: Mastering Emacs In-reply-to: Message-ID: <87zgashynw.fsf@masteringemacs.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLGBR01FT021:EE_|LO2P265MB5743:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: e402d2e3-2211-4e5e-edfe-08daf21fb7b1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OPPTY+nXKNU/NfPS0lL8DUsc8ZGioXdexIi304HAvYKqELgJWuWAiJ8DAZ2MA5yv/DtYLQXhFhC8z02EKYrBlC2rY+2EvWrKmLVOUZ5+Tcy8Yo86s9A5gdSJxoftYE43wbkQDGQuqs8a9uv5YHZgGsLhUKxTLDu1OeKMBlkzYd+IM8z4w1wYX5/ILIAPRonzitrER9ApKXi38SbbK6f5nnOZImeEz5UyKWnjZWWQ3DHNLfgjbHVhNdbhyczln6ZtV6SBZDFYpIwX3HwfKu13uQSJtCIl6IBIOjPYkKmHIIp4G81MfWBFLPXwSdu3lJhRRvSO6SHhxs3lArXG6hVDOHYasDWoytWuz+qRZVEG2PYecIe2yyahF2GdeQ5QsvS4a0+mbECDUhpHGv3lajp1S2K3B4BWxbhj08Yl9g8HznuW6qTjgPWbwsBXMEtXQxVOU+zrP3igdSTsLcLEuVE1xFfhNnGq2fGjyrU/1bVzz7Wk/Xm8/S5hx1rvc1KfwTbkK9rVcWNh4U0SX/Q3kLhFISQOKlr8Q9MSA7ZKBTgnPymfQTgpf4k/hLfwhObxq0VzDyz5z9r1+tKuoWZZQjS6YIH+JWZPoprg9ZlgHC6PDItlVJrvr7hUtzPXIvOeEId9RTnM5aNz1oyXsJ+TRT4P/ZJ1Et76nXIOpkke9rOQdZvIKszTb/Q3EJgSEWk+9CzCRg1quVySbJD/yE7OjbGsR9DM/kjfF3rzzewCj0gpA4rDbon7vtU2WRlJBcuEtwKmpakG1VVeWZ2U75koOIBCNTCXWYK/YK6HK2gQfABJEyY= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230022)(136003)(376002)(346002)(396003)(39830400003)(451199015)(46966006)(36840700001)(82310400005)(36756003)(8936002)(5660300002)(7636003)(2906002)(7596003)(41300700001)(47076005)(6862004)(36916002)(316002)(70206006)(4326008)(42186006)(70586007)(8676002)(356005)(186003)(6266002)(40480700001)(2616005)(26005)(336012)(83380400001)(86362001)(36860700001)(478600001)(966005)(6666004)(38230200001)(14776008)(79816003); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 08:58:46.5833 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e402d2e3-2211-4e5e-edfe-08daf21fb7b1 X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: CWLGBR01FT021.eop-gbr01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO2P265MB5743 X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Yuan Fu writes: > Mickey Petersen writes: > >> If you parse some text, retrieve a node -- using `treesit-node-at', >> for example -- and then edit the buffer, then the node you retrieved >> is marked outdated. >> >> However, tree-sitter is capable of handling that, to a greater or lesser= extent: >> >> https://tree-sitter.github.io/tree-sitter/using-parsers#editing >> >> It is therefore possible to refresh node instances that were created >> _before_ the edit. I suppose it could remain an explicit step that you >> must enter a special form and then Emacs will track node instances >> issued inside that form and refresh them when edits take place inside >> of it. >> >> As it stands, it is very hard to edit and maintain a node registry at >> the same time. (I'm using markers and overlays as a crude hack to work >> around it.) > > This is kind of a limitation of tree-sitter. The "node editing" isn=E2=80= =99t > like what you thought (it fooled me too when I first read it). > Tree-sitter=E2=80=99s incremental parsing works roughly like this: > > 1. You have a parsed tree, TREE, corresponding to some TEXT > 2. You make some edit to the TEXT, eg, TEXT=E2=80=99 =3D insert(TEXT, 1, = "abc") > 3. Now you need to "edit" the old tree with _positions_ of your edit: > edit(TREE, Insert(pos=3D1, len=3D3)) (Notice that this modifies the tree = in-place.) > 4. You reparse the edited tree and gets a new tree: > TREE=E2=80=99 =3D parse(TREE, TEXT=E2=80=99) (Notice that this returns a = new tree.) > > If you have a NODE from TREE, editing that node only updates position > information. That corresponds to the eidt(TREE, ...) step. There is no > equivalent of the parse(TREE, TEXT=E2=80=99) step for nodes: once the tre= e is > reparsed and a new tree is returned, none of the nodes in the old tree > gets carried to the new tree. In practice, tree-sitter reuses old tree=E2= =80=99s > data, but conceptually the old and new tree don=E2=80=99t share any node. > > IOW, the editing feature for nodes is for very specific situations, > where you edit the parse tree but didn=E2=80=99t reparse yet. In this cas= e, if > you want to make your node=E2=80=99s positions to be correct, you edit th= e node. > But once you reparse, there is no way to somehow "update" this old node > into its "equivalent" in the new tree. > > I=E2=80=99m not sure whether tree-sitter is capable to do what you want (= after > all the old and new tree are sharing data). But currently it doesn=E2=80= =99t > expose the feature to do that. > That's a shame. The documentation is a little bit ambiguous then. But if th= e library returns a brand-new tree and thus nodes, then I can see why this = won't work. One possible workaround is that outdated nodes are proxies for their underlying data (node type, range, text, anonymous/named) so that their actual state is kept around. That will allow `equal' checks to still succeed on an outdated and a "brand-new, but identical" node. Food for thought. > Yuan From unknown Sat Jun 21 10:40:29 2025 X-Loop: help-debbugs@gnu.org Subject: bug#60656: 30.0.50; tree-sitter: editing a buffer invalidates visited node instances References: <87o7r9jnbz.fsf@masteringemacs.org> In-Reply-To: <87o7r9jnbz.fsf@masteringemacs.org> Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 09 Jan 2023 20:31:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60656 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Mickey Petersen Cc: 60656@debbugs.gnu.org Received: via spool by 60656-submit@debbugs.gnu.org id=B60656.16732962342729 (code B ref 60656); Mon, 09 Jan 2023 20:31:01 +0000 Received: (at 60656) by debbugs.gnu.org; 9 Jan 2023 20:30:34 +0000 Received: from localhost ([127.0.0.1]:38291 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEymv-0000hx-IK for submit@debbugs.gnu.org; Mon, 09 Jan 2023 15:30:34 -0500 Received: from mail-pl1-f181.google.com ([209.85.214.181]:45572) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pEymt-0000he-Gr for 60656@debbugs.gnu.org; Mon, 09 Jan 2023 15:30:32 -0500 Received: by mail-pl1-f181.google.com with SMTP id g16so10811568plq.12 for <60656@debbugs.gnu.org>; Mon, 09 Jan 2023 12:30:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=E0McAiH6Tpy0ic6XLfk9PuHiwL6dq14nmuXDU2jP3XU=; b=AgcD/xhbTnEEGe4jRr5mW7dS9vnX5Hx0QKvYbwJtva6aeTmne++8BEN25QO8mB3LV2 bAdBRuMPH79qfWsX0k3Y+j4DPLXKZwRw3p0CVwcRagfbh9dWIspNyUSujHZ3AWZtXHm9 yuT3tkzSdFNZ6sPFkj3XFRFztDY4N3ov5+QufaR0FYToWoA7x24YNvNFAzApff5bYciZ jJYMDabRqtjABrxRixwcxOlL4RI6d3a2wgfyv6UMH6wmfC8A6cuJtVPO8s6dansaMEb8 CJIhF4d4C3aq6fBam2dUqIX2ul4gkA42hEhAjT5bwHGcgZuYSOtaIKNAUq89If5nR8T4 9eTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E0McAiH6Tpy0ic6XLfk9PuHiwL6dq14nmuXDU2jP3XU=; b=rm/iip/vPhGZsUWnphJuOqQXT6fcOpi7zaFjxxPVUd9o9cdBcdlXOXAD6uujk6AAXq gA3FMH6ztKczMw2TnICYkwz97wrM0AlcLAH0BCc800qLECtD2+wxEW6gVwlqE6fG3mVy x8Y5+oTw4duWtFa2NvBiD9EvUHtE+OZjZ4lYCkYS1cUI8N7bNrkwBiAXvltv6BOFJafi E2yAhqQ6Yxvuof+b89FL9ZulCX03Pu8g2SX0alNfmc3WtuCndyFJBRam+TP3AWxR+FvO uVyZhBbjjI8x4Ubvtirz/WFohEni4Jdg/QsMALqRbREGGy47ODg1V3KmOjwgrIoylMcV +/UQ== X-Gm-Message-State: AFqh2kpUWvgyfgqAQYzLyD3fQ2tCNqQuoU18rV0AFdyDnm6pxoU4iz6N 9L8LvY0xbFK+YbvAg/t95cOdP0a361k= X-Google-Smtp-Source: AMrXdXsHQmgAcuZG0rvF91SAnCcJK28mHpA++yVUu2Am9hnFSIDe2nbWXgJaj8Y4i+4xNZ2Ee8L2BQ== X-Received: by 2002:a17:902:b085:b0:192:d5dc:c84b with SMTP id p5-20020a170902b08500b00192d5dcc84bmr24377538plr.50.1673296225355; Mon, 09 Jan 2023 12:30:25 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id m11-20020a170902db0b00b00186a2274382sm6590308plx.76.2023.01.09.12.30.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Jan 2023 12:30:24 -0800 (PST) From: Yuan Fu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Message-Id: <579F5BE1-4735-4B8F-9CCE-A9FD9BA24422@gmail.com> Date: Mon, 9 Jan 2023 12:30:20 -0800 X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Mickey Petersen writes: > Yuan Fu writes: > >> Mickey Petersen writes: >> >>> If you parse some text, retrieve a node -- using `treesit-node-at', >>> for example -- and then edit the buffer, then the node you retrieved >>> is marked outdated. >>> >>> However, tree-sitter is capable of handling that, to a greater or = lesser extent: >>> >>> https://tree-sitter.github.io/tree-sitter/using-parsers#editing >>> >>> It is therefore possible to refresh node instances that were created >>> _before_ the edit. I suppose it could remain an explicit step that = you >>> must enter a special form and then Emacs will track node instances >>> issued inside that form and refresh them when edits take place = inside >>> of it. >>> >>> As it stands, it is very hard to edit and maintain a node registry = at >>> the same time. (I'm using markers and overlays as a crude hack to = work >>> around it.) >> >> This is kind of a limitation of tree-sitter. The "node editing" = isn=E2=80=99t >> like what you thought (it fooled me too when I first read it). >> Tree-sitter=E2=80=99s incremental parsing works roughly like this: >> >> 1. You have a parsed tree, TREE, corresponding to some TEXT >> 2. You make some edit to the TEXT, eg, TEXT=E2=80=99 =3D insert(TEXT, = 1, "abc") >> 3. Now you need to "edit" the old tree with _positions_ of your edit: >> edit(TREE, Insert(pos=3D1, len=3D3)) (Notice that this modifies the = tree in-place.) >> 4. You reparse the edited tree and gets a new tree: >> TREE=E2=80=99 =3D parse(TREE, TEXT=E2=80=99) (Notice that this = returns a new tree.) >> >> If you have a NODE from TREE, editing that node only updates position >> information. That corresponds to the eidt(TREE, ...) step. There is = no >> equivalent of the parse(TREE, TEXT=E2=80=99) step for nodes: once the = tree is >> reparsed and a new tree is returned, none of the nodes in the old = tree >> gets carried to the new tree. In practice, tree-sitter reuses old = tree=E2=80=99s >> data, but conceptually the old and new tree don=E2=80=99t share any = node. >> >> IOW, the editing feature for nodes is for very specific situations, >> where you edit the parse tree but didn=E2=80=99t reparse yet. In this = case, if >> you want to make your node=E2=80=99s positions to be correct, you = edit the node. >> But once you reparse, there is no way to somehow "update" this old = node >> into its "equivalent" in the new tree. >> >> I=E2=80=99m not sure whether tree-sitter is capable to do what you = want (after >> all the old and new tree are sharing data). But currently it = doesn=E2=80=99t >> expose the feature to do that. >> > > That's a shame. The documentation is a little bit ambiguous then. But > if the library returns a brand-new tree and thus nodes, then I can see > why this won't work. Yeah I wish tree-sitter can have it. Maybe you can raise an issue on tree-sitter=E2=80=99s github. The author seems to be rather busy, = though. > One possible workaround is that outdated nodes are proxies for their > underlying data (node type, range, text, anonymous/named) so that > their actual state is kept around. That will allow `equal' checks to > still succeed on an outdated and a "brand-new, but identical" node. > > Food for thought. If you can describe what high-level feature you want to accomplish (with node update), maybe I can provide some suggestions. Yuan