From unknown Sat Jun 14 03:54:09 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#61369 <61369@debbugs.gnu.org> To: bug#61369 <61369@debbugs.gnu.org> Subject: Status: Problem with keeping tree-sitter parse tree up-to-date Reply-To: bug#61369 <61369@debbugs.gnu.org> Date: Sat, 14 Jun 2025 10:54:09 +0000 retitle 61369 Problem with keeping tree-sitter parse tree up-to-date reassign 61369 emacs submitter 61369 Dmitry Gutov severity 61369 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 08 10:34:40 2023 Received: (at submit) by debbugs.gnu.org; 8 Feb 2023 15:34:40 +0000 Received: from localhost ([127.0.0.1]:56349 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPmT0-0003Ni-IH for submit@debbugs.gnu.org; Wed, 08 Feb 2023 10:34:40 -0500 Received: from lists.gnu.org ([209.51.188.17]:39434) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPmSv-0003NV-6r for submit@debbugs.gnu.org; Wed, 08 Feb 2023 10:34:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPmSs-0001jw-Q4 for bug-gnu-emacs@gnu.org; Wed, 08 Feb 2023 10:34:30 -0500 Received: from mail-ed1-x530.google.com ([2a00:1450:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pPmSq-00011O-Ok for bug-gnu-emacs@gnu.org; Wed, 08 Feb 2023 10:34:30 -0500 Received: by mail-ed1-x530.google.com with SMTP id eq11so20982029edb.6 for ; Wed, 08 Feb 2023 07:34:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:sender:from:to:cc:subject :date:message-id:reply-to; bh=8yqFsIaAtzWlw0v4W8DbnWzPEdLaCuUIMw05yJbF4ho=; b=SzEHqXMDtQCbl8Ggfkoxa8FG+UJZP/7Gobr0E8KT2+iz7olwp4NCdaOjlWZa9EVAyq ZQIRMAuwpPKEEC1znUeRM925NBlHN0C7/0LIPQLySAlQNQFIy/bROe6YIfMtEDW96iOI 1e/JEreWPVFZAVQsfIkkEYNeOfcEoIvnRTSCe/K1sdYiJJeefFYRBpzP9edDguZrG0d4 lkswcsuZf3FYVTZsJofF0bx409yIramg7gJlcSpPWthFEoLZKQBdfrObIFpI7TaBWAVd C8lPjBpxjAulwG/ahV3vbTw9woWFz6Pqhb4Uus03ad/ng1LNg1bgDsxU5k7fR8jN8eGJ VuGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8yqFsIaAtzWlw0v4W8DbnWzPEdLaCuUIMw05yJbF4ho=; b=nTD0pLZ10hONZyjWh4geW4LA++j3CBW3ffOzblRPiQK/BKncgpayX3F9ig19z+uM4K S0mp7KouTT+icOzlYt0aZ7T9YO6ij32a1M/qnw/2c6SQhCw3nLfuShCwDE7pjckIWgz9 otTU3KYTHh/e00CbEqGHsNdqTQEbS/EU3SmeUZTiqTMrWIgJcDXLFOAHeZ+FuP84meg/ be4RxPdXhoZCzXvwTBzorLMuZySQLxq4hi2cCg9+/ks7CzBzl6DNV1pqwkSItdfqTtBu t54T0Ug00J764ar8iiTYLUmOcsU9RX/C3XX2p70ym1dqNxmyRKmC3GI4eZfINr4rp/CL +F6g== X-Gm-Message-State: AO0yUKVkFteKJXk8h/wN7EWozVsCb5BE1/108rEfSPxNHwAEENT/c3II KA8UxzOBvYQmF5tdVpDdlv+MbaYznLQ= X-Google-Smtp-Source: AK7set8Y0yDjQAC3THEs6DFDILHcW58fOH/BaChq7Wgg5U7A9xrKUJGlhElstTcnjmqmlhfQJzDNdA== X-Received: by 2002:a50:9b55:0:b0:4aa:b3f2:726c with SMTP id a21-20020a509b55000000b004aab3f2726cmr7697374edj.30.1675870467019; Wed, 08 Feb 2023 07:34:27 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id q20-20020a50c354000000b004aacd6b88ccsm2348171edb.90.2023.02.08.07.34.26 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 08 Feb 2023 07:34:26 -0800 (PST) Message-ID: Date: Wed, 8 Feb 2023 17:34:25 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Content-Language: en-US To: bug-gnu-emacs@gnu.org From: Dmitry Gutov Subject: Problem with keeping tree-sitter parse tree up-to-date Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=2a00:1450:4864:20::530; envelope-from=raaahh@gmail.com; helo=mail-ed1-x530.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) 1. Have some buffer with text "use std::Path::{self, Path, PathBuf}; // good: std is a crate name ... (maybe something else " 2. Have this text in a different buffer (I used scratch): " let date = DateTime::::from_utc(date, chrono::Utc); " Meaning, the buffer starts with two empty lines. 3. Select the first line from the first buffer including the trailing newline, yank and then paste at the beginning of the second buffer. The second buffer will now look like this: "use std::Path::{self, Path, PathBuf}; // good: std is a crate name let date = DateTime::::from_utc(date, chrono::Utc); " The parse tree will, however, be (according to treesit-explore-mode): (source_file (use_declaration use argument: (scoped_use_list path: (scoped_identifier path: (identifier) :: name: (identifier)) :: list: (use_list { (self) , (identifier) , (identifier) })) ;) (line_comment) (let_declaration let pattern: (identifier) = value: (scoped_identifier path: (generic_type type: (type_identifier) :: type_arguments: (type_arguments < (scoped_type_identifier path: (identifier) :: name: (type_identifier)) >)) :: name: (identifier)) (ERROR ( (identifier) , (scoped_identifier path: (identifier) :: name: (identifier)) ;) ;)) But if I edit the buffer after that, e.g. add and remove a space at the beginning, the error node disappears: (source_file (use_declaration use argument: (scoped_use_list path: (scoped_identifier path: (identifier) :: name: (identifier)) :: list: (use_list { (self) , (identifier) , (identifier) })) ;) (line_comment) (let_declaration let pattern: (identifier) = value: (call_expression function: (scoped_identifier path: (generic_type type: (type_identifier) :: type_arguments: (type_arguments < (scoped_type_identifier path: (identifier) :: name: (type_identifier)) >)) :: name: (identifier)) arguments: (arguments ( (identifier) , (scoped_identifier path: (identifier) :: name: (identifier)) ))) ;)) From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 08 13:20:30 2023 Received: (at 61369) by debbugs.gnu.org; 8 Feb 2023 18:20:30 +0000 Received: from localhost ([127.0.0.1]:56626 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPp3V-0007wL-UV for submit@debbugs.gnu.org; Wed, 08 Feb 2023 13:20:30 -0500 Received: from out-82.mta0.migadu.com ([91.218.175.82]:46690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPp3Q-0007w4-Qv for 61369@debbugs.gnu.org; Wed, 08 Feb 2023 13:20:28 -0500 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thornhill.no; s=key1; t=1675880422; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pIlYZm5qQNMOWU+xFrEavNMrA/CrdoxlGwNGeB8Bjh0=; b=SaXuDB82p6AsP7xp54IclzxoqUQow200lreYrVZLC7PnI4ucT0WYT937YnqzWaw5Uieq4U bOW2g4topxKsT3sRZ/Ove8UZiK16XU+UndINSFsyf4ocw4fE3SWI+TltecUw2WHBbc91Ei 3d36PoX3BegYdW6wcJXwahiT0QZygQpbglanhgAxDU58L/zi4xM795Sycv1zo+tEn/Eskw BriZ04dBqHP3PDzkf5r2BTQG+8zj03ET2VRn0fQju5EoqOGS+8bTk5UCZj5Qdk8/9Mz/oc gVAAopUk1jkEkIGmNas1D5aTWou5QBYoYdxAsSmY7HzWo9Sx8Jfhb80ztgSYSQ== From: Theodor Thornhill To: Dmitry Gutov Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date In-Reply-To: (Dmitry Gutov's message of "Wed, 8 Feb 2023 17:34:25 +0200") References: Date: Wed, 08 Feb 2023 19:20:19 +0100 Message-ID: <87ttzw6mu4.fsf@thornhill.no> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Flow: FLOW_OUT X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Dmitry Gutov writes: > 1. Have some buffer with text > > "use std::Path::{self, Path, PathBuf}; // good: std is a crate name > ... (maybe something else > " > > 2. Have this text in a different buffer (I used scratch): > > " > > > let date = DateTime::::from_utc(date, chrono::Utc); > " > > Meaning, the buffer starts with two empty lines. > > 3. Select the first line from the first buffer including the trailing newline, > yank and then paste at the beginning of the second buffer. > > The second buffer will now look like this: > > "use std::Path::{self, Path, PathBuf}; // good: std is a crate name > > > let date = DateTime::::from_utc(date, chrono::Utc); > " I just want to confirm that I can reproduce this, and that if you skip the trailing newline from the use-statement, I don't get this behavior. So it seems like the newline is the crucial point, right? Theo From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 08 14:41:20 2023 Received: (at 61369) by debbugs.gnu.org; 8 Feb 2023 19:41:20 +0000 Received: from localhost ([127.0.0.1]:56715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPqJk-0001fy-HV for submit@debbugs.gnu.org; Wed, 08 Feb 2023 14:41:20 -0500 Received: from forward501b.mail.yandex.net ([178.154.239.145]:42360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pPqJh-0001fm-L4 for 61369@debbugs.gnu.org; Wed, 08 Feb 2023 14:41:19 -0500 Received: from myt5-f35de42dd9a5.qloud-c.yandex.net (myt5-f35de42dd9a5.qloud-c.yandex.net [IPv6:2a02:6b8:c00:27aa:0:640:f35d:e42d]) by forward501b.mail.yandex.net (Yandex) with ESMTP id 1A0165F08C; Wed, 8 Feb 2023 22:41:15 +0300 (MSK) Received: from mail.yandex.ru (mail.yandex.ru [80.244.28.130]) by myt5-f35de42dd9a5.qloud-c.yandex.net (mxback/Yandex) with HTTP id 3fjuOY0V34Y1-ZD14Idmo; Wed, 08 Feb 2023 22:41:14 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1675885274; bh=e7RFCXiomqJXXUj0wn+6ZZqe9KeGNQR0dqMCXQat3Ao=; h=Message-Id:References:Date:Cc:Subject:In-Reply-To:To:From; b=knSOEoHbBeDHYp3EUZ41AgcxKW/zGSEsqbJj/1KyaiG1wHzWlzFYtrgji1787ZkuV Zxwjf/KMtmTg+lHXeCWRism3cyea227GXYkdPq23tcZqKwjhRImyv5B9+2gsWzXqXs 2tbesvFec7c3o2JO74NKxiMaILv3EapP4/EkA/tg= Authentication-Results: myt5-f35de42dd9a5.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by 66ftscwz67pcsni4.myt.yp-c.yandex.net with HTTP; Wed, 08 Feb 2023 22:41:14 +0300 From: Dmitry Gutov To: Theodor Thornhill In-Reply-To: <87ttzw6mu4.fsf@thornhill.no> References: <87ttzw6mu4.fsf@thornhill.no> Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Wed, 08 Feb 2023 22:41:14 +0300 Message-Id: <3309321675885274@66ftscwz67pcsni4.myt.yp-c.yandex.net> Content-Transfer-Encoding: base64 Content-Type: text/html; charset=utf-8 X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: 61369 Cc: "61369@debbugs.gnu.org" <61369@debbugs.gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.3 (/) PGJyIC8+PGJyIC8+MjA6MjEsIDggRmVicnVhcnkgMjAyMywgIlRoZW9kb3IgVGhvcm5oaWxsIHZp YSBCdWcgcmVwb3J0cyBmb3IgR05VIEVtYWNzLCB0aGUgU3dpc3MgYXJteSBrbmlmZSBvZiB0ZXh0 IGVkaXRvcnMiICZsdDtidWctZ251LWVtYWNzQGdudS5vcmcmZ3Q7OjxiciAvPjxibG9ja3F1b3Rl IGNsYXNzPSIyMTBlN2E4NDhlOGZjYjQ1d21pLXF1b3RlIj48cD5EbWl0cnkgR3V0b3YgJmx0Ozxh IGhyZWY9Im1haWx0bzpkZ3V0b3ZAeWFuZGV4LnJ1Ij5kZ3V0b3ZAeWFuZGV4LnJ1PC9hPiZndDsg d3JpdGVzOjxiciAvPjxiciAvPjwvcD48YmxvY2txdW90ZSBjbGFzcz0iMjEwZTdhODQ4ZThmY2I0 NXdtaS1xdW90ZSI+wqAxLiBIYXZlIHNvbWUgYnVmZmVyIHdpdGggdGV4dDxiciAvPjxiciAvPsKg InVzZSBzdGQ6OlBhdGg6OntzZWxmLCBQYXRoLCBQYXRoQnVmfTsgIC8vIGdvb2Q6IHN0ZCBpcyBh IGNyYXRlIG5hbWU8YnIgLz7CoC4uLiAobWF5YmUgc29tZXRoaW5nIGVsc2U8YnIgLz7CoCI8YnIg Lz48YnIgLz7CoDIuIEhhdmUgdGhpcyB0ZXh0IGluIGEgZGlmZmVyZW50IGJ1ZmZlciAoSSB1c2Vk IHNjcmF0Y2gpOjxiciAvPjxiciAvPsKgIjxiciAvPjxiciAvPjxiciAvPsKgbGV0IGRhdGUgPSBE YXRlVGltZTo6Jmx0O2Nocm9ubzo6VXRjJmd0Ozo6ZnJvbV91dGMoZGF0ZSwgY2hyb25vOjpVdGMp OzxiciAvPsKgIjxiciAvPjxiciAvPsKgTWVhbmluZywgdGhlIGJ1ZmZlciBzdGFydHMgd2l0aCB0 d28gZW1wdHkgbGluZXMuPGJyIC8+PGJyIC8+wqAzLiBTZWxlY3QgdGhlIGZpcnN0IGxpbmUgZnJv bSB0aGUgZmlyc3QgYnVmZmVyIGluY2x1ZGluZyB0aGUgdHJhaWxpbmcgbmV3bGluZSw8YnIgLz7C oHlhbmsgYW5kIHRoZW4gcGFzdGUgYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgc2Vjb25kIGJ1ZmZl ci48YnIgLz48YnIgLz7CoFRoZSBzZWNvbmQgYnVmZmVyIHdpbGwgbm93IGxvb2sgbGlrZSB0aGlz OjxiciAvPjxiciAvPsKgInVzZSBzdGQ6OlBhdGg6OntzZWxmLCBQYXRoLCBQYXRoQnVmfTsgIC8v IGdvb2Q6IHN0ZCBpcyBhIGNyYXRlIG5hbWU8YnIgLz48YnIgLz48YnIgLz7CoGxldCBkYXRlID0g RGF0ZVRpbWU6OiZsdDtjaHJvbm86OlV0YyZndDs6OmZyb21fdXRjKGRhdGUsIGNocm9ubzo6VXRj KTs8YnIgLz7CoCI8YnIgLz48L2Jsb2NrcXVvdGU+PHA+PGJyIC8+SSBqdXN0IHdhbnQgdG8gY29u ZmlybSB0aGF0IEkgY2FuIHJlcHJvZHVjZSB0aGlzLCBhbmQgdGhhdCBpZiB5b3Ugc2tpcDxiciAv PnRoZSB0cmFpbGluZyBuZXdsaW5lIGZyb20gdGhlIHVzZS1zdGF0ZW1lbnQsIEkgZG9uJ3QgZ2V0 IHRoaXMgYmVoYXZpb3IuPGJyIC8+U28gaXQgc2VlbXMgbGlrZSB0aGUgbmV3bGluZSBpcyB0aGUg Y3J1Y2lhbCBwb2ludCwgcmlnaHQ/PC9wPjwvYmxvY2txdW90ZT48YnIgLz48ZGl2Plllcywgc2Ft ZS48L2Rpdj48ZGl2PjxiciAvPjwvZGl2PjxkaXY+VGhyIHRyYWlsaW5nIG5ld2xpbmUgaXMgbmVj ZXNzYXJ5LjwvZGl2PjxkaXY+PGJyIC8+PC9kaXY+PGRpdj5UaGUgZW1wdHkgbGluZXMgYXQgdGhl IGJlZ2lubmluZyBvZiB0aGUgYnVmZmVyIChiZWluZyBjb3BpZWQgdG8pIGFyZSBuZWNlc3Nhcnkg dG8gcmVwcm9kdWNlIHRoaXMgYXMgd2VsbC48L2Rpdj4= From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 09 20:22:35 2023 Received: (at 61369) by debbugs.gnu.org; 10 Feb 2023 01:22:36 +0000 Received: from localhost ([127.0.0.1]:33926 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pQI7X-0006C6-Fh for submit@debbugs.gnu.org; Thu, 09 Feb 2023 20:22:35 -0500 Received: from mail-pj1-f41.google.com ([209.85.216.41]:46721) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pQI7V-0006Bq-MT for 61369@debbugs.gnu.org; Thu, 09 Feb 2023 20:22:34 -0500 Received: by mail-pj1-f41.google.com with SMTP id rm7-20020a17090b3ec700b0022c05558d22so4050361pjb.5 for <61369@debbugs.gnu.org>; Thu, 09 Feb 2023 17:22:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=BcLQeueJ5SJFqDV2MQIKM4Bwd+Q3bif7sP9MQzfeCsM=; b=brAGRBYJQ9zji8L/k0C8X2lh31ZvHvW+HeBoxMLG/NLombRFyrx7lKzqyImmtgysGd Ij9StzMK0k3eo5m7TALkI/B2k4efg8rnxegk2a7CER39MjtAh6IMy8o0JbXhyOqgGsWQ izhPMOvHy15pXu0dDysb+0XYvv+E5Kd0YE0EIRwvPLWKdlAumWV5u7OLYClEy1H5TE7k /nDdORd4eFcfoNKOW3OIOPpiGYgC9+6m03q85E3Ke60Pe2DGG60Wb3eprhsOCdkA4sWI ZQKFc9Od5BXWSDcg5JZiQd7cjTVVZxf9Kwf9wA1bSpdi1goV1jsRsV/dDys+zuBLymxG kDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BcLQeueJ5SJFqDV2MQIKM4Bwd+Q3bif7sP9MQzfeCsM=; b=lEcSQHpxlhBrVqeOt14fJYyqsu7yn6fk0UHrZmBeoiJLBtvG7NcofcdZUrQRzVY1DA TXEUocULJxVip3f6tdX20yTF5y5bwfCU6R/RiN6IReWb1xhcT/ip8/sDJODq6Ow5u0/d v5dZrrMu86SFNuVHffwEx386auLTO/XVBySRx0B74/Vsx+X8aQTp4XK5sILRUwnXBPW1 U8Dzv5lx9tDxMFNRCSnsKeCZwtrIUOqDxMx+gEFtfwF859FojG9gbYWFB/3raVrvV7g3 Ay8kC92mHOhWsoWVg8ol1DvnF5NeX3Izs2FdYlEe8d1wzFwiN96c9i/QYTc0QOazj4tb MEWQ== X-Gm-Message-State: AO0yUKXHADnW34BSY0bGozlfbs44V2oOtsfNKFPOuPh1aNh4WFR9OEqT jg4l/ImKjZkB/dxdrX5P6aI= X-Google-Smtp-Source: AK7set+1pUNoSqIbmg4E6fPyzyrt7oJpqJTLdXOdTGvlVXxgtAL5hKglthXN0d1YlADfGlPsQsyweg== X-Received: by 2002:a05:6a20:4293:b0:c3:161a:b954 with SMTP id o19-20020a056a20429300b000c3161ab954mr10273511pzj.44.1675992147692; Thu, 09 Feb 2023 17:22:27 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id q29-20020a638c5d000000b004b4d4de54absm1877912pgn.59.2023.02.09.17.22.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Feb 2023 17:22:27 -0800 (PST) From: Yuan Fu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Message-Id: <934F7D4C-6AFF-4370-8C19-120764E4A372@gmail.com> Date: Thu, 9 Feb 2023 17:22:15 -0800 To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > I just want to confirm that I can reproduce this, and that if you = skip > the trailing newline from the use-statement, I don't get this = behavior. > So it seems like the newline is the crucial point, right? > > Yes, same. > > Thr trailing newline is necessary. > > The empty lines at the beginning of the buffer (being copied to) are = necessary to reproduce this as well. Hmmm, it might be related to how does tree-sitter does incremental parsing? If the newline is necessary, then I guess it=E2=80=99s not = because Emacs missed characters when reporting edits to tree-sitter. Yuan From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 09 20:38:14 2023 Received: (at 61369) by debbugs.gnu.org; 10 Feb 2023 01:38:14 +0000 Received: from localhost ([127.0.0.1]:33960 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pQIMg-0006es-Ed for submit@debbugs.gnu.org; Thu, 09 Feb 2023 20:38:14 -0500 Received: from mail-ej1-f45.google.com ([209.85.218.45]:44965) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pQIMe-0006ec-LF for 61369@debbugs.gnu.org; Thu, 09 Feb 2023 20:38:13 -0500 Received: by mail-ej1-f45.google.com with SMTP id hx15so11981181ejc.11 for <61369@debbugs.gnu.org>; Thu, 09 Feb 2023 17:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=Z+x4N1E9OWOJ4nHb+QKKnADJj7ovcn2rKk0097N/9qU=; b=E+jQXsgY0/px4mRJ/SDr3Yl8H7mRMe3jwsnwuKwlyvN+WNSgLTQ5VvjP3Js0G1bRxo 3zIyzG6E7HfJmOarLRk9j9zYI0n/bdG751oAx6YoO2IpLjyTIISLdCVh/Zs2imRGjFVj hJbrCBb74kcRFVg5Tn3hdiRkaMgU8AmRF36XnRPf3QPKccp0KlGnFowNxRot8rEnbEmE JmrnPoutTj5SnjQugJ3RhybU0/JEO8NJrqH5G9eshZfd4AOGSEb8tdByM/NhVrD9UMQb f2wABuKmp0GT6+Fkf35itQlECjbAFm3d6wqIfdwufK36oNdY0ryvpQ6stA5vG1Hog2Zy gODw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Z+x4N1E9OWOJ4nHb+QKKnADJj7ovcn2rKk0097N/9qU=; b=nmvhJLaqK/V9fq54SBeHQTAzl84daRJX3gWDBZ7lItbkF7ARWXjVfGaQT6rNq42hhY atZojzourJbJnho/Gg/+M1R+0m72pwaKZGTLzhvk8N2gw4t98+Xn4ovBSxKMQwSzud7a ymD16BMTmiiGMTritnPfy60npdyOE9m7/Wdwv57FLAftfZmlzJ9DJ2darfJ9Xyfwj5h2 oNPKVtwvk8e21xDYdFKlWiS0ZYb0/8wfn01wGKf8AEcWKZ0eN9jETjVKDcY+orTc3nYF 0eRfuG3480TcWqpln2SyqbcpjxgN12jvjCyD4TxygMKu5nLj2s/7cD668UvWTungvSo9 vxYA== X-Gm-Message-State: AO0yUKUlaSbJN+dt5mcF5eXyvkuRhQYaAXKxQE2b76EbURmNGarDB+j9 6oTn7gbfAqjADPueDUZVJxQ= X-Google-Smtp-Source: AK7set9mEEeYE493T6Q/BlziNFlagzYPyQUfKkrVxkb0qIz6lGWUn7KXcQwDn5nW+H963tim8hzthQ== X-Received: by 2002:a17:906:fad2:b0:886:221b:44de with SMTP id lu18-20020a170906fad200b00886221b44demr14618445ejb.0.1675993086671; Thu, 09 Feb 2023 17:38:06 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id a6-20020a17090680c600b008aef73e4494sm1645235ejx.108.2023.02.09.17.38.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 09 Feb 2023 17:38:05 -0800 (PST) Message-ID: <6ae6d1c3-8da0-ba37-1cf8-c89cc6660858@yandex.ru> Date: Fri, 10 Feb 2023 03:38:04 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US To: Yuan Fu References: <934F7D4C-6AFF-4370-8C19-120764E4A372@gmail.com> From: Dmitry Gutov In-Reply-To: <934F7D4C-6AFF-4370-8C19-120764E4A372@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 10/02/2023 03:22, Yuan Fu wrote: >> I just want to confirm that I can reproduce this, and that if you skip >> the trailing newline from the use-statement, I don't get this behavior. >> So it seems like the newline is the crucial point, right? >> >> Yes, same. >> >> Thr trailing newline is necessary. >> >> The empty lines at the beginning of the buffer (being copied to) are necessary to reproduce this as well. > Hmmm, it might be related to how does tree-sitter does incremental > parsing? If the newline is necessary, then I guess it’s not because > Emacs missed characters when reporting edits to tree-sitter. The newline is somewhat necessary: the scenario doesn't work, for example, if the pasted text doesn't include the newline but the buffer had an additional (third) one at the top. But the scenario also doesn't work if some other (any) character is removed from the yanked line before pasting: it could be even one after the comment instruction (//). OTOH, if I add an extra char to the yanked line, anywhere, I can skip the newline. E.g. I can paste use std::path::{self, Path, PathBuf}; // good: std is a crate namee without a newline and still see the exact same syntax error. So it looks more like an off-by-one error somewhere. Maybe in our code, but maybe in tree-sitter somewhere. From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 13 04:10:30 2023 Received: (at 61369) by debbugs.gnu.org; 13 Feb 2023 09:10:30 +0000 Received: from localhost ([127.0.0.1]:47626 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pRUr0-0000LU-Aa for submit@debbugs.gnu.org; Mon, 13 Feb 2023 04:10:30 -0500 Received: from mail-pf1-f181.google.com ([209.85.210.181]:44880) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pRUqx-0000Kz-Ff for 61369@debbugs.gnu.org; Mon, 13 Feb 2023 04:10:28 -0500 Received: by mail-pf1-f181.google.com with SMTP id h7so2368722pfc.11 for <61369@debbugs.gnu.org>; Mon, 13 Feb 2023 01:10:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=/kGtskZGqNSgp4Z6oW+T3V0dzNDausqKzAyg7PZgJ3Q=; b=CvA5ADvke7gPzdB4VZ8SCk5uiX6XY8bke04l4chUzGZOdwZp0r6dAIKiL4Z/nHrPSa 14qeFr04hrFSGfyjvnwoRBct5po7+tdzPO6Q/y/fKJT8uH/OZyCHHX5NwKwfFcLfChLA SJUEbHQ1FMVfGL2k9REBMVO7LHyMux2DDDbFVtxKobJr4fTJLGY66C/J+JztZ52aOO/l nD33hZmpyxriEcCGFLAZ4UuEB8DF33H3w4tMLkPa18buhfbaNAEBan9We8GqWX3TUuyW OhvxsxLzaJs09FFQQk38kGr07F23ss+s85OqedfgDaKoueXmeL5AW3cwykGvUccvGao5 F2dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/kGtskZGqNSgp4Z6oW+T3V0dzNDausqKzAyg7PZgJ3Q=; b=3Ml6A5aPCKxxWEs8YVkgi5X4PEaes8wKIuIHx74FENcnCyTVjC85Ld+JfY74vDXy/+ u5Z/3DPBJ+O6FFQCil2L7x/u+K0IjXcn3yOzyEO3NiiV2jxRMMiOvl+sLE/WFdSUgNmh /Po0UpsW0RXt1COwzy8rXf9bvEdz+DohedQbs/47H5pQlIUXWyC5ACVqeFWnpSa+3ebI gmjLy77q1Hq2i5sfaLQKNLbf7U6KaQkfALB1GogdDgkUXr+eZeNKDX9LeG/CfC7Mso1U oXxvePS4Zo9W8fNL5l/ddjlMfuB9IeP1BwxQcOoincsiCKV194YNgZEeG7Zl1l1Z/tPB /sHA== X-Gm-Message-State: AO0yUKXmcj/4sw8E1JUkx7ylDvWTjUv3LZUg0mSIERVEvYW7yiC3g+gU 3JkSYM3KWDQnM4AdT1wrpmA= X-Google-Smtp-Source: AK7set9+WnUpFXHplRy/ZhorDRtHDl0GWRarBbgqy3OyHmSS02/nAV6PuqlHfDtqY47nWmLiPO9LQw== X-Received: by 2002:a62:55c1:0:b0:5a0:c4b6:edd6 with SMTP id j184-20020a6255c1000000b005a0c4b6edd6mr19746290pfb.0.1676279421544; Mon, 13 Feb 2023 01:10:21 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id q22-20020a62ae16000000b005a8aab9ae7esm2429641pff.216.2023.02.13.01.10.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Feb 2023 01:10:20 -0800 (PST) From: Yuan Fu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Message-Id: <83A8EC13-E0E3-4280-8377-516BC23A59C5@gmail.com> Date: Mon, 13 Feb 2023 01:10:05 -0800 To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Dmitry Gutov writes: > On 10/02/2023 03:22, Yuan Fu wrote: >>> I just want to confirm that I can reproduce this, and that if you = skip >>> the trailing newline from the use-statement, I don't get this = behavior. >>> So it seems like the newline is the crucial point, right? >>> >>> Yes, same. >>> >>> Thr trailing newline is necessary. >>> >>> The empty lines at the beginning of the buffer (being copied to) are = necessary to reproduce this as well. >> Hmmm, it might be related to how does tree-sitter does incremental >> parsing? If the newline is necessary, then I guess it=E2=80=99s not = because >> Emacs missed characters when reporting edits to tree-sitter. > > The newline is somewhat necessary: the scenario doesn't work, for > example, if the pasted text doesn't include the newline but the buffer > had an additional (third) one at the top. > > But the scenario also doesn't work if some other (any) character is > removed from the yanked line before pasting: it could be even one > after the comment instruction (//). > > OTOH, if I add an extra char to the yanked line, anywhere, I can skip > the newline. E.g. I can paste > > use std::path::{self, Path, PathBuf}; // good: std is a crate namee > > without a newline and still see the exact same syntax error. > > So it looks more like an off-by-one error somewhere. Maybe in our > code, but maybe in tree-sitter somewhere. Some progress report: I added a function that reads the buffer like a parser would, like this: DEFUN ("treesit--parser-view", Ftreesit__parser_view, Streesit__parser_view, 1, 1, 0, doc: /* Return the view of PARSER. Read buffer like PARSER would into a string and return it. */) (Lisp_Object parser) { const ptrdiff_t visible_beg =3D XTS_PARSER (parser)->visible_beg; const ptrdiff_t visible_end =3D XTS_PARSER (parser)->visible_end; const ptrdiff_t view_len =3D visible_end - visible_beg; char *str_buf =3D xzalloc (view_len + 1); uint32_t read =3D 0; TSPoint pos =3D { 0 }; for (int idx =3D 0; idx < view_len; idx++) { const char *ch =3D treesit_read_buffer (XTS_PARSER (parser), idx, pos, &read); if (read =3D=3D 0) { xfree (str_buf); xsignal1 (Qtreesit_error, make_fixnum (idx)); } else str_buf[idx] =3D *ch; } Lisp_Object ret_str =3D make_string (str_buf, view_len); xfree (str_buf); return ret_str; } After I follow the steps and got the error node, I run this function on the parser, and the returned string looks good. Next I=E2=80=99ll try to log every character actually read by the parser = and see if anything seems fishy. Yuan From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 13 18:59:24 2023 Received: (at 61369) by debbugs.gnu.org; 13 Feb 2023 23:59:24 +0000 Received: from localhost ([127.0.0.1]:51993 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pRijD-0004UK-Ez for submit@debbugs.gnu.org; Mon, 13 Feb 2023 18:59:24 -0500 Received: from mail-pj1-f52.google.com ([209.85.216.52]:38467) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pRijB-0004U3-GH for 61369@debbugs.gnu.org; Mon, 13 Feb 2023 18:59:22 -0500 Received: by mail-pj1-f52.google.com with SMTP id a8-20020a17090a6d8800b002336b48f653so12537910pjk.3 for <61369@debbugs.gnu.org>; Mon, 13 Feb 2023 15:59:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=mNX0orqMOjoF7wfu9T7pQhz4iLGFczXNUl1I3AC4tLQ=; b=mc/g0JgCFAe9fLWFN1xZM+OumEkP5Mmwt3PL3qw6fuVcOudBvplnJ31fb0G5u2SEfr /rOfWY4dTYbur9UB4yg5FjdzdYD7kneG+4fgvAIvDlZwqaF8CA/WyXcfp2DK1ja7/JxL OC7dA9lEybDtF4abefYNLtGBXxgsMmZhvzBXdz3ujBlJXZpeSe2+5UoJFsnysOHIsvo6 tDdxj9WBoiarAFAoPjwuM1AKIa4o3YUIQq4UPhtR4RkfslnaWNjrBss1n1pzOG2lV6l9 ay1zeyUZXYaXRrcviPzNCR8T++oHTFsUS317JFSP6Bq3NtGYY8fuL6aefDbv32DKX6zb 4RVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=mNX0orqMOjoF7wfu9T7pQhz4iLGFczXNUl1I3AC4tLQ=; b=soVnaZEQLEY7QSNUEzTHB06wNEtGnB6cF5mcOZtNvCCcFHDT8GYSPoGuxk8NK1CEve LJ7+JJs6bE8NZXDWL1lUF3fiK+/hctBhURq87IflmLQX05fy/GZJyNYop0l8anociLyC D/Nw/pWhQ/IiBoRSqZUFItYq/ePNzZg9sLzES+KNak57Aqxyr6Iy3jZ0k2xWeZBh5dI7 mXoXlCEvsFcVzqwa8I0tk5FvnitRUA0q9DrHQocHmBju+cgAe84v+fS2QB+0hyuKNg0S aaxKSD5QlKvI4Z/JyTUq/1Bh+O1WXUs2YLgo656e5TxYlm4Sd2p5cfVhZtQjMFgF/2gx U8RA== X-Gm-Message-State: AO0yUKXScHb20t3M3Je2uH9jctR4wsf/o7YqcoBD/jv1Lxcqk96vZ7As 9UXdCpM73SBmBKzdTFfNkfU= X-Google-Smtp-Source: AK7set/nJzqEtwOIUuEN4pZ378ioPU30Y1ZjEJTxZfbAZhDaCbK1pIRaTw3JODq3xVsYTS65UqoZpA== X-Received: by 2002:a17:902:d2d0:b0:19a:5a0d:f760 with SMTP id n16-20020a170902d2d000b0019a5a0df760mr19546784plc.18.1676332755536; Mon, 13 Feb 2023 15:59:15 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id i11-20020a170902eb4b00b001992521f23esm4910784pli.100.2023.02.13.15.59.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Feb 2023 15:59:15 -0800 (PST) From: Yuan Fu Content-Type: multipart/mixed; boundary="Apple-Mail=_C4C62C22-6CBE-42EB-A4C4-AAA5F12BCE0A" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Message-Id: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> Date: Mon, 13 Feb 2023 15:59:02 -0800 To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Apple-Mail=_C4C62C22-6CBE-42EB-A4C4-AAA5F12BCE0A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Yuan Fu writes: > Dmitry Gutov writes: > >> On 10/02/2023 03:22, Yuan Fu wrote: >>>> I just want to confirm that I can reproduce this, and that if you = skip >>>> the trailing newline from the use-statement, I don't get this = behavior. >>>> So it seems like the newline is the crucial point, right? >>>> >>>> Yes, same. >>>> >>>> Thr trailing newline is necessary. >>>> >>>> The empty lines at the beginning of the buffer (being copied to) = are necessary to reproduce this as well. >>> Hmmm, it might be related to how does tree-sitter does incremental >>> parsing? If the newline is necessary, then I guess it=E2=80=99s not = because >>> Emacs missed characters when reporting edits to tree-sitter. >> >> The newline is somewhat necessary: the scenario doesn't work, for >> example, if the pasted text doesn't include the newline but the = buffer >> had an additional (third) one at the top. >> >> But the scenario also doesn't work if some other (any) character is >> removed from the yanked line before pasting: it could be even one >> after the comment instruction (//). >> >> OTOH, if I add an extra char to the yanked line, anywhere, I can skip >> the newline. E.g. I can paste >> >> use std::path::{self, Path, PathBuf}; // good: std is a crate = namee >> >> without a newline and still see the exact same syntax error. >> >> So it looks more like an off-by-one error somewhere. Maybe in our >> code, but maybe in tree-sitter somewhere. > > Some progress report: I added a function that reads the buffer like a > parser would, like this: > > DEFUN ("treesit--parser-view", > Ftreesit__parser_view, > Streesit__parser_view, 1, 1, 0, > doc: /* Return the view of PARSER. > Read buffer like PARSER would into a string and return it. */) > (Lisp_Object parser) > { > const ptrdiff_t visible_beg =3D XTS_PARSER (parser)->visible_beg; > const ptrdiff_t visible_end =3D XTS_PARSER (parser)->visible_end; > const ptrdiff_t view_len =3D visible_end - visible_beg; > > char *str_buf =3D xzalloc (view_len + 1); > uint32_t read =3D 0; > TSPoint pos =3D { 0 }; > for (int idx =3D 0; idx < view_len; idx++) > { > const char *ch =3D treesit_read_buffer (XTS_PARSER (parser), > idx, pos, &read); > if (read =3D=3D 0) > { > xfree (str_buf); > xsignal1 (Qtreesit_error, make_fixnum (idx)); > } > else > str_buf[idx] =3D *ch; > } > Lisp_Object ret_str =3D make_string (str_buf, view_len); > xfree (str_buf); > return ret_str; > } > > After I follow the steps and got the error node, I run this function = on > the parser, and the returned string looks good. > > Next I=E2=80=99ll try to log every character actually read by the = parser and see > if anything seems fishy. I don=E2=80=99t know if it=E2=80=99s good news or bad news, but it = doesn=E2=80=99t seem like a off-by-one. Here is what I did: 1. I applied the attached patch (patch.diff) so that = treesit_read_buffer, the function used by tree-sitter parser to read buffer contents, prints the position it read and the character it gets to stdout. 2. I open test.rs which contains " let date =3D DateTime::::from_utc(date, chrono::Utc); " as in the recipe. I have rust-ts-mode enabled, so Emacs prints the characters read by the parser to stdout. I type return several times to separate this first batch of output from the next, which is what I=E2=80=99= m interested in. 3. I paste "use std::Path::{self, Path, PathBuf}; // good: std is a crate name " at the beginning of the buffer. Now the parse tree contains that error node. I go to the terminal, copy the output out, which looks like: 0 117 1 115 2 101 3 32 0 117 1 115 2 101 ... 133 59 134 10 134 10 134 10 134 10 4. I paste this output (output.txt) into a buffer, and reconstruct the = text read by the parser with (setq str (reconstruct)), where reconstruct is: (defun reconstruct () (goto-char (point-min)) (let ((result "")) (while (< (point) (point-max)) (let* ((str (buffer-substring (point) (line-end-position))) (nums (string-split str)) (pos (string-to-number (car nums))) (char (string-to-number (cadr nums)))) (when (not (< pos (length result))) (setq result (concat result (make-string (- (1+ pos) (length result)) ?0)))) (setf (aref result pos) char)) (forward-line 1)) result)) 5. I insert str into a new buffer, and (to my disappointment) the content is identical to the buffer text. There are two surprises here: 1) there isn=E2=80=99t an off-by-one bug, = 2) the parser actually read the whole buffer, rather than reading only the new content. Then there are even less reason for it to create that error node. In addition, I inserted a new line in the Rust source buffer (test.rs) = (which fixes the error node), here is what the parser read after that insertion: "0000000000000000000000000000000000000000000000000000000000000000000 let 0000 =3D 000000000000000000000000000000000000000000000000000);" 0 means it didn=E2=80=99t read that position, we can see that the parser = read all the newlines, "let ", " =3D ", and ");". I can=E2=80=99t discern = anything interesting from that, tho. Yuan --Apple-Mail=_C4C62C22-6CBE-42EB-A4C4-AAA5F12BCE0A Content-Disposition: attachment; filename=output.txt Content-Type: text/plain; x-unix-mode=0644; name="output.txt" Content-Transfer-Encoding: 7bit 0 117 1 115 2 101 3 32 0 117 1 115 2 101 3 32 4 115 3 32 4 115 5 116 6 100 7 58 4 115 5 116 6 100 7 58 8 58 9 80 10 97 11 116 12 104 13 58 9 80 13 58 14 58 15 123 16 115 17 101 18 108 19 102 20 44 16 115 17 101 18 108 19 102 20 44 21 32 22 80 21 32 22 80 23 97 24 116 25 104 26 44 22 80 26 44 27 32 28 80 27 32 28 80 29 97 30 116 31 104 32 66 33 117 34 102 35 125 28 80 35 125 36 59 37 32 38 32 39 47 40 47 37 32 38 32 39 47 40 47 41 32 42 103 43 111 44 111 45 100 46 58 47 32 48 115 49 116 50 100 51 32 52 105 53 115 54 32 55 97 56 32 57 99 58 114 59 97 60 116 61 101 62 32 63 110 64 97 65 109 66 101 67 10 68 10 69 10 70 108 67 10 68 10 69 10 70 108 71 101 72 116 73 32 70 108 71 101 72 116 73 32 74 100 73 32 74 100 75 97 76 116 77 101 78 32 74 100 75 97 78 32 79 61 78 32 79 61 80 32 81 68 80 32 81 68 82 97 83 116 84 101 85 84 86 105 87 109 88 101 89 58 81 68 89 58 90 58 91 60 92 99 93 104 94 114 95 111 96 110 97 111 98 58 92 99 93 104 94 114 98 58 99 58 100 85 101 116 102 99 103 62 100 85 103 62 104 58 105 58 106 102 107 114 108 111 109 109 110 95 111 117 112 116 113 99 114 40 106 102 107 114 114 40 115 100 116 97 117 116 118 101 119 44 115 100 116 97 119 44 120 32 121 99 120 32 121 99 122 104 123 114 124 111 125 110 126 111 127 58 121 99 122 104 123 114 127 58 128 58 129 85 130 116 131 99 132 41 129 85 133 59 134 10 132 41 133 59 134 10 134 10 134 10 134 10 --Apple-Mail=_C4C62C22-6CBE-42EB-A4C4-AAA5F12BCE0A Content-Disposition: attachment; filename=patch.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="patch.diff" Content-Transfer-Encoding: 7bit diff --git a/src/treesit.c b/src/treesit.c index cab2f0d5354..ad87a6ae759 100644 --- a/src/treesit.c +++ b/src/treesit.c @@ -1101,6 +1101,13 @@ treesit_read_buffer (void *parser, uint32_t byte_index, assertion should never hit. */ eassert (len < UINT32_MAX); *bytes_read = (uint32_t) len; + + if (*bytes_read > 0) + { + printf ("%d %d\n", byte_index, *beg); + fflush (stdout); + } + return beg; } @@ -3432,6 +3439,37 @@ DEFUN ("treesit-subtree-stat", } } +DEFUN ("treesit--parser-view", + Ftreesit__parser_view, + Streesit__parser_view, 1, 1, 0, + doc: /* Return the view of PARSER. +Read buffer like PARSER would into a string and return it. */) + (Lisp_Object parser) +{ + const ptrdiff_t visible_beg = XTS_PARSER (parser)->visible_beg; + const ptrdiff_t visible_end = XTS_PARSER (parser)->visible_end; + const ptrdiff_t view_len = visible_end - visible_beg; + + char *str_buf = xzalloc (view_len + 1); + uint32_t read = 0; + TSPoint pos = { 0 }; + for (int idx = 0; idx < view_len; idx++) + { + const char *ch = treesit_read_buffer (XTS_PARSER (parser), + idx, pos, &read); + if (read == 0) + { + xfree (str_buf); + xsignal1 (Qtreesit_error, make_fixnum (idx)); + } + else + str_buf[idx] = *ch; + } + Lisp_Object ret_str = make_string (str_buf, view_len); + xfree (str_buf); + return ret_str; +} + #endif /* HAVE_TREE_SITTER */ DEFUN ("treesit-available-p", Ftreesit_available_p, @@ -3633,6 +3671,8 @@ syms_of_treesit (void) defsubr (&Streesit_search_forward); defsubr (&Streesit_induce_sparse_tree); defsubr (&Streesit_subtree_stat); + + defsubr (&Streesit__parser_view); #endif /* HAVE_TREE_SITTER */ defsubr (&Streesit_available_p); } --Apple-Mail=_C4C62C22-6CBE-42EB-A4C4-AAA5F12BCE0A-- From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 14 21:17:39 2023 Received: (at 61369) by debbugs.gnu.org; 15 Feb 2023 02:17:39 +0000 Received: from localhost ([127.0.0.1]:57483 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pS7MZ-0002RL-2G for submit@debbugs.gnu.org; Tue, 14 Feb 2023 21:17:39 -0500 Received: from mail-ej1-f45.google.com ([209.85.218.45]:36463) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pS7MY-0002R8-1Z for 61369@debbugs.gnu.org; Tue, 14 Feb 2023 21:17:38 -0500 Received: by mail-ej1-f45.google.com with SMTP id a3so10697291ejb.3 for <61369@debbugs.gnu.org>; Tue, 14 Feb 2023 18:17:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=4rOg51csOaz7slIQfB+JUREFaCifPXSm/P7mlS7Vd6A=; b=qDvOn90YAWYI8ANEjwPHJvw0x5CXZnuoXHFPN8rt/nFiZMi7JDXNyA/Y+n9eG26tUm V0rO0gl7NGPXJQPSuzHElSHaegmUgopgBh0wbehU0XS1kfyo0VBnhwtx5/6xmapCFhgF 5f8CUds18OxFTbjmgcoNyLyab0ZPiAMSL4RP4DKufhWA9Y9mxoeluLGZbucldpGLmlAl Xon3GLCHxEyIucfHW1SE/1oFGarcsM3YKaItGhYM4pchDQhGm/gIOmyvG947Snf9BAjB zwCuX+9Umw0USHYX9I6TtIZ8/YVd+zwuZjmZDFwVAawhKumGNc67L1/LTo/sgbZOB71T M4AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4rOg51csOaz7slIQfB+JUREFaCifPXSm/P7mlS7Vd6A=; b=n+CaZbbIPq8sZwvT2Burq9PV0SrzYauzzHrBkhqzU6S+4IHhUfNL7KkTdJU8X3uRCv Sv6Lp8VC00MvjRcyeXFaSXJfqBVYHvnG8WcvpJHKkNaL5Kno0x6wb62eNd4sQ4Sb/tAu ggv+fVH64C1C4D8F+VhJEMEKXsh+5tCVduvEie8mpS7AUHWMzHUAvYuzGZUkXvWhRdci kbfOH576adc3BIi/yCQN/7ctdsGy/Rd4gTHh8iHIDYSH108nbwUpycx2KDHvf89MioUb Mx75efHEDuM+Fyb93u1/GcNcEkaZPSImCk9hb3I5sNFhmtY7xcmd0QgjwEW4x7mriE3U GhVw== X-Gm-Message-State: AO0yUKWtwPveQYUoCE6f/oFDPJCUiEd/NpBmcUjJxMf0VhE2wfNOJyv9 Is9y6X6pKnk+tG9wHkgw5/U= X-Google-Smtp-Source: AK7set+nLfPkcRxhD0BL+t0CBphnN7qLi1w+01NA+LQ266cmhi/sOosiCzLGv2N8M51AyPoz9N3fFw== X-Received: by 2002:a17:906:850e:b0:7c1:765:9cfc with SMTP id i14-20020a170906850e00b007c107659cfcmr961666ejx.34.1676427451945; Tue, 14 Feb 2023 18:17:31 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id bp8-20020a170907918800b008806a3c22c5sm1876978ejb.25.2023.02.14.18.17.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Feb 2023 18:17:30 -0800 (PST) Message-ID: <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> Date: Wed, 15 Feb 2023 04:17:29 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US To: Yuan Fu References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> From: Dmitry Gutov In-Reply-To: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 14/02/2023 01:59, Yuan Fu wrote: > There are two surprises here: 1) there isn’t an off-by-one bug, 2) the > parser actually read the whole buffer, rather than reading only the new > content. Then there are even less reason for it to create that error > node. The parser reads the whole buffer, but if it tries to reparse based on the previous parse tree with incorrect positions, it might get into an invalid state as a result. I've tried gdb-ing treesit_tree_edit_1 (after dropping the 'inline' qualifier), and here's what I see: - If I paste the test line without the trailing newline or not, the value. - If I paste the test line with the trailing newline, the value of new_end_byte is still 67. But then it is followed by this call right away: Thread 1 "emacs" hit Breakpoint 3, treesit_tree_edit_1 (tree=tree@entry=0x5555574139b0, start_byte=start_byte@entry=134, old_end_byte=old_end_byte@entry=134, new_end_byte=135) at treesit.c:739 - If I 'undo' after that, the call is as expected: Thread 1 "emacs" hit Breakpoint 3, treesit_tree_edit_1 (tree=0x555557435cd0, start_byte=start_byte@entry=0, old_end_byte=old_end_byte@entry=68, new_end_byte=new_end_byte@entry=0) at treesit.c:739 739 { So I tried again to figure out the odd call, with the backtrace: Thread 1 "emacs" hit Breakpoint 3, treesit_tree_edit_1 (tree=tree@entry=0x5555575b64f0, start_byte=start_byte@entry=134, old_end_byte=old_end_byte@entry=134, new_end_byte=269) at treesit.c:739 739 { (gdb) backtrace #0 treesit_tree_edit_1 (tree=tree@entry=0x5555575b64f0, start_byte=start_byte@entry=134, old_end_byte=old_end_byte@entry=134, new_end_byte=269) at treesit.c:739 #1 0x00005555557cb085 in treesit_sync_visible_region (parser=parser@entry=XIL(0x555556fc329d)) at treesit.c:931 #2 0x00005555557ccf28 in treesit_ensure_parsed (parser=XIL(0x555556fc329d)) at treesit.c:1025 #3 Ftreesit_parser_root_node (parser=XIL(0x555556fc329d)) at treesit.c:1507 treesit.c:739 points to a treesit_tree_edit_1 call which is predicated on this condition: if (visible_end < BUF_ZV_BYTE (buffer)) ...which shouldn't be the case since the buffer is small enough to fit in the default window. It might already be the consequence of passing the wrong value of new_end_byte to ts_tree_edit, though. Going back to the first call, the backtrace looks like this: Thread 1 "emacs" hit Breakpoint 3, treesit_tree_edit_1 (tree=0x5555574f0ff0, start_byte=start_byte@entry=0, old_end_byte=old_end_byte@entry=0, new_end_byte=new_end_byte@entry=67) at treesit.c:739 739 { (gdb) backtrace #0 treesit_tree_edit_1 (tree=0x5555574f0ff0, start_byte=start_byte@entry=0, old_end_byte=old_end_byte@entry=0, new_end_byte=new_end_byte@entry=67) at treesit.c:739 #1 0x00005555557cc991 in treesit_record_change (start_byte=1, old_end_byte=1, new_end_byte=69) at treesit.c:806 #2 0x00005555556f8bb7 in insert_from_string_1 (string=XIL(0x55555744c4f4), pos=0, pos_byte=0, nchars=68, nbytes=68, inherit=, before_markers=false) at insdel.c:1084 Seems like treesit_record_change turns new_end_byte=69 into new_end_byte=67 inside treesit_tree_edit_1. It seems to fail in this calculation: ptrdiff_t new_end_offset = (min (visible_end, max (visible_end, new_end_byte)) - visible_beg); because visible_end is still 68 there. It value gets updated later, closer to the end of this function. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 15 17:44:16 2023 Received: (at 61369) by debbugs.gnu.org; 15 Feb 2023 22:44:17 +0000 Received: from localhost ([127.0.0.1]:34377 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pSQVc-0008BF-Hj for submit@debbugs.gnu.org; Wed, 15 Feb 2023 17:44:16 -0500 Received: from mail-ed1-f46.google.com ([209.85.208.46]:34592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pSQVb-0008B2-Bq for 61369@debbugs.gnu.org; Wed, 15 Feb 2023 17:44:15 -0500 Received: by mail-ed1-f46.google.com with SMTP id fj20so427595edb.1 for <61369@debbugs.gnu.org>; Wed, 15 Feb 2023 14:44:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=pJyLwPsnS29X5WSWI3oj2I39UujXbowb/bqXSSZqdJE=; b=XxuEx7/agpq+ibZ9+KnIhKAD93XNzhL9VJYYzr7bKYKmyu3nIs402JTVNY34QeHWWl p2Vw/QB99hQFRgl6kCmiWyR+3vPQ9KG4ED5UnzvxkXJQZi6xF9vOw03N89AD4QcWWBWG PTcC+ZOen+BdBOdJnD5V2A33CF6F+RWPf9t1xjuucG5e++ywYQr4K/JJ7Bq4NqFAojgt Ez1mIM8HHcc8y2/YXqAQfNbku7gtuzwGz8v5k2Dn/DSxJ4S5jXo+QeOrUKOImHY2YPUC TjQE4EUY3AnNlii5QMsLxwOzYXSP3LeKpgX7OfcQ2Pen4uuo2BnfZWKFt8ZFwyCS1uSq pEbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pJyLwPsnS29X5WSWI3oj2I39UujXbowb/bqXSSZqdJE=; b=Q7iTtbYCN+Hu+RvaW3rxrXkmpNc1tUlhsE0WZly4sV0AAS1+Wfy1qvJAxEwOfF6YLG b6xW0Bn3wyMNwD55obeVpuFXEqgp8zS12V6uojU2ATApB+v/K9zc2EWA/+iN/obQxqzK TCj4eGJxZHnPE+XWaN5ZxcQvpSLZtljT2PoKWspArF/XgDOw2eApAzWtUjyYPROy2V9q 0U0MOQrc5GNrOPBuINsEa7CqXpkFJpewopPZEyYEnG2ejzvV3XMZMQOi7RX5/1PuYxIe cg2GoV9BavaplTpPaxfSbELNTyhAXrAJ3ajk1nKuQLxcIATrgAi3fOnuKY9L/jcC3fIT 2sIw== X-Gm-Message-State: AO0yUKWkgJel0rBVI9cdTljJom/OYx8ibTILXg3/sUkal2IvpipLd2mp zC2gO6W4FTQHvn23jNptoGs= X-Google-Smtp-Source: AK7set+uoV/J6C5sZ/ymG8OQNRpJtQWugFAyyEljotZ2i5n+YBLmcqBJ2fmubWVxCTI6xuF9Tg8DHQ== X-Received: by 2002:a05:6402:32f:b0:4ab:4be9:5dcf with SMTP id q15-20020a056402032f00b004ab4be95dcfmr3954460edw.4.1676501049578; Wed, 15 Feb 2023 14:44:09 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id t14-20020a50d70e000000b004acb890553fsm37775edi.26.2023.02.15.14.44.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 15 Feb 2023 14:44:08 -0800 (PST) Message-ID: Date: Thu, 16 Feb 2023 00:44:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US From: Dmitry Gutov To: Yuan Fu References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> In-Reply-To: <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 15/02/2023 04:17, Dmitry Gutov wrote: > Seems like treesit_record_change turns new_end_byte=69 into > new_end_byte=67 inside treesit_tree_edit_1. > > It seems to fail in this calculation: > >   ptrdiff_t new_end_offset = (min (visible_end, >                    max (visible_end, new_end_byte)) >                   - visible_beg); > > because visible_end is still 68 there. It value gets updated later, > closer to the end of this function. So FWIW the patch below fixes the problem. But I'm not sure about change's clipping by the current restriction, or how it should be handled exactly. diff --git a/src/treesit.c b/src/treesit.c index cab2f0d5354..9f15b88a8bd 100644 --- a/src/treesit.c +++ b/src/treesit.c @@ -794,9 +794,7 @@ treesit_record_change (ptrdiff_t start_byte, ptrdiff_t old_end_byte, ptrdiff_t old_end_offset = (min (visible_end, max (visible_beg, old_end_byte)) - visible_beg); - ptrdiff_t new_end_offset = (min (visible_end, - max (visible_beg, new_end_byte)) - - visible_beg); + ptrdiff_t new_end_offset = max (visible_beg, new_end_byte) - visible_beg; eassert (start_offset <= old_end_offset); eassert (start_offset <= new_end_offset); From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 17 17:32:44 2023 Received: (at 61369) by debbugs.gnu.org; 17 Feb 2023 22:32:44 +0000 Received: from localhost ([127.0.0.1]:41812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pT9HY-0004SO-H9 for submit@debbugs.gnu.org; Fri, 17 Feb 2023 17:32:44 -0500 Received: from mail-pf1-f181.google.com ([209.85.210.181]:37409) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pT9HW-0004SA-OV for 61369@debbugs.gnu.org; Fri, 17 Feb 2023 17:32:43 -0500 Received: by mail-pf1-f181.google.com with SMTP id 71so1438434pfy.4 for <61369@debbugs.gnu.org>; Fri, 17 Feb 2023 14:32:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=DzX11qRZn3+gIyZu5pHGhyOqZ3b+HlT7dzXzXFZMsVo=; b=RpvB+QzNo1VaKW6Jj84iMIe5hG6TzWpYMFJGTqM9YMEtn0a//rb4L3DIUaDp20gD7B /hQgXGEDJ/ORph7lgnM8Pwsu5vGKaE/2EaEZPDMKGVn0l8piHld4fBRnoBfohN0Uh9p8 MN3sKiccTUSA/wMpIoDizPxhGv7ek7tQeVxCHi9tWzcE3bA96Gf2u/prK+QAB8JmLsdQ Vm5b1+Hscw9GFlITud0Np1BDrJfbc1Y2hj/nVyhi5G7DI306Q47M/KLkAGKsSKBnimOS KIGrKkLcburEdHJ3wlg6G61aJlw40mDn90X+KDvJZcrHYLuBLiV4a4Y+kIVDszBmkQIC ZuXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DzX11qRZn3+gIyZu5pHGhyOqZ3b+HlT7dzXzXFZMsVo=; b=Ue7I1H/iMcEfGKjseBKOn3kAcOYeIyJJ/GD/YttPSr3+h97Aic1e/iRBBfIeLEIBxF 1Y/mr3Se9J7vd0Q7g/rsqMB/p8Steiv4OzmvcSNv7iz8EnKFqMjsmImm/bfBSJPfTNrk Z41V0N/T8V1RVUGut83iwtcwePzQkuSDHTOfuopP7g5sNWrnd9yC9kD5nzYjPh3p0Yxo pBFEKcYo9v/Hh7MNYRdWtkYYM1OJOVtcZ/sH0k10bV+RUE4m+stDXWsrR33YmEIY/EZz 6QoX+IGntpDK4rV+LG3ZyBwiMBqO/PvL3btmxWxa50OpiPJCo+w+HkSH4m7Z6s2/dSgu bzIg== X-Gm-Message-State: AO0yUKV3uObs1C21trUwW7kDK0XydUkfZQgWkxKHWJkqcbHW7OxwO5aJ cnF9GkBtwPwYL9kSX6o7Hx0= X-Google-Smtp-Source: AK7set+4ypCKfJfebiEXsv4DZHG/oXTX8u4JJQUlvv0je72RFpmDzfZCTb6Aax8ovuCQX8dtqW4IDA== X-Received: by 2002:aa7:95b2:0:b0:5a8:acca:d5ce with SMTP id a18-20020aa795b2000000b005a8accad5cemr2054780pfk.28.1676673156765; Fri, 17 Feb 2023 14:32:36 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id 17-20020aa79211000000b005abc0d426c4sm440276pfo.54.2023.02.17.14.32.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Feb 2023 14:32:36 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date From: Yuan Fu In-Reply-To: Date: Fri, 17 Feb 2023 14:32:22 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: Theodor Thornhill , 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Feb 15, 2023, at 2:44 PM, Dmitry Gutov wrote: >=20 > On 15/02/2023 04:17, Dmitry Gutov wrote: >> Seems like treesit_record_change turns new_end_byte=3D69 into = new_end_byte=3D67 inside treesit_tree_edit_1. >> It seems to fail in this calculation: >> ptrdiff_t new_end_offset =3D (min (visible_end, >> max (visible_end, new_end_byte)) >> - visible_beg); >> because visible_end is still 68 there. It value gets updated later, = closer to the end of this function. >=20 > So FWIW the patch below fixes the problem. But I'm not sure about = change's clipping by the current restriction, or how it should be = handled exactly. >=20 > diff --git a/src/treesit.c b/src/treesit.c > index cab2f0d5354..9f15b88a8bd 100644 > --- a/src/treesit.c > +++ b/src/treesit.c > @@ -794,9 +794,7 @@ treesit_record_change (ptrdiff_t start_byte, = ptrdiff_t old_end_byte, > ptrdiff_t old_end_offset =3D (min (visible_end, > max (visible_beg, old_end_byte)) > - visible_beg); > - ptrdiff_t new_end_offset =3D (min (visible_end, > - max (visible_beg, new_end_byte)) > - - visible_beg); > + ptrdiff_t new_end_offset =3D max (visible_beg, new_end_byte) - = visible_beg; > eassert (start_offset <=3D old_end_offset); > eassert (start_offset <=3D new_end_offset); Thank you very much! I thought that clipping the change into the fixed = visible range, and rely on treesit_sync_visible_region to add back the = clipped =E2=80=9Ctail=E2=80=9D when we extend the visible range would be = equivalent to not clipping, but I guess clipping and re-adding affects = how incremental parsing works inside tree-sitter. I don=E2=80=99t think this change would have any adverse effect, because = if you think of it, inserting text in a narrowed region always extends = the region, rather than pushing text at the end out of the narrowed = region. So the right thing to do here is in fact not clipping = new_end_offset. I pushed this change. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 17 19:11:32 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 00:11:32 +0000 Received: from localhost ([127.0.0.1]:41886 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTApA-0007CL-AD for submit@debbugs.gnu.org; Fri, 17 Feb 2023 19:11:32 -0500 Received: from mail-wr1-f48.google.com ([209.85.221.48]:41541) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTAp8-0007C7-Aa for 61369@debbugs.gnu.org; Fri, 17 Feb 2023 19:11:30 -0500 Received: by mail-wr1-f48.google.com with SMTP id i15so2408698wry.8 for <61369@debbugs.gnu.org>; Fri, 17 Feb 2023 16:11:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=VAuWSqJ0jWOUE/VurEJYQUAInvJmTyU6TSrwqBdoH4g=; b=LLhKHdrirC8y/8iEsAwyXNSTON8qgNL9tM3O5954DOMxZD8+FAVRshF/5M36guRzDA fLeUaP7yZ56nIn1w1iCop7uehemltkBlO/fNMKV3MRIxQnq8W2JtHtj5XiTl0a/nGNXB b/xkutB3Wvh1zW0rF6NwETzDFENd0vlqibKO57OxeNOhxW8wN0dvtp0tpyHfEagsT2Jp Qmtu9fLTJFIiz96B1QyUw+cAAelb6/7uC+ievTc6e3UqvShLKqANzmMP6zGJu2NkfmU6 hwLYhrzeon1uI2CK6UhmPD8JwskZe2ppLE9zTbuPZyNV0YvD2lHmHLl048sGB09cxcze nwEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VAuWSqJ0jWOUE/VurEJYQUAInvJmTyU6TSrwqBdoH4g=; b=tRi0PiuAcno4YdV14JEGj4T7mTNCkwyAtmmX0J6ZgNFmB0LMyVpAEWANIwNU7egtE3 DBPle4DWekca0cJq0YP5vH4wSB7sl8oAJDCzOhexuD2PwUq+akrAUDSzE1Q8a6XGZSKI /FnKCBU2EDDwRwWizH4ApczosjEqrBOrj8tl83WxXKDdilE+sjjMAAtQO2/H8X3cr9dY RUuk7B/G0q3UcNwPDjlJJNwR4VLfOFxfy9ZQIyVKetgUNknxOYn1CeKh8l/V4IklfcCC 9k20yphfSQ/i+0KJg1FMuWXC99dDOdIbSLiTMK9S8GTQNlYLGBywiGqXp79HVWe2YOlx ucUg== X-Gm-Message-State: AO0yUKU+ClV9KyXc+c44S19MdENdFuHq4D16b5vZCLzmu/+XSUtSwAkf 1PEw+xE/InbK2nTgrnHjiQg= X-Google-Smtp-Source: AK7set8v8Rb59RNl/wvdvG6ooFekuyFHfO4xb7yGnPmhSZSljy+h1OamGHtbSNT+sb0FGCKHVh+6WQ== X-Received: by 2002:a5d:6a06:0:b0:2c5:4ffa:ba55 with SMTP id m6-20020a5d6a06000000b002c54ffaba55mr466617wru.69.1676679084349; Fri, 17 Feb 2023 16:11:24 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id r10-20020a5d694a000000b002c5539171d1sm5351475wrw.41.2023.02.17.16.11.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Feb 2023 16:11:23 -0800 (PST) Message-ID: <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> Date: Sat, 18 Feb 2023 02:11:22 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US To: Yuan Fu References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> From: Dmitry Gutov In-Reply-To: <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: Theodor Thornhill , 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 18/02/2023 00:32, Yuan Fu wrote: > Thank you very much! I thought that clipping the change into the fixed visible range, and rely on treesit_sync_visible_region to add back the clipped “tail” when we extend the visible range would be equivalent to not clipping, but I guess clipping and re-adding affects how incremental parsing works inside tree-sitter. It seems like the "repairing" sync used a different range, one that didn't include the character number 68 inserted from the beginning. It just synced the 1 or 2 characters at the end of the buffer, the difference between the computed visible_end and the actual BUF_ZV_BYTE. > I don’t think this change would have any adverse effect, because if you think of it, inserting text in a narrowed region always extends the region, rather than pushing text at the end out of the narrowed region. So the right thing to do here is in fact not clipping new_end_offset. I figured it could be a problem if both old_end_byte and new_end_byte extend past the current restriction. But I'm not sure whether that could actually happen in practice. The obvious attempts (undo a change outside of the narrowing, or revert the buffer when narrowing is in effect) didn't play out, but I'm not sure whether there is an actual hard limit on modifying the text outside of the current restriction. > I pushed this change. Thanks. Good to see it make it in. From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 17 20:14:43 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 01:14:43 +0000 Received: from localhost ([127.0.0.1]:41973 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBoJ-0000Ny-5E for submit@debbugs.gnu.org; Fri, 17 Feb 2023 20:14:43 -0500 Received: from mail-pl1-f169.google.com ([209.85.214.169]:45037) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBoH-0000Nl-Ck for 61369@debbugs.gnu.org; Fri, 17 Feb 2023 20:14:41 -0500 Received: by mail-pl1-f169.google.com with SMTP id b6so3188315plg.11 for <61369@debbugs.gnu.org>; Fri, 17 Feb 2023 17:14:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xjBWnHpujWbJ3EaYlwqGOiipSJ57WN8m57ecJl7VHzQ=; b=UaV7ABoiN8OzhlMvhjsYMDvKcFjl8kw89ueJJDbTbskeerKW1jaVHt17rFTtzyzqeV VUz+eCr1uLWMVLe1X9HmDZMvihA1e3pxAMOrXrdyPna90pCG8iSBYEjjkgHc6Tw4bo0r SSJxZS/TFTj8WX/hfIxl4LJF9ggEoRKvjAG0bAXwkZ8mThObqZp5CZLT4tlH8aCyB7N0 LiJCKKZgAFK+OLUkgWq4RVP3gbzl9boaw5J4Qu+aEabjyK8g6/T/6l8BHDvZ+1fKJuBa 5Ie3ukcUl54CvBIN6TDfK9iwbKKKwJDkpM5AeifkybRqtmAh31sSupnh4a1RbcaGsTHp FG8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xjBWnHpujWbJ3EaYlwqGOiipSJ57WN8m57ecJl7VHzQ=; b=vZs4vetfK1BqAxDkEz353+U3lT5A634ogaPc9KgWS/Yj4ry+1Jk0LkQKGE/uHoqeIA hRGqoX9DK/KtUbV3402MA5LWDdiQEUMbU6/6+DF38iU1fp8VJ1EOuAjZ78Vn6yS+fNQ0 SsIrAF7tCt3K5LO3wzF0E1ChD/RcbJ25HfG2aLXHuzd29wQ0ZkUqhliL564l5kOUjzSL U4dXyr3apnQmI0NnTZQsXmFLbpusk1lp56wg7jqQbM1Ndg08yFGW3EnjcgJH8ha+S9di gx52oWdBSNOQwqMnUzWZEnO0ilyz6i5PhDt6xkTRIgvVJxhqJkxwQgwg+fDclMoCksMJ Wr/g== X-Gm-Message-State: AO0yUKX2MAAct5+jexoA7v032Au9oP5SD9Pfbkde4XLmVTqiJJ39jouY pZV/O4V+Pc8uLAaOWQopc7A= X-Google-Smtp-Source: AK7set9TkHS0YZrDH+quIt5md21yxR3XCwRgKy+YUYeWwMxmKkmz4dz45zeZAsGo4Aqql/4ANGb8uQ== X-Received: by 2002:a17:903:187:b0:19a:a6cd:35a8 with SMTP id z7-20020a170903018700b0019aa6cd35a8mr8445133plg.25.1676682875446; Fri, 17 Feb 2023 17:14:35 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id v8-20020a170902b7c800b001994554099esm6048plz.173.2023.02.17.17.14.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Feb 2023 17:14:35 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date From: Yuan Fu In-Reply-To: <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> Date: Fri, 17 Feb 2023 17:14:22 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: Theodor Thornhill , 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Feb 17, 2023, at 4:11 PM, Dmitry Gutov wrote: >=20 > On 18/02/2023 00:32, Yuan Fu wrote: >> Thank you very much! I thought that clipping the change into the = fixed visible range, and rely on treesit_sync_visible_region to add back = the clipped =E2=80=9Ctail=E2=80=9D when we extend the visible range = would be equivalent to not clipping, but I guess clipping and re-adding = affects how incremental parsing works inside tree-sitter. >=20 > It seems like the "repairing" sync used a different range, one that = didn't include the character number 68 inserted from the beginning. >=20 > It just synced the 1 or 2 characters at the end of the buffer, the = difference between the computed visible_end and the actual BUF_ZV_BYTE. That should be enough, no? Because other text didn=E2=80=99t change, = they just moved. And tree-sitter should know that they moved. Or maybe = I=E2=80=99m misunderstanding what you mean. >=20 >> I don=E2=80=99t think this change would have any adverse effect, = because if you think of it, inserting text in a narrowed region always = extends the region, rather than pushing text at the end out of the = narrowed region. So the right thing to do here is in fact not clipping = new_end_offset. >=20 > I figured it could be a problem if both old_end_byte and new_end_byte = extend past the current restriction. That should be fine (ie, technically correct), since when we widen, the = clipped text are reparsed by tree-sitter as new text. >=20 > But I'm not sure whether that could actually happen in practice. The = obvious attempts (undo a change outside of the narrowing, or revert the = buffer when narrowing is in effect) didn't play out, but I'm not sure = whether there is an actual hard limit on modifying the text outside of = the current restriction. It is my impression that Emacs in general enforces the narrowing = restriction strictly. And we are still correct when exceptions occur. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 17 20:26:13 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 01:26:13 +0000 Received: from localhost ([127.0.0.1]:42004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBzR-0000hg-95 for submit@debbugs.gnu.org; Fri, 17 Feb 2023 20:26:13 -0500 Received: from mail-wm1-f43.google.com ([209.85.128.43]:34700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBzL-0000hG-NB for 61369@debbugs.gnu.org; Fri, 17 Feb 2023 20:26:11 -0500 Received: by mail-wm1-f43.google.com with SMTP id bg22-20020a05600c3c9600b003dff4480a17so1522100wmb.1 for <61369@debbugs.gnu.org>; Fri, 17 Feb 2023 17:26:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=OHoTsq5DDmCnJJ5XScuMc1tfGAMWhCAE7QlYlaXeAjg=; b=JHW2u/Oy9VftBe8DkRikTDUhAaUO7mRK9maW5+5P5IN89E9kkt7gsWmoSHZOfaQlMT XJZ/Yq77iuKeDqNMGEd+WURQcYorVKD4oEC0ixT5Zqpie9afx+OxBji+ltNi3yz1ZoJL tww2dLZqLQAN5ycx3mclWEIVsTUzoHcY59xMYxuE8IenO2jEw887y2E95ymMpU3+40ga 3A2b3q3SS7ZK6qNZxHohsXYQYSutPmspd3SxCQK2HAIFs1cGPjO0QqkhV9kIQLrB2Mzq p92ci35OMysLiRzu+gCA+eA+C1LW/8LmN87eXRd2ePhwtNC3xWbcsX+fcS176e+Z/+OV KJIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OHoTsq5DDmCnJJ5XScuMc1tfGAMWhCAE7QlYlaXeAjg=; b=xSVVLhipAYlDUbD7U7MkXipX5aqWwM7XL7LKPBwm4Yr1J0Ng32lm5/cDatxeth3WAW 0p5L39b9ouoPJNTGaSXvM0xyvMNcIYzsKOGHU0GCgwE1gwDOQi1Wj99vXscfEvx9RsA1 +iMWZPGlvsrVdUmekuf1hgpzMWu9aLFWQIt62eEGQ3qcj6fgbDyvIjOqgYoxjzsdBS1r 6XYDrPT7uCezELGiMYjvIBX25L2bcJ+gXjnjgA/n17V6FdSSnZTrDsnkZGccjxkiMJCx oItPx302LefeLdK6tSfoOyLm6vxLw++5PCC0Gm364PN4vMVVMEEQejRh7kGcQP232p2q NT+Q== X-Gm-Message-State: AO0yUKU68Y9fGwMxxzvM+zj1q8fmdYWpDQPZkxWijk6ZUujhU4Ol6VXP 73ZjC94/TG3F/VCYSCYl6G4= X-Google-Smtp-Source: AK7set/kkSQ8DfySm7OXvPC+OTrq2xdYLO02I7Nud+h3hV3eixz5a2u2RTnz7/OwaWYixs2bpIlSJg== X-Received: by 2002:a05:600c:746:b0:3de:d52:2cd2 with SMTP id j6-20020a05600c074600b003de0d522cd2mr1363160wmn.4.1676683561747; Fri, 17 Feb 2023 17:26:01 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id k37-20020a05600c1ca500b003dffe312925sm3201335wms.15.2023.02.17.17.26.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Feb 2023 17:26:01 -0800 (PST) Message-ID: <7ee28606-18cc-ce4f-e601-3954489c4f4c@yandex.ru> Date: Sat, 18 Feb 2023 03:25:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US To: Yuan Fu References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> From: Dmitry Gutov In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: Theodor Thornhill , 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 18/02/2023 03:14, Yuan Fu wrote: > > >> On Feb 17, 2023, at 4:11 PM, Dmitry Gutov wrote: >> >> On 18/02/2023 00:32, Yuan Fu wrote: >>> Thank you very much! I thought that clipping the change into the fixed visible range, and rely on treesit_sync_visible_region to add back the clipped “tail” when we extend the visible range would be equivalent to not clipping, but I guess clipping and re-adding affects how incremental parsing works inside tree-sitter. >> >> It seems like the "repairing" sync used a different range, one that didn't include the character number 68 inserted from the beginning. >> >> It just synced the 1 or 2 characters at the end of the buffer, the difference between the computed visible_end and the actual BUF_ZV_BYTE. > > That should be enough, no? Because other text didn’t change, they just moved. And tree-sitter should know that they moved. Or maybe I’m misunderstanding what you mean. But the "unsynced" character is at position 68. And we just tell tree-sitter to update positions 134-136. So it stays ignorant of the changed char in the middle of the buffer. It's not just about not knowing about the change either (the character in question is a newline, so its absence wouldn't lead to a syntax error), but about wrong offsets in the old parse tree, based on which the new tree is generated. That probably creates a wrong picture of the source text in the parser. >>> I don’t think this change would have any adverse effect, because if you think of it, inserting text in a narrowed region always extends the region, rather than pushing text at the end out of the narrowed region. So the right thing to do here is in fact not clipping new_end_offset. >> >> I figured it could be a problem if both old_end_byte and new_end_byte extend past the current restriction. > > That should be fine (ie, technically correct), since when we widen, the clipped text are reparsed by tree-sitter as new text. I guess the effect I was thinking of is that XTS_PARSER (lisp_parser)->visible_end would end up with a higher value than BUF_ZV_BYTE. Not sure if it's a problem. >> >> But I'm not sure whether that could actually happen in practice. The obvious attempts (undo a change outside of the narrowing, or revert the buffer when narrowing is in effect) didn't play out, but I'm not sure whether there is an actual hard limit on modifying the text outside of the current restriction. > > It is my impression that Emacs in general enforces the narrowing restriction strictly. And we are still correct when exceptions occur. Very good. From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 18 02:16:02 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 07:16:02 +0000 Received: from localhost ([127.0.0.1]:42294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTHRy-0001yr-4Q for submit@debbugs.gnu.org; Sat, 18 Feb 2023 02:16:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39008) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTHRw-0001yG-QO for 61369@debbugs.gnu.org; Sat, 18 Feb 2023 02:16:01 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pTHRq-0000W7-RD; Sat, 18 Feb 2023 02:15:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=bWHuopdDqUSs8XuvYBl1itdBQqY0pwosSuwkiLwBvxc=; b=lLKoFOYJ1zT1 TrTiKhG0tyWjP8Ni2RLvZl9DpeLRuJ6bHuXRabhH5aKVlQDzazTgYRsXsitUd9c5CH60vgqD/+bIi FhlkiYVKF3DK0boyMbmJ0TpEpurWSv7cVaxXWY5BOkzhhcFGNE2oIPCLHpZFepysw/e5cp05sHfC/ Dl4S5lZtz1m26il6PYqoSQAyVYVZrNiUtZUEG90IA0KdNDV265GuYWMf0CznNB31rH8fp3WS0aqfc uWCz4KM8gmR47nDhA0NdQXe5s62FM5dJ4YIXvaH1xoQgxfUufl0Aryn4P/gRX/LQEIDFrlayqf1MH zFSioDD012z9yXYMUWwBWQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pTHRp-0006bx-GO; Sat, 18 Feb 2023 02:15:54 -0500 Date: Sat, 18 Feb 2023 09:15:56 +0200 Message-Id: <83sff3zbo3.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-Reply-To: <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> (message from Dmitry Gutov on Sat, 18 Feb 2023 02:11:22 +0200) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 61369 Cc: casouri@gmail.com, theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: Theodor Thornhill , 61369@debbugs.gnu.org > Date: Sat, 18 Feb 2023 02:11:22 +0200 > From: Dmitry Gutov > > I'm not sure whether there is an actual hard limit on modifying the > text outside of the current restriction. It is, for most/all practical purposes. If you try to modify text outside of the current restriction, you risk many parts of code barfing or signaling errors on you. For example, conversion from character to byte positions and vice versa will stop working (and in a build with --enable-checking will actually raise SIGABRT). From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 18 05:05:25 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 10:05:25 +0000 Received: from localhost ([127.0.0.1]:42541 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTK5s-0006eY-Lm for submit@debbugs.gnu.org; Sat, 18 Feb 2023 05:05:25 -0500 Received: from mail-pj1-f51.google.com ([209.85.216.51]:44781) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTK5r-0006eL-A7 for 61369@debbugs.gnu.org; Sat, 18 Feb 2023 05:05:23 -0500 Received: by mail-pj1-f51.google.com with SMTP id pw17-20020a17090b279100b00236a0d55d3aso919415pjb.3 for <61369@debbugs.gnu.org>; Sat, 18 Feb 2023 02:05:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8crWgUWJa70DqUiHhq8795aGrWnhA7KpyuPTqHK9w80=; b=U7NJPGzYVdAVufo5eufaKHMiRp9GFNTA7Li2BamV+zZJjCMRNkjTjRpOlr56bLdhQO Jb4M8IDBGn8qfWdfMSY3Is10BS4ow0mBI678yFR4XcF2Yzv12Zhm+1K/i0m7tABVXLJw 2eYaDhUtld4ZsmWfsHToTWch9kcWGU1Qw7LXyWgs4rvezAABtiCE5WzkvE6zwwgytt/g saveoGeL22Gbb6IGrJ8P3nw1hWL3znsBUhevnoofzWOt7zegYLY6FR67AP4xw9BReVeX dTuO4nxJUalrtHos7nGGc4F+1xaCvsypD8tESYZ1d/4oZV+8jHBZ9Vp9t68HCCFSJ9KS jfWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8crWgUWJa70DqUiHhq8795aGrWnhA7KpyuPTqHK9w80=; b=VefcVWhsF7WXluH77X8DiZ06b36L3XjHR4deYJ9yf2XMol7sqD31n4Id1OE/zNCuas 0XKNvA3lamXaNJqjOJt5L9UitiXv11juvpw4Xj8vGAcQNOSt8dSI7E/9g7uceFNeQuAd xuh/eFmXCPY/ZRuAK9ID46LQOYwSvU6sgtpbz4GUXsR2qJlK0aFRpP3qL4ZPEyexcyGm 54vxQJ713uht0Uv6lST59aa/tId71HytL/3BkXM1nOXGJyaIbswzc7Sq2qwD9DIqc5FC UWWRiec/M+quvz8jJq8BztwgN8/tJaR7sZDThpmeEQrPAFe5K89PFWoSqB8z27BHCby8 HZTg== X-Gm-Message-State: AO0yUKWA5SxmtKzjP6eK3Xg3a34r5i8HNecQrA6l50BR1K/V3t2xeQvB wz0kcyOtJHPiAeO3e3TrVeg= X-Google-Smtp-Source: AK7set/uO3PUAy5nAkKyoxs144TGHL14eTCjn9vGsRifcvbA73j6kttLAAxtp1iEt2dko3hfrOM1BQ== X-Received: by 2002:a17:90b:4c49:b0:234:eeb:8df4 with SMTP id np9-20020a17090b4c4900b002340eeb8df4mr132105pjb.14.1676714717194; Sat, 18 Feb 2023 02:05:17 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id a12-20020a17090aa50c00b002341ae23ad7sm773742pjq.1.2023.02.18.02.05.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 18 Feb 2023 02:05:16 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date From: Yuan Fu In-Reply-To: <7ee28606-18cc-ce4f-e601-3954489c4f4c@yandex.ru> Date: Sat, 18 Feb 2023 02:05:04 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <6FF39BA3-247A-46D9-B3A1-ECFE17A09778@gmail.com> References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> <7ee28606-18cc-ce4f-e601-3954489c4f4c@yandex.ru> To: Dmitry Gutov X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61369 Cc: Theodor Thornhill , 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Feb 17, 2023, at 5:25 PM, Dmitry Gutov wrote: >=20 > On 18/02/2023 03:14, Yuan Fu wrote: >>> On Feb 17, 2023, at 4:11 PM, Dmitry Gutov wrote: >>>=20 >>> On 18/02/2023 00:32, Yuan Fu wrote: >>>> Thank you very much! I thought that clipping the change into the = fixed visible range, and rely on treesit_sync_visible_region to add back = the clipped =E2=80=9Ctail=E2=80=9D when we extend the visible range = would be equivalent to not clipping, but I guess clipping and re-adding = affects how incremental parsing works inside tree-sitter. >>>=20 >>> It seems like the "repairing" sync used a different range, one that = didn't include the character number 68 inserted from the beginning. >>>=20 >>> It just synced the 1 or 2 characters at the end of the buffer, the = difference between the computed visible_end and the actual BUF_ZV_BYTE. >> That should be enough, no? Because other text didn=E2=80=99t change, = they just moved. And tree-sitter should know that they moved. Or maybe = I=E2=80=99m misunderstanding what you mean. >=20 > But the "unsynced" character is at position 68. >=20 > And we just tell tree-sitter to update positions 134-136. So it stays = ignorant of the changed char in the middle of the buffer. >=20 > It's not just about not knowing about the change either (the character = in question is a newline, so its absence wouldn't lead to a syntax = error), but about wrong offsets in the old parse tree, based on which = the new tree is generated. That probably creates a wrong picture of the = source text in the parser. Ok, I made some visualization to understand it, and yeah you are right. = I=E2=80=99ll need to modify the comment a bit. |visible range| updated range ------------- |aaaaaa| |bbbbbbbbaaaa|aa start: 0, old_end: 0, new_end: 6 ------ =20 |bbbbbbbbaaaaaa| start: 12, old_end: 12, new_end: 14 -- >=20 >>>> I don=E2=80=99t think this change would have any adverse effect, = because if you think of it, inserting text in a narrowed region always = extends the region, rather than pushing text at the end out of the = narrowed region. So the right thing to do here is in fact not clipping = new_end_offset. >>>=20 >>> I figured it could be a problem if both old_end_byte and = new_end_byte extend past the current restriction. >> That should be fine (ie, technically correct), since when we widen, = the clipped text are reparsed by tree-sitter as new text. >=20 > I guess the effect I was thinking of is that >=20 > XTS_PARSER (lisp_parser)->visible_end >=20 > would end up with a higher value than BUF_ZV_BYTE. Not sure if it's a = problem. It shouldn=E2=80=99t be, since BUF_ZV_BYTE should automatically grow = when user inserts text. Even if it does, we always call = treesit_sync_visible_region to sync up visible_beg/end with = BUF_(Z)V_BYTE before parsing, so it shouldn=E2=80=99t be a problem. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 18 12:21:50 2023 Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 17:21:51 +0000 Received: from localhost ([127.0.0.1]:44856 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTQuE-0004WK-KN for submit@debbugs.gnu.org; Sat, 18 Feb 2023 12:21:50 -0500 Received: from mail-wm1-f53.google.com ([209.85.128.53]:43760) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTQuC-0004W7-SK for 61369@debbugs.gnu.org; Sat, 18 Feb 2023 12:21:49 -0500 Received: by mail-wm1-f53.google.com with SMTP id bg25-20020a05600c3c9900b003e21af96703so452607wmb.2 for <61369@debbugs.gnu.org>; Sat, 18 Feb 2023 09:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=eDAInN9U2Ef70rBBbpNPIqtm1q9W4EpICvSnUEIEzI4=; b=MYXROo3C15KV+Ygkmsk2Js8c/3I+t/LFo8c89Ysv/BJhPSTmJ4zXv88r+bkk8TqT9j yxYOmV5SWCdEOBi9YjBFGhyuc178Qru/5Dpa1cXiPLGclfLzoN/FncWy2sEJuv3f1As/ B2GByqms3P8538PF/do9gfgUrCMBVAUXu4cqxAZ5ezKGc3mw4mlZHxIdcaNXOYJOwq+e WSHAU9TyU3YNSBF228FM70gLgY+gZitXvDPOIsaSXpNPIl9KeiXHGMRGZAwmyvGCVInf VRQKoY8w8he3EFmrtBTI8txcJI+jT23xKFRhqg7VMOWmUXYxTNyc2W8emHY84Igw8Key O0uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eDAInN9U2Ef70rBBbpNPIqtm1q9W4EpICvSnUEIEzI4=; b=CpfUCyVSItQ0ZIHqcO/CGIlQ2AHLB5JjiEJiL87rFX6eB/08ebtSoQP34ViVr0n00F tIGmB4mpFWz8AMWkyaaiaO5ZT5dSyRF3VBwCoiiS3B76DqUriwDpXYwNFyeOuCNZNToj 8U2LTwmmv1hyLmKUP6eUM6jV5oA6ZpCer9CFnDhyVAR43FZjmUTzJitTO8akH8SEtDHk N8Mj9PmhIVvwPEs38VMtvXd3Yt5Pn5xd4Qz84Tf0vj0f3OJQZXrPZm7pk5101NMxAkTe gp1kLF/cGl17DSwHmgKsOb2L7Plv7dXsye4Vc0Wvz1yCreacxWSiRgcZHyxEY+ggCOwa /jOQ== X-Gm-Message-State: AO0yUKWZhFdWJ2bBbHIqdJiLjxLu5PUThj7ZSwRS91UFO8wbV9p7KctD gK5MgbB4GrVrVVH0CgUQr0M= X-Google-Smtp-Source: AK7set8mEH82akD+apwaF6XafnPISPbgeVrypedjLlR/O/HZpVE7HAEv9CpI+uUCKE4XXqEdp59Dcw== X-Received: by 2002:a05:600c:755:b0:3e2:1fe9:8d1b with SMTP id j21-20020a05600c075500b003e21fe98d1bmr4916363wmn.6.1676740902835; Sat, 18 Feb 2023 09:21:42 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id b4-20020a05600c4e0400b003d1d5a83b2esm5270594wmq.35.2023.02.18.09.21.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 18 Feb 2023 09:21:42 -0800 (PST) Message-ID: <2e141270-ab5c-8987-6c40-c2665a4e7ca2@yandex.ru> Date: Sat, 18 Feb 2023 19:21:40 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Content-Language: en-US To: Eli Zaretskii References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> <83sff3zbo3.fsf@gnu.org> From: Dmitry Gutov In-Reply-To: <83sff3zbo3.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61369 Cc: casouri@gmail.com, theo@thornhill.no, 61369@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) On 18/02/2023 09:15, Eli Zaretskii wrote: >> Cc: Theodor Thornhill,61369@debbugs.gnu.org >> Date: Sat, 18 Feb 2023 02:11:22 +0200 >> From: Dmitry Gutov >> >> I'm not sure whether there is an actual hard limit on modifying the >> text outside of the current restriction. > It is, for most/all practical purposes. If you try to modify text > outside of the current restriction, you risk many parts of code > barfing or signaling errors on you. For example, conversion from > character to byte positions and vice versa will stop working (and in a > build with --enable-checking will actually raise SIGABRT). All right, thank you both.