From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 21 01:12:31 2024 Received: (at submit) by debbugs.gnu.org; 21 Sep 2024 05:12:31 +0000 Received: from localhost ([127.0.0.1]:36898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1srsQ2-0005TZ-Kk for submit@debbugs.gnu.org; Sat, 21 Sep 2024 01:12:30 -0400 Received: from lists.gnu.org ([209.51.188.17]:37398) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1srsQ0-0005TR-Q1 for submit@debbugs.gnu.org; Sat, 21 Sep 2024 01:12:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1srsPg-0003DJ-F3 for bug-gnu-emacs@gnu.org; Sat, 21 Sep 2024 01:12:08 -0400 Received: from mail-uksouthazlp170110003.outbound.protection.outlook.com ([2a01:111:f403:c205::3] helo=LO2P265CU024.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1srsPe-00080i-FC for bug-gnu-emacs@gnu.org; Sat, 21 Sep 2024 01:12:07 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qLc/vyopDIwxSF+WARAirzTfjcgqWfDY+VhpHHj9LnN9sUtuKH7tRdoFQCM8slCmy1+O+//sNlktnzX8EDgCHPbdSvh0l9KES7/i6mImNd40XhJKj5Bth6mzhVCXHZ90skG0eYEf/8qt11peqX+gBsIbqt/ce17uTdDS7BwhR5wL009b+1BzEoVdUdcsVMMVjtoj6yeL2SGG9BQg+yEaTNzpGBsna+oTZiAQikYuVDw28dPlOVZAZZlhKVV+H785XfLypJZl1qZZW9Jr2u29s9K7ig6na4PxrrLrXmeQ7TiifbPlx5lU/JxDtbTo93824DTGW5mDM4F489GCbIAXDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T0vgEIHGoXAcab5cFt8MLq1IM9EJg+63PQLzhzGblw0=; b=mrvHeeV7Na0ST/+XnzwhEsv+K/pCCwODJZbpzzgsc1gpshPrVYzfi+kiRJLIlJ2t6F4P8gFzW6QDVEMojPA5vVDHP4CYhTmcIHQ4Zx9xrd3VArEzcsKTpboygLTbyxcqCFklTJlKFb4P9cc0kbhqvYBjadKXiShX1ldKbxqH0Es5uo4nXl7UXxULnDt+eSKpsuaWOct//Kpjd3Lz5QBT9I+fc2RrWRXuLZhupVPvOdfmKFkbVvJcqChYaSypfIKmf7umXZaHeKmB6gMXhr4co8jUmJdt1T/8ATcl2Ou9sWmorVhN8macCixbVsVK0ok/oO02ggOHq/J3cVxYpXMuEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T0vgEIHGoXAcab5cFt8MLq1IM9EJg+63PQLzhzGblw0=; b=jBEW6b0/AoGM11nw5ysi5p0srmJ4+kVGWk0BxBx1AEkQziRVAZkSlTHIapCBa9yvvFlTPO2JcdfmVebT+ve2G0g1Zl1Nsa9BhbFQVBbTaIkJ2PNI5yw5yX9sgw9Kqiiy3el7/eCJ6T7oHVivX17YHqAnUmOT3JIF6wgc5VZ8VVo= Received: from LO4P265CA0009.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ad::12) by CWLP265MB6705.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1ec::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.23; Sat, 21 Sep 2024 05:07:02 +0000 Received: from LO1PEPF000022FD.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ad:cafe::be) by LO4P265CA0009.outlook.office365.com (2603:10a6:600:2ad::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.29 via Frontend Transport; Sat, 21 Sep 2024 05:07:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by LO1PEPF000022FD.mail.protection.outlook.com (10.167.240.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Sat, 21 Sep 2024 05:07:02 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 226ED114002; Sat, 21 Sep 2024 06:07:02 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS autolearn=ham autolearn_force=no version=3.4.2 From: Mickey Petersen To: bug-gnu-emacs@gnu.org Subject: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes X-Debbugs-Cc: Date: Sat, 21 Sep 2024 06:06:58 +0100 Message-ID: <87plox4mtp.fsf@masteringemacs.org> Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO1PEPF000022FD:EE_|CWLP265MB6705:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: c1923c0b-81c1-464b-e0e5-08dcd9fb3a9f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|61400799027|82310400026|376014|36860700013|14776008|79816003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?a81i49klccb3E445Kq6oG4ErA9wYL+px9ktk+3e7u468Uzpo1iuzMekwkC7d?= =?us-ascii?Q?oH34izRPfNB0BZzk9eW5hIy4T6IM/mpl7NvDfWKempsDfA8Yw0w0nBDlYV4b?= =?us-ascii?Q?y1LUxZPQpCf0Jqa8Tf8I8r3B4UqJZU4ToN77mg5T4RtdmxbHvCDEUxyiUFLn?= =?us-ascii?Q?ZDrHLjqJ3PIJuey4TtUiQTudSw46iMzJPNVXwEpvc91nhiCd0eQOCSAsu30n?= =?us-ascii?Q?zs6Wz8lmvIyXLbPJ+kRDoFOokCIgQbZe31moD7+/rROgzZA2h7q0rJMWCg5k?= =?us-ascii?Q?pQp6Fjh8fZvzrTzOD3olV5iLh9Azu6as95/fmvRpaqAnJYVVFMP0ZH8sdovD?= =?us-ascii?Q?Qpu+Ferg2RBxg8UdV9L9MmCM94ASdmqyaVocf9TiA/p17YiQU6pcaCH7cpUf?= =?us-ascii?Q?vRHhk/qfOrs77BE3B+hWwiaTaU2vItv6/1dSiJCPFQ+tI3wHukcwNH9HSZbD?= =?us-ascii?Q?EVxwfyuZL57DBfzPJIB1kGE/Ex2oxsoTAnx4rx1n4/Om8HKW0O8TwiOK30sY?= =?us-ascii?Q?B1shPIATfHeNYYATxVGzIPadJ8FxGDu2oxXiEEDl/BlUZfATLV2jYUy73OUY?= =?us-ascii?Q?s+dyS1ksx/PAGdAi3QE2yDSJ1x2f5lcg3xciAepXxS3kOzfEP2OuHJ+R0xUP?= =?us-ascii?Q?rlmf1y4NSHg+5gt8HOvTUaKIX/o1g408OgmBt3bL1EDMEiy7H3Sf3ic/IZ7J?= =?us-ascii?Q?0XYrjMStL3cUVv/GJgPRp3OEalf1Upr6pXhpnt7Ntjr+q/QCulweEaIctWzI?= =?us-ascii?Q?6/B/mghJs+k93APnTs3j71KI9Ah4qRHC9YSMKTd9NXbFwJVPmcOQQVLMC49J?= =?us-ascii?Q?1zsR/N2WC9ZI93hnemyztkCJIfXseJeEQvbm45mbqnJ6z1VAZZKAPZYAFmha?= =?us-ascii?Q?NbRYXXtc0yPpofq1UiYpxetYkjjXepqCRi1PmU+7FGCNlpzgvB6WcxuNZ7a+?= =?us-ascii?Q?wPjHhTmFC4scqPhE0WpBucdWWPDRNE7IN4DGQKMU7OScO4WVcFBTTvF/XdxO?= =?us-ascii?Q?5B9rWxxm7e+CRSkInJ02AFwV+6q2iXL5ZDgY+ZTwFA9Is8euOfICrlr9G+wF?= =?us-ascii?Q?uVvjHJRBq9+G8IIEslyWLMfVsN5L4HwUoUVTkLn+oghIpJ2VhbgSLIVbbI5s?= =?us-ascii?Q?85NU4X7XUnlNYvgoY6xGxPB0lSGhEob87V0XMeOr5f0FpKOrV3dwzB71q5nf?= =?us-ascii?Q?Q0i+yYOl7/lC6xn1GQwJU0sPYcm1D5OjKxhYUPzTJhSfF0VikW1678UkwJPh?= =?us-ascii?Q?2JawRtDwUnwKWHBmDUZfU0lo592eaYfEVdN7LJUZNNS95ULFzg7paM+gDQ/E?= =?us-ascii?Q?TBvRxBdAhXhvr/5H0PrYt8xELaSCN9IPYUQR99s4xk+SWUz//wW+k61iBtUs?= =?us-ascii?Q?Fjixn7BfWoaySE7xZE0gQwZ6AzHOEU8uouDgd6RYPpHQWCZNGi1nQ7oM4VoN?= =?us-ascii?Q?nxv68mwjZoI=3D?= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230040)(61400799027)(82310400026)(376014)(36860700013)(14776008)(79816003); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2024 05:07:02.2874 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c1923c0b-81c1-464b-e0e5-08dcd9fb3a9f X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: LO1PEPF000022FD.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWLP265MB6705 Received-SPF: pass client-ip=2a01:111:f403:c205::3; envelope-from=mickey@masteringemacs.org; helo=LO2P265CU024.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Examples with javascript-mode. It holds for all modes i tested with a TS equivalent. Let -!- be the starting point and ^N be the subsequent position after a movement command. -!-export const add = (a, b) => a + b; Repeated `C-M-f' yields export const add = (a, b) => a + b; ^1 ^2 ^3 ^4 ^5 ^6 In other words, it works as it always has. Meanwhile, in `js-ts-mode': export const add = (a, b) => a + b; ^1 ^2 ^3 ^4 >From ^1 and back with `C-M-b' export const add-!- = (a, b) => a + b; export const add = (a, b) => a + b; ^1 At this point, `C-M-b' no longer goes back. It is stuck. Another example: -!-console.log("Addition result:", result1); With `C-M-f': console.log("Addition result:", result1); ^1 ^2 This affects every single -sexp function that uses either `forward-sexp-function' or `transpose-sexp-function' to do its job. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 03:44:40 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 07:44:41 +0000 Received: from localhost ([127.0.0.1]:50977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stjB2-0002CY-9l for submit@debbugs.gnu.org; Thu, 26 Sep 2024 03:44:40 -0400 Received: from mail-oo1-f53.google.com ([209.85.161.53]:60588) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stjB0-0002C6-3N for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 03:44:38 -0400 Received: by mail-oo1-f53.google.com with SMTP id 006d021491bc7-5d5c7f24372so380899eaf.0 for <73404@debbugs.gnu.org>; Thu, 26 Sep 2024 00:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727336585; x=1727941385; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=9BYnUuo77WxF5NtFm5j2/xu/bHsG/8PbViQG5Sih8OI=; b=PHfH4WaVdTLKF5Z9uYL8EjgRt6L14004ObwZaRxRELJm2Zc01Wx9PMOLROOZdwX2/0 A7T+4q9TIXQGZH0mgiN7dYiIyOW7yAfo7CiVHjYXBzJw4VOYcAatTx2je8Qyu+yZS+mu njrvUBSex2MYJh/sH+/Qps85B1/HG1jL2uLYQk/OagVdbe8k45+vuooO+usLyaMgowfa 2V1wtOrgupB79NBBDf++H09LwuvjQh3sEq35s9zxS/bphQ2Zer9UosrhliShS9wWOIFV QWS6uhu/BRYooHHylhgCekaIeopkJKpKEegmhxjJia7+tST/Ojp94dNIK055fkVwwDaR qhHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727336585; x=1727941385; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9BYnUuo77WxF5NtFm5j2/xu/bHsG/8PbViQG5Sih8OI=; b=GuFTFajxQiGJkoUDvdFFhXEYS51FZBhYLpZb6lqmJfu7TycRZ4mFssfvrCEuV8TqC3 1KyR5ufu5sHuYjg6xKxmlM2kDMlQvj9qnFBvwXPXBvORY5Yj32+urojC9Pf0TaQ4156y W1CtOsA5MYe8OBkVig//1xQ+ubECXiW6+OldJhhJYBDuCCZGUPBgLSfiUXibrB72wuhf 5iax8ThLNwUfQ8jNgpd+DERgfvwl+XsBIAjhZKieOe5aRLDtyGe1r8hiTpOUOn/RlYg5 bECToQLdLcTHKviW+WKCjWGpinpgVvcnVfKcExstCYsVfbhzfyHCvJ/09AAYoJfJ5hVd hcFg== X-Gm-Message-State: AOJu0YzlDaiK4FEYPdiWZqmRFZnyJ6jefZWOe9ZfcHVS5ISq8XmNinH7 tZh//R5oOIvb9tWUweakGRIAYIpJ+8mtncO2zwqsK1BTSdBBezGh X-Google-Smtp-Source: AGHT+IF4jQdbztoi7KLWdvp62DuuFNB+x91J5LWSCAyafdN48OhRqSnN8mxaWL/9gCpOSIAWdp0Wxg== X-Received: by 2002:a05:6870:8089:b0:25a:eca3:6b5e with SMTP id 586e51a60fabf-286e129f9d1mr4068254fac.9.1727336585205; Thu, 26 Sep 2024 00:43:05 -0700 (PDT) Received: from smtpclient.apple ([2601:646:8f81:6120:b979:1835:aae4:102]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7e6b7c5c06fsm3760346a12.55.2024.09.26.00.43.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Sep 2024 00:43:04 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87plox4mtp.fsf@masteringemacs.org> Date: Thu, 26 Sep 2024 00:42:53 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> To: Mickey Petersen X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Sep 20, 2024, at 10:06=E2=80=AFPM, Mickey Petersen = wrote: >=20 >=20 >=20 > Examples with javascript-mode. It holds for all modes i tested with a > TS equivalent. Let -!- be the starting point and ^N be the subsequent > position after a movement command. >=20 > -!-export const add =3D (a, b) =3D> a + b; >=20 > Repeated `C-M-f' yields >=20 > export const add =3D (a, b) =3D> a + b; >=20 > ^1 ^2 ^3 ^4 ^5 ^6 >=20 >=20 > In other words, it works as it always has. >=20 > Meanwhile, in `js-ts-mode': >=20 > export const add =3D (a, b) =3D> a + b; > ^1 ^2 ^3 ^4 >=20 > =46rom ^1 and back with `C-M-b' >=20 > export const add-!- =3D (a, b) =3D> a + b; >=20 > export const add =3D (a, b) =3D> a + b; > ^1 >=20 > At this point, `C-M-b' no longer goes back. It is stuck. >=20 >=20 > Another example: >=20 > -!-console.log("Addition result:", result1); >=20 > With `C-M-f': >=20 > console.log("Addition result:", result1); >=20 > ^1 ^2 >=20 >=20 > This affects every single -sexp function that uses either > `forward-sexp-function' or `transpose-sexp-function' to do its job. >=20 > Thanks. >=20 I=E2=80=99m aware of this problem and it=E2=80=99s quite inconvenient at = times, but right now I don=E2=80=99t have a good solution for it. Ideas = are welcome. Basically tree-sitter=E2=80=99s sexp movement works on subtrees. It = determines the position of the point in the whole parse tree and goes = forward/back across the next subtree in the parse tree. If there=E2=80=99s= no more sibling subtrees in the same level to move over, sexp movement = stops like in lisp. The parse tree is invisible and often groups token = in unexpected ways, so many times the sexp movement isn=E2=80=99t = intuitive. We might need to add a user option so people can easily turn off = tree-sitter sexp movement, since it isn=E2=80=99t a strict upgrade from = the generic sexp movement=E2=80=94it=E2=80=99s more of a different = flavored sexp movement. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 06:10:22 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 10:10:22 +0000 Received: from localhost ([127.0.0.1]:55441 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stlS1-0003FW-88 for submit@debbugs.gnu.org; Thu, 26 Sep 2024 06:10:22 -0400 Received: from mail-ukwestazon11021123.outbound.protection.outlook.com ([52.101.100.123]:19911 helo=CWXP265CU009.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stlHz-0002YH-BC for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 06:00:00 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UTfeCCUfRibQoN0hUT5Vyzbk6DUiQUj5XYn1e4Q1lFcZt2A88yFtXoQf40srr/8MzhbapPlIYHRc+z+qv40dhD30madpoukaxUWgNtCCqUvJJaDmsdmVlPYC+HnKKHUVui+gmullb6Tn7gH2h7TyiDmZ/4fjSddgbMmUjMWEt31dsDk0fvNaivijko/G5sDaMdk1iOUCJ4RlVWVznA+7yfaTJQrVjuyweIsOVKTgQ9aCV4lULjeGtzH8cT99vivK3GQfzVV28QbTqEWtCncYCmcp1Rp7JEshF3ITpqG4i3OUmqbB8LwKeohgBYGzCUITqoaTddKvBR2GFCvBW5lbCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TdkEuSv7SxPK1/iCVt6IT+GDYBicIQDQeX4zMONkDEA=; b=M2Ut6oP5dtbFv0c+H4MZdgSBd6ajyzpyQd5E30Moeykv/bFUlqeVXCdQHv/HQQb4NQO1amFtB7S4LQvUqMiuZX0+Qvo2hqgDZiI5ZQ0eHOwyCTOxYBLqxdJB6EpJU1g3Aw6maB799M7W/udEoLil4AHsrPpwxJvvH6QSwUK4GcLKLndwrt/mHj5+Srak0KF5g/V5JbSyZQgmVhVV39CN6aH0xvS26ziTBjLLGWmvtKnDALiFajxijHoW45I4cZgWTOMcfRTm+v8SlItBOaprMI4wdtjCLCuVgmcAtSQ0JmySEC2dVea4kCbsZNY1WTWyNI92woWRkKA5BYsV0gOiCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=debbugs.gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TdkEuSv7SxPK1/iCVt6IT+GDYBicIQDQeX4zMONkDEA=; b=00N7qzfTynOP51RXaWWnXC5mRhf/3eFqJtRol8iUbkqHbW9Ma1p1/WRuzlHDACIRauV8oosPureJLgVN2rY5cBwwdC2MAtaIk92gRUX5jxkr1mR7DI/CqBEQ+8FiHvN5tdIC4CLLZpZmNUW21nzqwJl+3jyRUE+EEmKtYxMCIVs= Received: from LO2P265CA0064.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::28) by LO3P265MB1996.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:10e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.21; Thu, 26 Sep 2024 09:59:05 +0000 Received: from LO1PEPF000028CC.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60:cafe::6e) by LO2P265CA0064.outlook.office365.com (2603:10a6:600:60::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.17 via Frontend Transport; Thu, 26 Sep 2024 09:59:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by LO1PEPF000028CC.mail.protection.outlook.com (10.167.240.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.15 via Frontend Transport; Thu, 26 Sep 2024 09:59:05 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 223CE114002; Thu, 26 Sep 2024 10:59:05 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 References: <87plox4mtp.fsf@masteringemacs.org> User-agent: mu4e @VERSION@; emacs 30.0.50 From: Mickey Petersen To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes Date: Thu, 26 Sep 2024 10:56:35 +0100 Organization: Mastering Emacs In-reply-to: Message-ID: <87frpm20t7.fsf@masteringemacs.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO1PEPF000028CC:EE_|LO3P265MB1996:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 6c05bfe7-55a0-48ed-b42a-08dcde11db4f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|61400799027|82310400026|79816003|14776008; X-Microsoft-Antispam-Message-Info: =?utf-8?B?UXJuZ0EzaVFCNlVyK1NXZ3RweVZkbFhna2ZqSHlycFRIWlNGMEgzSHBBQWti?= =?utf-8?B?ZUpvc3FVVy84UnBVZGlkak9mTkNQdGF2eG9nZDlISG5qSW40RTNqN0c3Lysy?= =?utf-8?B?QWQ5ZUNIejNNNGl5dVNFbkFmQUQrOG9nNmdPRTRxWTF4dzA1OU1mRnI3Nm5N?= =?utf-8?B?ZjBXODZwRE5pWlo4c3VKakluaGRQemhEN21YeXhvandhVklnVWhKaVdQY043?= =?utf-8?B?L01obTM5K2JlTE1kQm01ODBwamlkaDRxQ0VpdUdDOVQxalZPZ2pDa0RpKzJO?= =?utf-8?B?cFYwUHJFV1NRN0hnMEJwN1Jod1pzS0c0ZlJ0K0R0TDl5bmkrNW5wcG1UYmpY?= =?utf-8?B?b3l6aFJnZEs1MDduVjlOQ3B4SUNDc2ovUVVIRFBJTWVKRXpkREFkSzN1N0Rr?= =?utf-8?B?cWZXTWhVZDMzdUFUaEt2YXptMUFvWklJbHloeVpLTW1EYjhLcEhwM2VQTU9o?= =?utf-8?B?QmVkcWxNUCs2Z0RYbHdLRVhja25kQW9ndGtMcVUzRlJZNmF4Mm5YOVdKMTJk?= =?utf-8?B?NnB1K0lBOFZqcE41Q25WVnE3TG0weWJKWkpjU1ZHczZxcklOU2p6bk1PbzAv?= =?utf-8?B?TTR1ODhjY2VVbEVaOXQ1YXdiK1R6T1BSZFN5VnRkd3hVNUFmc2tvVmF3VTVY?= =?utf-8?B?b0tzcFZWOTNWcU1iUVB2QTgyZTBia3hKbm1ZOEpmUzFacnlXQ0pycjlKQjZ1?= =?utf-8?B?WlVBNC80emxOMVh0WUZHS01FeHpoN3kvNXN5K3pVY0x4bVdHTXNyT1ZPMTlS?= =?utf-8?B?RFJvVzE4cDJqRVVzZUdPN0NSeGhvdkRmNnVHK2NDSEpXeURyTjJuYk1wTHV5?= =?utf-8?B?a0Zzb1RmdHplejE5OGRHZ2phMXBOR3FsaU0vZmtIMndCZ0FEM09ueCtUcXJz?= =?utf-8?B?Q0FBQjJtZHhCU3lMYk15cFhqTm8wbnpHZUVTT0RNU0dvOWRnaG5kU05DbGxh?= =?utf-8?B?aUIwR0lWbkJwTU9rVC9YNytwWUJWckNTcHJhU2ZBUHVZM2pIcEQwaFIyOWdX?= =?utf-8?B?c1MrYVBNM1pmcktJVGIrWmZpaVFwKzhmelhDeS9rR1YvSThpK0JPVWVlNWpn?= =?utf-8?B?b1BJVHM4RzM5d0dZbHB4N05lRUtaREEyL2kyZ0h5Y3BnblRUN1Vjd0ZlM3A5?= =?utf-8?B?ZEdpRmRNN3c0cWdxZ3NGSUNsQjk4ZG5rWXR1UlhOSlc5OVEyS2FwSENGNTF1?= =?utf-8?B?Q2tBYVlvY0JnRGkxUUNWV2JUemt2M2FySU11TWFUQk52UGRoWFlYelNadm5H?= =?utf-8?B?RGdHaFJkTU5hTXMzOWNQcENNQytvWlI1cHRIbGRCRlRKeTlFOTF5U01NbEls?= =?utf-8?B?aDRZZ3o5VllnTUVjQ29sbzhPZDZVS0plb254WVNKYW9PamozV2ZoODlYbXpH?= =?utf-8?B?WWdCWGJUdXE2MnA5Mi91RDZqY2ZmWGZ4eDZiR1RqZGNhSVRlanhiNmhwR1Bl?= =?utf-8?B?VUZzRHd5dFROcEpaTkN4V0dtYmVYNUc5MHFDNGhWcnB2RE1DekJ4a1JwSGdH?= =?utf-8?B?Y3MxRzdGblo2Nm5ML296VHNUOVV3bExDd3V0aHlDVldkeVk4ODNraXhKVGZR?= =?utf-8?B?Mlp3djhpUk1CRk9WcmJqWDh3TXB6eGJOdmJpMVk4MmxDcW14cFAwWWcvd3pp?= =?utf-8?B?bHMrUjBNR2M3R0Q0NVBJTFJPdnJUU0JCcHJKTlFFbzVaTFNhS0hJeFRDRno2?= =?utf-8?B?TFcvbjFrbTlIenRxZmhYc1JpZ25FK1duM3dta2Z5MWFzWGR3V2xxZGdmZ1hj?= =?utf-8?B?SGVreW1iTTBFdWppKzJ4MXI2V2hSc0ZUNlpDWU9BaXgzL2JndVFiR2hvOFV1?= =?utf-8?B?S2J2dE0vMWJIYU95b1R3WU02S3l2L3MwZlZLbXRIR3BQMjdMcWtOZ2t1Qy93?= =?utf-8?B?YmY2eUFzbk91Zkp1VGdSSlhJZzJMTVJNditqbEVWTC9yZC96Tmc2ZGp0SFQ3?= =?utf-8?Q?I3kkRE6BasU=3D?= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230040)(376014)(36860700013)(61400799027)(82310400026)(79816003)(14776008); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2024 09:59:05.2667 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c05bfe7-55a0-48ed-b42a-08dcde11db4f X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: LO1PEPF000028CC.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO3P265MB1996 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Yuan Fu writes: >> On Sep 20, 2024, at 10:06=E2=80=AFPM, Mickey Petersen wrote: >> >> >> >> Examples with javascript-mode. It holds for all modes i tested with a >> TS equivalent. Let -!- be the starting point and ^N be the subsequent >> position after a movement command. >> >> -!-export const add =3D (a, b) =3D> a + b; >> >> Repeated `C-M-f' yields >> >> export const add =3D (a, b) =3D> a + b; >> >> ^1 ^2 ^3 ^4 ^5 ^6 >> >> >> In other words, it works as it always has. >> >> Meanwhile, in `js-ts-mode': >> >> export const add =3D (a, b) =3D> a + b; >> ^1 ^2 ^3 ^4 >> >> From ^1 and back with `C-M-b' >> >> export const add-!- =3D (a, b) =3D> a + b; >> >> export const add =3D (a, b) =3D> a + b; >> ^1 >> >> At this point, `C-M-b' no longer goes back. It is stuck. >> >> >> Another example: >> >> -!-console.log("Addition result:", result1); >> >> With `C-M-f': >> >> console.log("Addition result:", result1); >> >> ^1 ^2 >> >> >> This affects every single -sexp function that uses either >> `forward-sexp-function' or `transpose-sexp-function' to do its job. >> >> Thanks. >> > > I=E2=80=99m aware of this problem and it=E2=80=99s quite inconvenient at = times, but right now I don=E2=80=99t have a good solution for it. Ideas are= welcome. > > Basically tree-sitter=E2=80=99s sexp movement works on subtrees. It deter= mines > the position of the point in the whole parse tree and goes > forward/back across the next subtree in the parse tree. If there=E2=80=99= s no > more sibling subtrees in the same level to move over, sexp movement > stops like in lisp. The parse tree is invisible and often groups token > in unexpected ways, so many times the sexp movement isn=E2=80=99t intuiti= ve. > Hi Yuan, In my opinion, that's not what `sexp' movement is. Sexp movement is movement by balanced expressions -- and a fallback to word-like behaviour absent that -- and this is not that. It would be better to relegate this sort of thing to its own set of keybindings. > We might need to add a user option so people can easily turn off > tree-sitter sexp movement, since it isn=E2=80=99t a strict upgrade from t= he > generic sexp movement=E2=80=94it=E2=80=99s more of a different flavored s= exp movement. It should be opt-in, not opt-out. > > Yuan From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 06:55:01 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 10:55:01 +0000 Received: from localhost ([127.0.0.1]:56016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stm9F-0005t5-0U for submit@debbugs.gnu.org; Thu, 26 Sep 2024 06:55:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46578) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stm9D-0005sr-CW for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 06:55:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1stm8b-0006N0-M6; Thu, 26 Sep 2024 06:54:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=YbzFjl4B6d8m0e7Tz7S13UhgPn3JtsZGBh1JD8WKs9A=; b=nnea3bZnVBNtLGRlcJ3b HzKjIEpe1LuWKPS+2hepRn6Jsez9O69o037yuDipcDzuvBxA+lsdieKeDR8v1Kfzay5EWeNcuzEsJ e5sKRtzKjNFN3r7FGJ7Ysg2oJapOe/9m9hwq2H79WiE2zkxRDX+I1fUF3xjzj4mUD2QRVCeMsAy48 8X4bIVx72ba4E914JckZwSMUhHkUF8giP6S7iEW5X6nikAkmCN96F4nmj3mulM5XYWLYUOqi0ZqS3 Dr1oi1P0IknhEEkGFjib2O24eSLd6G9JrMIivsngxJiftbs765F8WdS/rybrAnzNPP0baHOvCA+Cl FYV/lp7tVIfr2Q==; Date: Thu, 26 Sep 2024 13:53:54 +0300 Message-Id: <8634lmbs8t.fsf@gnu.org> From: Eli Zaretskii To: Mickey Petersen In-Reply-To: <87frpm20t7.fsf@masteringemacs.org> (message from Mickey Petersen on Thu, 26 Sep 2024 10:56:35 +0100) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 73404@debbugs.gnu.org > From: Mickey Petersen > Date: Thu, 26 Sep 2024 10:56:35 +0100 > > In my opinion, that's not what `sexp' movement is. > > Sexp movement is movement by balanced expressions -- and a fallback to > word-like behaviour absent that -- and this is not that. It would be > better to relegate this sort of thing to its own set of keybindings. The term "balanced expression" is not well defined in languages other than Lisp and Lisp-like ones. It is clear what expected when point is on a brace or a parenthesis, but entirely NOT clear when you start from something else. For example: int foo = bar + 2 * baz; Suppose you start with point at "foo": what would you expect forward-sexp to do? nothing? > > We might need to add a user option so people can easily turn off > > tree-sitter sexp movement, since it isn’t a strict upgrade from the > > generic sexp movement—it’s more of a different flavored sexp movement. > > It should be opt-in, not opt-out. I disagree. Moving by sub-trees is a natural generalization of sexp movement for languages where parentheses and braces are rare and far in-between. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 08:26:29 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 12:26:29 +0000 Received: from localhost ([127.0.0.1]:58746 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stnZl-0003T1-4Y for submit@debbugs.gnu.org; Thu, 26 Sep 2024 08:26:29 -0400 Received: from mail-ukwestazon11020112.outbound.protection.outlook.com ([52.101.195.112]:20007 helo=CWXP265CU008.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stnZi-0003SW-2n for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 08:26:27 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=R2ekTKIgSbqArKeWq0qfBbKO2icNC6pM57stJzilf9qO9CRiNxmPt91Kj27JGF3Ix5O5WfpxexXi50VK5rvOtvzXJCNRBiIPmqutGLr7P2JvICs67Pq57pK/lwQ68QVZZPVCF9IULfznRERiedZylPjcOcehQx5sGDrtnL/MqrCh7DLSM+WZR6SequniwkY5y5CYUwDCM3UDUFSGGh2V6KEyhkBB8brnt06MXwujF4Z84CizGIT30V5J7R8f/aGJTcVze4499pM4JbZElg5P0Af7LUlY1okbHHyBv9FACF4aLpoKB98c+z6YATYBdiH4JvgVPYpcFtksRHizBfcXEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aJWWfCh8W7UgZpAHEhRnZYgAxLeV03Ug+zHWqGO+6Y8=; b=B9uGVyOMY9M2TJXHQVPSmgEAihGsODa+3fYL7meug4TDqeD4otTfpKyKVGGfHZHNvsjTKECbN8PuZEvMZZ1FU3uPqrSaBxHs9VpCHzsbXsODchdvrRTBQQpIv1pNwAijuTqaIcu8hSDpgJU5oW64U2cATMgdJnBIyBiwvAmj91SdqoYY5/ETGsgMTRY5nPCN23pADWK18vjcW1pdpy16AtnTn2Y9z/zgOZhI5CweaJl1FRHSdmyiSqWP1O/mSrUFzYnvBSFEn0ZxKLbqfUmtDMRS1/19uPL4qDSfrmoayLlUeiY1O5zpeSrZ14G8SvuhBWT8Y3p3TZTt+aIrlzy0zA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=debbugs.gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aJWWfCh8W7UgZpAHEhRnZYgAxLeV03Ug+zHWqGO+6Y8=; b=N37TpcKMDKOFmy9RtiAkab8wOcNfTP8DldYqPhCID/qFTGXQ6G/I+lLdhsBry78USRj28391VsTAIx9OZ0KyJkqIrzMPiGCemETqYlrBHzGk6pQVmOSOj1DnXD9Fc3pgOO8/QT7lqXe+5Iev2NSyC1RzqcgCw8OfmHcvsT7FuN8= Received: from LO4P123CA0426.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18b::17) by LO9P265MB7647.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:3a3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.27; Thu, 26 Sep 2024 12:25:50 +0000 Received: from LO1PEPF000028CC.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:18b:cafe::8d) by LO4P123CA0426.outlook.office365.com (2603:10a6:600:18b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.21 via Frontend Transport; Thu, 26 Sep 2024 12:25:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by LO1PEPF000028CC.mail.protection.outlook.com (10.167.240.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.15 via Frontend Transport; Thu, 26 Sep 2024 12:25:49 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 784EA114002; Thu, 26 Sep 2024 13:25:49 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> User-agent: mu4e @VERSION@; emacs 30.0.50 From: Mickey Petersen To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes Date: Thu, 26 Sep 2024 13:13:53 +0100 Organization: Mastering Emacs In-reply-to: <8634lmbs8t.fsf@gnu.org> Message-ID: <87bk0a1u0o.fsf@masteringemacs.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO1PEPF000028CC:EE_|LO9P265MB7647:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 296f569f-d0d2-4a47-7ee1-08dcde265b0a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|376014|36860700013|61400799027|14776008|79816003; X-Microsoft-Antispam-Message-Info: =?utf-8?B?RzJXa2I3czByUUJldDhOcHpTTmZFbFJRUEZXSHVTekJpTlJnbDR4RjAvZmRv?= =?utf-8?B?NUh0U3ppNWxwVCsycFk4UnhBcEpvR29oT3phcUtuY1lnRmtqWUcwZ3lCV3F2?= =?utf-8?B?cDl6TmdsSlBWR0JxMFhFVlpYenBoRldNdlZHN1RPSG1PTXVRK3U0MmZ4ZHpW?= =?utf-8?B?dG5XL2gyZzB6bjBnZnF1bk03c1B5bko5czM0cXNGT2pnTU1Vb2dBZ3c0S2Z6?= =?utf-8?B?QlNYenpmV2xyYUFiK2RMS2pEV0pGOC9mNTFXRnN6L3BlNURwVmpDSVgxZDd0?= =?utf-8?B?RzFtbjh3NTNGMkFhSHVjQ01YaS92YjFzQjBtakUrY1dMaU5iems2aEFpekVK?= =?utf-8?B?VjdkMHNHUTZ0VTAvRlI2UUhnSk4xN3lXYzRoYXFidWYzbG5La09GS3FkamlW?= =?utf-8?B?WUxSbmtvb3pEM0FsRnVrZExnUmZuOWdoNFlOcWhmR044ZUc4SUlxOXRqNWdJ?= =?utf-8?B?SjRuU0tNbkxHV0lZdlgwOWl3bzBITHBadG1jWkZxb0pVUkwrelBnUkk5VEpM?= =?utf-8?B?SFhRL2VKZnhpRHFMYzVHOWhFR3V2RHNPWFc2OFU0V2c4R3lQSW4vNkFEQXQr?= =?utf-8?B?QXU2eUVBNE9UM0ROb3hZTVFYSC9LRFNkdEtTRERqT3JHQ0dGWnhPRndJSS9i?= =?utf-8?B?aXAvNXd5cE9HNXYweEZLeUk4cHlYUElBU2lqcWcveFVydHU3V3hMcEYwcnRR?= =?utf-8?B?R3RLdTRUb3ZQUWxlRHFCV2FqM2k4MWlGQ1k0M0N5SjhUMVYvTEJBUE1vR3hW?= =?utf-8?B?U3RuMU9kbysyZWJLSjZvNCsySXBDbXhPSFBBSU94T3huVzZFWk5RQjFLNkN1?= =?utf-8?B?MW95UkxtbXRZdk5BakIxUHh4TW5aVXl2SmdDSHcxYzJNMUFjQWhzOENobUdU?= =?utf-8?B?VVdvS0NSVnRXL0ZmaG4zYmdzd0xLMkE0WnFUR0VZSmd5SDBwRFBQdmJueUw5?= =?utf-8?B?alVmdkdRaVQwNnNadHY5WVFGWDdPOFFNMndsRHgwQnd1REQyeUNTMGU1UTUr?= =?utf-8?B?TFNtSTE2K0k2eXZLUENCVERlSDZBZUpvVzlRYzZPY2RkZ0sxeXY1N2srdm5i?= =?utf-8?B?MHFMVDBWVVZQWVpKTE45czdJejVOODhmcXdCbkl5Ry8vMm9sVmQvRDJvUHhx?= =?utf-8?B?NEFWUENoTkkzTUFtdjlLTEs1ekV1UlF0bzIzZ2g1emtwUi9URC80dE41RUFI?= =?utf-8?B?WSs1NXVaRTF5L25NZ2hTd2RNbHh6dGtXKzVCY1JEaUd3RytPQ3JjaFhOTHl0?= =?utf-8?B?UzVTRHBlbWFoZkQ3YW1CUnYwL1hEcHNkRHZHUXpHbVFhNlNOY2ErSGpENG4x?= =?utf-8?B?S2x2aGptQjQya2Rmb3NLdE1FU2J0QzRqcXkxZm5LT2NzcTBmL2FYM1VPMEpw?= =?utf-8?B?UFFHeDlmcWtpR045KzF4eTAxT2tFYTNFWFZGcU03SEsrVmtoSU9zMEZlSHNx?= =?utf-8?B?Ylo4WnBrRDhOckhrcmVGbDQyZENsZXhiWWExU0tRZDA4SHdlVTZGRmZLN1NZ?= =?utf-8?B?UW83N2huYTgyci9nMjEveFVqRFJFUVd2dFhlYkowUWUzQlE0MTVLRkZ3Rmpp?= =?utf-8?B?OUZHd2ozRUI1SENYdVdOVXh1Unc3TG5CZ2FOa1hzMTd0Y0lORVJjNmc5U2lB?= =?utf-8?B?dU96NjZ1MEozQXBxMHFLS3Rici9wbzVSSWxDaFdqUnVTWFNaWHA5b0ZUeVRP?= =?utf-8?B?NmJYaEdGNnVCUHpwcWwxeExXTE53OUlGaWJMRnZpa2g4enVTZmRCdVpqaENp?= =?utf-8?B?TDFtZk8xeDFtUHAxNSsvVU9iQkk2S29CR3U4bXNNWnRzdkUwTGlkYW12L2RE?= =?utf-8?B?RlNuanBra0tFMmtUR3lqcHlERDRHVkpIbVRFQk4rOC9nNjBQWFJqUUlSYkNM?= =?utf-8?B?bjkxMVRDdjBiNFJEMW5zUUczODRMV3Y1TFN6NkpSbHQ2STJ3dzdIYmVmY3dH?= =?utf-8?Q?GKAKx9jCfnc=3D?= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230040)(82310400026)(376014)(36860700013)(61400799027)(14776008)(79816003); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2024 12:25:49.6680 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 296f569f-d0d2-4a47-7ee1-08dcde265b0a X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: LO1PEPF000028CC.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO9P265MB7647 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> Cc: 73404@debbugs.gnu.org >> From: Mickey Petersen >> Date: Thu, 26 Sep 2024 10:56:35 +0100 >> >> In my opinion, that's not what `sexp' movement is. >> >> Sexp movement is movement by balanced expressions -- and a fallback to >> word-like behaviour absent that -- and this is not that. It would be >> better to relegate this sort of thing to its own set of keybindings. > > The term "balanced expression" is not well defined in languages other > than Lisp and Lisp-like ones. It is clear what expected when point is > on a brace or a parenthesis, but entirely NOT clear when you start > from something else. For example: > > int foo =3D bar + 2 * baz; > > Suppose you start with point at "foo": what would you expect > forward-sexp to do? nothing? > I expect it to behave as it presently does: default to word-like behaviour such as M-@ / M-f etc. Balanced expression is not well defined, de jure, but it is in practical terms, making it de facto rather well understood and supported. It behaves reasonably consistently across languages, and I use *-sexp commands thousands of times a day in a wide range of major modes= and contexts, both in code and also prose. Most people who use *-sexp (or *-word commands for that matter) in major modes come to recognise how they work and know what happens to the text/point in their buffer before they run them. I would challenge anyone, given even small samples of code, to do the same with the current TS only implementation. >> > We might need to add a user option so people can easily turn off >> > tree-sitter sexp movement, since it isn=E2=80=99t a strict upgrade fro= m the >> > generic sexp movement=E2=80=94it=E2=80=99s more of a different flavore= d sexp movement. >> >> It should be opt-in, not opt-out. > > I disagree. Moving by sub-trees is a natural generalization of sexp > movement for languages where parentheses and braces are rare and far > in-between. Yes, if one can intuit the sub trees' structure, which is not so simple; and if the selection of commands are sufficiently expressive enough to let you navigate the tree. I am not sure they are. The CSTs are deep, wide, and nodes' ranges frequently overlap; they are multi-dimensional structures that map to a simple 2-dimensional 'grid' in your buffer. Making heads or tails of that is no easy feat. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 09:47:48 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 13:47:48 +0000 Received: from localhost ([127.0.0.1]:33818 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stoqR-0000qV-8b for submit@debbugs.gnu.org; Thu, 26 Sep 2024 09:47:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43088) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stoqP-0000q2-Aw for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 09:47:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1stops-00015I-EM; Thu, 26 Sep 2024 09:47:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=MQK8z6L1yU9zsurrJOLUPUSvPp+uYQ/Qrj9OT14SknA=; b=BxsOmYDPZU1b AkWaQ2wAeeF1AsprWzsOrepSfXfFHVYj1k6TbNFCUV/VGm+EHWqPP7MRROZsMu5WBedbuLHLzF1q7 JOEO7c1if8S1OZgk7jf1AzPy33fj9L6WuRykRLz5MjaHbdWKV8iNKS6AuaWS4YX7QGQLit3ZN9cax T8xDbSjoonY5a+eRUp3mmva+l3iDEfwQ1cr7SjrqSbS8u98GjO6XGmYKbk8QyGHHPz6gjcYQFO1kh tUqguYj5qf01Z/Fc5b7inlmEXSR0sjneHBk/s5Vnr4Xvg7k10+W8i+hYlSoDHXvwZ3RJij0imSSaF EMAHz/MUYxCGwobsK2P81Q==; Date: Thu, 26 Sep 2024 16:46:52 +0300 Message-Id: <86tte2a5o3.fsf@gnu.org> From: Eli Zaretskii To: Mickey Petersen In-Reply-To: <87bk0a1u0o.fsf@masteringemacs.org> (message from Mickey Petersen on Thu, 26 Sep 2024 13:13:53 +0100) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, > NO_RELAYS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no > version=3.4.2 > From: Mickey Petersen > Cc: casouri@gmail.com, 73404@debbugs.gnu.org > Date: Thu, 26 Sep 2024 13:13:53 +0100 > > > Eli Zaretskii writes: > > > int foo = bar + 2 * baz; > > > > Suppose you start with point at "foo": what would you expect > > forward-sexp to do? nothing? > > > > I expect it to behave as it presently does: default to word-like > behaviour such as M-@ / M-f etc. Then we just lost an opportunity to have more useful commands, because we already have M-f and M-@. > Balanced expression is not well defined, de jure, but it is in > practical terms, making it de facto rather well understood and > supported. It behaves reasonably consistently across languages, and I > use *-sexp commands thousands of times a day in a wide range of major modes and > contexts, both in code and also prose. I think the ability to move by parse sub-trees is also very useful. > Most people who use *-sexp (or *-word commands for that matter) in > major modes come to recognise how they work and know what happens to > the text/point in their buffer before they run them. > > I would challenge anyone, given even small samples of code, to do the > same with the current TS only implementation. That's just a matter of getting used to the new semantics. > > I disagree. Moving by sub-trees is a natural generalization of sexp > > movement for languages where parentheses and braces are rare and far > > in-between. > > Yes, if one can intuit the sub trees' structure, which is not so > simple; and if the selection of commands are sufficiently expressive > enough to let you navigate the tree. I am not sure they are. There are enough situations where moving by words will also surprise you. For example, did you know that M-f stops when it finds a character from a different script? And yet we still use these commands. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 11:25:09 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 15:25:09 +0000 Received: from localhost ([127.0.0.1]:40043 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stqMe-0008Ag-MN for submit@debbugs.gnu.org; Thu, 26 Sep 2024 11:25:09 -0400 Received: from mail-ukwestazon11021102.outbound.protection.outlook.com ([52.101.100.102]:29421 helo=CWXP265CU009.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stqMb-00089w-Lj for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 11:25:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U86vlHcaZ3U9tulS1iowMsBXejps2Gqv2NHquXMTlSmL7wBNiO8v3F1FZ0Mkxb8xoRJRvyrj7BGrkt91Z6KQ6YwMCfQhiqmLNLJI7lVSHQ9l4VjtOep42c7HVjmaawWLV2p87mn5aZUBjuLW0+uz72zGAsM7oKgEf7v1rc3B81nx0JoyMX1hBUydzwHPNrJb2rVJfoKJZLqVdB2nS0ZFRm/CpvIGucPLT/80t7toT7tUpOL9OvZRN1P10k4e9IwpWKnkK1ixElIfF50iqHgZISoDAL72QX+9Eyuv4BlyM6C2qLud6fkyN4gMXZeGKV4TG+PfvBmrPFZzkbKkEHVgaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=c+wILEaEqETF5Gj/Tbu99sZvpIhUKhLsaGMsZWQQ0FA=; b=jyonaRSZG2/RixPqcXd+Doi/2LfqtmNtJzqiqBLgU00nmOYV2g3ON4tr3dJFWY4GCimnissuDlz0/3eKCDQPaU03N0Fx+hRtn0eQda+/YtSLmEmssxSGPXYgUZVzBuXN5r6T70YvXDX6Cl9nCArNDo0a0555rVLcbWbpoa2jU0M0TpDirpjeAeVYWjzERlg7eBSLUAdxNZL0eCdeNaXi4Hqp7ypfrrGBPlijk6UXel46FpCOWpRBQm9tcMfSDEzdixLiggXxaYF8ryZpexmoCH4XtCVinsEnttZQWf7xcZhvIYf8Xpgl8plZQq0Pj2jSd5bY1KBoxN7vkn0DgQZ3mQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=debbugs.gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c+wILEaEqETF5Gj/Tbu99sZvpIhUKhLsaGMsZWQQ0FA=; b=eh1XVUYGHzhyF4qrLrRr0M+18KLC57T6SD6QvS+V9Z/NR9rA1+/vYC23ml+yS9q8skHs/6HQG1jAt0aZSusfnG355Ua8dcsHWeQ5pjvhIFHV7c2jfxwXuRCb7f4TtHh9ayMnuOaPP85aX6XczNUaxnJWux4xHYUpvo54BNWG6YQ= Received: from LO4P123CA0053.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::22) by LO7P265MB7938.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:40f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.27; Thu, 26 Sep 2024 15:24:30 +0000 Received: from LO1PEPF000022FE.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:152:cafe::62) by LO4P123CA0053.outlook.office365.com (2603:10a6:600:152::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.21 via Frontend Transport; Thu, 26 Sep 2024 15:24:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by LO1PEPF000022FE.mail.protection.outlook.com (10.167.240.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.15 via Frontend Transport; Thu, 26 Sep 2024 15:24:30 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id 1D47F114002; Thu, 26 Sep 2024 16:24:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> User-agent: mu4e @VERSION@; emacs 30.0.50 From: Mickey Petersen To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes Date: Thu, 26 Sep 2024 16:21:33 +0100 Organization: Mastering Emacs In-reply-to: <86tte2a5o3.fsf@gnu.org> Message-ID: <877cay1lqt.fsf@masteringemacs.org> Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO1PEPF000022FE:EE_|LO7P265MB7938:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 84322977-2a1d-49e3-fe0d-08dcde3f50fe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|61400799027|376014|82310400026|79816003|14776008; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?FJnCto/CjphgZWH90GzTeDyU64W+E8JLwCvc7vI/vRIiztUColWxcYTFSpyr?= =?us-ascii?Q?reZI323ip+RrXhWTS8lVmY196jxfIncQPB5eDHZwYvLZ8n4Ym5pY74/n4Ohz?= =?us-ascii?Q?l0SIQV32/NC/zXdqiPNuCGUAj1IxH60dmqyD6IquZv6Ck2cWJmdHR1j6GxGL?= =?us-ascii?Q?RlUcAUAAj8kMLjZ+0NIUHa7eZd16hq9naaq93u+0kQTXsYi4MIZKC58/NO8N?= =?us-ascii?Q?dlZydaSThslm/C1zkSepR0DD572vdB4h6tcOS88SCSFFsLq9NppW2RBA7i1F?= =?us-ascii?Q?3lIBBoE9ub66rGv5vG1H7SbODGKnwQP79pOKvZn1eOfIBU6Ho6/QzNz39QgK?= =?us-ascii?Q?OC65ilt1zw3gPLcYbRo87oE1jxXeqzzscKDuMnCyLhaJbbUfv++pW3h9komG?= =?us-ascii?Q?NHjoOMF4BsMpTMPNXlH2STYh2VWuoRQOBCQZuAzg3xVb/LU1QNbkxIH+Tecd?= =?us-ascii?Q?vk8EiUqXxvhOGG905aqj6MwiC4ED0jpfvuL3V8o30MCF/9OKtKIXeA6xGa58?= =?us-ascii?Q?cUddzJ7vVBsbtPGjv4AqzoaQ3cmqwQKHVmVovZ9xiNsqZalRkJFQHt1k/XTz?= =?us-ascii?Q?e4WQVZODwRQ7RclSmKXwCWr2mLGRVaQF599IOwd/CyiaLsSIF3KHam+uh0tL?= =?us-ascii?Q?q6V/nnmuWP4LoyJ61pOvNqM8Bny1rONd8PNr78Z5XYtU1VS551D85cmo5fB9?= =?us-ascii?Q?5mgjVsyqD1DgJd5+51p5lg0uqLwRiOA0ia6sWFj6UnKWcJYwYgrqH1v4QEsH?= =?us-ascii?Q?l/fG8488qub3Fd1pbqNmzvKGiOddPX4KXDYFk+dcoa7jZXTR2W5Wby0IBz8s?= =?us-ascii?Q?I5x/b+kDCsIbWVyd/7PH/vfFcXS1AzGyB+SoDNRmwDcNCauvZQjf/ps2WV68?= =?us-ascii?Q?vCFpMaZ5P1afO5b9KY8A4OHWxBNtyftoyz3jjMpJ++YyNfKQ6aRoXwJgGlMc?= =?us-ascii?Q?g63QGWrSu4bDVS5SHropdYcSstaannwinN5r2GY9oj/kSIJQyXnGSVbaVbgP?= =?us-ascii?Q?oxL34RzMuBrJPXEOYoq4JTpRE7gc3AiJAbw/crjbvIt3J0X8heqE9VUzg7VO?= =?us-ascii?Q?y/FuGr5cD1iU9z9zOWiVaN13VJnECNTnoi9WmCgXq8Hrke17QB/vgmBMc7Kw?= =?us-ascii?Q?RAH2ljH0NXtnAAn4AG6F8cZ1yg0j5c5SjcWNdFUKic0oJGU+kD73X6swAQgb?= =?us-ascii?Q?e2wuNu0653Of5WpXLVGUGrQITTP8ELE/6PzDBqC9kgeoCqvsd6GkoKLTThYn?= =?us-ascii?Q?l8d3HpuNXYQrw4rXQTKBM0fojcSbacTjSXPmdnCwI0KiTCR2YV2yOqYdx6IH?= =?us-ascii?Q?YS0xsASa9WLEbB3SOu32i5ECgfvQz13v+XS7dm4ZdJLNj0ZhdHl5BCZoOfae?= =?us-ascii?Q?MnhXsdeFve9+7BVaD+fK7Agm41XZ89hZ/sNS8F/D3CJ6wwxxuB/d2EqjRRWl?= =?us-ascii?Q?Av4ipNBnjIc0QPSFzykMJTziC/2F0egmRdeiL9xUKVqK33j+3wFU8ZlPNJoK?= =?us-ascii?Q?iGH1HmMfksoyP+c=3D?= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230040)(36860700013)(61400799027)(376014)(82310400026)(79816003)(14776008); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2024 15:24:30.2493 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84322977-2a1d-49e3-fe0d-08dcde3f50fe X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: LO1PEPF000022FE.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P265MB7938 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, >> NO_RELAYS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no >> version=3.4.2 >> From: Mickey Petersen >> Cc: casouri@gmail.com, 73404@debbugs.gnu.org >> Date: Thu, 26 Sep 2024 13:13:53 +0100 >> >> >> Eli Zaretskii writes: >> >> > int foo = bar + 2 * baz; >> > >> > Suppose you start with point at "foo": what would you expect >> > forward-sexp to do? nothing? >> > >> >> I expect it to behave as it presently does: default to word-like >> behaviour such as M-@ / M-f etc. > > Then we just lost an opportunity to have more useful commands, because > we already have M-f and M-@. > >> Balanced expression is not well defined, de jure, but it is in >> practical terms, making it de facto rather well understood and >> supported. It behaves reasonably consistently across languages, and I >> use *-sexp commands thousands of times a day in a wide range of major modes and >> contexts, both in code and also prose. > > I think the ability to move by parse sub-trees is also very useful. > Agreed. What matters is whether the crop of new sexp commands, such as they are, perform satisfactorily. Do you think the examples I listed in the original bug report match your expectations? If so, then it is probably OK to close the bug report. >> Most people who use *-sexp (or *-word commands for that matter) in >> major modes come to recognise how they work and know what happens to >> the text/point in their buffer before they run them. >> >> I would challenge anyone, given even small samples of code, to do the >> same with the current TS only implementation. > > That's just a matter of getting used to the new semantics. > >> > I disagree. Moving by sub-trees is a natural generalization of sexp >> > movement for languages where parentheses and braces are rare and far >> > in-between. >> >> Yes, if one can intuit the sub trees' structure, which is not so >> simple; and if the selection of commands are sufficiently expressive >> enough to let you navigate the tree. I am not sure they are. > > There are enough situations where moving by words will also surprise > you. For example, did you know that M-f stops when it finds a > character from a different script? And yet we still use these > commands. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 26 11:46:07 2024 Received: (at 73404) by debbugs.gnu.org; 26 Sep 2024 15:46:07 +0000 Received: from localhost ([127.0.0.1]:41127 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stqgx-0001Ht-5o for submit@debbugs.gnu.org; Thu, 26 Sep 2024 11:46:07 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52926) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1stqgv-0001HN-DS for 73404@debbugs.gnu.org; Thu, 26 Sep 2024 11:46:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1stqgO-0001P0-7g; Thu, 26 Sep 2024 11:45:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=yy0QlwTMur6cRUHgHjQ7qeunlN+hXY7W9aCoTTZndUU=; b=AF2D4Na3R10t h22zf5iYA1cw5ql1CT9k14HXewibjX82FbKzW7S+cFA8+UwfDwrkhnLZQOhJJ05+GmNWXCFnmd0/8 mWkDHHellRRCMwlzXMFLyrAMyumb99r10wX1SuouJp5Chw0MjT7EkEv7tFQ4sHiG7+cuIfeIvtpUC xxk5Oykb9YGxvBPVcsSPtiVQw2CB/FyOV3SWqpQFkks0kKvbwjPyXdU9j+YyvqyvphZf+ruaNRTUS 4EP+flbxX2Guf+D5Z72Q51FZeQcWsc4DUpkkOcSLYPQaPcgiCah/I5tLnQwYxi+AtE/JR4RX1iUQb edWzrxY3dsfPsOPU+6lsaA==; Date: Thu, 26 Sep 2024 18:45:28 +0300 Message-Id: <86frpma06f.fsf@gnu.org> From: Eli Zaretskii To: Mickey Petersen In-Reply-To: <877cay1lqt.fsf@masteringemacs.org> (message from Mickey Petersen on Thu, 26 Sep 2024 16:21:33 +0100) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mickey Petersen > Cc: casouri@gmail.com, 73404@debbugs.gnu.org > Date: Thu, 26 Sep 2024 16:21:33 +0100 > > > I think the ability to move by parse sub-trees is also very useful. > > > > Agreed. What matters is whether the crop of new sexp commands, such as they > are, perform satisfactorily. > > Do you think the examples I listed in the original bug report match > your expectations? If so, then it is probably OK to close the bug report. Yes, I do, but let's wait for others to chime in if they have opinions on this. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 27 01:44:55 2024 Received: (at 73404) by debbugs.gnu.org; 27 Sep 2024 05:44:56 +0000 Received: from localhost ([127.0.0.1]:45239 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1su3mh-0006ak-H0 for submit@debbugs.gnu.org; Fri, 27 Sep 2024 01:44:55 -0400 Received: from mail-pl1-f177.google.com ([209.85.214.177]:53549) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1su3me-0006aU-Q7 for 73404@debbugs.gnu.org; Fri, 27 Sep 2024 01:44:54 -0400 Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-206bd1c6ccdso16212675ad.3 for <73404@debbugs.gnu.org>; Thu, 26 Sep 2024 22:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727415798; x=1728020598; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4FDo8tsHntPZVrDxLpyN3MgQeMknqG+toQWLkoWJwZE=; b=KzzEaAYFOWwXMU5eF7TS5lDqkgDDHGtHBIbZNfAdkk6Lj2Tbyk2pwf77tbb499Gfcw 0gWWo8M+a+ocynz+K+/Pw/8XfRr3ceVGZKiByUc9p0rSPcE8WwLmGUcte1MEjNB0VFUg b9cclqldlu/EUpOVyNLZN8lAsPCvx8wnRddoyHzkuAlHrkwqkgeX1kpGnK4PyK/wO+s3 6fzN72dzPrfjr1vdlhttsSvDtYgFjOlh8cWBoUIrgxcYsz+jk9EB3DHX280z7/ILLieR FySeDkcTldrYxf8QudkPW9lQ9dS8zec1wIwFG2D6WFlW/SGU+pHjuzAbGSEEQkuiYhxt aBMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727415798; x=1728020598; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4FDo8tsHntPZVrDxLpyN3MgQeMknqG+toQWLkoWJwZE=; b=jwvABOAiI6Q+ACZ7odVhscWbTafNVtSI4P5ylYmcJc0075/nPTpXaeJVUg4Pk4QkJp BKmBSNg3WZjJfm/qqD+IH4uGYj+v/aSnx01Q4QLyEeAMLZBzmnTD5GV3JxHCIJWwADry WuxR4ceo1a3Z/n7si6R165qowp6CmcbR4pQQzGuSVMV7VLxOxg4nNV+U0MnxOG7lSSEv At8I/YwMhobLxcdDDN7lxQcBNQ8oEzqldSyouRdYWb1i9ihnpau+MzhvqbBbyR6XYCP9 d90hNyrWkKFCYS+p+IxIlBsr2iHco9JamXTjldGPMeWOO1e+c2upZgL7qOhkugl9Nmqj YqAQ== X-Forwarded-Encrypted: i=1; AJvYcCVWPGsojViE1AFdFCfYo2KYU8f02VBdP5PS22j6JXCE9jdQvF2WqFpBYgeLi1li1Uc6Kj19Ww==@debbugs.gnu.org X-Gm-Message-State: AOJu0YxnkdXkf7ww+kFg8t3CMffE9gGg1rH5cNsJtuC31uMD9EWaEIpT UMD352eP6HAeH5UnBmfEZYwcUznfchL6KXyub/57SwX1K/irpd2i X-Google-Smtp-Source: AGHT+IECFmqakNiD2bLZAh54wlsSBM5oQF1i/czkrCWPi6foOCmcEpw0VqDvFKX6k/x5MJVgX5pe4Q== X-Received: by 2002:a17:902:e751:b0:207:82c1:44b8 with SMTP id d9443c01a7336-20b37b912c1mr35629195ad.56.1727415798425; Thu, 26 Sep 2024 22:43:18 -0700 (PDT) Received: from smtpclient.apple ([2601:646:8f81:6120:880f:a981:5530:3a90]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20b37e5183dsm6620925ad.256.2024.09.26.22.43.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Sep 2024 22:43:17 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <86frpma06f.fsf@gnu.org> Date: Thu, 26 Sep 2024 22:43:06 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Sep 26, 2024, at 8:45=E2=80=AFAM, Eli Zaretskii = wrote: >=20 >> From: Mickey Petersen >> Cc: casouri@gmail.com, 73404@debbugs.gnu.org >> Date: Thu, 26 Sep 2024 16:21:33 +0100 >>=20 >>> I think the ability to move by parse sub-trees is also very useful. >>>=20 >>=20 >> Agreed. What matters is whether the crop of new sexp commands, such = as they >> are, perform satisfactorily. Note that you can affect the behavior of tree-sitter sexp movement by = defining the sexp =E2=80=9Cthing=E2=80=9D in treesit-thing-settings. = Js-ts-mode defines one (js--treesit-sexp-nodes) and it only consider = some nodes as sexp. You might be able to tweak the sexp movement to your = liking by changing it, or directly modifying the definition for `sexp=E2=80= =99 in treesit-thing-settings. >>=20 >> Do you think the examples I listed in the original bug report match >> your expectations? If so, then it is probably OK to close the bug = report. >=20 > Yes, I do, but let's wait for others to chime in if they have opinions > on this. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 13:01:58 2024 Received: (at 73404) by debbugs.gnu.org; 29 Sep 2024 17:02:01 +0000 Received: from localhost ([127.0.0.1]:41260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1suxIs-0004tM-S1 for submit@debbugs.gnu.org; Sun, 29 Sep 2024 13:01:58 -0400 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:48617) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1suxIn-0004rD-Du for 73404@debbugs.gnu.org; Sun, 29 Sep 2024 13:01:48 -0400 Received: by mail.gandi.net (Postfix) with ESMTPSA id 94F5EC0002; Sun, 29 Sep 2024 17:01:06 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Thu, 26 Sep 2024 22:43:06 -0700") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> Date: Sun, 29 Sep 2024 19:56:22 +0300 Message-ID: <86ikueiekp.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Note that you can affect the behavior of tree-sitter sexp movement by > defining the sexp “thing” in treesit-thing-settings. Js-ts-mode defines one > (js--treesit-sexp-nodes) and it only consider some nodes as sexp. You might > be able to tweak the sexp movement to your liking by changing it, or > directly modifying the definition for `sexp’ in treesit-thing-settings. > >>> Do you think the examples I listed in the original bug report match >>> your expectations? If so, then it is probably OK to close the bug report. >> >> Yes, I do, but let's wait for others to chime in if they have opinions >> on this. Here are some ideas how to cover more use cases. Suppose that a user wants to disable tree-sitter sexp movement completely to use the default forward-sexp-default-function. The natural way to do this would be set the list of nodes to nil: (setq js--treesit-sexp-nodes nil) However, this currently doesn't work, and requires a change like this: @@ -2290,10 +2290,12 @@ treesit-forward-sexp (treesit-node-at (point) (treesit-language-at (point))))) (or (when (and node-at-point ;; Make sure point is strictly inside node. - (< (treesit-node-start node-at-point) - (point) - (treesit-node-end node-at-point)) - (treesit-node-match-p node-at-point 'text t)) + (<= (treesit-node-start node-at-point) + (point) + (treesit-node-end node-at-point)) + (or (treesit-node-match-p node-at-point 'text t) + (not (treesit-node-match-p node-at-point 'sexp t)) + )) (forward-sexp-default-function arg) t) (if (> arg 0) Now, the next case: what if the user wants to use the default forward-sexp-default-function except for the 'binary_expression' like "a + b" where `C-M-f' should move from "a" to the end of "b": export const add = (a, b) => -!-a + b; should move to export const add = (a, b) => a + b; ^1 The best way for the user would be to customize: (setq js--treesit-sexp-nodes '("binary_expression")) But this is not yet handled by the condition above: (not (treesit-node-match-p node-at-point 'sexp t)) because 'node-at-point' is "identifier". So we need to use 'treesit-parent-until' to check if all parent nodes match 'js--treesit-sexp-nodes'. Then it will find the parent "binary_expression". I believe something like this will make treesit-forward-sexp more customizable. From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 01 10:32:04 2024 Received: (at 73404) by debbugs.gnu.org; 1 Oct 2024 14:32:04 +0000 Received: from localhost ([127.0.0.1]:51681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svdv1-0002VP-Up for submit@debbugs.gnu.org; Tue, 01 Oct 2024 10:32:04 -0400 Received: from mail-ot1-f44.google.com ([209.85.210.44]:44460) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svduz-0002Ux-Uy for 73404@debbugs.gnu.org; Tue, 01 Oct 2024 10:32:02 -0400 Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-710e489860bso2895499a34.1 for <73404@debbugs.gnu.org>; Tue, 01 Oct 2024 07:32:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727793057; x=1728397857; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=lKu6RK5D5CPROq7ZBpITKnqSjn1wOF98HEslkScTRj4=; b=EIDsFCO56F2u4bIMhMO5dpdYZZ7yJEF8SWPxMq9QmSHTddv/cfSxUoKKahUij+PuXk KGhOCXb2X9dbnIP8V4pFwSVaec+4jpVt3c8mP7sEuVgY8qgMsFU6wonnhj5f4cefia0t nBIFhpKGrouq3QLajb5eIrjSzhGBxaFk/91bpOrZtf25Jf6rSdfWrOOr3A7MvAIKhrG+ jxdvot/Ggrzz9xR/0t9FoqWECFEz3s5sslF8S6RxUxXkMA7YVzzU7sXvUAuG85Tymo5h tu1qcTiJLrmEFi/zD55JN6A0F1LQoUo6wJNbHtBRT9tAx+0uWRsxUn+9Mz59CB4c8RTG 3N/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727793057; x=1728397857; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lKu6RK5D5CPROq7ZBpITKnqSjn1wOF98HEslkScTRj4=; b=RJ15e6chHsU+/MHH+qC22iuSTYu3MArAqNV7Wx6dGfeZyj1ad1JrqLjTDfi0O5FQHy fikKo0Wugb5IuPP9UPreNv8OX258xgj3bilBZOlnJnHeYbI9pPEX+5tGBob1RNg+dqkK CWSgSJG0dkO2c3JZdVyhNxj3GII58WtUYvL26Pa73T+wjmRp7Yjj4TDAuQ7AMU0eVKBP nXGBc/BzNlt52nct3Vo8+feJx44sXCz+3oiPd4XLGhzefE04DMkbCyJh8N1tO5MvQFfM TIcVDvnI/JzE/ismhSQ/ediP24bCUKpm/LENR92U/bOTt7/BgZFheAnRlY4s6GpMOkbC v5Gw== X-Forwarded-Encrypted: i=1; AJvYcCXo7Bg7xpeROLpzZgxIQ+cpFlN0tt4b2R1gthgK/6/FWp4zT7h305AIqbXKzcKWfaKIxP+VgA==@debbugs.gnu.org X-Gm-Message-State: AOJu0YxYFADUml1fDjpagcb0vkKJF/zVlpTqL1gdX65CtrTCZLvBEFRR 8wu7rdVwnekpyjkOa31oj0uBCTpDNA7eefeudRzM3ggrfPrmJh+oAZsroQ== X-Google-Smtp-Source: AGHT+IH07RBr6uDpe5Ts/fnIwqP9ZmG5aUo8DKFts72GgY6X/6Zax9kS+7uW5qFdOp3XXvRnCM1f0Q== X-Received: by 2002:a17:902:e551:b0:20b:65d6:d268 with SMTP id d9443c01a7336-20b65d6d4bcmr145334865ad.53.1727755068549; Mon, 30 Sep 2024 20:57:48 -0700 (PDT) Received: from smtpclient.apple ([2601:646:8f81:6120:6dd0:d187:4760:9860]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20b37e0d7f2sm60861935ad.180.2024.09.30.20.57.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Sep 2024 20:57:47 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <86ikueiekp.fsf@mail.linkov.net> Date: Mon, 30 Sep 2024 20:57:36 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Sep 29, 2024, at 9:56=E2=80=AFAM, Juri Linkov = wrote: >=20 >> Note that you can affect the behavior of tree-sitter sexp movement by >> defining the sexp =E2=80=9Cthing=E2=80=9D in treesit-thing-settings. = Js-ts-mode defines one >> (js--treesit-sexp-nodes) and it only consider some nodes as sexp. You = might >> be able to tweak the sexp movement to your liking by changing it, or >> directly modifying the definition for `sexp=E2=80=99 in = treesit-thing-settings. >>=20 >>>> Do you think the examples I listed in the original bug report match >>>> your expectations? If so, then it is probably OK to close the bug = report. >>>=20 >>> Yes, I do, but let's wait for others to chime in if they have = opinions >>> on this. >=20 > Here are some ideas how to cover more use cases. >=20 > Suppose that a user wants to disable tree-sitter sexp movement > completely to use the default forward-sexp-default-function. > The natural way to do this would be set the list of nodes to nil: >=20 > (setq js--treesit-sexp-nodes nil) >=20 > However, this currently doesn't work, and requires a change like this: >=20 > @@ -2290,10 +2290,12 @@ treesit-forward-sexp > (treesit-node-at (point) (treesit-language-at (point))))) > (or (when (and node-at-point > ;; Make sure point is strictly inside node. > - (< (treesit-node-start node-at-point) > - (point) > - (treesit-node-end node-at-point)) > - (treesit-node-match-p node-at-point 'text t)) > + (<=3D (treesit-node-start node-at-point) > + (point) > + (treesit-node-end node-at-point)) > + (or (treesit-node-match-p node-at-point 'text t) > + (not (treesit-node-match-p node-at-point = 'sexp t)) > + )) > (forward-sexp-default-function arg) > t) > (if (> arg 0) >=20 > Now, the next case: what if the user wants to use the default > forward-sexp-default-function except for the 'binary_expression' > like "a + b" where `C-M-f' should move from "a" to the end of "b": >=20 > export const add =3D (a, b) =3D> -!-a + b; >=20 > should move to >=20 > export const add =3D (a, b) =3D> a + b; >=20 > ^1 >=20 > The best way for the user would be to customize: >=20 > (setq js--treesit-sexp-nodes '("binary_expression")) >=20 > But this is not yet handled by the condition above: >=20 > (not (treesit-node-match-p node-at-point 'sexp t)) >=20 > because 'node-at-point' is "identifier". > So we need to use 'treesit-parent-until' > to check if all parent nodes match > 'js--treesit-sexp-nodes'. Then it will find > the parent "binary_expression". >=20 > I believe something like this will make > treesit-forward-sexp more customizable. The user can modify treesit-thing-settings to alter the behavior of sexp = navigation, they don=E2=80=99t necessarily need to use = js--treesit-sexp-nodes. Maybe we should add a test for = (treesit-thing-defined-p 'sexp nil) in treesit-forward-sexp?=20 Your second example sounds useful, but right now the premise of = tree-sitter sexp movement is to use the parse tree primarily, and only = use the default sexp movement for comments and strings. What you = envisioned seems to be the other way around: use default sexp movement = by default, and only use tree-sitter movement under certain conditions. = Is that few lines of change able to make such big difference in the = logic? Yuan= From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 01 13:56:08 2024 Received: (at 73404) by debbugs.gnu.org; 1 Oct 2024 17:56:08 +0000 Received: from localhost ([127.0.0.1]:52756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svh6W-0002os-1x for submit@debbugs.gnu.org; Tue, 01 Oct 2024 13:56:08 -0400 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:51885) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svh6R-0002oT-MG for 73404@debbugs.gnu.org; Tue, 01 Oct 2024 13:56:06 -0400 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6A2831BF203; Tue, 1 Oct 2024 17:55:57 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Mon, 30 Sep 2024 20:57:36 -0700") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> Date: Tue, 01 Oct 2024 20:49:39 +0300 Message-ID: <86ed4zg1cc.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > The user can modify treesit-thing-settings to alter the behavior of > sexp navigation, they don’t necessarily need to use > js--treesit-sexp-nodes. Maybe we should add a test for > (treesit-thing-defined-p 'sexp nil) in treesit-forward-sexp? I tried to do something like this, and everything works nicely with: @@ -2289,11 +2289,10 @@ treesit-forward-sexp (node-at-point (treesit-node-at (point) (treesit-language-at (point))))) (or (when (and node-at-point - ;; Make sure point is strictly inside node. - (< (treesit-node-start node-at-point) - (point) - (treesit-node-end node-at-point)) - (treesit-node-match-p node-at-point 'text t)) + (or (treesit-node-match-p node-at-point 'text t) + (not (treesit-thing-at + (if (> arg 0) (point) (1- (point))) + (treesit-thing-definition 'sexp nil))))) (forward-sexp-default-function arg) t) (if (> arg 0) The new logic is the following: if there is no sexp thing defined at point, then fall back to 'forward-sexp-default-function'. Then after (setq js--treesit-sexp-nodes '("binary_expression")) 'C-M-f' in e.g. export const add = (a, b) => -!-a + b; moves point to export const add = (a, b) => a + b-!-; The condition (if (> arg 0) (point) (1- (point))) above is necessary to allow 'C-M-b' to move back to: export const add = (a, b) => -!-a + b; Also the condition to make sure point is strictly inside node was removed to handle the case when point was at the beginning of the buffer: -!- export const add = (a, b) => a + b; to move after export-!- const add = (a, b) => a + b; by 'forward-sexp-default-function'. > Your second example sounds useful, but right now the premise of tree-sitter > sexp movement is to use the parse tree primarily, and only use the default > sexp movement for comments and strings. What you envisioned seems to be the > other way around: use default sexp movement by default, and only use > tree-sitter movement under certain conditions. Is that few lines of change > able to make such big difference in the logic? I think we need to support both ways: 1. opt-out - where sexp-thing definition is used by default, and only text-thing allows users to override it; 2. opt-in - where 'forward-sexp-default-function' is used by default, and user can explicitly define what sexp-things are preferable for navigation by treesit. Then in the latter case the users could prefer to use treesit sexp navigation only for constructions with "invisible parens". For example, in Ruby there are two interchangeable syntaxes for code blocks: 1. curly braces {...} that are already handled by 'forward-sexp-default-function'; 2. do...end that can't be handled by 'forward-sexp-default-function', so treesit is coming to the rescue for the case of such implicit braces. From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 02 02:15:35 2024 Received: (at 73404) by debbugs.gnu.org; 2 Oct 2024 06:15:35 +0000 Received: from localhost ([127.0.0.1]:56321 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svse6-0002Bc-LE for submit@debbugs.gnu.org; Wed, 02 Oct 2024 02:15:35 -0400 Received: from mail-pl1-f172.google.com ([209.85.214.172]:61914) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svse5-0002BQ-Da for 73404@debbugs.gnu.org; Wed, 02 Oct 2024 02:15:34 -0400 Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-20bc2970df5so7692805ad.3 for <73404@debbugs.gnu.org>; Tue, 01 Oct 2024 23:15:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727849667; x=1728454467; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=HQI5CKJhw8HN4YBu4hfpxTlfh7VeWb6JscWOg0y7VTk=; b=HaeZT9RX9J55DIYrRx70djifEowggqeeAT5zTfLlJLD81oqMbyQfxxDCaWrxGuQgYH Ahak3d89BBShO4Om7JpoNTV9An7xZpZL40aKunxczKSvhZQq3z49Re6xlIpNg4hkq85H fJs10UY2z6iWfCusAPJv8nppWcX8t9oF2NBu41SeqmmCzbYiLtQl0++S6hxcOuj1ianR fs4wMXYqNd3zBcA7rixoXGbFWwEKdtH3l+DzzLjdTjnxXKWNf6y06JnuapbFGqbBKWaE X2Zhidf/SBkFoq1vsKGUk5d2oSJUgxW+Zpq6HKr/AgUfy3omeFehod0m3OsLqVj81UTQ P9Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727849667; x=1728454467; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HQI5CKJhw8HN4YBu4hfpxTlfh7VeWb6JscWOg0y7VTk=; b=Uj+7y9LybjT1Arj68/whYwDnte9c7wRgTMr5z7zZSnaF4GMQV1eZC5SQTDdf4fcTw3 88rVWYx3QSLYgVJGVZAxjzT04OZ0Rw8guDRgYTm/JFQ/ZpK+8IXlhoBRLHfYiHdFzlGB gM/n0AJgQhub+t6odzxhWQzuVMAWZDBZ7mSXSfCWxBKSgpB9lM6Xnujmwy/Rjvbwo1Gy BqDtUy3XZ1dAYrStlRXG4FQje0oV1K2GITQR6ynZMzFA4U31tFW2QxhacNha0XyJmfn6 ZHUz2HxDZeFpBAO7RKWBOeMecVc/BVS1OPPwJ/uuUoAM1zhLukeP5OspzC9Nsp+BnPrH OxQQ== X-Forwarded-Encrypted: i=1; AJvYcCXSoQnicfOneQ+b6ozxUdolHQA9psftTvAnacYQClQ2aVq9XAvsTs/pP4sqgMUBQftx+5o7jg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YxUoPVEoa885vo0h/BM9aB3IMsT750OAR3+lA8whjWcR7FJmwna XTyGKIow+Bsh1rtTzzlO74RtrMb35MSnPWYNosEfVEPnMfce2J1c X-Google-Smtp-Source: AGHT+IHmNSgFXj3FDx15Ll7rHPXqVxN3GCG+gqBhkH6Fvv2xsF5s2jsIaWfmUbUTGpLWoMwVTo6iMA== X-Received: by 2002:a17:90a:c287:b0:2e0:b02e:d1e0 with SMTP id 98e67ed59e1d1-2e18456f0dfmr2691815a91.16.1727849667166; Tue, 01 Oct 2024 23:14:27 -0700 (PDT) Received: from smtpclient.apple ([2601:646:8f81:6120:6dd0:d187:4760:9860]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e18f747292sm745286a91.7.2024.10.01.23.14.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Oct 2024 23:14:26 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <86ed4zg1cc.fsf@mail.linkov.net> Date: Tue, 1 Oct 2024 23:14:15 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <08933A47-CCCD-4317-9EEA-DCB0C1D9E544@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Oct 1, 2024, at 10:49=E2=80=AFAM, Juri Linkov = wrote: >=20 >> The user can modify treesit-thing-settings to alter the behavior of >> sexp navigation, they don=E2=80=99t necessarily need to use >> js--treesit-sexp-nodes. Maybe we should add a test for >> (treesit-thing-defined-p 'sexp nil) in treesit-forward-sexp? >=20 > I tried to do something like this, and everything works nicely with: >=20 > @@ -2289,11 +2289,10 @@ treesit-forward-sexp > (node-at-point > (treesit-node-at (point) (treesit-language-at (point))))) > (or (when (and node-at-point > - ;; Make sure point is strictly inside node. > - (< (treesit-node-start node-at-point) > - (point) > - (treesit-node-end node-at-point)) > - (treesit-node-match-p node-at-point 'text t)) > + (or (treesit-node-match-p node-at-point 'text t) > + (not (treesit-thing-at > + (if (> arg 0) (point) (1- (point))) > + (treesit-thing-definition 'sexp = nil))))) > (forward-sexp-default-function arg) > t) > (if (> arg 0) >=20 > The new logic is the following: if there is no sexp thing defined at = point, > then fall back to 'forward-sexp-default-function'. >=20 > Then after (setq js--treesit-sexp-nodes '("binary_expression")) > 'C-M-f' in e.g. >=20 > export const add =3D (a, b) =3D> -!-a + b; >=20 > moves point to >=20 > export const add =3D (a, b) =3D> a + b-!-; >=20 > The condition (if (> arg 0) (point) (1- (point))) above > is necessary to allow 'C-M-b' to move back to: >=20 > export const add =3D (a, b) =3D> -!-a + b; >=20 > Also the condition to make sure point is strictly inside node > was removed to handle the case when point was at the beginning > of the buffer: >=20 > -!- > export const add =3D (a, b) =3D> a + b; >=20 > to move after >=20 > export-!- const add =3D (a, b) =3D> a + b; >=20 > by 'forward-sexp-default-function'. Sounds good. Feel free to install on master if you think it works well = :-) >=20 >> Your second example sounds useful, but right now the premise of = tree-sitter >> sexp movement is to use the parse tree primarily, and only use the = default >> sexp movement for comments and strings. What you envisioned seems to = be the >> other way around: use default sexp movement by default, and only use >> tree-sitter movement under certain conditions. Is that few lines of = change >> able to make such big difference in the logic? >=20 > I think we need to support both ways: >=20 > 1. opt-out - where sexp-thing definition is used by default, > and only text-thing allows users to override it; >=20 > 2. opt-in - where 'forward-sexp-default-function' is used by default, > and user can explicitly define what sexp-things are preferable > for navigation by treesit. >=20 > Then in the latter case the users could prefer to use > treesit sexp navigation only for constructions with > "invisible parens". For example, in Ruby there are > two interchangeable syntaxes for code blocks: >=20 > 1. curly braces {...} that are already handled > by 'forward-sexp-default-function'; >=20 > 2. do...end that can't be handled by 'forward-sexp-default-function', > so treesit is coming to the rescue for the case of such > implicit braces. Sounds good to me, I wonder if there are clever way to implement this. = If there isn=E2=80=99t, we=E2=80=99d need to define two sets = treesit-sexp functions and add a custom option to control which one to = use. Seems a bit clunky to me. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 30 14:45:53 2024 Received: (at control) by debbugs.gnu.org; 30 Nov 2024 19:45:53 +0000 Received: from localhost ([127.0.0.1]:49178 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tHTPd-0005FX-4n for submit@debbugs.gnu.org; Sat, 30 Nov 2024 14:45:53 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:58049) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tHTPc-0005FF-7P; Sat, 30 Nov 2024 14:45:52 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id D014A60002; Sat, 30 Nov 2024 19:45:39 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#74366: 31.0.50; tree-sitter In-Reply-To: <86wmgl82p6.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 30 Nov 2024 11:56:37 +0200") Organization: LINKOV.NET References: <8634jrgskk.fsf@gnu.org> <87serrhtr9.fsf@mail.linkov.net> <86wmgl82p6.fsf@gnu.org> Date: Sat, 30 Nov 2024 21:44:55 +0200 Message-ID: <87y1107bgo.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: control Cc: casouri@gmail.com, 74366@debbugs.gnu.org, theo@thornhill.no, jiangyuezhang@outlook.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) forcemerge 73404 74366 thanks >> >> open ts file, and enable typescript-ts-mode. >> >> just like >> >> ``` >> >> function hello() { >> >> console.log('hello'), >> >>     return 'hello'; >> >> } >> >> ``` >> >> Jump to point before {, and try C-M-f (forward-sexp or treesitter-forward-sexp), it just jump to >> >> end of ), not }. It was expected to end of }. >> > >> > Here it jumps to after "'hello';". >> >> This should work with the fix from bug#73404. > > Should we merge this bug with bug#73404? Ok, merging. Sorry for not yet finishing the fix. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 05 13:53:57 2024 Received: (at 73404) by debbugs.gnu.org; 5 Dec 2024 18:53:57 +0000 Received: from localhost ([127.0.0.1]:40923 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tJGz6-00057G-TF for submit@debbugs.gnu.org; Thu, 05 Dec 2024 13:53:57 -0500 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:44091) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tJGz4-00056y-La for 73404@debbugs.gnu.org; Thu, 05 Dec 2024 13:53:55 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id CB1FD1BF204; Thu, 5 Dec 2024 18:53:27 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86ed4zg1cc.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 01 Oct 2024 20:49:39 +0300") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> Date: Thu, 05 Dec 2024 20:52:18 +0200 Message-ID: <87zflac68t.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > The new logic is the following: if there is no sexp thing defined at point, > then fall back to 'forward-sexp-default-function'. > > Then after (setq js--treesit-sexp-nodes '("binary_expression")) > 'C-M-f' in e.g. > > export const add = (a, b) => -!-a + b; > > moves point to > > export const add = (a, b) => a + b-!-; Unfortunately, I still can't find a way to handle such case that from export const add = (a, b) -!- => a + b; typing 'C-M-f' should jump to the end of the next sexp (to the end of whole "binary_expression"): export const add = (a, b) => a + b-!-; since only tree-sitter knows about "binary_expression", so 'forward-sexp-default-function' can't be used here. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 05 14:55:02 2024 Received: (at 73404) by debbugs.gnu.org; 5 Dec 2024 19:55:02 +0000 Received: from localhost ([127.0.0.1]:41055 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tJHwE-0008En-Di for submit@debbugs.gnu.org; Thu, 05 Dec 2024 14:55:02 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:59373) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tJHwC-0008EB-Fc for 73404@debbugs.gnu.org; Thu, 05 Dec 2024 14:55:01 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 60D2BFF805; Thu, 5 Dec 2024 19:54:51 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87zflac68t.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 05 Dec 2024 20:52:18 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> Date: Thu, 05 Dec 2024 21:53:38 +0200 Message-ID: <87jzcdlxdp.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> The new logic is the following: if there is no sexp thing defined at point, >> then fall back to 'forward-sexp-default-function'. >> >> Then after (setq js--treesit-sexp-nodes '("binary_expression")) >> 'C-M-f' in e.g. >> >> export const add = (a, b) => -!-a + b; >> >> moves point to >> >> export const add = (a, b) => a + b-!-; > > Unfortunately, I still can't find a way to handle such case > that from > > export const add = (a, b) -!- => a + b; > > typing 'C-M-f' should jump to the end of the next sexp > (to the end of whole "binary_expression"): > > export const add = (a, b) => a + b-!-; > > since only tree-sitter knows about "binary_expression", > so 'forward-sexp-default-function' can't be used here. Actually, I have one idea of possible heuristics: 1. first try 'forward-sexp-default-function' 2. if it crosses the boundary of sexp defined by 'treesit-thing-settings' then use 'treesit-end-of-thing' instead This should work. Ok, will try. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 10 12:49:18 2024 Received: (at 73404) by debbugs.gnu.org; 10 Dec 2024 17:49:18 +0000 Received: from localhost ([127.0.0.1]:59487 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tL4MI-0002Ae-4p for submit@debbugs.gnu.org; Tue, 10 Dec 2024 12:49:18 -0500 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:42269) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tL4MF-0002AH-IO for 73404@debbugs.gnu.org; Tue, 10 Dec 2024 12:49:16 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id DC1B51BF204; Tue, 10 Dec 2024 17:48:46 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87jzcdlxdp.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 05 Dec 2024 21:53:38 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> Date: Tue, 10 Dec 2024 19:20:43 +0200 Message-ID: <87o71jocgs.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain >> export const add = (a, b) -!- => a + b; >> >> typing 'C-M-f' should jump to the end of the next sexp >> (to the end of whole "binary_expression"): >> >> export const add = (a, b) => a + b-!-; >> >> since only tree-sitter knows about "binary_expression", >> so 'forward-sexp-default-function' can't be used here. > > Actually, I have one idea of possible heuristics: > > 1. first try 'forward-sexp-default-function' > 2. if it crosses the boundary of sexp defined by 'treesit-thing-settings' > then use 'treesit-end-of-thing' instead > > This should work. Ok, will try. This is implemented now in the attached patch, and it works nicely. The main rule is the following: 'forward-sexp-default-function' should not go out of the current thing, neither go inside a sibling. So we use 'treesit-end-of-thing' in such cases. But when inside a thing or outside a thing, use the default function. This supposes that such things as "identifier" in js should be removed from 'treesit-thing-settings' since identifiers should be navigated the same way as such keywords as "export" and "const" using 'forward-sexp-default-function'. What should remain in 'treesit-thing-settings' are only grouping constructs such as "parenthesized_expression" and "statement_block". Removing "identifier" from 'treesit-thing-settings' exposed a problem in 'treesit-navigate-thing'. This line ((and (null next) (null prev)) parent) tries to go out of the current thing to its parent, thus breaking the main principle that 'forward-sexp' should move forward across siblings only. But removing this line fixed the problem: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=treesit-forward-sexp.patch diff --git a/lisp/treesit.el b/lisp/treesit.el index db8f7a7595d..4fcdbe7fc56 100644 --- a/lisp/treesit.el +++ b/lisp/treesit.el @@ -2373,21 +2373,41 @@ treesit-forward-sexp What constitutes as text and source code sexp is determined by `text' and `sexp' in `treesit-thing-settings'." (interactive "^p") - (let ((arg (or arg 1)) - (pred (or treesit-sexp-type-regexp 'sexp)) - (node-at-point - (treesit-node-at (point) (treesit-language-at (point))))) - (or (when (and node-at-point - ;; Make sure point is strictly inside node. - (< (treesit-node-start node-at-point) - (point) - (treesit-node-end node-at-point)) - (treesit-node-match-p node-at-point 'text t)) - (forward-sexp-default-function arg) - t) - (if (> arg 0) - (treesit-end-of-thing pred (abs arg) 'restricted) - (treesit-beginning-of-thing pred (abs arg) 'restricted)) + (let* ((arg (or arg 1)) + (pred (or treesit-sexp-type-regexp 'sexp)) + (current-thing (treesit-thing-at (point) pred t)) + (default-pos + (condition-case _ + (save-excursion + (forward-sexp-default-function arg) + (point)) + (scan-error nil))) + (default-pos (unless (eq (point) default-pos) default-pos)) + (sibling-pos + (save-excursion + (and (if (> arg 0) + (treesit-end-of-thing pred (abs arg) 'restricted) + (treesit-beginning-of-thing pred (abs arg) 'restricted)) + (point)))) + (sibling (when sibling-pos + (if (> arg 0) + (treesit-thing-prev sibling-pos pred) + (treesit-thing-next sibling-pos pred))))) + + ;; 'forward-sexp-default-function' should not go out of the current thing, + ;; neither go inside the next thing, neither go over the next thing + (or (when (and default-pos + (or (null current-thing) + (if (> arg 0) + (< default-pos (treesit-node-end current-thing)) + (> default-pos (treesit-node-start current-thing)))) + (or (null sibling) + (if (> arg 0) + (< default-pos (treesit-node-start sibling)) + (> default-pos (treesit-node-end sibling))))) + (goto-char default-pos)) + (when sibling-pos + (goto-char sibling-pos)) ;; If we couldn't move, we should signal an error and report ;; the obstacle, like `forward-sexp' does. If we couldn't ;; find a parent, we simply return nil without moving point, @@ -2849,8 +2869,7 @@ treesit-navigate-thing (if (eq tactic 'restricted) (setq pos (funcall advance - (cond ((and (null next) (null prev)) parent) - ((> arg 0) next) + (cond ((> arg 0) next) (t prev)))) ;; For `nested', it's a bit more work: ;; Move... --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 11 01:32:50 2024 Received: (at 73404) by debbugs.gnu.org; 11 Dec 2024 06:32:50 +0000 Received: from localhost ([127.0.0.1]:60865 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLGH8-0007GB-HX for submit@debbugs.gnu.org; Wed, 11 Dec 2024 01:32:50 -0500 Received: from mail-pl1-f170.google.com ([209.85.214.170]:54479) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLGH2-0007Fx-JW for 73404@debbugs.gnu.org; Wed, 11 Dec 2024 01:32:45 -0500 Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2163bd70069so33415205ad.0 for <73404@debbugs.gnu.org>; Tue, 10 Dec 2024 22:32:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733898698; x=1734503498; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=KthLNndSnFM9j2SwKPlZ5N7rIaMVCa98IPmti5HnFP4=; b=ZVtHdjKWB6yJclQM5g5zIQ2nLyozrzu7N4vwGqqZkk/YWnMzMupsYczkclaki1MKLN zr4RjahbaJ1fksfp2eIH0CnXDJC9vhCc5RpO0SF71cIDdU982s8rAoYidl/VC3acufJX +U1k4mqVYapjXYUcKZzO2a9D4lSvCYfpFs4LLxZE6W9f949b1I5qM+lgxEh2sqyK/r5u gequ1AYAok7aDbZBxNdjIXDIzqm9YGwmxaDhOv4errcFf9sWu0MH6Rd58iKFdXcubVcY I5nwlX5WGnZ5xNihgmvCzjsj9opnU+nNXgQG9arIhWOf+1miXgt0KNOIEZwJy+/XgVx8 ERQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733898698; x=1734503498; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KthLNndSnFM9j2SwKPlZ5N7rIaMVCa98IPmti5HnFP4=; b=QpfX+ga3a5lkJlzNooshqEVIb/USHrHTl0t4TwA8t5mudQmKBokqCqfmQtCgghVNqF V2kg8GEhCVC4jAx+t7XSwWTNhJcrWSunkH3KkTMO4sPTa+0AL7DDBJ6TZJnV7RmqM4NY Zy9ovajN9aFPtX86YX21Vxov79X++vWYRCflGv4TUXqL2ncnF4bLsRUvSwLwHGhocXy0 WTiSA0uLJj09c8WZvBfQD5CIS8gHN6KL1WbOdalkrkz11g0+abLtPFApl+Y5LoBLVa5v 4SoDl2VEDZ1B0AQI6sq+hZFemw/oeqcZR/sIgfKt9kK+hm6/qMbMVQ3FXdyobIJOSoIn RekA== X-Forwarded-Encrypted: i=1; AJvYcCXGdRdZDnVYqx9hWhP7AGU0YrHH6uOdTDhYqRgmRB2LeJPrvx8xNzG35Cpiy+0ZChUVHFE1PQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0YxowwA+6YsPbnRHhj93AXCY2yB+OXkjatqNkf5GDHqRbLYkdhxf nqBDDAObkuhPbLcUEnwWooyS44t46T0hvg+00o8UUB/GSVGXUdIt X-Gm-Gg: ASbGncvblgadAn1C7c6u7KV5GtP6IqgHdItc5RMBMSepOxewOh6oSIMQVuop29oD7Pk a/qaKw7zk3oIjPpF5u/ylQcWr/dkfqcaoXzRjsN/8yfgWY4gErVYDA+3OfzAftTXZev2ZRWJ6Wi YewqsAwQYwb9Wd3JMafelwnfUGCZkoac2F9NOeuAPIb8t5xr12+PxfP7Ox8av3jafGQlihUcGPW U8uoPdZs0akIZPjHTfCoiJU5DGXBeGNPQ8/P34MlS7yV2xop+fobMM+lZVvv1ejwSYl9xAYXTQD Rg== X-Google-Smtp-Source: AGHT+IHKszyUzGB0G8jGDuK4heO36K2xAh+fzTY8PWwBFKvxRhjisQYfUz6njTHYRxpbPC9Ct5qOSw== X-Received: by 2002:a17:903:945:b0:215:7fad:49ab with SMTP id d9443c01a7336-2177851f6a0mr32503325ad.10.1733898698497; Tue, 10 Dec 2024 22:31:38 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:f90e:3b71:6ee2:6197]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21653b0d4f5sm42787205ad.70.2024.12.10.22.31.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2024 22:31:37 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87o71jocgs.fsf@mail.linkov.net> Date: Tue, 10 Dec 2024 22:31:26 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 10, 2024, at 9:20=E2=80=AFAM, Juri Linkov = wrote: >=20 >>> export const add =3D (a, b) -!- =3D> a + b; >>>=20 >>> typing 'C-M-f' should jump to the end of the next sexp >>> (to the end of whole "binary_expression"): >>>=20 >>> export const add =3D (a, b) =3D> a + b-!-; >>>=20 >>> since only tree-sitter knows about "binary_expression", >>> so 'forward-sexp-default-function' can't be used here. >>=20 >> Actually, I have one idea of possible heuristics: >>=20 >> 1. first try 'forward-sexp-default-function' >> 2. if it crosses the boundary of sexp defined by = 'treesit-thing-settings' >> then use 'treesit-end-of-thing' instead >>=20 >> This should work. Ok, will try. >=20 > This is implemented now in the attached patch, and it works nicely. >=20 > The main rule is the following: 'forward-sexp-default-function' > should not go out of the current thing, neither go inside a sibling. > So we use 'treesit-end-of-thing' in such cases. But when inside > a thing or outside a thing, use the default function. >=20 > This supposes that such things as "identifier" in js should be > removed from 'treesit-thing-settings' since identifiers should be > navigated the same way as such keywords as "export" and "const" > using 'forward-sexp-default-function'. >=20 > What should remain in 'treesit-thing-settings' are only grouping > constructs such as "parenthesized_expression" and "statement_block". Ah, this matches my idea of defining sexp in other languages as = =E2=80=9Crepeatable construct/list-like construct=E2=80=9D. We went with = =E2=80=9Cevery syntactic construct=E2=80=9D at the time, which I = didn=E2=80=99t object to, but I=E2=80=99m definitely happier with the = repeatable construct approach. Including Stefan and Theo since they were = part of the original sexp navigation discussion. My only concern is that would the result be a bit = unpredictable/confusing when we mix the result of two logic together in = such an involved way? We can push to master and try it out for a while. = I use tree-sitter sexp navigation for work every day, albeit strictly = for navigating list-like constructs=E2=80=94I use forward/backward-word = for smaller navigation. >=20 > Removing "identifier" from 'treesit-thing-settings' exposed a problem > in 'treesit-navigate-thing'. This line >=20 > ((and (null next) (null prev)) parent) >=20 > tries to go out of the current thing to its parent, > thus breaking the main principle that 'forward-sexp' > should move forward across siblings only. But removing > this line fixed the problem: Thanks, LGTM. >=20 > diff --git a/lisp/treesit.el b/lisp/treesit.el > index db8f7a7595d..4fcdbe7fc56 100644 > --- a/lisp/treesit.el > +++ b/lisp/treesit.el > @@ -2373,21 +2373,41 @@ treesit-forward-sexp > What constitutes as text and source code sexp is determined > by `text' and `sexp' in `treesit-thing-settings'." > (interactive "^p") > - (let ((arg (or arg 1)) > - (pred (or treesit-sexp-type-regexp 'sexp)) > - (node-at-point > - (treesit-node-at (point) (treesit-language-at (point))))) > - (or (when (and node-at-point > - ;; Make sure point is strictly inside node. > - (< (treesit-node-start node-at-point) > - (point) > - (treesit-node-end node-at-point)) > - (treesit-node-match-p node-at-point 'text t)) > - (forward-sexp-default-function arg) > - t) > - (if (> arg 0) > - (treesit-end-of-thing pred (abs arg) 'restricted) > - (treesit-beginning-of-thing pred (abs arg) 'restricted)) > + (let* ((arg (or arg 1)) > + (pred (or treesit-sexp-type-regexp 'sexp)) > + (current-thing (treesit-thing-at (point) pred t)) > + (default-pos > + (condition-case _ > + (save-excursion > + (forward-sexp-default-function arg) > + (point)) > + (scan-error nil))) > + (default-pos (unless (eq (point) default-pos) default-pos)) > + (sibling-pos > + (save-excursion > + (and (if (> arg 0) > + (treesit-end-of-thing pred (abs arg) = 'restricted) > + (treesit-beginning-of-thing pred (abs arg) = 'restricted)) > + (point)))) > + (sibling (when sibling-pos > + (if (> arg 0) > + (treesit-thing-prev sibling-pos pred) > + (treesit-thing-next sibling-pos pred))))) > + > + ;; 'forward-sexp-default-function' should not go out of the = current thing, > + ;; neither go inside the next thing, neither go over the next = thing > + (or (when (and default-pos > + (or (null current-thing) > + (if (> arg 0) > + (< default-pos (treesit-node-end = current-thing)) > + (> default-pos (treesit-node-start = current-thing)))) > + (or (null sibling) > + (if (> arg 0) > + (< default-pos (treesit-node-start = sibling)) > + (> default-pos (treesit-node-end = sibling))))) > + (goto-char default-pos)) > + (when sibling-pos > + (goto-char sibling-pos)) > ;; If we couldn't move, we should signal an error and report > ;; the obstacle, like `forward-sexp' does. If we couldn't > ;; find a parent, we simply return nil without moving point, > @@ -2849,8 +2869,7 @@ treesit-navigate-thing > (if (eq tactic 'restricted) > (setq pos (funcall > advance > - (cond ((and (null next) (null prev)) parent) > - ((> arg 0) next) > + (cond ((> arg 0) next) > (t prev)))) > ;; For `nested', it's a bit more work: > ;; Move... From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 11 10:12:28 2024 Received: (at 73404) by debbugs.gnu.org; 11 Dec 2024 15:12:28 +0000 Received: from localhost ([127.0.0.1]:35690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLOO4-0006CS-0H for submit@debbugs.gnu.org; Wed, 11 Dec 2024 10:12:28 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:65324) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLOO1-0006CB-RB for 73404@debbugs.gnu.org; Wed, 11 Dec 2024 10:12:26 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 1167010004C; Wed, 11 Dec 2024 10:12:18 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1733929937; bh=i1qE7E84/FYGblDOZpssF4Iya+aYuLoJlcFqiXBV+Jg=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=IA3oIqhnzFBbSic1qFMgPL0FxJGldLXxAMkmvosn/5/GflGQJJ3iewQA8KOFZMENU 5yymyZUoNI/xMcGlGVwsRAU0cgt8aAaZJUUXtWSymmEtVxKzTv2LYuU4X1GvQbZ3J3 qVdLvmI7ndhO5PafjZfWhjgAr+wTQTive+OPFTMZv97mY39yJgx152RigE6WOIJ8A+ GxFjGv1UKDsoZvpKXDLj4O3T+dNa1KrwhNw1hr55eSezL0Z4DZ321mnmuh0y/w2Vqw f1M1Wd8QHqaKTuseddHvsCrnpAXjSS+WVbsiD3GD3weGOBs383NNePSTXz1ikuLPFB 0Y0W/ggp20ESA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 6A4F5100043; Wed, 11 Dec 2024 10:12:17 -0500 (EST) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 54A1C12047F; Wed, 11 Dec 2024 10:12:17 -0500 (EST) From: Stefan Monnier To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Tue, 10 Dec 2024 22:31:26 -0800") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> Date: Wed, 11 Dec 2024 10:12:15 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.214 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Ah, this matches my idea of defining sexp in other languages as =E2=80=9C= repeatable > construct/list-like construct=E2=80=9D. We went with =E2=80=9Cevery synt= actic construct=E2=80=9D at > the time, which I didn=E2=80=99t object to, but I=E2=80=99m definitely ha= ppier with the > repeatable construct approach. Including Stefan and Theo since they were > part of the original sexp navigation discussion. FWIW, we have both `forward-list` and `forward-list` and the new behavior you suggest sounds closer to the historical behavior of `forward-list` than `forward-sexp`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 11 10:30:10 2024 Received: (at 73404) by debbugs.gnu.org; 11 Dec 2024 15:30:10 +0000 Received: from localhost ([127.0.0.1]:35728 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLOfB-0007Jt-R8 for submit@debbugs.gnu.org; Wed, 11 Dec 2024 10:30:10 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:34145) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLOf7-0007Ha-JK for 73404@debbugs.gnu.org; Wed, 11 Dec 2024 10:30:07 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CDF284422F5; Wed, 11 Dec 2024 10:29:59 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1733930998; bh=rpydzkDtoL+nRLu6T7+iyFxMwlsoJQ8G9oZz85K6OOw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=R/z2vBSGyc2TT1rsIsaSUVLDFgKLDoAjkzSxe/cwUqzp770tXvKqWTshxC4npnDkd HogzAEXOZf28Fz+FPWTcExs3hNWzJ2357elcCa5gLwjkd51NtUDschPCUa9l7z9Z0j A9fa7oMx2g3dnJEZSVnZZhddCMO3ptQJzxaKGK2no8itk6EQ9fq+RBvHGN28eD94d0 5EP11F8xlprIT/xUO15BIdxo9t4nZzWQpgYr4VUvvWRd+fWcaIu6STyR6c+dvU7f/e WTwankCP5u/AZ+FX77AL+f8Vy8ATXeQ0yx/J2v/95IqBo0qCOZ07KU14PypNVwz4BK bmXvdHDCPPjjw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DFBB9442263; Wed, 11 Dec 2024 10:29:58 -0500 (EST) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id CEE7F1203DF; Wed, 11 Dec 2024 10:29:58 -0500 (EST) From: Stefan Monnier To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Wed, 11 Dec 2024 10:12:15 -0500") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> Date: Wed, 11 Dec 2024 10:29:58 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.214 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > FWIW, we have both `forward-list` and `forward-list` and the new ^^^^ sexp Otherwise it sounds smarter than it is. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 11 11:51:50 2024 Received: (at 73404) by debbugs.gnu.org; 11 Dec 2024 16:51:50 +0000 Received: from localhost ([127.0.0.1]:36040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLPwD-0004lN-Pa for submit@debbugs.gnu.org; Wed, 11 Dec 2024 11:51:50 -0500 Received: from mail-uksouthazon11021105.outbound.protection.outlook.com ([52.101.95.105]:51795 helo=LO2P265CU024.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLPwB-0004l0-6x for 73404@debbugs.gnu.org; Wed, 11 Dec 2024 11:51:48 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PezuJ1QdMycQk06l9fer7KTEA3NDAwI6YmeR9f2qdTp5bt36BXAe0sBxCnzMxxSbv9TEpgaBKIQ7LF6Oljife/JS5LO0j9Cko63su+4GORwkQab9H3+3UBCXqU/esCI2Kklvnao//84neLsFZbiN1p5U/36FvsU9DcOMXrchHGnTX+BU6wc/zWmlCmEHa9rfFZwR+J/SXnu25mZISmD6ZcLPodQGsluRDI7iH8OPQ1wnLHvTPhhIvf1yHOQ1y4rjht0FVN5fqeNnl/VgdvUyCoMUZPj7tWWIvJ5bnsMeoWm5K2uPpBWkYqpYrQ4szcWTjHfLxm4lf1cS9osv3K1T9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/nslWIYnPSuNZ7CXxwVAqBax6IplmBkeHsPyQeNV9zU=; b=JKaGLAdEgodWTfhSEaeZp74bY7S5GLA3x+l4NaTpnGsDuMaiY3PccT3D/SbF36flyS+RfOSoHTxZ0j1uG0hw9wyFxBxSdC+Uu773VWR/mRhhbjvPDwjxdoKbLnj6webnTCiYqIxauui8z9YUPNYvQ+bMyLXoKzyVlf278smzJOWLuEgQTDb5P8lzS1aR+eRHoNRGF+Lnht6u0HUP8TlXFf3/MNSpMeNxOWFU6bZRFe2hoDK/r+1JfaDppHab5zI0bUomHInDPxX7A3KtPdNcc52zp7Grk6mY4pgr6snAeTeqB+aDa8mVMrcDowPMItBPm0d/Qv3UU+3LIhOq3pw6Kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 178.79.136.144) smtp.rcpttodomain=debbugs.gnu.org smtp.mailfrom=masteringemacs.org; dmarc=pass (p=none sp=none pct=100) action=none header.from=masteringemacs.org; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semantical.onmicrosoft.com; s=selector1-semantical-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/nslWIYnPSuNZ7CXxwVAqBax6IplmBkeHsPyQeNV9zU=; b=iJmdS13vwi8wE3N10CUBmwqlLhZ6Fr9Yjw5CDUzW0goCHOKCnufwlu1AW1KIRkWAgYt+9jznQs5BJIjjKcF9GsrYLTCqIf8dRdFG5ZCtMZFyUlR1uaOR2WAI4YH9hpcM0Z6CZ/KnzRqT5mjIYFahCtD7xatkLNp/BR8IibDkfV0= Received: from LO4P123CA0552.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:319::14) by CWLP265MB6308.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1c4::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.15; Wed, 11 Dec 2024 16:51:39 +0000 Received: from LO1PEPF000022FF.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:319:cafe::df) by LO4P123CA0552.outlook.office365.com (2603:10a6:600:319::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8251.15 via Frontend Transport; Wed, 11 Dec 2024 16:51:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 178.79.136.144) smtp.mailfrom=masteringemacs.org; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=masteringemacs.org; Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org designates 178.79.136.144 as permitted sender) receiver=protection.outlook.com; client-ip=178.79.136.144; helo=semantical.co.uk; pr=C Received: from semantical.co.uk (178.79.136.144) by LO1PEPF000022FF.mail.protection.outlook.com (10.167.240.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.15 via Frontend Transport; Wed, 11 Dec 2024 16:51:38 +0000 Received: by semantical.co.uk (Postfix, from userid 5001) id A09C8114003; Wed, 11 Dec 2024 16:51:38 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on semantical.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RECEIVED, NO_RELAYS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> User-agent: mu4e @VERSION@; emacs 31.0.50 From: Mickey Petersen To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes Date: Wed, 11 Dec 2024 16:50:12 +0000 Organization: Mastering Emacs In-reply-to: Message-ID: <87h67aqi22.fsf@masteringemacs.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO1PEPF000022FF:EE_|CWLP265MB6308:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 235b15c1-54d7-4465-ff74-08dd1a0414d8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|61400799027|36860700013|376014|14776008|79816003; X-Microsoft-Antispam-Message-Info: =?utf-8?B?RUMrSVcwdnFSek5uTWo1MHcxbGo1VHFiUDZPVGdTTi84UURJOVJBUHlvNzdr?= =?utf-8?B?ZzhYcWYxOTNmcnJiWWVBV3E0SlJzSkI3K1lKcExNY1ZZVVBQU1JOVWVkMlNU?= =?utf-8?B?a3B6Umx0ZW5sdFFhZEI5Y3RYd1c2RHY0bnk1alpFdWZtckQ1cFN5ZU9iUTl6?= =?utf-8?B?SE9OTmFBY3U4amxXdW5MVlF2QkNVZkh5UE9ncWkyMDlVUVNRNTFEL3A4Vkor?= =?utf-8?B?eGN5U0YxazlETjRheGRwZy9ZWFhTOWpTRHIrMzBFZ0hSVXF2THYyR1NRK3pH?= =?utf-8?B?UGdCa0wyU2YrUEdGd0F6RURYK056dzhvOFQyb0t0VUxBT2dwWjJyRmFDOTNs?= =?utf-8?B?ZDVCb3h1b0FDYytPSXpGVm4vTFpJVmdwUThtY3lxZndScjVZb3lsR3RBanhG?= =?utf-8?B?UnN1NWlYc2Q2SE5aZWhQYlpicUt4Yk4xekVIaGxoQzBsN0oxdWtIQ1RxV21s?= =?utf-8?B?RldIYmZGNUtMTFRXcEdNYy9HaVlkbEJwaUE1enNFT3ZqVzJ5UUJ6bFp2YUFI?= =?utf-8?B?TWg4cE5UNXhzMmF4dHdFMFJxcW1DTHo1a2xjSDkxSzNTbzNHWmU0VGVKZUJC?= =?utf-8?B?bHhFc3Bnc01xeEJMQzFkZ01aZXNmeTJ2TEg1K2ZqQjRzT1J1V3g1Tk16S2Fo?= =?utf-8?B?NTYzNmxrZmtiK2xxUTQ3VzFHdTc3WkMyRW9TSWh3OUY5clhvVlk4Qng0c1RB?= =?utf-8?B?UHdGMGNBK0ZaVDVLMy9BeUZ2eWRIa1IyUFg3ekFsT2JwdXg4dDF3elhkTVJv?= =?utf-8?B?a3RYZ0FtenMwVUpwOHM5MVZ3R2NlS2NTTjlVTmEzcHN4VVNsT2I4S3VWU2VL?= =?utf-8?B?NEJJZkhrZktlbUFPOXFFQjd3d01ZYU1oYU9yVStKZFZTa0hESkF1ZE1ZL0Mw?= =?utf-8?B?YVBXZHk4U3VYbkVRcllPUkJwaHp2Z1NxMUcvNFhnMUZhV1krYzFjQmY1bytl?= =?utf-8?B?TXhUV0dNYjQyenF1T2NLU091akI1UzBHeWRRTjJVV1BtZittYWxEUWlxblV5?= =?utf-8?B?RDlBVWxxY1VXdzI4QjM4NzVmV21WK29qTXIreitOb0lZRko0WlpML3pkZXNr?= =?utf-8?B?NlE3WGs2QlFvMysrUzJvekFGcWZpRm1NWHgrTUYyM2RBMHh4WW5jYXJwczR0?= =?utf-8?B?d0VENnVvbzVKVnNqUWtyMXpqUlZTV2dsZE1NNlhLV25sN2dvaXhCMDF5ZHJY?= =?utf-8?B?S2poYi9FWk5ETnB2bGxyaHJ3WHU4eWkvOHc2WDJRUWY5Z3R4b1BXWklDZ2xk?= =?utf-8?B?TElyTC8vNk1wVmpjZmtSYkpxSU1XOEhsOVU4K1Raa01pMkd0WERVekFsa2tX?= =?utf-8?B?NTdwRFk0cXpKZVExRmVSUlBqRUwrd0JOS2FkT1NZTEI3dGNDSXc5Ulc5SGJo?= =?utf-8?B?cGF0SzA0ZHVFOVJVZm05WGdoa3d3K0cwZkxXa2djci9lUm4vS0RqYjVPeXhm?= =?utf-8?B?M3NZamZBT0tKQlFTS0xOSTkwVDg3QStzK2FNMzZaazRuaUI0dDBPb2VvdXhm?= =?utf-8?B?cWRPTFdKdGMxOUMxNUluNS9jQVh6MUR1OHZYZkJ2MnNpK2ZqTFRmcHBkRjJw?= =?utf-8?B?Nk1mVWMvY2ZXZlh0R0xkeWNyZnhHNVM0VFJXU2w2MUdDWjNwL2Q0Zkh5UHhS?= =?utf-8?B?eHkwSVhEc0l4OUlLOGJlcjRYdFk2Vkx0QUR4WnlITWlwRGEzSWxvMnBWN21K?= =?utf-8?B?MjNhVGdFSk14Mk1jYmJSM3BvK2U1dDRJTjVBcTMwL0ROMDhuR3RFbFF2THNi?= =?utf-8?B?RDRNUTNmZHI0MDl1NVJiNTMzNkpqRmN0dzhmM295eDl5K1dYL0ZTeW9yRUM5?= =?utf-8?B?MTBsYjZWNzh4QVRIVy9UR1l1TTdKUmIzRWtta0JibTdqUmpSWXcydWlFODlo?= =?utf-8?B?Qzh3SE5JenhlUjdkN0VoNXU5YjRKcWpVbEpVd3lpZy9hOW16WnBmY0ZqSHU1?= =?utf-8?B?bnpKdU1WRVNqVGliTFNyY2JoOXBIVEdSbFZ5QlRwUnBCcnRPczA1RDR1UWVR?= =?utf-8?Q?0gA6V1WpojAVrh5/QUtXrzVwl2IMkk=3D?= X-Forefront-Antispam-Report: CIP:178.79.136.144; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:semantical.co.uk; PTR:semantical.co.uk; CAT:NONE; SFS:(13230040)(82310400026)(61400799027)(36860700013)(376014)(14776008)(79816003); DIR:OUT; SFP:1102; X-OriginatorOrg: masteringemacs.org X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Dec 2024 16:51:38.7966 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 235b15c1-54d7-4465-ff74-08dd1a0414d8 X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144]; Helo=[semantical.co.uk] X-MS-Exchange-CrossTenant-AuthSource: LO1PEPF000022FF.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWLP265MB6308 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: 73404@debbugs.gnu.org, Yuan Fu , Theodor Thornhill , Eli Zaretskii , Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Stefan Monnier writes: >> Ah, this matches my idea of defining sexp in other languages as =E2=80= =9Crepeatable >> construct/list-like construct=E2=80=9D. We went with =E2=80=9Cevery syn= tactic construct=E2=80=9D at >> the time, which I didn=E2=80=99t object to, but I=E2=80=99m definitely h= appier with the >> repeatable construct approach. Including Stefan and Theo since they were >> part of the original sexp navigation discussion. > > FWIW, we have both `forward-list` and `forward-list` and the new > behavior you suggest sounds closer to the historical behavior of > `forward-list` than `forward-sexp`. > Indeed, in Combobulate `-list' is explicitly used for sibling navigation, and `-sexp' instead does what it does in plain major modes, but tweaked ever so slightly. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 11 13:28:51 2024 Received: (at 73404) by debbugs.gnu.org; 11 Dec 2024 18:28:51 +0000 Received: from localhost ([127.0.0.1]:36204 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLRS7-0004ez-HO for submit@debbugs.gnu.org; Wed, 11 Dec 2024 13:28:51 -0500 Received: from mail-pf1-f180.google.com ([209.85.210.180]:43490) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLRS4-0004eK-Ax for 73404@debbugs.gnu.org; Wed, 11 Dec 2024 13:28:50 -0500 Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-725e71a11f7so831306b3a.1 for <73404@debbugs.gnu.org>; Wed, 11 Dec 2024 10:28:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733941662; x=1734546462; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=I55dRULHAyFgp2759isFlN9Revh+rhm3TQoz06NNpJA=; b=H9LlXBKdaNmjwh0zOpQE32aeWKI+s+c2yEy56eGhoR1+qLLzqN8S3Aak2t06RVxZel nqRdW/C+2nza2wBggOfF3D4/6ElqGqMOS0JEs9fsWohS6cvyFfRda+5zYthcVXo7Hex4 hhkUL01ZL7ewWcQDhgyRSA/WY0gsvIsmfunCCubdx5HmyY/0W1nK6NcAebqOh0fmA4fT ESaH2sJDl54T3tllh4bF3rezHjoP2aQeG3jR/pgExKK5nfW7NHZ9GhQQvRbqCMEKFcOD XfA7ZeW8iA4ZPQ5M3BcX3EXNygtiH+4alqZzgAWmQvQ1WhwMY2cNapvVvQLD2t9kiNML 25WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733941662; x=1734546462; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I55dRULHAyFgp2759isFlN9Revh+rhm3TQoz06NNpJA=; b=VzshE+yxQ6gSo+BYsErMjW4u8u4e795T1kcIBoFp/ilfHh4e/3KDjqjL3iw386DMce fHZOA1urpog5Gl1TQVv5B0NwjRE3nvQXBvFxm/Hqbz0wbrkOIik6wGopYlf1wAmKUKLt Ez6S4gnVLERF7a+1Y/yUD/xJn4JvJnLXxF1dEuAYd0wO6PLZeGD2g3DcaW8y9ycOrgcC 3dS2t8c45CTFUoA/grzTlf/HbqoOiiLQjrSjkgzZFOdOVmFOAXdkkmJrVg+snhjX4sBG xbTEZh1mLJkPpj+00LFpxJoxPKcDMVSzvRRVG4VcQtFR7Hl+9UQR78pNMbZDxuLv9Wiy vJjQ== X-Forwarded-Encrypted: i=1; AJvYcCVPcnsNFwUQIZhrGOinC5q3sVdHaRDqNaXX8U0+HwcrLCe98EV6zB6ntn/B3QTDGJ6Z5pQung==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyoT4naXOMEaMFkL9VSoH7piU7A+yh/K3theQfeFsB+KILB9NCG BzDZDyEfiz9o2TijdODJYf9dCAfZ9lGNEb1JKCiLEg15L6czvwLz X-Gm-Gg: ASbGncvfVfyJC4s5tM8nd85/hSFLsRCQPmwqEipDG4L0IXyFlp0oRP6clMSsfTbEPIJ 2sSpqkEE6u2J5waubURq4eyKyrmpnpXKnftaYxemFXxAR7AADlx+pkSJAZLL1q9YNHf3AYNVNep v5tY/Wf1Vc4Gj35F62PD+VxwkNQUWxFNG8/OvcItC4m+Wqxi2SBseX/cKRmwYU6IbARLj8WM2hC DptGZhFOvtAAiwpCLibL22SFIwoWaSZC438BgdZoJC0yImplUYGXexeIav4CU95YSEw/iqK2ONK xg== X-Google-Smtp-Source: AGHT+IGDGDXlS7HJ3zbLtIfrgznf50Zn8SZRAHBdOvtDrw2IDMJRuc2AMnXKIXY7QFJjpYGpSqGyCQ== X-Received: by 2002:a05:6a20:918d:b0:1e1:bd9b:40cf with SMTP id adf61e73a8af0-1e1cea1d239mr415486637.8.1733941662133; Wed, 11 Dec 2024 10:27:42 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:f90e:3b71:6ee2:6197]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fd3fb02955sm7167598a12.81.2024.12.11.10.27.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Dec 2024 10:27:41 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: Date: Wed, 11 Dec 2024 10:27:29 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> To: Stefan Monnier X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 11, 2024, at 7:12=E2=80=AFAM, Stefan Monnier = wrote: >=20 >> Ah, this matches my idea of defining sexp in other languages as = =E2=80=9Crepeatable >> construct/list-like construct=E2=80=9D. We went with =E2=80=9Cevery = syntactic construct=E2=80=9D at >> the time, which I didn=E2=80=99t object to, but I=E2=80=99m = definitely happier with the >> repeatable construct approach. Including Stefan and Theo since they = were >> part of the original sexp navigation discussion. >=20 > FWIW, we have both `forward-list` and `forward-list` and the new > behavior you suggest sounds closer to the historical behavior of > `forward-list` than `forward-sexp`. >=20 >=20 > Stefan >=20 Actually, what=E2=80=99s the difference between forward-list and = forward-sexp? I always thought they are the same at least for Lisp. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 02:23:30 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 07:23:30 +0000 Received: from localhost ([127.0.0.1]:37372 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLdXm-0003K0-8L for submit@debbugs.gnu.org; Thu, 12 Dec 2024 02:23:30 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:36763) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLdXh-0003Je-Vh for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 02:23:28 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id BDD0960003; Thu, 12 Dec 2024 07:23:12 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> (Yuan Fu's message of "Wed, 11 Dec 2024 10:27:29 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> Date: Thu, 12 Dec 2024 09:17:44 +0200 Message-ID: <87frmtbc9z.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >>> Ah, this matches my idea of defining sexp in other languages as “repeatable >>> construct/list-like construct”. We went with “every syntactic construct” at >>> the time, which I didn’t object to, but I’m definitely happier with the >>> repeatable construct approach. Including Stefan and Theo since they were >>> part of the original sexp navigation discussion. >> >> FWIW, we have both `forward-list` and `forward-list` and the new >> behavior you suggest sounds closer to the historical behavior of >> `forward-list` than `forward-sexp`. > > Actually, what’s the difference between forward-list and forward-sexp? > I always thought they are the same at least for Lisp. forward-sexp moves over a balanced parenthetical group like forward-list does. Plus forward-sexp also moves over an atom such as a symbol, a number. The problem is that treesit adds too much structural information to such simple things as a symbol and a number. For example, in js a simple keyword "export" gets the "(export_statement export" subtree, Another keyword "const" gets "(lexical_declaration kind: const", etc. Therefore for such symbols forward-sexp needs to bypass the structure and use simpler syntactic information to move over them like on a flat list. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 02:41:20 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 07:41:20 +0000 Received: from localhost ([127.0.0.1]:37445 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLdp2-0004OU-1b for submit@debbugs.gnu.org; Thu, 12 Dec 2024 02:41:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLdoz-0004OB-JO for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 02:41:18 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tLdos-0006dK-8p; Thu, 12 Dec 2024 02:41:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=wcp5LueQE6cSq3gzt/YgZ1PnzYzSX/i44lwvpLKA7oU=; b=W7xb5TbEIMLgxI0A1efo 2W7Y+7TQPqEvpoaWToaPaZ5ojwssxADXvolQcTCsFGVePDIGsMSLDUQTNjyq/qmMwcY351mjBIQ/0 Ys7//NnSxlejOeFXnj4m2ahxwHQHe6XkFdd6tVrevwnZ53CVEjMiMNJXfXUBv1t0YSMRmuFLsdgLt 2lJvzWwbihRPFIX5U9J3X34hNQJ3enCZSz0nr+55XySROsXq7gmilMJtJbmRhFpE0mYMUvxzzEqRl e+YYbistvu14uVMwAc3RmIk7aF/0kHtWxMtCpd37dSiN67XTfr6WZr7N8uj0/hRHW9sywgzEp6ZCY 9ruLZ5fGbawMvA==; Date: Thu, 12 Dec 2024 09:40:57 +0200 Message-Id: <86bjxh1h86.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87frmtbc9z.fsf@mail.linkov.net> (message from Juri Linkov on Thu, 12 Dec 2024 09:17:44 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, casouri@gmail.com, mickey@masteringemacs.org, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: Stefan Monnier , Eli Zaretskii > , Mickey Petersen , > 73404@debbugs.gnu.org, Theodor Thornhill > Date: Thu, 12 Dec 2024 09:17:44 +0200 > > >>> Ah, this matches my idea of defining sexp in other languages as “repeatable > >>> construct/list-like construct”. We went with “every syntactic construct” at > >>> the time, which I didn’t object to, but I’m definitely happier with the > >>> repeatable construct approach. Including Stefan and Theo since they were > >>> part of the original sexp navigation discussion. > >> > >> FWIW, we have both `forward-list` and `forward-list` and the new > >> behavior you suggest sounds closer to the historical behavior of > >> `forward-list` than `forward-sexp`. > > > > Actually, what’s the difference between forward-list and forward-sexp? > > I always thought they are the same at least for Lisp. > > forward-sexp moves over a balanced parenthetical group like > forward-list does. Plus forward-sexp also moves over an atom > such as a symbol, a number. > > The problem is that treesit adds too much structural information > to such simple things as a symbol and a number. For example, in js > a simple keyword "export" gets the "(export_statement export" subtree, > Another keyword "const" gets "(lexical_declaration kind: const", etc. > > Therefore for such symbols forward-sexp needs to bypass the structure > and use simpler syntactic information to move over them like on a flat list. If you mean we should ignore the information provided by tree-sitter and instead use our own syntactic information, then that sounds wrong to me, FWIW. Why cannot we understand enough of the tree-sitter structural information to move like we want? Presumably, the structural information provided by tree-sitter is a portion of a parse tree, which to me means we should be able to move between the parse tree's nodes as long as we understand the tree and can interpret it in our terms. Aren't there some grammar-agnostic traits of tree-sitter nodes that would allow us to interpret the nodes in language-independent terms? If that is not available, then each major mode will have to provide treesit.el with a way to interpret the tree-sitter nodes of the corresponding grammar in a way that will allow sexp movement, thus providing an abstraction layer that treesit.el could use for the movement commands. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 03:03:59 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 08:04:00 +0000 Received: from localhost ([127.0.0.1]:37484 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLeAx-0005wA-HW for submit@debbugs.gnu.org; Thu, 12 Dec 2024 03:03:59 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:55787) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLeAu-0005vh-AD for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 03:03:57 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id C18291C0008; Thu, 12 Dec 2024 08:03:28 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86bjxh1h86.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 12 Dec 2024 09:40:57 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> Date: Thu, 12 Dec 2024 09:58:03 +0200 Message-ID: <87y10l8h6k.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, casouri@gmail.com, mickey@masteringemacs.org, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> forward-sexp moves over a balanced parenthetical group like >> forward-list does. Plus forward-sexp also moves over an atom >> such as a symbol, a number. >> >> The problem is that treesit adds too much structural information >> to such simple things as a symbol and a number. For example, in js >> a simple keyword "export" gets the "(export_statement export" subtree, >> Another keyword "const" gets "(lexical_declaration kind: const", etc. >> >> Therefore for such symbols forward-sexp needs to bypass the structure >> and use simpler syntactic information to move over them like on a flat list. > > If you mean we should ignore the information provided by tree-sitter > and instead use our own syntactic information, then that sounds wrong > to me, FWIW. Why cannot we understand enough of the tree-sitter > structural information to move like we want? Presumably, the > structural information provided by tree-sitter is a portion of a parse > tree, which to me means we should be able to move between the parse > tree's nodes as long as we understand the tree and can interpret it in > our terms. > > Aren't there some grammar-agnostic traits of tree-sitter nodes that > would allow us to interpret the nodes in language-independent terms? > If that is not available, then each major mode will have to provide > treesit.el with a way to interpret the tree-sitter nodes of the > corresponding grammar in a way that will allow sexp movement, thus > providing an abstraction layer that treesit.el could use for the > movement commands. Maybe it would be possible to use something like 'flatten-tree' on the treesit's syntax tree? But this will require the addition of a lot of rules to specify what nodes should be flattened. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 03:24:54 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 08:24:55 +0000 Received: from localhost ([127.0.0.1]:37526 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLeVC-0007P5-Mu for submit@debbugs.gnu.org; Thu, 12 Dec 2024 03:24:54 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:54591) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLeV9-0007Oa-VW for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 03:24:52 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id AEF64FF808; Thu, 12 Dec 2024 08:24:23 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87y10l8h6k.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 12 Dec 2024 09:58:03 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> Date: Thu, 12 Dec 2024 10:14:59 +0200 Message-ID: <87ldwl8g60.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Maybe it would be possible to use something like 'flatten-tree' > on the treesit's syntax tree? But this will require the addition > of a lot of rules to specify what nodes should be flattened. A better idea: instead of 'sexp' in treesit-thing-settings define separately 'list' and 'atom', e.g. replace (setq-local treesit-thing-settings `((javascript (sexp ,(js--regexp-opt-symbol js--treesit-sexp-nodes))))) with (setq-local treesit-thing-settings `((javascript (list ,(js--regexp-opt-symbol js--treesit-list-nodes)) (atom ,(js--regexp-opt-symbol js--treesit-atom-nodes))))) From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 11:45:00 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 16:45:00 +0000 Received: from localhost ([127.0.0.1]:40062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLmJ9-0003eT-Ob for submit@debbugs.gnu.org; Thu, 12 Dec 2024 11:45:00 -0500 Received: from relay1-d.mail.gandi.net ([217.70.183.193]:48249) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLmJ8-0003e7-70 for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 11:44:59 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 80C0C240006; Thu, 12 Dec 2024 16:44:50 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87ldwl8g60.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 12 Dec 2024 10:14:59 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> Date: Thu, 12 Dec 2024 18:31:04 +0200 Message-ID: <87wmg53rdj.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, casouri@gmail.com, mickey@masteringemacs.org, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > A better idea: instead of 'sexp' in treesit-thing-settings > define separately 'list' and 'atom', e.g. replace > > (setq-local treesit-thing-settings > `((javascript > (sexp ,(js--regexp-opt-symbol js--treesit-sexp-nodes))))) > > with > > (setq-local treesit-thing-settings > `((javascript > (list ,(js--regexp-opt-symbol js--treesit-list-nodes)) > (atom ,(js--regexp-opt-symbol js--treesit-atom-nodes))))) Still the problem is that using the atoms from the tree doesn't provide backward-compatibility with non-ts modes and how C-M-f moves on non-list atoms there. One way would be to extract anonymous text leaf nodes such as "export" and "const" from (export_statement "export" declaration: (lexical_declaration kind: "const" But still need to check symbol/word syntax to omit such nodes as "+" from (binary_expression left: (identifier) operator: "+" Therefore to provide backward-compatibility with non-ts modes in regard to C-M-f navigation, navigation on atoms should follow the Sword/Ssymbol rules of 'scan_lists' with non-nil 'sexpflag'. So an atom should be a syntactical entity, not structural. This means that treesit-forward-sexp should use the 'list' thing with syntactical atoms. For example, for 'C-M-f' on var p = { case: 'zzzz', -!-default: 'donkey', tee: 'ornery' }; in js-ts-mode it would be unexpected to move to var p = { case: 'zzzz', default: 'donkey'-!-, tee: 'ornery' }; because js-mode moves to var p = { case: 'zzzz', default-!-: 'donkey', tee: 'ornery' }; But anyway 'list' should be customizable as 'sexp' already is. OTOH, transpose-sexps should use treesit 'sexp' with more fine-grained lists that are not suitable for forward-sexp. (And we have no transpose-lists.) For example, it's expected for transpose-sexps to transpose a pair of key:value defined by 'sexp': var p = { default: 'donkey', -!-case: 'zzzz', tee: 'ornery' }; that will be a big improvement when comparing to js-mode. This also will close bug#60655. And if someone want to transpose adjacent atoms (symbols or words), there is transpose-words. In summary: The current implementation in treesit-forward-sexp is more like forward-list. So let's rename it to treesit-forward-list, then create a new implementation of treesit-forward-sexp that uses the 'list' thing together with syntactical atoms. Another variant is to leave treesit-forward-sexp as is, and create a new function treesit-forward-sexp-with-list that uses the 'list' thing. And anyway let's keep the current implementation of treesit-transpose-sexps that uses the 'sexp' thing. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 12:52:51 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 17:52:51 +0000 Received: from localhost ([127.0.0.1]:40199 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLnMo-0007Vk-Lo for submit@debbugs.gnu.org; Thu, 12 Dec 2024 12:52:51 -0500 Received: from [217.70.183.199] (port=34671 helo=relay9-d.mail.gandi.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLnMl-0007VB-6m for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 12:52:49 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 9C2B2FF804; Thu, 12 Dec 2024 17:52:26 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87wmg53rdj.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 12 Dec 2024 18:31:04 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> Date: Thu, 12 Dec 2024 19:49:30 +0200 Message-ID: <87a5d0n651.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-GND-Sasl: juri@linkov.net X-Spam-Score: 0.6 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.4 (/) --=-=-= Content-Type: text/plain > Another variant is to leave treesit-forward-sexp as is, > and create a new function treesit-forward-sexp-with-list > that uses the 'list' thing. This patch keep the current function treesit-forward-sexp, and creates a new function treesit-forward-sexp-list that uses the 'sexp-list' thing to navigate lists while using forward-sexp-default-function to navigate atoms: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=treesit-forward-sexp-list.patch diff --git a/lisp/treesit.el b/lisp/treesit.el index db8f7a7595d..f064be55b9c 100644 --- a/lisp/treesit.el +++ b/lisp/treesit.el @@ -2400,6 +2400,68 @@ treesit-forward-sexp (treesit-node-start boundary) (treesit-node-end boundary))))))) +(defun treesit-forward-sexp-list (&optional arg) + "Tree-sitter implementation for `forward-sexp-function'. + +ARG is described in the docstring of `forward-sexp-function'. + +If point is inside a text environment where tree-sitter is not +supported, go forward a sexp using `forward-sexp-default-function'. +If point is inside code, use tree-sitter functions with the +following behavior. If there are no further sexps to move across, +signal `scan-error' like `forward-sexp' does. If point is already +at top-level, return nil without moving point. + +What constitutes as text and source code sexp is determined +by `text' and `sexp' in `treesit-thing-settings'." + (interactive "^p") + (let* ((arg (or arg 1)) + (pred (or treesit-sexp-type-regexp 'sexp-list)) + (current-thing (treesit-thing-at (point) pred t)) + (default-pos + (condition-case _ + (save-excursion + (forward-sexp-default-function arg) + (point)) + (scan-error nil))) + (default-pos (unless (eq (point) default-pos) default-pos)) + (sibling-pos + (save-excursion + (and (if (> arg 0) + (treesit-end-of-thing pred (abs arg) 'restricted) + (treesit-beginning-of-thing pred (abs arg) 'restricted)) + (point)))) + (sibling (when sibling-pos + (if (> arg 0) + (treesit-thing-prev sibling-pos pred) + (treesit-thing-next sibling-pos pred))))) + + ;; 'forward-sexp-default-function' should not go out of the current thing, + ;; neither go inside the next thing, neither go over the next thing + (or (when (and default-pos + (or (null current-thing) + (if (> arg 0) + (< default-pos (treesit-node-end current-thing)) + (> default-pos (treesit-node-start current-thing)))) + (or (null sibling) + (if (> arg 0) + (< default-pos (treesit-node-start sibling)) + (> default-pos (treesit-node-end sibling))))) + (goto-char default-pos)) + (when sibling-pos + (goto-char sibling-pos)) + ;; If we couldn't move, we should signal an error and report + ;; the obstacle, like `forward-sexp' does. If we couldn't + ;; find a parent, we simply return nil without moving point, + ;; then functions like `up-list' will signal "at top level". + (when-let* ((parent (treesit-thing-at (point) pred t)) + (boundary (if (> arg 0) + (treesit-node-child parent -1) + (treesit-node-child parent 0)))) + (signal 'scan-error (list "No more sexp to move across" + (treesit-node-start boundary) + (treesit-node-end boundary))))))) + (defun treesit-transpose-sexps (&optional arg) "Tree-sitter `transpose-sexps' function. ARG is the same as in `transpose-sexps'. @@ -2849,7 +2911,7 @@ treesit-navigate-thing (if (eq tactic 'restricted) (setq pos (funcall advance - (cond ((and (null next) (null prev)) parent) + (cond ((and (null next) (null prev) (not (eq thing 'sexp-list))) parent) ((> arg 0) next) (t prev)))) ;; For `nested', it's a bit more work: @@ -3246,6 +3308,9 @@ treesit-major-mode-setup (setq-local forward-sexp-function #'treesit-forward-sexp) (setq-local transpose-sexps-function #'treesit-transpose-sexps)) + (when (treesit-thing-defined-p 'sexp-list nil) + (setq-local forward-sexp-function #'treesit-forward-sexp-list)) + (when (treesit-thing-defined-p 'sentence nil) (setq-local forward-sentence-function #'treesit-forward-sentence)) diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el index dbf721e8d0f..c4d33564e80 100644 --- a/lisp/progmodes/js.el +++ b/lisp/progmodes/js.el @@ -3878,6 +3878,19 @@ js--treesit-sexp-nodes "Nodes that designate sexps in JavaScript. See `treesit-thing-settings' for more information.") +(defvar js--treesit-sexp-list-nodes + '("formal_parameters" + "arguments" + "statement_block" + "parenthesized_expression" + "switch_body" + "array" + "object" + "string" + "regex") + "Nodes that designate lists in JavaScript. +See `treesit-thing-settings' for more information.") + (defvar js--treesit-jsdoc-beginning-regexp (rx bos "/**") "Regular expression matching the beginning of a jsdoc block comment.") @@ -3921,6 +3934,7 @@ js-ts-mode (setq-local treesit-thing-settings `((javascript (sexp ,(js--regexp-opt-symbol js--treesit-sexp-nodes)) + (sexp-list ,(js--regexp-opt-symbol js--treesit-sexp-list-nodes)) (sentence ,(js--regexp-opt-symbol js--treesit-sentence-nodes)) (text ,(js--regexp-opt-symbol '("comment" "string_fragment")))))) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 12 14:14:02 2024 Received: (at 73404) by debbugs.gnu.org; 12 Dec 2024 19:14:03 +0000 Received: from localhost ([127.0.0.1]:40345 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLodO-0003My-AI for submit@debbugs.gnu.org; Thu, 12 Dec 2024 14:14:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLodK-0003MG-8o for 73404@debbugs.gnu.org; Thu, 12 Dec 2024 14:14:00 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tLodC-00025O-Md; Thu, 12 Dec 2024 14:13:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=PMSfyoQmLyXPqCGELZZRod4ZVsqLF7jtxCnqpva+WFI=; b=sYD+hFeVpX// DCSUMy7dRczt0A8wqKlR5r4fOi3FDsH6Ep83DUZtgZmbSORLtsYyDdnDX0YJIOB3I+80kY1wN2pOv VpUayrmewhadi3ND+sCk6b4IkoA1FVtsoh9WKTn7QAZBaiQiIVWgPVO8EzMHKBqLTmrA5wLWxMEus aXWCsje33zuyq48Knb7rnwMJ4p9GTQptCSo+5AoO5qTbIjIN9a9Rg8XNzF1JjECl9xirECaDR5jFk fn7eclHWYwv7AxLGwWKtK6F9k3kMcyatYNbqPk2HG4V3bjnDpU3UzLWn4+tOYidB+PKxtovh7OsYq K90ZapnUlOJnDnNf5EH2+g==; Date: Thu, 12 Dec 2024 21:13:07 +0200 Message-Id: <865xnozpdo.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87a5d0n651.fsf@mail.linkov.net> (message from Juri Linkov on Thu, 12 Dec 2024 19:49:30 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: theo@thornhill.no, casouri@gmail.com, mickey@masteringemacs.org, > monnier@iro.umontreal.ca, 73404@debbugs.gnu.org > Date: Thu, 12 Dec 2024 19:49:30 +0200 > > > Another variant is to leave treesit-forward-sexp as is, > > and create a new function treesit-forward-sexp-with-list > > that uses the 'list' thing. > > This patch keep the current function treesit-forward-sexp, > and creates a new function treesit-forward-sexp-list > that uses the 'sexp-list' thing to navigate lists while > using forward-sexp-default-function to navigate atoms: Introducing a new command raises the question how should users navigate using the two. Will C-M- invoke forward-sexp or the new command (or maybe one or the other in some dwin-ish way)? What are the problems with teaching treesit-forward-sexp do what the new command does? IOW, why do we need a new command? From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 13 02:10:24 2024 Received: (at 73404) by debbugs.gnu.org; 13 Dec 2024 07:10:24 +0000 Received: from localhost ([127.0.0.1]:41347 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLzoe-00057J-5K for submit@debbugs.gnu.org; Fri, 13 Dec 2024 02:10:24 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:59881) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tLzoc-000574-1m for 73404@debbugs.gnu.org; Fri, 13 Dec 2024 02:10:22 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 7995260006; Fri, 13 Dec 2024 07:10:12 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <865xnozpdo.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 12 Dec 2024 21:13:07 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> <865xnozpdo.fsf@gnu.org> Date: Fri, 13 Dec 2024 09:06:04 +0200 Message-ID: <87zfl083rn.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> This patch keep the current function treesit-forward-sexp, >> and creates a new function treesit-forward-sexp-list >> that uses the 'sexp-list' thing to navigate lists while >> using forward-sexp-default-function to navigate atoms: > > Introducing a new command raises the question how should users > navigate using the two. Will C-M- invoke forward-sexp or the > new command (or maybe one or the other in some dwin-ish way)? > > What are the problems with teaching treesit-forward-sexp do what the > new command does? IOW, why do we need a new command? Actually, it's not a new command, but a new function for C-M- via forward-sexp-function. This is from treesit-major-mode-setup: (when (treesit-thing-defined-p 'sexp nil) (setq-local forward-sexp-function #'treesit-forward-sexp) (setq-local transpose-sexps-function #'treesit-transpose-sexps)) (when (treesit-thing-defined-p 'sexp-list nil) (setq-local forward-sexp-function #'treesit-forward-sexp-list)) When a ts-mode defines the 'sexp-list' thing, only then it's used. Otherwise the current implementation with 'sexp' is used for C-M-. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 14 06:02:35 2024 Received: (at 73404) by debbugs.gnu.org; 14 Dec 2024 11:02:35 +0000 Received: from localhost ([127.0.0.1]:45704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMPus-0002do-RS for submit@debbugs.gnu.org; Sat, 14 Dec 2024 06:02:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46878) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMPup-0002dV-Ch for 73404@debbugs.gnu.org; Sat, 14 Dec 2024 06:02:33 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tMPui-0001qv-O9; Sat, 14 Dec 2024 06:02:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=bpAHJBDO/TCtAU/RF5m6AAYcb7crOHEAdGUiOqdobtE=; b=p8ugF7EIDqIa TUjul+vnfd8a9IHLAPv8kT7NjIMBP0p3Wy5h58jZvqITr8YphZFqeqcvkCjRwxu2Ia7NFBqpDy+Ej ZVPZFwhpztuv22VreznGQ4ra0nimg6IYxxUBMpQgIzSg3mzMmOLyVlpP8WijoMyyrlUmBbNP5i8Yy 3JeypxRT8LKTh43EQY9xL5E08zCoaZsftVofHwMspq5Ii9XTP/BFtoMtvzWDukZJhidQcYHrWohE1 s8kM2gant5PzdE9Irz9SZC2WR8CGy/cl+O8p98/q7YQIIGLvXFGoPq5h+TZCdJCQqdT71VBwQ4iG6 fMy45hFDXQt/YYVznQWNlA==; Date: Sat, 14 Dec 2024 13:02:19 +0200 Message-Id: <86ed2av878.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87zfl083rn.fsf@mail.linkov.net> (message from Juri Linkov on Fri, 13 Dec 2024 09:06:04 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> <865xnozpdo.fsf@gnu.org> <87zfl083rn.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: theo@thornhill.no, casouri@gmail.com, mickey@masteringemacs.org, > monnier@iro.umontreal.ca, 73404@debbugs.gnu.org > Date: Fri, 13 Dec 2024 09:06:04 +0200 > > >> This patch keep the current function treesit-forward-sexp, > >> and creates a new function treesit-forward-sexp-list > >> that uses the 'sexp-list' thing to navigate lists while > >> using forward-sexp-default-function to navigate atoms: > > > > Introducing a new command raises the question how should users > > navigate using the two. Will C-M- invoke forward-sexp or the > > new command (or maybe one or the other in some dwin-ish way)? > > > > What are the problems with teaching treesit-forward-sexp do what the > > new command does? IOW, why do we need a new command? > > Actually, it's not a new command, but a new function for C-M- > via forward-sexp-function. It's an interactive function, so it's a command. > This is from treesit-major-mode-setup: > > (when (treesit-thing-defined-p 'sexp nil) > (setq-local forward-sexp-function #'treesit-forward-sexp) > (setq-local transpose-sexps-function #'treesit-transpose-sexps)) > > (when (treesit-thing-defined-p 'sexp-list nil) > (setq-local forward-sexp-function #'treesit-forward-sexp-list)) > > When a ts-mode defines the 'sexp-list' thing, only then it's used. > Otherwise the current implementation with 'sexp' is used for C-M-. Then why is the new function interactive? From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 14 13:16:14 2024 Received: (at 73404) by debbugs.gnu.org; 14 Dec 2024 18:16:15 +0000 Received: from localhost ([127.0.0.1]:48276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMWgY-0007fv-DG for submit@debbugs.gnu.org; Sat, 14 Dec 2024 13:16:14 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:49197) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMWgT-0007fd-2r for 73404@debbugs.gnu.org; Sat, 14 Dec 2024 13:16:13 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id F1B3BFF802; Sat, 14 Dec 2024 18:15:39 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86ed2av878.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 14 Dec 2024 13:02:19 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> <865xnozpdo.fsf@gnu.org> <87zfl083rn.fsf@mail.linkov.net> <86ed2av878.fsf@gnu.org> Date: Sat, 14 Dec 2024 20:14:54 +0200 Message-ID: <87jzc25dzx.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain >> Actually, it's not a new command, but a new function for C-M- >> via forward-sexp-function. > > It's an interactive function, so it's a command. > >> When a ts-mode defines the 'sexp-list' thing, only then it's used. >> Otherwise the current implementation with 'sexp' is used for C-M-. > > Then why is the new function interactive? Ah, I didn't notice it's interactive! treesit-forward-sexp-list was based on treesit-forward-sexp that has the interactive spec added by Theo in the commit 207901457c01. I guess that Theo decided to make this function interactive for such use case that users could use it separately or bind it to a key. What really needs to be interactive is the new function treesit-forward-list added in the following patch because there is no special variable forward-list-function for forward-list like there is forward-sexp-function for forward-sexp. So users might want to use it separately: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=treesit-forward-list.patch diff --git a/lisp/treesit.el b/lisp/treesit.el index 18200acf53f..a1c012b6d2f 100644 --- a/lisp/treesit.el +++ b/lisp/treesit.el @@ -2366,6 +2366,11 @@ treesit-sexp-type-regexp however, smaller in scope than sentences. This is used by `treesit-forward-sexp' and friends.") +(defun treesit-forward-list (&optional arg) + (interactive "^p") + (let ((treesit-sexp-type-regexp 'sexp-list)) + (treesit-forward-sexp arg))) + (defun treesit-forward-sexp (&optional arg) "Tree-sitter implementation for `forward-sexp-function'. @@ -2382,18 +2387,8 @@ treesit-forward-sexp by `text' and `sexp' in `treesit-thing-settings'." (interactive "^p") (let ((arg (or arg 1)) - (pred (or treesit-sexp-type-regexp 'sexp)) - (node-at-point - (treesit-node-at (point) (treesit-language-at (point))))) - (or (when (and node-at-point - ;; Make sure point is strictly inside node. - (< (treesit-node-start node-at-point) - (point) - (treesit-node-end node-at-point)) - (treesit-node-match-p node-at-point 'text t)) - (forward-sexp-default-function arg) - t) - (if (> arg 0) + (pred (or treesit-sexp-type-regexp 'sexp))) + (or (if (> arg 0) (treesit-end-of-thing pred (abs arg) 'restricted) (treesit-beginning-of-thing pred (abs arg) 'restricted)) ;; If we couldn't move, we should signal an error and report @@ -2408,6 +2403,63 @@ treesit-forward-sexp (treesit-node-start boundary) (treesit-node-end boundary))))))) +(defun treesit-forward-sexp-list (&optional arg) + "Tree-sitter implementation for `forward-sexp-function'. + +ARG is described in the docstring of `forward-sexp-function'. + +If point is inside a text environment where tree-sitter is not +supported, go forward a sexp using `forward-sexp-default-function'. +If point is inside code, use tree-sitter functions with the +following behavior. If there are no further sexps to move across, +signal `scan-error' like `forward-sexp' does. If point is already +at top-level, return nil without moving point. + +What constitutes as text and source code sexp is determined +by `text' and `sexp' in `treesit-thing-settings'." + (interactive "^p") + (let* ((arg (or arg 1)) + (pred 'sexp-list) + (default-pos + (condition-case _ + (save-excursion + (forward-sexp-default-function arg) + (point)) + (scan-error nil))) + (default-pos (unless (eq (point) default-pos) default-pos)) + (sibling-pos + (when default-pos + (save-excursion + (and (if (> arg 0) + (treesit-end-of-thing pred (abs arg) 'restricted) + (treesit-beginning-of-thing pred (abs arg) 'restricted)) + (point))))) + (sibling (when sibling-pos + (if (> arg 0) + (treesit-thing-prev sibling-pos pred) + (treesit-thing-next sibling-pos pred)))) + (sibling (when (and sibling + (if (> arg 0) + (<= (point) (treesit-node-start sibling)) + (>= (point) (treesit-node-start sibling)))) + sibling)) + (current-thing (when default-pos + (treesit-thing-at (point) pred t)))) + + ;; 'forward-sexp-default-function' should not go out of the current thing, + ;; neither go inside the next thing, neither go over the next thing + (or (when (and default-pos + (or (null current-thing) + (if (> arg 0) + (< default-pos (treesit-node-end current-thing)) + (> default-pos (treesit-node-start current-thing)))) + (or (null sibling) + (if (> arg 0) + (<= default-pos (treesit-node-start sibling)) + (>= default-pos (treesit-node-end sibling))))) + (goto-char default-pos)) + (treesit-forward-list arg)))) + (defun treesit-transpose-sexps (&optional arg) "Tree-sitter `transpose-sexps' function. ARG is the same as in `transpose-sexps'. @@ -2857,7 +2909,7 @@ treesit-navigate-thing (if (eq tactic 'restricted) (setq pos (funcall advance - (cond ((and (null next) (null prev)) parent) + (cond ((and (null next) (null prev) (not (eq thing 'sexp-list))) parent) ((> arg 0) next) (t prev)))) ;; For `nested', it's a bit more work: @@ -3254,6 +3306,9 @@ treesit-major-mode-setup (setq-local forward-sexp-function #'treesit-forward-sexp) (setq-local transpose-sexps-function #'treesit-transpose-sexps)) + (when (treesit-thing-defined-p 'sexp-list nil) + (setq-local forward-sexp-function #'treesit-forward-sexp-list)) + (when (treesit-thing-defined-p 'sentence nil) (setq-local forward-sentence-function #'treesit-forward-sentence)) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 18 02:43:34 2024 Received: (at 73404) by debbugs.gnu.org; 18 Dec 2024 07:43:34 +0000 Received: from localhost ([127.0.0.1]:33227 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tNoiU-00034E-7x for submit@debbugs.gnu.org; Wed, 18 Dec 2024 02:43:34 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:42477) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tNoiR-00033t-JB for 73404@debbugs.gnu.org; Wed, 18 Dec 2024 02:43:32 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 583BC1C0003; Wed, 18 Dec 2024 07:43:00 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87a5d0n651.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 12 Dec 2024 19:49:30 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> Date: Wed, 18 Dec 2024 09:37:39 +0200 Message-ID: <87zfktphks.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, mickey@masteringemacs.org, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) While testing treesit-forward-sexp-list, I discovered that thing-navigation functions are not restricted to named nodes. I wonder if there a reason to find anonymous nodes as things? The problem was found with the node "unless" in Ruby: unless cond a += 1 else b -= 1 end Here the named node 'unless' has exactly the same name as the anonymous node with the text "unless": (unless "unless" condition: (identifier) Finding anonymous nodes breaks forward-sexp when point is on "unless": un-!-less cond a += 1 else b -= 1 end because (treesit-thing-at (point) 'sexp t) finds # instead of # Also this breaks backward-sexp and backward-up-list because treesit--thing-sibling finds the anonymous node "unless" as a previous sibling instead of the named node 'unless' as a parent. Would the right solution be to check if the found thing is a named node? With something like: diff --git a/lisp/treesit.el b/lisp/treesit.el index 18200acf53f..9ad879ee40c 100644 --- a/lisp/treesit.el +++ b/lisp/treesit.el @@ -2711,6 +2774,7 @@ treesit--thing-sibling (lambda (n) (>= (treesit-node-start n) pos)))) (iter-pred (lambda (node) (and (treesit-node-match-p node thing t) + (treesit-node-check node 'named) (funcall pos-pred node)))) (sibling nil)) (when cursor @@ -2760,6 +2824,7 @@ treesit-thing-at (let* ((cursor (treesit-node-at pos)) (iter-pred (lambda (node) (and (treesit-node-match-p node thing t) + (treesit-node-check node 'named) (if strict (< (treesit-node-start node) pos) (<= (treesit-node-start node) pos)) From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 18 23:05:33 2024 Received: (at 73404) by debbugs.gnu.org; 19 Dec 2024 04:05:33 +0000 Received: from localhost ([127.0.0.1]:36902 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tO7n2-0005CO-W3 for submit@debbugs.gnu.org; Wed, 18 Dec 2024 23:05:33 -0500 Received: from mail-pf1-f172.google.com ([209.85.210.172]:43067) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tO7n0-0005C8-4Y for 73404@debbugs.gnu.org; Wed, 18 Dec 2024 23:05:31 -0500 Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-725dc290c00so1215682b3a.0 for <73404@debbugs.gnu.org>; Wed, 18 Dec 2024 20:05:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734581064; x=1735185864; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=z2TIDlaz7NNbHJehin0C2XwrYBL/bg3tGB5mCt7W1gA=; b=GGE2L/GAq4MK6V2xWAAfs0SygybqPQ5ZKyX7fGvcUKlKn+SA9cVrLQGMIkPYtAEWkF PK8/3HbDKRKiTkmwK8c+e2hqN06ifbx33rKMiHpUd4wPdqJ0CSfWIc6VdPLwluMa7zf6 PLbkyqINFppOVI4NFOyLwYSqO3UzccNWHDJLUJ/2zd/8OkFTNuTcJmGX3ti31CHkuCPW dwGGmr9YZf++oQLg+ATpUkulnuwKjlu2NLeimppzFHw57SitVFziD5Pwg0rY3Rir05pq DiTdIBSvMHhUUG++D8+SrHU9yq0HZCEwYZ0R5n6pL+f1OEjJALEOMdemCKWCBToT0rO3 r80A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734581064; x=1735185864; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z2TIDlaz7NNbHJehin0C2XwrYBL/bg3tGB5mCt7W1gA=; b=dcCyB4tYvLTk4xdVoSYhzJZt90UReVJu9XLxSMNc3fwta5cEaKjV90lCkA9eQ3uy3H Su5SMHfWfpyzZYu9esjNFjFpHDcKcWVk1cY0To521+xtK8gHK+4xc63jmh+8YcSIPu3O Nduvat5W+8Du/84i8wwn7CBgo3XDvwYHumwrfHckvUE6bHCPGa8kE1bUaSaUiZ9NFkDx iXNNbX33qGSbEAOKnO5OVmjqfWt7bETTVE4MtLQqOPfg2j0+eERywVsdesd14MODA0yM Vj6eAS9/u+7zWJFAABGJ3/XF1LQxJI5ob8M9yIjLEMQZcY2ik+85+5dC//1IK5X5Az70 UC1g== X-Forwarded-Encrypted: i=1; AJvYcCWBf6rdhJgUxzXy48bKduhjUjFdidXegIEnsi1B7oI9OkpPAieeEONJdehnI5PWzht2AIzlbQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yw4OiSIJW2sErI0D9W+VXsO+9a3AOdMPor+4/5TyDMJRqsr9JYO 5CBc8wDN6rlj7xyJBp3UIYVG9Kzh+H4YVK4nSuzexOjn2zYFY9Xp X-Gm-Gg: ASbGncv8dHATnkB5Tx54YpxwYHrTeg7+Ja8llaeE7DgfpMa16Y+pYgPp/1SxeynU6cz mZSNQyMkgH5zR7sdLiqixIATIFQ03rUDss8e4UaedH0WyFGTZ2yzTlCszViuy+AL+ApLQklbnR4 bP970+QVW26wHWnYZU6hWsnZRAWaZELHTgKIcZgpywaqmtG4yxkvLgpZbVgEMRB/JQJVQ8PER3Q XzuJ/eJJmIobP0eeZd6CugW99NJ5/Kid5bNuFJQFKUJfAKNd7La8K9jGYhvmztGSa1INbutt9MP QQvS X-Google-Smtp-Source: AGHT+IEPAunrXxUN4hH1ZhwnPsTi7w7y1GBZcCUsdd7fJbI2+bVbifz03zct6ymu5213Zu2p5wCmXg== X-Received: by 2002:a17:902:f687:b0:215:a808:61cf with SMTP id d9443c01a7336-219da7f007bmr24064355ad.25.1734581064109; Wed, 18 Dec 2024 20:04:24 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:d586:f3b3:85da:8dc9]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc96294bsm3145675ad.36.2024.12.18.20.04.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2024 20:04:23 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87zfktphks.fsf@mail.linkov.net> Date: Wed, 18 Dec 2024 20:04:11 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> <87zfktphks.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, Mickey Petersen , monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 17, 2024, at 11:37=E2=80=AFPM, Juri Linkov = wrote: >=20 > While testing treesit-forward-sexp-list, I discovered that > thing-navigation functions are not restricted to named nodes. >=20 > I wonder if there a reason to find anonymous nodes as things? We should rather ask is there any reason to not find anonymous nodes as = things? Even ruby-ts-mode defines a bunch of anonymous nodes as sexp, = no? In any case, excluding anonymous nodes from things doesn=E2=80=99t = sound right. >=20 > The problem was found with the node "unless" in Ruby: >=20 > unless cond > a +=3D 1 > else > b -=3D 1 > end >=20 > Here the named node 'unless' has exactly the same name > as the anonymous node with the text "unless": >=20 > (unless "unless" condition: (identifier) I feel like Ruby=E2=80=99s grammar should call the named node something = else, like unless_statement. >=20 > Finding anonymous nodes breaks forward-sexp when point is on "unless": >=20 > un-!-less cond > a +=3D 1 > else > b -=3D 1 > end >=20 > because (treesit-thing-at (point) 'sexp t) finds >=20 > # >=20 > instead of >=20 > # >=20 > Also this breaks backward-sexp and backward-up-list > because treesit--thing-sibling finds > the anonymous node "unless" as a previous sibling > instead of the named node 'unless' as a parent. >=20 > Would the right solution be to check if the found thing > is a named node? With something like: >=20 > diff --git a/lisp/treesit.el b/lisp/treesit.el > index 18200acf53f..9ad879ee40c 100644 > --- a/lisp/treesit.el > +++ b/lisp/treesit.el > @@ -2711,6 +2774,7 @@ treesit--thing-sibling > (lambda (n) (>=3D (treesit-node-start n) pos)))) > (iter-pred (lambda (node) > (and (treesit-node-match-p node thing t) > + (treesit-node-check node 'named) > (funcall pos-pred node)))) > (sibling nil)) > (when cursor > @@ -2760,6 +2824,7 @@ treesit-thing-at > (let* ((cursor (treesit-node-at pos)) > (iter-pred (lambda (node) > (and (treesit-node-match-p node thing t) > + (treesit-node-check node 'named) > (if strict > (< (treesit-node-start node) pos) > (<=3D (treesit-node-start node) pos)) A better solution IMO is to add some way to distinguish between named = and anonymous nodes. I can think of two ways, either add =E2=80=9Cand=E2=80= =9D and =E2=80=9Cnamed/anonymous=E2=80=9D predicate, so (and named = =E2=80=9Cunless=E2=80=9D) only matches the named =E2=80=9Cunless=E2=80=9D = node; or we add a special syntax such that =E2=80=9C(unless)=E2=80=9D = only matches named nodes, and =E2=80=9C\=E2=80=9Dunless\=E2=80=9D=E2=80=9D= only matches anonymous nodes. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 19 02:19:54 2024 Received: (at 73404) by debbugs.gnu.org; 19 Dec 2024 07:19:54 +0000 Received: from localhost ([127.0.0.1]:37227 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOAp7-0006sd-O1 for submit@debbugs.gnu.org; Thu, 19 Dec 2024 02:19:53 -0500 Received: from relay1-d.mail.gandi.net ([217.70.183.193]:48833) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOAp6-0006s9-5O for 73404@debbugs.gnu.org; Thu, 19 Dec 2024 02:19:52 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id EEB37240005; Thu, 19 Dec 2024 07:19:42 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Wed, 18 Dec 2024 20:04:11 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <5192B278-66C0-48AE-B881-E57CCBB6B501@gmail.com> <87frmtbc9z.fsf@mail.linkov.net> <86bjxh1h86.fsf@gnu.org> <87y10l8h6k.fsf@mail.linkov.net> <87ldwl8g60.fsf@mail.linkov.net> <87wmg53rdj.fsf@mail.linkov.net> <87a5d0n651.fsf@mail.linkov.net> <87zfktphks.fsf@mail.linkov.net> Date: Thu, 19 Dec 2024 09:14:33 +0200 Message-ID: <87ikrgtbjq.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: theo@thornhill.no, Mickey Petersen , monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > A better solution IMO is to add some way to distinguish between named and > anonymous nodes. I can think of two ways, either add “and” and > “named/anonymous” predicate, so (and named “unless”) only matches the named > “unless” node; or we add a special syntax such that “(unless)” only matches > named nodes, and “\”unless\”” only matches anonymous nodes. I agree, so will create a new feature request. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 19 02:36:19 2024 Received: (at 73404) by debbugs.gnu.org; 19 Dec 2024 07:36:19 +0000 Received: from localhost ([127.0.0.1]:37260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOB50-0007kJ-QT for submit@debbugs.gnu.org; Thu, 19 Dec 2024 02:36:19 -0500 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:41779) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOB4y-0007k4-FQ for 73404@debbugs.gnu.org; Thu, 19 Dec 2024 02:36:16 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 1AE0A20003; Thu, 19 Dec 2024 07:35:47 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Tue, 10 Dec 2024 22:31:26 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> Date: Thu, 19 Dec 2024 09:34:15 +0200 Message-ID: <87wmfwqg7e.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> What should remain in 'treesit-thing-settings' are only grouping >> constructs such as "parenthesized_expression" and "statement_block". > > Ah, this matches my idea of defining sexp in other languages as “repeatable > construct/list-like construct”. We went with “every syntactic construct” at > the time, which I didn’t object to, but I’m definitely happier with the > repeatable construct approach. Including Stefan and Theo since they were > part of the original sexp navigation discussion. > > My only concern is that would the result be a bit unpredictable/confusing > when we mix the result of two logic together in such an involved way? We > can push to master and try it out for a while. I use tree-sitter sexp > navigation for work every day, albeit strictly for navigating list-like > constructs—I use forward/backward-word for smaller navigation. > >> tries to go out of the current thing to its parent, >> thus breaking the main principle that 'forward-sexp' >> should move forward across siblings only. But removing >> this line fixed the problem: > > Thanks, LGTM. Ok, so now pushed to master in such backwards-compatible way that when a ts-mode doesn't define the 'sexp-list' thing, then the existing 'sexp' is used. Also added 'sexp-list' to c-ts-mode, js-ts-mode, ruby-ts-mode and html-ts-mode. Addition of 'sexp-list' to other ts-modes is underway. From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 24 14:09:46 2024 Received: (at 73404) by debbugs.gnu.org; 24 Dec 2024 19:09:46 +0000 Received: from localhost ([127.0.0.1]:34223 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQAHq-0000yF-1A for submit@debbugs.gnu.org; Tue, 24 Dec 2024 14:09:46 -0500 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:54355) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQAHn-0000xx-TA for 73404@debbugs.gnu.org; Tue, 24 Dec 2024 14:09:44 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 1ED93E0003; Tue, 24 Dec 2024 19:09:34 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87wmfwqg7e.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 19 Dec 2024 09:34:15 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> Date: Tue, 24 Dec 2024 21:05:37 +0200 Message-ID: <87ikr8hpdu.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Ok, so now pushed to master in such backwards-compatible way > that when a ts-mode doesn't define the 'sexp-list' thing, > then the existing 'sexp' is used. Also I'm looking into allowing more list-navigation commands to be usable in ts-modes. E.g. instead of limiting list-navigation only to the current 'forward-sexp', another useful command is 'down-list'. Currently to get inside an HTML element in html-ts-mode is possible by using 'M-e' (forward-sentence) to skip HTML tag. This is not quite obvious. More natural would be to support 'C-M-d' (down-list). But this requires overriding more low-level 'scan-lists' and 'scan-sexps' in treesit, instead of overriding the current top-level 'forward-sexp-function'. Also treesit-thing-settings for 'down-list' would require pairs of node names: one node to define a list, and another node to skip a node to get inside the list. For html-ts-mode a list node is "element", and the node to skip is "tag". So for example: -!-text 'C-M-f' will use node "element" to move to the beginning of "element": text -!- then to get inside also need to skip the tag: text -!- From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 24 16:15:52 2024 Received: (at 73404) by debbugs.gnu.org; 24 Dec 2024 21:15:52 +0000 Received: from localhost ([127.0.0.1]:34849 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQCFr-0007dd-M8 for submit@debbugs.gnu.org; Tue, 24 Dec 2024 16:15:52 -0500 Received: from mail-pl1-f181.google.com ([209.85.214.181]:46162) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQCFn-0007dK-8z for 73404@debbugs.gnu.org; Tue, 24 Dec 2024 16:15:49 -0500 Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-219f8263ae0so24062345ad.0 for <73404@debbugs.gnu.org>; Tue, 24 Dec 2024 13:15:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735074881; x=1735679681; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Mk8gkCPRAKEXpcLUrTuci4jPVSOlcx59CpBFlshxcIA=; b=dt5apz5bjhL8v5ej+/AcYmOutym9AZuFUM2A05qh1LbD2sYpLlQEYzPdphYwKrXIB7 ObfrdzZCsutVYBTZ2Z0zqixbjgh3Kzb/lHyyWu3wliqXZL9Pn3iOISGwyefhXurxec5j d5GRNFu61yVuapOueL3v1Bve47iFRUna0Dl+lvzbbektav3DtgWosikwHfokt5qeOZXl KuMuDDHWohnBgw1XHFS+qRKBhIS9n5MpTBnNHEzg333grqLu3jSSpOEh65vT5p0AP7Yt apIIKSyzZSTDLlTkkz2amLFSgNNbSA+d43pkGh+I9xt9YMQXoD2NTbT9DjhCAFqaf+qT khaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735074881; x=1735679681; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mk8gkCPRAKEXpcLUrTuci4jPVSOlcx59CpBFlshxcIA=; b=iIIoJ00+sbRZnmlcpFpJI+nIV/lO5v1HwsGtdVyEcnrrp//pD+z0h4i8i7ymHi8KXE DpoaiglNedhIMT4m2wEO5bI7XLWwBpGBUI36/+Ax9JlszdFP4xPX1YivS/X3zcY9l/2s y5plilGGlPjcpcDUFfBe1ZPzQx2HFHDNspyV1HcNklb2uC4xtiLqLnZMBSl/2rvivUbS 6NL8hz8Jcz5eYeyAY9zxHqSmdwnNt0CGeGWnEbHdCwJ+OQUwAuKuIN0PVGJHA2Zcn4ZY NQVFFv+Cu5EqK+hueKlIjtVBYNd3GhXrnqH/d2mG5dCL7R1qMKBo/+wsr2AHnWeC4FSs t3CQ== X-Forwarded-Encrypted: i=1; AJvYcCWPpExv78JH2QFufPcxsCRP0zaiN1I1gGkFbwE1KNsUEwC1x/AD9Q/S11kcLUUR6iRF6jUtbQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yzl4Zv0XSVRhVDwNX9HMjgKo5np4xmO+VxTZyUaLK6HaNxDIpdQ xOQEFKCjrPZTkVYWR6Ucf8SgsMNut59cpQqVUC8dZRQmmSEY+qlf X-Gm-Gg: ASbGncuk/ca8kAIK/O/6drMrJZMzOeoN4a/yVmEadZzP50ZSQzHahiZHYa/R1CrPcqp 9zdIEwKOvHtwX/Jo54iFohT9xZP6pc+rAVpKAHb0JlkKObI8KeRIwLnJmp3N/e4jDZPEI2shb0I SyRxO/WE8EHQJ4YBb52xVWYwXFTRleY6FOASCCZrvfgRS+bu0hMpTyRdNP35000e8YW6pMWa/xr KdxXQWPjXM9MyE9KNGNE3l14NsK7z+pJvoSoMbWEMJc1SME2v59nAXO21Tivg1lopwnNu9haiIQ Fx1f X-Google-Smtp-Source: AGHT+IFYrcw0UyHAXmb9JfmQaoEM74V3YhwlrSkza8tyUgGTg74aVoTeUOOLX+fin8PsX3WZvOgsdA== X-Received: by 2002:a17:903:11c5:b0:210:fce4:11ec with SMTP id d9443c01a7336-219e6e89208mr306321635ad.1.1735074881536; Tue, 24 Dec 2024 13:14:41 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:1d98:6810:9846:b152]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc96eb40sm94222655ad.86.2024.12.24.13.14.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Dec 2024 13:14:40 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87ikr8hpdu.fsf@mail.linkov.net> Date: Tue, 24 Dec 2024 13:14:29 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 24, 2024, at 11:05=E2=80=AFAM, Juri Linkov = wrote: >=20 >> Ok, so now pushed to master in such backwards-compatible way >> that when a ts-mode doesn't define the 'sexp-list' thing, >> then the existing 'sexp' is used. >=20 > Also I'm looking into allowing more list-navigation commands > to be usable in ts-modes. E.g. instead of limiting list-navigation > only to the current 'forward-sexp', another useful command is > 'down-list'. >=20 > Currently to get inside an HTML element in html-ts-mode > is possible by using 'M-e' (forward-sentence) to skip HTML tag. > This is not quite obvious. >=20 > More natural would be to support 'C-M-d' (down-list). >=20 > But this requires overriding more low-level 'scan-lists' and > 'scan-sexps' in treesit, instead of overriding the current top-level > 'forward-sexp-function'. >=20 > Also treesit-thing-settings for 'down-list' would require > pairs of node names: one node to define a list, and another node > to skip a node to get inside the list. >=20 > For html-ts-mode a list node is "element", and the node to skip > is "tag". So for example: >=20 > -!-text >=20 > 'C-M-f' will use node "element" to move to the beginning of "element": >=20 > text -!- >=20 > then to get inside also need to skip the tag: >=20 > text -!- Right, the thing navigation only supports going up/outside, but not = going down/inside. We can add a new thing for beginning and end of = balanced pairs. Then down-list will be going from the start of a = balanced-pair-open to the end of it. Up-list will be going from the = start of a balanced-pair-close to the end of it. We might also want a way to jump from pair-open to pair-end. Going to = pair-open=E2=80=99s parent=E2=80=99s end will be almost always correct. = (Except for the grammars that do weird stuff, like tree-sitter-c=E2=80=99s= for statement, the condition is not a node by itself, but split into = opening parentheses, initializer, loop condition, increment, closing = paren, all of which direct child of the for_statement, not grouped into = a condition node.)=20 Yuan= From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 03:11:13 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 08:11:13 +0000 Received: from localhost ([127.0.0.1]:36054 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQMU5-0005rD-5d for submit@debbugs.gnu.org; Wed, 25 Dec 2024 03:11:13 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:37047) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQMU2-0005qp-VB for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 03:11:11 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 7A8B61C0002; Wed, 25 Dec 2024 08:10:41 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> (Yuan Fu's message of "Tue, 24 Dec 2024 13:14:29 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> Date: Wed, 25 Dec 2024 09:44:09 +0200 Message-ID: <871pxw430m.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> For html-ts-mode a list node is "element", and the node to skip >> is "tag". So for example: >> >> -!-text >> >> 'C-M-f' will use node "element" to move to the beginning of "element": >> >> text -!- >> >> then to get inside also need to skip the tag: >> >> text -!- > > Right, the thing navigation only supports going up/outside, but not going > down/inside. We can add a new thing for beginning and end of balanced > pairs. Then down-list will be going from the start of a balanced-pair-open > to the end of it. Up-list will be going from the start of > a balanced-pair-close to the end of it. Probably we could use just such heuristics that 'down-list' should skip the first node of the balanced pair. This should work for most ts-modes. For example, for 'jsx_element' the first child to skip is 'jsx_opening_element'. For 'argument_list' the first child to skip is an anonymous node "(". For 'statement_block' the first child to skip is an anonymous node "{". From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 03:36:07 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 08:36:07 +0000 Received: from localhost ([127.0.0.1]:36130 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQMsB-0007Da-DF for submit@debbugs.gnu.org; Wed, 25 Dec 2024 03:36:07 -0500 Received: from mail-pl1-f178.google.com ([209.85.214.178]:60870) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQMs9-0007D2-BU for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 03:36:06 -0500 Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-21654fdd5daso59770455ad.1 for <73404@debbugs.gnu.org>; Wed, 25 Dec 2024 00:36:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735115700; x=1735720500; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Td3TEQ/VcOwz+4fSDL5XVfh8qKI2DJXlhIvk5pWYgVE=; b=OndX+2JYkZTfBkpEd6Uq3zedfcJrMzaeDsAr3c+aQL7So2W/5S04cm2QU0RIBvLpTc 2s12afxMEYLGFsctiPMG3sMR4cicGi+DPPcCT1Aq0bhZHni2Zqp5aQLR9mmRx5yYicEi ttUMi8ALTP4nEWVEwvKoYMav5Iw8iR5DmsMaW7wuZb+1Y1M2YFZ0skgzktEtIk8JYi2o UDN6yy05tUMfnWKUNe/cn2C5tNAUvAWNscSyRYAyaGU1CU1gg0vgsIIMojonf9ScceO7 vTyLPYbFHcYcVpbuKZltAcsHiPAKx9TWatKO+N/iPDfa/+HbeeMuNw6NLnLEYJxgg8G0 j6mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735115700; x=1735720500; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Td3TEQ/VcOwz+4fSDL5XVfh8qKI2DJXlhIvk5pWYgVE=; b=ipAuzgLhstOoMB50GSn8pywr/iSwlM9rRQiUhWE/aozhbGeVa1f72ZAplber6ZIycm pDWL25MqtGv9LEG6ihAMQL3t5aYj7SH6B2/cN+c1VJuSK5jtSPSyq5bu7VSLLtkR3Ey3 c8a0H9MMAsaPLr+VVBQracDLtADJiz6IO0EXuqTjBfG3w7Yn0gS5WL0xNUS8Uu7/Gg8k sKxveR2ZMgvPuWQ4I+KlhcV3oFT3UgOApoPSvxsZm9gC6EfxbDkhtXoXP3rXmI5qnIQG rY4s8GkVula907ADtNo6vEJ3veD62si3TCBC5MAjCK8eN5gNQSvbBKlDdPTa0qrTrdz1 mWxQ== X-Forwarded-Encrypted: i=1; AJvYcCUeLJjO+svRAuQoTcCkRItXB9WjQrYS7Dh5+TRSADoflI4EweqcUMT9HDfHoGtKEkT+DTq2Rg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyhJsAHh4bQKxg3CGQzJ2ubSgCrZE1GZhxFKB+EkBtYfrAMHGT6 rzH031Evlqo2Kw8gDfnG/BUD7Rs5NEl/QefD95kKzOGYOMCt5ruz X-Gm-Gg: ASbGncv0QzB9QxHAXSmiWYbPwc3etycnYKA5lcSOSicjBAalIJrpaTlP6JpX3cmx7Wa bPi9zudYVvHb91/u5lJ5/CTRYSfcmLrbYKQPTl0uVe4asWjZ0wwS/vJstCV3m4YfNDeJKmSoX6F b30cifn5JYRtN/+eue0Fz4IluEBSXFnZJR+fd5PICZS5g0cEZVATnhG/YZRDreFVQVnCkoVcJl1 xV5S7Hb+a6VrMdwd57g9/Lr/nIqQkdYLXW7990JZCqnz+Yb3imXYU4GmWbgfBIw7OksF7eJHYNR lVA= X-Google-Smtp-Source: AGHT+IEQxDrqTmAVO2fyucC6K1xKoyEDcDFlrgPRKzkD4CPzaoZWBwgHKqhOyudrTq5lfopPOI6bwA== X-Received: by 2002:a05:6300:6681:b0:1e0:c6c0:1e1f with SMTP id adf61e73a8af0-1e5e07f8a98mr27768655637.36.1735115699839; Wed, 25 Dec 2024 00:34:59 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:e179:285e:283:33ae]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad81641esm10809401b3a.3.2024.12.25.00.34.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Dec 2024 00:34:59 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <871pxw430m.fsf@mail.linkov.net> Date: Wed, 25 Dec 2024 00:34:45 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8D171A7A-D565-4BEF-AE56-1308C69FE5AC@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> <871pxw430m.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 24, 2024, at 11:44=E2=80=AFPM, Juri Linkov = wrote: >=20 >>> For html-ts-mode a list node is "element", and the node to skip >>> is "tag". So for example: >>>=20 >>> -!-text >>>=20 >>> 'C-M-f' will use node "element" to move to the beginning of = "element": >>>=20 >>> text -!- >>>=20 >>> then to get inside also need to skip the tag: >>>=20 >>> text -!- >>=20 >> Right, the thing navigation only supports going up/outside, but not = going >> down/inside. We can add a new thing for beginning and end of balanced >> pairs. Then down-list will be going from the start of a = balanced-pair-open >> to the end of it. Up-list will be going from the start of >> a balanced-pair-close to the end of it. >=20 > Probably we could use just such heuristics that 'down-list' should = skip > the first node of the balanced pair. This should work for most = ts-modes. >=20 > For example, for 'jsx_element' the first child to skip is = 'jsx_opening_element'. > For 'argument_list' the first child to skip is an anonymous node "(". > For 'statement_block' the first child to skip is an anonymous node = "{=E2=80=9C. Yes, that should work for the vast majority of grammars. I can=E2=80=99t = think of an counter example other than for_statment in tree-sitter-c. Yuan From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 12:19:34 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 17:19:34 +0000 Received: from localhost ([127.0.0.1]:38691 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQV2j-0006uN-Ot for submit@debbugs.gnu.org; Wed, 25 Dec 2024 12:19:34 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61681) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQV2i-0006u7-OE for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 12:19:33 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id D990B100042; Wed, 25 Dec 2024 12:19:26 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1735147166; bh=3SacxGVIx6dZuXx4r3cWA9z6lfsXLbGNtJ5EwgjLJ2w=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=gAPiYE24JZOpUZUYrSPaTlaePwF6wAIoeSRBaer7iDr80wrFwN4sARwzY50Wow/vs bXv66tgbRWINH1xkSIiNe9wGlZGxEsf2dZIm9Z0hJmkhJ6RqmNsD1Ofyb/NmlC9TUH VYsb7D6yz8S/XFBdG/59UZw7ZxhC7OiOEHwdF/uC9V8iukvNsKseKcDkp44D9+xmGw XMUPX6X8b5GaTj9lC9XCbGd5xnfpDInqlxAU2Jq+sVOG+pGaXXyjkWDHQG87zn7nqp kNCKhyBoP55ZGrVrwKHYyNy+5Bhwi88t4CUOLtRSMd5MQNazPgRw4PU5wHTqCcXaEp 7o9Wfr7mDHUmg== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id F27E510002E; Wed, 25 Dec 2024 12:19:25 -0500 (EST) Received: from pastel (104-195-225-43.cpe.teksavvy.com [104.195.225.43]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id AEDD412064B; Wed, 25 Dec 2024 12:19:25 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87ikr8hpdu.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 24 Dec 2024 21:05:37 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> Date: Wed, 25 Dec 2024 12:19:25 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.753 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Also I'm looking into allowing more list-navigation commands > to be usable in ts-modes. E.g. instead of limiting list-navigation > only to the current 'forward-sexp', another useful command is > 'down-list'. Gentle reminder that `forward-sexp` is not a "list-navigation" function. That would be `forward-list`. We very often use sexp commands and functions to manipulate non-lists such as identifiers. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 12:50:42 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 17:50:42 +0000 Received: from localhost ([127.0.0.1]:38725 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQVWr-0008LI-N0 for submit@debbugs.gnu.org; Wed, 25 Dec 2024 12:50:42 -0500 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:36719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQVWq-0008L5-8u for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 12:50:40 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 0C26DC0002; Wed, 25 Dec 2024 17:50:10 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <8D171A7A-D565-4BEF-AE56-1308C69FE5AC@gmail.com> (Yuan Fu's message of "Wed, 25 Dec 2024 00:34:45 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> <871pxw430m.fsf@mail.linkov.net> <8D171A7A-D565-4BEF-AE56-1308C69FE5AC@gmail.com> Date: Wed, 25 Dec 2024 19:36:48 +0200 Message-ID: <87r05vznir.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> Probably we could use just such heuristics that 'down-list' should skip >> the first node of the balanced pair. This should work for most ts-modes. >> >> For example, for 'jsx_element' the first child to skip is 'jsx_opening_element'. >> For 'argument_list' the first child to skip is an anonymous node "(". >> For 'statement_block' the first child to skip is an anonymous node "{“. > > Yes, that should work for the vast majority of grammars. I can’t think > of an counter example other than for_statment in tree-sitter-c. Indeed, the 'for_statement' is a hard problem, and I see no good solution. I already encountered in different languages the same problem that you described: > We might also want a way to jump from pair-open to pair-end. Going to > pair-open’s parent’s end will be almost always correct. (Except for the > grammars that do weird stuff, like tree-sitter-c’s for statement, the > condition is not a node by itself, but split into opening parentheses, > initializer, loop condition, increment, closing paren, all of which direct > child of the for_statement, not grouped into a condition node.) Maybe indeed worth to try defining pair-open and pair-end as some children nodes of the 'for_statement', i.e. only the opening paren and the closing paren. Something like (completely untested): (and (member (treesit-node-text node) '("(" ")")) (equal (treesit-node-type (treesit-node-parent node)) "for_statement")) From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 13:05:26 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 18:05:26 +0000 Received: from localhost ([127.0.0.1]:38756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQVl8-0000aE-IQ for submit@debbugs.gnu.org; Wed, 25 Dec 2024 13:05:26 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:42067) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQVl6-0000Zy-BK for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 13:05:24 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 84DCA60002; Wed, 25 Dec 2024 18:04:54 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Wed, 25 Dec 2024 12:19:25 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> Date: Wed, 25 Dec 2024 20:01:37 +0200 Message-ID: <87y103wsji.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> Also I'm looking into allowing more list-navigation commands >> to be usable in ts-modes. E.g. instead of limiting list-navigation >> only to the current 'forward-sexp', another useful command is >> 'down-list'. > > Gentle reminder that `forward-sexp` is not a "list-navigation" function. > That would be `forward-list`. We very often use sexp commands and > functions to manipulate non-lists such as identifiers. Do you think it would be better to override low-level functions 'scan-lists' and 'scan-sexps' with new variables like 'scan-lists-function' and 'scan-sexps-function', instead of adding more variables for overriding top-level commands such as a new variable 'forward-list-function' and 'down-list-function', like the existing 'forward-sexp-function'? From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 25 14:29:31 2024 Received: (at 73404) by debbugs.gnu.org; 25 Dec 2024 19:29:31 +0000 Received: from localhost ([127.0.0.1]:38844 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQX4V-0004Mb-Am for submit@debbugs.gnu.org; Wed, 25 Dec 2024 14:29:31 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:16134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQX4S-0004MJ-QT for 73404@debbugs.gnu.org; Wed, 25 Dec 2024 14:29:29 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C3FA34408FA; Wed, 25 Dec 2024 14:29:21 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1735154960; bh=qVTLt6f7V97XUnnEdIXPtH3CvYYP2fUYKQX2ByX6nBA=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=iCUVJF1TXrQcQm6FBBajGQ7JqcR6yina10/XoWbCcUiov8CRFggeeqK/xxRcpqeqi w+ks3f6NhiJ9bmsCXMYWe6dNBJqnG/lHNT1PWJFC5Tb8Cf7sKeW7Bi4wpEBDrEMOix 7hD4ZPRpa/rhgqO8wj5No0+PPYYj+Q5i3Em1JnbDNXGRVrIQRgnZ/W/D6VHwyt7ZRk qgv1AVq8evMPPv0eCPTNSo9gggnEy7aH5oKUQ3SaEs6vQuZHj/RwJ0Mm79a105cKbX zWR/F5aJ1sYcZD+dCj5HbB8BahcxN8ehNOJd0FUc9+IG6FrWOKJV6eB6Os+vXo7qRP +2+qcw5DRWeZA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C3C01440096; Wed, 25 Dec 2024 14:29:20 -0500 (EST) Received: from pastel (104-195-225-43.cpe.teksavvy.com [104.195.225.43]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 4D28C120422; Wed, 25 Dec 2024 14:29:20 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87y103wsji.fsf@mail.linkov.net> (Juri Linkov's message of "Wed, 25 Dec 2024 20:01:37 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <87y103wsji.fsf@mail.linkov.net> Date: Wed, 25 Dec 2024 14:29:19 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Gentle reminder that `forward-sexp` is not a "list-navigation" function. >> That would be `forward-list`. We very often use sexp commands and >> functions to manipulate non-lists such as identifiers. > Do you think it would be better to override low-level functions 'scan-lists' > and 'scan-sexps' with new variables like 'scan-lists-function' > and 'scan-sexps-function', instead of adding more variables for > overriding top-level commands such as a new variable 'forward-list-function' > and 'down-list-function', like the existing 'forward-sexp-function'? Don't know. What I do know is that in general we'd also want an `up-sexp` operation. Currently we have an ugly kludge in `up-list` to try and use `forward-sexp-function` (which is ugly both because `forward-sexp-function` doesn't really provide the functionality we need, and because it mixes up sexp and list navigation), and it would be good to clean it up. AFAIK `up-list` is the only place where I've needed sexp-based navigation and where `forward-sexp-function` didn't do the job. In theory I guess `down-list` is another, but I've never found a use for it. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 27 03:27:45 2024 Received: (at 73404) by debbugs.gnu.org; 27 Dec 2024 08:27:45 +0000 Received: from localhost ([127.0.0.1]:44604 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tR5hA-00035O-Aa for submit@debbugs.gnu.org; Fri, 27 Dec 2024 03:27:44 -0500 Received: from relay1-d.mail.gandi.net ([217.70.183.193]:47813) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tR5h6-000356-RI for 73404@debbugs.gnu.org; Fri, 27 Dec 2024 03:27:42 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id C992A240002; Fri, 27 Dec 2024 08:27:32 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Wed, 25 Dec 2024 14:29:19 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <87y103wsji.fsf@mail.linkov.net> Date: Fri, 27 Dec 2024 09:54:11 +0200 Message-ID: <87h66pr1qc.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain >>> Gentle reminder that `forward-sexp` is not a "list-navigation" function. >>> That would be `forward-list`. We very often use sexp commands and >>> functions to manipulate non-lists such as identifiers. >> Do you think it would be better to override low-level functions 'scan-lists' >> and 'scan-sexps' with new variables like 'scan-lists-function' >> and 'scan-sexps-function', instead of adding more variables for >> overriding top-level commands such as a new variable 'forward-list-function' >> and 'down-list-function', like the existing 'forward-sexp-function'? > > Don't know. I tried to override `scan-lists` and `scan-sexps` with advices (a proof of concept attached below), and it works nicely. For example, `C-M-d` moves down to the HTML element in html-ts-mode. But then I realized there are not too many places where such overriding might be useful. In fact, there is only 1 place in `show-paren--default` that uses `scan-sexps`. But even this occurrence can't be used, so I created `treesit-show-paren-data` for `show-paren-data-function` in bug#75122. And overriding `scan-lists` is useful only in `forward-list`, `down-list` and `up-list`. That's all. So clearly instead of overriding `scan-lists` and `scan-sexps`, better would be to add 3 new variables: `forward-list-function`, `down-list-function` and `up-list-function`. This gives the users more flexibility to choose for example navigation for C-M-f with a limited number of treesit lists + syntax symbols, and for C-M-n everything that is a list in treesit. This means that with start_atimer (-!-enum atimer_type type, struct timespec timestamp) C-M-f could skip only the next symbol and move to start_atimer (enum-!- atimer_type type, struct timespec timestamp) while C-M-n could skip the whole next `parameter_declaration` start_atimer (enum atimer_type type-!-, struct timespec timestamp) or vice versa depending on the values of all new options `...-function` in ts-modes. > What I do know is that in general we'd also want an `up-sexp` operation. > Currently we have an ugly kludge in `up-list` to try and use > `forward-sexp-function` (which is ugly both because > `forward-sexp-function` doesn't really provide the functionality we > need, and because it mixes up sexp and list navigation), and it would be > good to clean it up. Agreed, this distinction is required for treesit. Hopefully, this can be achieved by separating `forward-sexp-function` and `up-list-function` that in ts-modes could be set to new functions either `treesit-up-list` or `treesit-up-sexp`. PS: will try to refactor everything in this attachment to new `forward-list-function`, `down-list-function` and `up-list-function`: --=-=-= Content-Type: application/emacs-lisp Content-Disposition: inline; filename=scan-lists-advice.el Content-Transfer-Encoding: quoted-printable (defvar treesit--recursive-scan-call nil) (define-advice scan-lists (:around (ofun from count depth) treesit) (if (or treesit--recursive-scan-call (not treesit-thing-settings)) (funcall ofun from count depth) (save-excursion (goto-char from) (let ((treesit--recursive-scan-call t)) (cond ((=3D depth 0) (treesit-forward-list count) (point)) ((< depth 0) (when (treesit-forward-list count) (let* ((node (if (> count 0) (treesit-thing-prev (point) 'sexp-list) (treesit-thing-next (point) 'sexp-list))) (beg-node (when node (treesit-node-child node 0)))) (when beg-node (treesit-node-end beg-node))))) ((> depth 0) (let ((node (treesit-thing-at (point) 'sexp-list))) (while (and node (eq (point) (treesit-node-start node))) (setq node (treesit-parent-until node 'sexp-list))) (when node (treesit-node-start node))))))))) (define-advice scan-sexps (:around (ofun from count) treesit) (if (or treesit--recursive-scan-call (not treesit-thing-settings)) (funcall ofun from count) (save-excursion (goto-char from) (let ((treesit--recursive-scan-call t)) (treesit-forward-sexp-list count)) (point)))) (define-advice treesit-major-mode-setup (:after (&rest _) reset) (setq-local forward-sexp-function nil)) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 27 03:28:07 2024 Received: (at 73404) by debbugs.gnu.org; 27 Dec 2024 08:28:07 +0000 Received: from localhost ([127.0.0.1]:44608 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tR5hW-00036V-Ko for submit@debbugs.gnu.org; Fri, 27 Dec 2024 03:28:06 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:44729) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tR5hU-00035t-JP for 73404@debbugs.gnu.org; Fri, 27 Dec 2024 03:28:05 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 76B091C0003; Fri, 27 Dec 2024 08:27:36 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87r05vznir.fsf@mail.linkov.net> (Juri Linkov's message of "Wed, 25 Dec 2024 19:36:48 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <856C1B51-D9BC-4794-95CF-28E9B9B59325@gmail.com> <871pxw430m.fsf@mail.linkov.net> <8D171A7A-D565-4BEF-AE56-1308C69FE5AC@gmail.com> <87r05vznir.fsf@mail.linkov.net> Date: Fri, 27 Dec 2024 09:59:52 +0200 Message-ID: <87r05to7vb.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> I can’t think of an counter example other than for_statment in >> tree-sitter-c. > > Indeed, the 'for_statement' is a hard problem, and I see > no good solution. A possible solution would be to find a way to create "virtual" nodes. This means transforming the existing syntax tree by inserting addition nodes to it. For example, transforming (for_statement "for" "(" condition: (assignment_expression left: (identifier) operator: "=" right: (number_literal)) ; body: (binary_expression left: (identifier) operator: "<" right: (number_literal)) ; (update_expression operator: "++" argument: (identifier)) ")" into (for_statement "for" (for_parameters "(" condition: (assignment_expression left: (identifier) operator: "=" right: (number_literal)) ; body: (binary_expression left: (identifier) operator: "<" right: (number_literal)) ; (update_expression operator: "++" argument: (identifier)) ")" by inserting the virtual node "for_parameters". Not sure if such transformation is supported by tree-sitter. Generally, we have two problems with syntax trees created by the authors of tree-sitter grammars: 1. Insufficient structure 2. Excessive structure For insufficient structure a possible solution would be to insert virtual nodes like above. And for excessive structure we need to flatten some nodes. For example, in c-ts-mode: static -!-struct atimer *free_atimers; 'C-M-f' unexpectedly jumped not to the next symbol, but to static struct atimer-!- *free_atimers; because of too much structure built in the syntax tree: (declaration type: (storage_class_specifier "static") declarator: (struct_specifier "struct" name: (type_identifier)) (pointer_declarator "*" declarator: (identifier)) ";") So the solution could be to flatten 'struct_specifier' to move 'type_identifier' to be a sibling in the same list inside 'declaration': (declaration type: (storage_class_specifier "static") declarator: (struct_specifier "struct") (type_identifier) (pointer_declarator "*" declarator: (identifier)) ";") Also not sure if such thing is possible in tree-sitter. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 29 12:59:50 2024 Received: (at 73404) by debbugs.gnu.org; 29 Dec 2024 17:59:50 +0000 Received: from localhost ([127.0.0.1]:56071 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tRxZu-0004gr-4c for submit@debbugs.gnu.org; Sun, 29 Dec 2024 12:59:50 -0500 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:46681) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tRxZs-0004gd-4u for 73404@debbugs.gnu.org; Sun, 29 Dec 2024 12:59:48 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id C36ECE0003; Sun, 29 Dec 2024 17:59:17 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87h66pr1qc.fsf@mail.linkov.net> (Juri Linkov's message of "Fri, 27 Dec 2024 09:54:11 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <87ikr8hpdu.fsf@mail.linkov.net> <87y103wsji.fsf@mail.linkov.net> <87h66pr1qc.fsf@mail.linkov.net> Date: Sun, 29 Dec 2024 19:58:01 +0200 Message-ID: <87ikr21idy.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Yuan Fu , Mickey Petersen , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > So clearly instead of overriding `scan-lists` and `scan-sexps`, > better would be to add 3 new variables: `forward-list-function`, > `down-list-function` and `up-list-function`. Ok, now these 3 functions are added in lisp.el and overridden in treesit.el. >> What I do know is that in general we'd also want an `up-sexp` operation. >> Currently we have an ugly kludge in `up-list` to try and use >> `forward-sexp-function` (which is ugly both because >> `forward-sexp-function` doesn't really provide the functionality we >> need, and because it mixes up sexp and list navigation), and it would be >> good to clean it up. > > Agreed, this distinction is required for treesit. Hopefully, > this can be achieved by separating `forward-sexp-function` > and `up-list-function` that in ts-modes could be set to new > functions either `treesit-up-list` or `treesit-up-sexp`. Like `treesit-forward-sexp-list` uses `forward-sexp-default-function` to move between symbols inside lists, I'm going to add syntax-based fallback in `treesit-down-list` and `treesit-up-list` as well. This is useful in strings and comments. This is what `up-list` already does. Unfortunately, currently there is such limitation in `down-list`: (when (ppss-comment-or-string-start (syntax-ppss)) (user-error "This command doesn't work in strings or comments")) But this is not a problem for `treesit-down-list`. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 02:17:28 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 07:17:29 +0000 Received: from localhost ([127.0.0.1]:57157 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSA1o-000881-K6 for submit@debbugs.gnu.org; Mon, 30 Dec 2024 02:17:28 -0500 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:33047) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSA1m-00087h-Ua for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 02:17:27 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6386D20002; Mon, 30 Dec 2024 07:17:16 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87wmfwqg7e.fsf@mail.linkov.net> (Juri Linkov's message of "Thu, 19 Dec 2024 09:34:15 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> Date: Mon, 30 Dec 2024 09:15:42 +0200 Message-ID: <8734i5fyv1.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Also added 'sexp-list' to c-ts-mode, js-ts-mode, ruby-ts-mode and > html-ts-mode. Addition of 'sexp-list' to other ts-modes is underway. BTW, my initial intention was to add the thing 'list'. But then I discovered that (treesit-thing-next (point) 'list) uses the function 'list' instead of the thing 'list'. However, the current 'sexp-list' as a two-word composite is too ugly. Now I found a better replacement: 'group'. This word is already used in lisp.el such as in the docstring of 'forward-list': "Move forward across one balanced group of parentheses." And the error messages: "No next group". So exactly the same message will be used by (format-message "No next %S" pred) where 'pred' will be 'group'. IMHO, this looks better: (setq-local treesit-thing-settings `((html (sexp ,(regexp-opt '("element" "text" "attribute" "value"))) (group ,(regexp-opt '("element""))) (sentence "tag") (text ,(regexp-opt '("comment" "text")))))) From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 03:02:14 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 08:02:14 +0000 Received: from localhost ([127.0.0.1]:57213 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSAj7-0001wq-Pp for submit@debbugs.gnu.org; Mon, 30 Dec 2024 03:02:14 -0500 Received: from mail-pl1-f178.google.com ([209.85.214.178]:44217) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSAj3-0001wV-Sw for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 03:02:12 -0500 Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2163dc5155fso112342685ad.0 for <73404@debbugs.gnu.org>; Mon, 30 Dec 2024 00:02:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735545664; x=1736150464; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=deZrfdjuG/JSq679l39a0tAcRsZnxiTHz05Pjzu9Xtw=; b=ifNMIwlOtB7hA8NnLCjbOe9G5vDuQsxpYDrkLm1FZ/igIOfQ1SkmGB7XDd/1X733ei 43xO4jvzZcVKTalL+3csWaCUobLyTlfy2aVPKv975e2pVrPTRDVebD1/wUF0sWO2AavW qqon18NiSW9kZS7N3pjsA3QuLmYIFmM+0fTwiHD6cPs2tSG8MbUF/aP26m8ek5z73euA 2Ho03CL1AZEr7/hJhD6Q53fRbOCJh6w2P6B8xyYNy/sEDynR4DvaQUJImXCalJFhgnom 6iLoxW+OlJSw5GoQFWWE+quCHSxzrnKFagztY9EtrM1BR9KTRfjMJUYw9frYZaGuHbZy qz1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735545664; x=1736150464; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=deZrfdjuG/JSq679l39a0tAcRsZnxiTHz05Pjzu9Xtw=; b=pod/EHPYihWZScITiqFEoJrtJXsa5bBGLijg/RKsjDwt36JbjMI2JujOVUux4FBHC6 o20QLVDPui50eTRUnMTKMBUqiUllvde3duB6rSsY+8i4gQJAAcJybZRpf+xTWzl+UzSc LTYit8tZk7bZxMVIJq8LmlDamSx4Mhnyf6jj92N9rXoWnRR2W58ufBHGKio9sdZkCDU8 WK0xPwe/xxWg8eyBDiLEZEs/i28Jo+qnlIA2JYhrnnMMzF8bviBB3bbaCnyjPWk12QsZ xlCKuYcca6Cq7/G2/d02dhSFjt1T9FKVWN1EJ0vDjknjv121KxVgDHyZdki8XMtXELmB rhPg== X-Forwarded-Encrypted: i=1; AJvYcCWblv4Mh9+GuJNrBIaI0T22puhaYUbxl6zrwt/NTVh5Es97m5FZMXIZzu264f+jksCtc+9Uiw==@debbugs.gnu.org X-Gm-Message-State: AOJu0YxX35lwINn+uwG4t49hrO2OgC+VrpoC9gLc176A2LPp29XnhFx7 lc4ydRLrUNhTNIIUTeCAvFoNYqe5DkMg/2UgJUq/CMbQ4KF9r5CN X-Gm-Gg: ASbGncsaoKFxNUb9k88MhMC1pTD6IMVT/KtVMNtvhHjb3+NG5zC4A40pE+jxbfw24Ey yJ2KsZROow6MW8pXtbUxg6z9ECamVQik3ZJQEjdepHjx9OKnC+bCibjjDXc7jHjp+S7J36T26wh mtMkutaOs6Qj0KtmhOng1HMUrEWbR6crBKeoJBz1tNUzUohdohPYWjzbwqhtfkR1/a6w/t/OEkZ Sizie74B5tDG0f8PVKiigGn044/holEhneA14s1G1O1cVSRCfMhJ6GjzUYankHRXmK67EYfRxNH yLI= X-Google-Smtp-Source: AGHT+IH1I/ub7m9CwUFSHzcV2oWc1ei6i7JA5GuT6VAaoKyRuvwWP1OWIyi2+JWo/Hx8lozSKRknRg== X-Received: by 2002:a05:6a00:35c3:b0:726:64a5:5f67 with SMTP id d2e1a72fcca58-72abde09893mr50185210b3a.12.1735545664009; Mon, 30 Dec 2024 00:01:04 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:57e:6e82:c5e4:f5c6]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad8163fdsm18651115b3a.6.2024.12.30.00.01.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Dec 2024 00:01:03 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <8734i5fyv1.fsf@mail.linkov.net> Date: Mon, 30 Dec 2024 00:00:51 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Dec 29, 2024, at 11:15=E2=80=AFPM, Juri Linkov = wrote: >=20 >> Also added 'sexp-list' to c-ts-mode, js-ts-mode, ruby-ts-mode and >> html-ts-mode. Addition of 'sexp-list' to other ts-modes is underway. >=20 > BTW, my initial intention was to add the thing 'list'. > But then I discovered that (treesit-thing-next (point) 'list) > uses the function 'list' instead of the thing 'list'. >=20 > However, the current 'sexp-list' as a two-word composite is too ugly. > Now I found a better replacement: 'group'. This word is already used > in lisp.el such as in the docstring of 'forward-list': >=20 > "Move forward across one balanced group of parentheses." >=20 > And the error messages: "No next group". >=20 > So exactly the same message will be used by >=20 > (format-message "No next %S" pred) >=20 > where 'pred' will be 'group'. IMHO, this looks better: >=20 > (setq-local treesit-thing-settings > `((html > (sexp ,(regexp-opt '("element" "text" "attribute" = "value"))) > (group ,(regexp-opt '("element""))) > (sentence "tag") > (text ,(regexp-opt '("comment" "text")))))) I don=E2=80=99t have an opinion on this. Groups sounds a bit abstract, = but so does sexp-list. So as long as we do a good job explaining the = concept, it should be fine. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 08:40:38 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 13:40:38 +0000 Received: from localhost ([127.0.0.1]:57823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSG0b-000254-Ol for submit@debbugs.gnu.org; Mon, 30 Dec 2024 08:40:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46614) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSG0Y-00024m-BY for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 08:40:35 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSG0P-0005GV-Te; Mon, 30 Dec 2024 08:40:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=5OgL+0EFlT//zNyDd2NHz9xBVgvUpC4NUo7sap4J0JY=; b=IabY+M0iTu4T Rz8KzqNRtXMvpzYnBecdNtc12j4VrBu4yoctyN1L01A+3Eak+aprxkXcntyYaaOicIb+92coZ8Z/i EOs1NrjfDZ9VnzwVSszSeEz1CGO2YOQ5qDCi72fxgb7JkkqyyOQwHuVoLNzrx80p3uTwNeC4t6Mzl aG4DFOXxQs3DllYe1t2TU5UmrUl3kmzfwi7zuM+f8IVXnUxUbE/zrVzQwzN5v/O+EjmSPnunli/py k1krKFF8ogwDSpUYknctv2aI5+AFPjTsu3NSNeaE7xAAdFl8Bh9XWsWTDZYFyE1oSv0ESsOI1Y96F GvsURED9c7ySFxKh5aM6hQ==; Date: Mon, 30 Dec 2024 15:40:22 +0200 Message-Id: <864j2lp9vd.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <8734i5fyv1.fsf@mail.linkov.net> (message from Juri Linkov on Mon, 30 Dec 2024 09:15:42 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: Theodor Thornhill , Eli Zaretskii , > Mickey Petersen , 73404@debbugs.gnu.org, > Stefan Monnier > Date: Mon, 30 Dec 2024 09:15:42 +0200 > > BTW, my initial intention was to add the thing 'list'. > But then I discovered that (treesit-thing-next (point) 'list) > uses the function 'list' instead of the thing 'list'. > > However, the current 'sexp-list' as a two-word composite is too ugly. > Now I found a better replacement: 'group'. This word is already used > in lisp.el such as in the docstring of 'forward-list': > > "Move forward across one balanced group of parentheses." > > And the error messages: "No next group". "Group" is too abstract and too removed from the actual thing. I'm quite sure we can do better. What are the possible kinds of "groups", which are not balanced parenthetical expressions? Can you show a more-or-less exhaustive list of them? With that, we could try looking for a proper terminology. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 11:31:13 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 16:31:13 +0000 Received: from localhost ([127.0.0.1]:59622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSIfc-0002k5-Uz for submit@debbugs.gnu.org; Mon, 30 Dec 2024 11:31:12 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:53958) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSIfW-0002jU-Km for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 11:31:06 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id B525010004C; Mon, 30 Dec 2024 11:30:54 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1735576253; bh=FMUTCq0ViqlC9n0hUL5nRvexo8TQtMXIK4kBEMZXkOc=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=LgbkfkTNfL5UIKaBmk2uY2ri6ebNP19FYQqoHzT0GfNqIA2F4Qj4e3PzOFdOeuA+v BOd+vqEDyJgt75oSNZFD2AwldNmubPBkgCXk0RIP+b1OZRlyUXz6sXEHuardHpVCBl YsSqFQayewrVd192wbbcLpH9Lf1eUbzNxc2CHk2WcGtfmVxaqYTbfXsKF4qkj1bKhC aU3RwHnhmtVqSXaO7rjmPFGzA0pY4DIO0Zge/Fpi/7kAQBQmzD1bjEkpAj3kXGkwRL YzODW+TA9TlrmnIvM1AGIV57HsjZyuJr+Z7MVzZ4v6tM1xWAz5qcSAITPpoiE8MuFH YBRucRtqZM4Ow== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 5E8A3100035; Mon, 30 Dec 2024 11:30:53 -0500 (EST) Received: from pastel (104-195-225-43.cpe.teksavvy.com [104.195.225.43]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 1D45E120484; Mon, 30 Dec 2024 11:30:53 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <8734i5fyv1.fsf@mail.linkov.net> (Juri Linkov's message of "Mon, 30 Dec 2024 09:15:42 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> Date: Mon, 30 Dec 2024 11:30:52 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.482 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > BTW, my initial intention was to add the thing 'list'. > But then I discovered that (treesit-thing-next (point) 'list) > uses the function 'list' instead of the thing 'list'. I don't have a good suggestion for the actual naming. Regarding the fact that this arg can take a either symbol or a function (which suffers from a risk of ambiguity, like you discovered), I think it's very important to try and avoid the intersection of the two, and a "standard" way to do that is to use keywords, like `:list`. Stefan PS: Tho, strictly speaking you can `(defun :list ...)`. I just hope noone ever does that (although I plead guilty to getting dangerously close to it when I suggested this very idea to Philip for `setup.el`). From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 14:04:34 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 19:04:35 +0000 Received: from localhost ([127.0.0.1]:59988 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSL46-0001go-Iz for submit@debbugs.gnu.org; Mon, 30 Dec 2024 14:04:34 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:57983) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSL40-0001gP-BU for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 14:04:33 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id A17E5FF802; Mon, 30 Dec 2024 19:04:17 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <864j2lp9vd.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 30 Dec 2024 15:40:22 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <864j2lp9vd.fsf@gnu.org> Date: Mon, 30 Dec 2024 20:54:39 +0200 Message-ID: <87seq52e8g.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> BTW, my initial intention was to add the thing 'list'. >> But then I discovered that (treesit-thing-next (point) 'list) >> uses the function 'list' instead of the thing 'list'. >> >> However, the current 'sexp-list' as a two-word composite is too ugly. >> Now I found a better replacement: 'group'. This word is already used >> in lisp.el such as in the docstring of 'forward-list': >> >> "Move forward across one balanced group of parentheses." >> >> And the error messages: "No next group". > > "Group" is too abstract and too removed from the actual thing. I'm > quite sure we can do better. The proper name is "list". Since it can't be used directly, the second best variant is "group". > What are the possible kinds of "groups", which are not balanced > parenthetical expressions? All groups should be balanced. Even in languages that don't use parens and brackets, e.g. "def" and "end" are balanced while these are not parenthetical expressions. > Can you show a more-or-less exhaustive list of them? A exhaustive list is not yet finished. But basically most groups are parenthetical expressions while some don't use parens. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 14:04:54 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 19:04:54 +0000 Received: from localhost ([127.0.0.1]:59991 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSL4P-0001ha-Vj for submit@debbugs.gnu.org; Mon, 30 Dec 2024 14:04:54 -0500 Received: from relay2-d.mail.gandi.net ([217.70.183.194]:55527) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSL4O-0001hM-JW for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 14:04:53 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 2D65540005; Mon, 30 Dec 2024 19:04:23 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Mon, 30 Dec 2024 11:30:52 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> Date: Mon, 30 Dec 2024 20:59:23 +0200 Message-ID: <87seq50zcs.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Yuan Fu , Theodor Thornhill , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> BTW, my initial intention was to add the thing 'list'. >> But then I discovered that (treesit-thing-next (point) 'list) >> uses the function 'list' instead of the thing 'list'. > > I don't have a good suggestion for the actual naming. > > Regarding the fact that this arg can take a either symbol or a function > (which suffers from a risk of ambiguity, like you discovered), I think > it's very important to try and avoid the intersection of the two, and > a "standard" way to do that is to use keywords, like `:list`. Then for backward-compatibility for existing things could support both variants, e.g. 'sentence' and ':sentence', 'text' and ':text' (luckily no one yet defined such functions). And for the new thing to use ':list', e.g. (setq-local treesit-thing-settings `((html (:sexp ,(regexp-opt '("element" "text" "attribute" "value"))) (:list ,(regexp-opt '("element""))) (:sentence "tag") (:text ,(regexp-opt '("comment" "text")))))) Another variant to avoid ambiguity would be to use a special syntax in calls, e.g. instead of (treesit-thing-next pos 'list) to use (treesit-thing-next pos '(thing list)). From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 30 14:37:11 2024 Received: (at 73404) by debbugs.gnu.org; 30 Dec 2024 19:37:11 +0000 Received: from localhost ([127.0.0.1]:60043 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSLZf-0003Fe-20 for submit@debbugs.gnu.org; Mon, 30 Dec 2024 14:37:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40696) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tSLZb-0003FQ-TT for 73404@debbugs.gnu.org; Mon, 30 Dec 2024 14:37:08 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSLZS-0003FQ-VZ; Mon, 30 Dec 2024 14:36:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=vL9MaPVk396BmGn02SaoGznlHm0AHWmWqRLQyfa/qXY=; b=AfGJWqMHN48x eh0GO/OXCA2pQDD2WJd1smh8o2puzVocZlbzaHo9RZhshAxTPLtyGonms/qrBeFyVthSbjC9zQP7m XlkNSFuKHkIKwp3pc5SkUXzmxdVcku4ZXNqLulaJx99t918bqe0edW2glNRcVSQ5qAIouAQQIls8o 6A/zvzhXbq4UXIcxL4KMtKzCdyIHhU53ka/0rskhkfq5Vbfu61tFRsfQQ8NhKYIk/5pEPD7bcCczN jYvurPidmllIaLPZTYUezb2vIjE8kiK5I03DQ0TOM44hBKfriduDWcMN0h3KgkA2koPhJnSTQxTVF xdJW0V/Fxr0zXjw4jhy/Sw==; Date: Mon, 30 Dec 2024 21:36:55 +0200 Message-Id: <861pxpneso.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87seq52e8g.fsf@mail.linkov.net> (message from Juri Linkov on Mon, 30 Dec 2024 20:54:39 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <864j2lp9vd.fsf@gnu.org> <87seq52e8g.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: casouri@gmail.com, theo@thornhill.no, mickey@masteringemacs.org, > 73404@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Mon, 30 Dec 2024 20:54:39 +0200 > > > "Group" is too abstract and too removed from the actual thing. I'm > > quite sure we can do better. > > The proper name is "list". Since it can't be used directly, > the second best variant is "group". Let's delay this part of the discussion until we see enough examples of those "lists" or "groups". > > What are the possible kinds of "groups", which are not balanced > > parenthetical expressions? > > All groups should be balanced. Even in languages that don't use > parens and brackets, e.g. "def" and "end" are balanced > while these are not parenthetical expressions. > > > Can you show a more-or-less exhaustive list of them? > > A exhaustive list is not yet finished. But basically most > groups are parenthetical expressions while some don't use parens. OK, maybe "exhaustive" is too much. Can you post a list that you have now? From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 04 12:53:52 2025 Received: (at 73404) by debbugs.gnu.org; 4 Jan 2025 17:53:52 +0000 Received: from localhost ([127.0.0.1]:57181 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tU8LM-0004XX-O5 for submit@debbugs.gnu.org; Sat, 04 Jan 2025 12:53:51 -0500 Received: from relay4-d.mail.gandi.net ([2001:4b98:dc4:8::224]:48085) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tU8LH-0004Wz-VJ for 73404@debbugs.gnu.org; Sat, 04 Jan 2025 12:53:44 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 46C68E0002; Sat, 4 Jan 2025 17:53:34 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Mon, 30 Dec 2024 00:00:51 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> Date: Sat, 04 Jan 2025 19:46:12 +0200 Message-ID: <875xmumpzv.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> BTW, my initial intention was to add the thing 'list'. >> But then I discovered that (treesit-thing-next (point) 'list) >> uses the function 'list' instead of the thing 'list'. >> >> However, the current 'sexp-list' as a two-word composite is too ugly. >> Now I found a better replacement: 'group'. This word is already used >> in lisp.el such as in the docstring of 'forward-list': > > I don’t have an opinion on this. Groups sounds a bit abstract, but so > does sexp-list. Indeed, the right name is 'list'. So let's use it. The minimal change that doesn't require updating all existing definitions of treesit-thing-settings, is just to add an exception for 'list': diff --git a/src/treesit.c b/src/treesit.c index b3214dad836..2d15e64180d 100644 --- a/src/treesit.c +++ b/src/treesit.c @@ -3618,7 +3618,7 @@ treesit_traverse_validate_predicate (Lisp_Object pred, } if (STRINGP (pred)) return true; - else if (FUNCTIONP (pred)) + else if (FUNCTIONP (pred) && !BASE_EQ (pred, Qlist)) return true; else if (SYMBOLP (pred)) { @@ -3722,7 +3722,7 @@ treesit_traverse_match_predicate (TSTreeCursor *cursor, Lisp_Object pred, const char *type = ts_node_type (node); return fast_c_string_match (pred, type, strlen (type)) >= 0; } - else if (FUNCTIONP (pred)) + else if (FUNCTIONP (pred) && !BASE_EQ (pred, Qlist)) { Lisp_Object lisp_node = make_treesit_node (parser, node); return !NILP (CALLN (Ffuncall, pred, lisp_node)); From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 04 14:05:45 2025 Received: (at 73404) by debbugs.gnu.org; 4 Jan 2025 19:05:45 +0000 Received: from localhost ([127.0.0.1]:57387 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tU9Sz-0007rS-2P for submit@debbugs.gnu.org; Sat, 04 Jan 2025 14:05:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47222) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tU9Sw-0007rB-6d for 73404@debbugs.gnu.org; Sat, 04 Jan 2025 14:05:43 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU9Sn-0002Pv-Q9; Sat, 04 Jan 2025 14:05:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=Nrpw0aCwVZEIBxUW7c5WeLcM0NQX5hb3hFbpDNLZmjw=; b=XZj2iukZfHBfqlM+gGJx c8gRsJFoKZqWWSeMbJkod1Ek4DxCX/r+adQMnWM02lgY4GKq6m9i1uMJ2meTAPjbNgbrJVxmw3wiF P6BgJEc+xME1Af79k5NIX4UpjB8GpiiSB5uaFavBuGab5ctq4A7nYiZJ0a8ReRRIfaG7xrrYsOS5m SaJNKloS45fxtX3b0kJNbF6FQ9ogR57eRjywUKp10KcWNIHulqeQV2acX0q2Il3c8404h9zWeO1Jb jZrzdW/lWIslTUzgeuhcmQCeoGL+jN2al/S8hTVsfDPWfqhcQHavvAULrO7b9gmOXwYgCCoD4Cw7T vmmgQNENb/EirQ==; Date: Sat, 04 Jan 2025 21:05:22 +0200 Message-Id: <86ikqubdsd.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <875xmumpzv.fsf@mail.linkov.net> (message from Juri Linkov on Sat, 04 Jan 2025 19:46:12 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: Theodor Thornhill , Eli Zaretskii , > Mickey Petersen , 73404@debbugs.gnu.org, > Stefan Monnier > Date: Sat, 04 Jan 2025 19:46:12 +0200 > > >> BTW, my initial intention was to add the thing 'list'. > >> But then I discovered that (treesit-thing-next (point) 'list) > >> uses the function 'list' instead of the thing 'list'. > >> > >> However, the current 'sexp-list' as a two-word composite is too ugly. > >> Now I found a better replacement: 'group'. This word is already used > >> in lisp.el such as in the docstring of 'forward-list': > > > > I don’t have an opinion on this. Groups sounds a bit abstract, but so > > does sexp-list. > > Indeed, the right name is 'list'. So let's use it. Please don't change the terminology yet. I'd like to try these commands with some tree-sitter modes and see what names we could use for them. I didn't yet have time for that, but OTOH there's no rush. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 02:45:20 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 07:45:20 +0000 Received: from localhost ([127.0.0.1]:59617 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tULK3-0001yv-PT for submit@debbugs.gnu.org; Sun, 05 Jan 2025 02:45:20 -0500 Received: from relay2-d.mail.gandi.net ([2001:4b98:dc4:8::222]:40269) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tULJx-0001vq-CJ for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 02:45:18 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 2484440002; Sun, 5 Jan 2025 07:45:02 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86ikqubdsd.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 04 Jan 2025 21:05:22 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> Date: Sun, 05 Jan 2025 09:32:58 +0200 Message-ID: <87wmf9912l.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> >> BTW, my initial intention was to add the thing 'list'. >> >> But then I discovered that (treesit-thing-next (point) 'list) >> >> uses the function 'list' instead of the thing 'list'. >> >> >> >> However, the current 'sexp-list' as a two-word composite is too ugly. >> >> Now I found a better replacement: 'group'. This word is already used >> >> in lisp.el such as in the docstring of 'forward-list': >> > >> > I don’t have an opinion on this. Groups sounds a bit abstract, but so >> > does sexp-list. >> >> Indeed, the right name is 'list'. So let's use it. > > Please don't change the terminology yet. This doesn't change the terminology. The ugly name was added a week ago. We need to fix it to the right name. The right name is 'list' to conform to the function name 'forward-list'. > I'd like to try these commands with some tree-sitter modes and see > what names we could use for them. The command is 'forward-list'. With 'list' it will do in ts-modes the same that 'forward-list' already does in non-ts modes. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 02:45:21 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 07:45:21 +0000 Received: from localhost ([127.0.0.1]:59620 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tULK5-0001zE-C0 for submit@debbugs.gnu.org; Sun, 05 Jan 2025 02:45:21 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:55157) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tULK3-0001yd-DO for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 02:45:20 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 44F891C0002; Sun, 5 Jan 2025 07:45:09 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <875xmumpzv.fsf@mail.linkov.net> (Juri Linkov's message of "Sat, 04 Jan 2025 19:46:12 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> Date: Sun, 05 Jan 2025 09:37:49 +0200 Message-ID: <87pll190ui.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain > The minimal change that doesn't require updating > all existing definitions of treesit-thing-settings, > is just to add an exception for 'list': > [...] > @@ -3618,7 +3618,7 @@ treesit_traverse_validate_predicate (Lisp_Object pred, > - else if (FUNCTIONP (pred)) > + else if (FUNCTIONP (pred) && !BASE_EQ (pred, Qlist)) I admit that hard-coding one symbol is not the right thing to do. So here is a better patch that checks for the symbol property: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=treesit-predicate.patch diff --git a/lisp/treesit.el b/lisp/treesit.el index e643eb48654..169607b150e 100644 --- a/lisp/treesit.el +++ b/lisp/treesit.el @@ -2516,6 +2516,8 @@ treesit-forward-sexp-list (treesit--scan-error pred arg))) (setq cnt (- cnt inc))))) +(put 'list 'treesit-predicate t) + (defun treesit-forward-list (&optional arg) "Move forward across a list. What constitutes a list is determined by `sexp-list' in diff --git a/src/treesit.c b/src/treesit.c index b3214dad836..4d2cd45e1b2 100644 --- a/src/treesit.c +++ b/src/treesit.c @@ -3618,7 +3618,7 @@ treesit_traverse_validate_predicate (Lisp_Object pred, } if (STRINGP (pred)) return true; - else if (FUNCTIONP (pred)) + else if (FUNCTIONP (pred) && !(SYMBOLP (pred) && Fget (pred, Qtreesit_predicate))) return true; else if (SYMBOLP (pred)) { @@ -3722,7 +3722,7 @@ treesit_traverse_match_predicate (TSTreeCursor *cursor, Lisp_Object pred, const char *type = ts_node_type (node); return fast_c_string_match (pred, type, strlen (type)) >= 0; } - else if (FUNCTIONP (pred)) + else if (FUNCTIONP (pred) && !(SYMBOLP (pred) && Fget (pred, Qtreesit_predicate))) { Lisp_Object lisp_node = make_treesit_node (parser, node); return !NILP (CALLN (Ffuncall, pred, lisp_node)); @@ -4333,6 +4333,8 @@ syms_of_treesit (void) DEFSYM (Qtreesit_invalid_predicate, "treesit-invalid-predicate"); DEFSYM (Qtreesit_predicate_not_found, "treesit-predicate-not-found"); + DEFSYM (Qtreesit_predicate, "treesit-predicate"); + DEFSYM (Qor, "or"); #ifdef WINDOWSNT --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 06:46:16 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 11:46:16 +0000 Received: from localhost ([127.0.0.1]:60190 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUP5E-0005vr-4G for submit@debbugs.gnu.org; Sun, 05 Jan 2025 06:46:16 -0500 Received: from mx.kolabnow.com ([212.103.80.154]:58552) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUP5B-0005vV-Tr for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 06:46:14 -0500 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id E8030209201D; Sun, 5 Jan 2025 12:46:07 +0100 (CET) Authentication-Results: ext-mx-out011.mykolab.com (amavis); dkim=pass (2048-bit key) reason="pass (just generated, assumed good)" header.d=kolabnow.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kolabnow.com; h= content-transfer-encoding:content-type:content-type:mime-version :message-id:date:date:references:in-reply-to:subject:subject :from:from:received:received:received; s=dkim20240523; t= 1736077567; x=1737891968; bh=A766Puq6PPrgBCLIKKkvjppzC+Xmxqwz7I1 fEoHFdYY=; b=ooSrn0MuzQpEEGXGdaMoKJO119nOEP/NSa+fcvZfhnOqEhSQD++ UZEnkgxfmddsqmub7cjnvPt8xKE5smIbsxctbgApGqZMGfyNzDpPSoyUgw7raalC Fl31fMxj1sJ9+CYNBVnKNoo7tQXeZz+BauQTWwyfecIY1MafS5s9HgnSK8uLC3ag PyJr2U6V/HfeufOKblLn+hLNFaBkmtAjw5l6oHrR/laHIfgmBGthG56eJBXlFC7M XuAMFr/48KodGbqGqYUyI4s6CqGuHDx44GFcyqC5NC47GqhL8AYNxf4G6NOVZugn t1xzTo8m59fPgLM6YkJpD7pyLmfpO8hS4PQ== X-Virus-Scanned: amavis at mykolab.com X-Spam-Flag: NO X-Spam-Score: -1 X-Spam-Level: X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] autolearn=ham autolearn_force=no Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out011.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id wfrrco_IP5xs; Sun, 5 Jan 2025 12:46:07 +0100 (CET) Received: from int-mx011.mykolab.com (unknown [10.9.13.11]) by mx.kolabnow.com (Postfix) with ESMTPS id 07EC22092013; Sun, 5 Jan 2025 12:46:04 +0100 (CET) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx011.mykolab.com (Postfix) with ESMTPS id B36A3313E1FF; Sun, 5 Jan 2025 12:46:04 +0100 (CET) From: Theodor Thornhill To: Juri Linkov , Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87wmf9912l.fsf@mail.linkov.net> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> Date: Sun, 05 Jan 2025 12:46:03 +0100 Message-ID: <87a5c5v5z8.fsf@thornhill.no> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Juri Linkov writes: >>> >> BTW, my initial intention was to add the thing 'list'. >>> >> But then I discovered that (treesit-thing-next (point) 'list) >>> >> uses the function 'list' instead of the thing 'list'. >>> >>=20 >>> >> However, the current 'sexp-list' as a two-word composite is too ugly. >>> >> Now I found a better replacement: 'group'. This word is already used >>> >> in lisp.el such as in the docstring of 'forward-list': >>> > >>> > I don=E2=80=99t have an opinion on this. Groups sounds a bit abstrac= t, but so >>> > does sexp-list. >>>=20 >>> Indeed, the right name is 'list'. So let's use it. >> >> Please don't change the terminology yet. > > This doesn't change the terminology. The ugly name was added a week ago. > We need to fix it to the right name. The right name is 'list' to > conform to the function name 'forward-list'. > >> I'd like to try these commands with some tree-sitter modes and see >> what names we could use for them. > > The command is 'forward-list'. With 'list' it will do in ts-modes > the same that 'forward-list' already does in non-ts modes. One issue I've had when exploring these things earlier is that I don't know the "spec" for how navigation in programming languages should work in Emacs. I mean, there are lots of examples in the source code, but no true specification. Should we possibly try to define some sane defaults here, so that it will be simpler to deal with idiosyncrasies of the tree sitter parsers? Lisp isn't in my experience the best language to define this, as it is just too simple. Also, many of the functions are related to word processing, like sentence, word, paragraph etc, which aren't really useful in programming, imo. What is a sentence in java, lisp, or python, for example? Also, maybe sexp is too lisp/ast-like of a term, so much so that it is hard to reason about what a sexp is or isn't. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 07:18:30 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 12:18:30 +0000 Received: from localhost ([127.0.0.1]:60250 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUPaP-0007gj-Pe for submit@debbugs.gnu.org; Sun, 05 Jan 2025 07:18:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55152) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUPaN-0007gT-Gg for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 07:18:28 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUPaG-0001l4-2b; Sun, 05 Jan 2025 07:18:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=FGKR5h/wyOBIhstqMgjfkgptHmeX9nTVx/vTAGYUNos=; b=TI1XVz5uQQfm tghepZ50XM5zMjalEPOfG0RFroPoAe1g//2Cpq3P1whWo0VWYNgcCcJ1HEX+Uj4BhRdXbXzMAtxLr znZkqJtfA7b1XHe1sbcPvfKEFTkt0ll60zsReeV3voKyySmzJnW4uxOaaCN1jIwOo8juaT6N+M6f8 Zt0w1A/GQyBs7ONzX5tSOVN2IniB6O572pkfeRA3bdfYsiCB1uUCnL9vJyBi6NkxM60IXAhCxt2qg pCYu8TqMXOkpMKO8vPiQsV9R7DHrG91ojVXYYDW+MDmVw9ESxDAG0oTIuIFLKHwQlF3GGFoe+PB6C uttf52RdMcQqJeQxqDSPVA==; Date: Sun, 05 Jan 2025 14:18:17 +0200 Message-Id: <86cyh18nee.fsf@gnu.org> From: Eli Zaretskii To: Theodor Thornhill In-Reply-To: <87a5c5v5z8.fsf@thornhill.no> (message from Theodor Thornhill on Sun, 05 Jan 2025 12:46:03 +0100) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca, juri@linkov.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] > autolearn=ham autolearn_force=no > From: Theodor Thornhill > Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, > monnier@iro.umontreal.ca > Date: Sun, 05 Jan 2025 12:46:03 +0100 > > Juri Linkov writes: > > > The command is 'forward-list'. With 'list' it will do in ts-modes > > the same that 'forward-list' already does in non-ts modes. > > One issue I've had when exploring these things earlier is that I don't > know the "spec" for how navigation in programming languages should work > in Emacs. I mean, there are lots of examples in the source code, but no > true specification. Should we possibly try to define some sane defaults > here, so that it will be simpler to deal with idiosyncrasies of the tree > sitter parsers? Lisp isn't in my experience the best language to define > this, as it is just too simple. Also, many of the functions are related > to word processing, like sentence, word, paragraph etc, which aren't > really useful in programming, imo. What is a sentence in java, lisp, or > python, for example? Also, maybe sexp is too lisp/ast-like of a term, so > much so that it is hard to reason about what a sexp is or isn't. Is this goal achievable in practice,l given the sometimes radical differences between languages? Maybe we only can have examples and some dwim-ish behavior in most cases? From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 07:30:48 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 12:30:49 +0000 Received: from localhost ([127.0.0.1]:60277 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUPmK-0008SB-AK for submit@debbugs.gnu.org; Sun, 05 Jan 2025 07:30:48 -0500 Received: from mx.kolabnow.com ([212.103.80.154]:60074) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUPmG-0008Ro-SX for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 07:30:46 -0500 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id 8FE9D3004C6E; Sun, 5 Jan 2025 13:30:38 +0100 (CET) Authentication-Results: ext-mx-out013.mykolab.com (amavis); dkim=pass (2048-bit key) reason="pass (just generated, assumed good)" header.d=kolabnow.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kolabnow.com; h= content-type:content-type:mime-version:message-id:date:date :references:in-reply-to:subject:subject:from:from:received :received:received; s=dkim20240523; t=1736080236; x=1737894637; bh=u+CF+m6p29qajvaIzzYv5vtx/udeV3SaxZ2CJRLfWi8=; b=P6lUaS423m9o szrup0WDtYJiH6W4LktiktIzaBYjnNRwCtyZ4nSSi68qj/L6Z3NKbtnYKxRa+knv CfNoV1BpywXzSGJhmZz+GXA18nManc5U1qH8sY0txDdaKzETVhzdiH+NHKSQYpGC czFEH4r2uVTLL4t5su2KqskmGEmEb4ny337rHVXUj0U5IlymZ+f3Tw5xjt6u/HEP GWbOoN2dDimSaOTKORkRJZ/7/rMgkUKdNlQOGHsCpWXmb/TA05onjtVjIwTNZG2Z nY6iOkH9YuCh+uGmlK5pvetUaV2BfYQTt2YgKgJ+ARrTJNqbeddxp4aAzKMYehS9 bU4wwJonnQ== X-Virus-Scanned: amavis at mykolab.com X-Spam-Flag: NO X-Spam-Score: -1 X-Spam-Level: X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] autolearn=ham autolearn_force=no Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out013.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id wBUqGKpmYrQ0; Sun, 5 Jan 2025 13:30:36 +0100 (CET) Received: from int-mx009.mykolab.com (unknown [10.9.13.9]) by mx.kolabnow.com (Postfix) with ESMTPS id 309583004C6B; Sun, 5 Jan 2025 13:30:32 +0100 (CET) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx009.mykolab.com (Postfix) with ESMTPS id CC54A2125AB9; Sun, 5 Jan 2025 13:30:32 +0100 (CET) From: Theodor Thornhill To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86cyh18nee.fsf@gnu.org> References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <86cyh18nee.fsf@gnu.org> Date: Sun, 05 Jan 2025 13:30:31 +0100 Message-ID: <874j2dv3x4.fsf@thornhill.no> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca, juri@linkov.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] >> autolearn=ham autolearn_force=no >> From: Theodor Thornhill >> Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, >> monnier@iro.umontreal.ca >> Date: Sun, 05 Jan 2025 12:46:03 +0100 >> >> Juri Linkov writes: >> >> > The command is 'forward-list'. With 'list' it will do in ts-modes >> > the same that 'forward-list' already does in non-ts modes. >> >> One issue I've had when exploring these things earlier is that I don't >> know the "spec" for how navigation in programming languages should work >> in Emacs. I mean, there are lots of examples in the source code, but no >> true specification. Should we possibly try to define some sane defaults >> here, so that it will be simpler to deal with idiosyncrasies of the tree >> sitter parsers? Lisp isn't in my experience the best language to define >> this, as it is just too simple. Also, many of the functions are related >> to word processing, like sentence, word, paragraph etc, which aren't >> really useful in programming, imo. What is a sentence in java, lisp, or >> python, for example? Also, maybe sexp is too lisp/ast-like of a term, so >> much so that it is hard to reason about what a sexp is or isn't. > > Is this goal achievable in practice,l given the sometimes radical > differences between languages? Maybe we only can have examples and > some dwim-ish behavior in most cases? I'm not sure if it is achievable or not, but I think it'd be nice to avoid any "hand-waviness" to confuse us while we talk about sexps/lists/etc. I could try to document thoroughly how lisp-modes and cc modes implement these and see if we have some glaring discrepancies between just those two? I'm just thinking that describing the as-is could be very beneficial, so that we know we're talking the same abstractions. It could start just as a scratch/movements.org file, not touching any code until you, stefan and others agree that we're talking about the same stuff. Then we can at least consult this document when we stumble upon things like jsx-text, whitespace etc. After this we can proceed with mickey's combobulate and friends to inform us on what the best way to deal with it is? Maybe we can discover som simplifications in core too. WDYT? Theo From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 08:02:48 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 13:02:48 +0000 Received: from localhost ([127.0.0.1]:60329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUQHF-0001gI-7z for submit@debbugs.gnu.org; Sun, 05 Jan 2025 08:02:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34692) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUQHC-0001g5-Kc for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 08:02:43 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUQH5-0004sP-4u; Sun, 05 Jan 2025 08:02:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=jsx5MiHaYUlnwSxBl5CAhKbvXg+d5qUSdQPUbLKK5jA=; b=Qt36DPRxvTiZgQM8J08t RQUXNlCafRDZZR3sPGPmZvLu2R890EAIgwRvk7CVPaBOfHZ4Q/IuPsppc1S/03jkm6ghzBB2XrBU7 Eoxc9jaDYfNu/buLYq9FbyQHyaIBBvr9aVgHDEH//q57NhDRSyPs3qLbcJJdytwdks/MczwFsm4rD WWQPiwj0xneNYzJsYI6EAwGQv4RthIIyt5des/vO7J9pK3nIv084w45/XtWkY2xk4Jtq0tkejzXJK D3Wj6yR1VevcnyXYTlErsXMyXhjoDRLunortnDCRpqRaTwWF6/wPX2ziQuKjtNKV30KOPvP4GOy9e zOshL70U/e+jug==; Date: Sun, 05 Jan 2025 15:02:31 +0200 Message-Id: <867c798lco.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87wmf9912l.fsf@mail.linkov.net> (message from Juri Linkov on Sun, 05 Jan 2025 09:32:58 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: casouri@gmail.com, theo@thornhill.no, mickey@masteringemacs.org, > 73404@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Sun, 05 Jan 2025 09:32:58 +0200 > > >> >> BTW, my initial intention was to add the thing 'list'. > >> >> But then I discovered that (treesit-thing-next (point) 'list) > >> >> uses the function 'list' instead of the thing 'list'. > >> >> > >> >> However, the current 'sexp-list' as a two-word composite is too ugly. > >> >> Now I found a better replacement: 'group'. This word is already used > >> >> in lisp.el such as in the docstring of 'forward-list': > >> > > >> > I don’t have an opinion on this. Groups sounds a bit abstract, but so > >> > does sexp-list. > >> > >> Indeed, the right name is 'list'. So let's use it. > > > > Please don't change the terminology yet. > > This doesn't change the terminology. The ugly name was added a week ago. > We need to fix it to the right name. The right name is 'list' to > conform to the function name 'forward-list'. > > > I'd like to try these commands with some tree-sitter modes and see > > what names we could use for them. > > The command is 'forward-list'. With 'list' it will do in ts-modes > the same that 'forward-list' already does in non-ts modes. Thanks. I tried this now in c-ts-mode, and it seems to behave strangely in some cases. First, AFAICT it supports "lists" enclosed in "(..)", "{..}" and "<..>", but not "[..]". Is that expected? Why don't we support "[..]"? Next, it many times signals an error "No next group", which is less useful than it could be. For example: w32_get_internal_run_time (void) { if (get_process_times_fn) { FILETIME create, exit, kernel, user; HANDLE proc = GetCurrentProcess (); if ((*get_process_times_fn) (proc, &create, &exit, &kernel, &user)) { return ltime (total.QuadPart); } ^ } return Fcurrent_time (); } If you put point on the semi-colon marked by "^", C-M-n signals an error. Can't we instead move to the next "list" after "Fcurrent_time"? Also, it sometimes "misses" what looks like a "list". For example: s_pfn_Open_Process_Token = (OpenProcessToken_Proc) get_proc_addr (hm_advapi32, "OpenProcessToken"); If you put point at the beginning of the line, C-M-n moves to the end of the 2nd parenthesized group, not to the end of the first group. As for terminology: the Emacs user manual calls these "groupings delimited by parentheses (or whatever else serves as delimiters in the language you are working with)", or in short "parenthetical group". Why cannot we keep this terminology? It sounds like it describes its subject as accurate as we can come up with. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 09:41:14 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 14:41:14 +0000 Received: from localhost ([127.0.0.1]:60477 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tURoX-0006ov-SI for submit@debbugs.gnu.org; Sun, 05 Jan 2025 09:41:14 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:13354) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tURoU-0006od-DK for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 09:41:12 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 32AFF80911; Sun, 5 Jan 2025 09:41:01 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736088060; bh=+H/g1i+V7IaDaLTqaDCK3+LGsMFIXSRzEG2jrgN8ZTw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=NPkR5MTDKSX1Aym0EvMO63J4cG12/UkasuiUKXwb2jwsWEPal7Tw+gwUlOHJV+A/X pNh2gMm+hoixZd39Iji+gGRXOzYsEK9t/tqIxDaT9HJndTxdTIEbYO0p36ix160hO7 UeUeF0PR4qI+/m0Sb4R7qRtOxx3zKSZY5CNAaekJAIKNMkmdd01AEaTDVkNG4EuiFT J9loCEtm1jdMGQMxFbzSSZOzyN+BcBsqJUgiNZOc5QJTJuSGZz5XLSCeUCBY1RwtXf NuZUne22oINZ/PhRE9qVA348cJVJncnVJxhnhk9yWI5GFXw3OTuaa6y8JQsJ5yqAJB of3uoNuYClqvg== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 4CA96806BF; Sun, 5 Jan 2025 09:41:00 -0500 (EST) Received: from pastel (104-195-230-250.cpe.teksavvy.com [104.195.230.250]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 0FF54120480; Sun, 5 Jan 2025 09:41:00 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87pll190ui.fsf@mail.linkov.net> (Juri Linkov's message of "Sun, 05 Jan 2025 09:37:49 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> Date: Sun, 05 Jan 2025 09:40:56 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.059 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Yuan Fu , Mickey Petersen , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I admit that hard-coding one symbol is not the right thing to do. > So here is a better patch that checks for the symbol property: That's better but I still wonder why we don't want to use a keyword. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 12:00:09 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 17:00:09 +0000 Received: from localhost ([127.0.0.1]:34973 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUTyz-0005Yk-0I for submit@debbugs.gnu.org; Sun, 05 Jan 2025 12:00:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33634) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUTyu-0005XT-9E for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 12:00:07 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUTym-0001VE-LZ; Sun, 05 Jan 2025 11:59:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ZPhB7GIUGfp5iLilvMcU6zffK2sZ4AscRzs0u3bqoPc=; b=JeQTNPH2MfGA H9twVuu9Ya2j48/CaTv26NmVX7Zm/hR64eVw6V3qxSFXe1iXXR+OGwTUAWUOKaPb3m3v4w/dn5BVn rDpi4TFwaLOyiphKvL9qo4UsB/djXHKssiSUzR07sR0THeHtpWY7O5qgO0N+qe84CrS7/5biL4LNC lpgESQHm3q0HFJLrb6xNrG61XX1yd0A/wp4gTuc9d+CAI0isj/aVt3Vjg7HAAcfwZlV1kH4tZTG8D 3lhG4Yo37LfRNAl5Z0W31AvdaRbe/5uwjGV9OLdNBqvWUrkqMTa/T86JagyVU8GVaMhlhEmuzRKUn 7GaeSr55FthzP6aY7i6VlQ==; Date: Sun, 05 Jan 2025 18:59:54 +0200 Message-Id: <86r05h6vsl.fsf@gnu.org> From: Eli Zaretskii To: Theodor Thornhill In-Reply-To: <874j2dv3x4.fsf@thornhill.no> (message from Theodor Thornhill on Sun, 05 Jan 2025 13:30:31 +0100) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <86cyh18nee.fsf@gnu.org> <874j2dv3x4.fsf@thornhill.no> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca, juri@linkov.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Theodor Thornhill > Cc: juri@linkov.net, casouri@gmail.com, mickey@masteringemacs.org, > 73404@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Sun, 05 Jan 2025 13:30:31 +0100 > > > Is this goal achievable in practice, given the sometimes radical > > differences between languages? Maybe we only can have examples and > > some dwim-ish behavior in most cases? > > I'm not sure if it is achievable or not, but I think it'd be nice to > avoid any "hand-waviness" to confuse us while we talk about > sexps/lists/etc. I could try to document thoroughly how lisp-modes and > cc modes implement these and see if we have some glaring discrepancies > between just those two? They are not "discrepancies", and certainly not "glaring", but there are differences. E.g., in c-ts-mode, we support "lists" enclosed in (..), {..}, and <..> (but not [..], for some reason). You will not find anything like that in Lisp-like languages. So, if we want to have a broader picture, we will need to try more modes. Can you or someone produce an exhaustive list of all TS modes we have that implement these movement commands? Then we could try the commands in each one of those modes and see if there are common patterns that we can name and describe. For now, my conclusion was that (almost) every parenthetical grouping supported by the language grammar can be moved across with these commands. But I only tried Lisp and c-ts-mode. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 13:20:25 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 18:20:25 +0000 Received: from localhost ([127.0.0.1]:35142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUVEe-0001If-P0 for submit@debbugs.gnu.org; Sun, 05 Jan 2025 13:20:25 -0500 Received: from mx.kolabnow.com ([212.103.80.154]:54600) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUVEc-0001Hf-KB for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 13:20:24 -0500 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id 73F4D3004C6C; Sun, 5 Jan 2025 19:20:16 +0100 (CET) Authentication-Results: ext-mx-out013.mykolab.com (amavis); dkim=pass (2048-bit key) reason="pass (just generated, assumed good)" header.d=kolabnow.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kolabnow.com; h= content-type:content-type:mime-version:message-id:date:date :references:in-reply-to:subject:subject:from:from:received :received:received; s=dkim20240523; t=1736101214; x=1737915615; bh=S3j0UJpO09yQeNNRJQMZNeUR4NYYAxwJ+3KDuU8c5v8=; b=GKd7YCzQdgGu ImGMpA8dSD3KjcBJb4JjF3yncOF95vIoN6Riij73lXxnglF3GHYot4cl1Z1A/3jz uU/+5+kSTjkJ+NW3hsyWy8ICJm5x8/fK7E+sgIoD1NDbp6pS8OT2f3Lk3mw6qh72 w8kifEhXA/GGv3qOgKiGBgnZZL8/+Az6TKWEtBklQ7/nbXSjXPu9KMOCj00zrTjA uplfY9JkQ/Isz3juCnKk4D6/uFQYhi4udTAx/her/sQNHD3/WBhF0B8EgagYyLPV 4cS7SXwVvxvk7gevWVbIzwMOK88JJU1mrOu7JmmjWxhQOJLUr0XPZ2uAR+U91c8S nCXX/s+1dw== X-Virus-Scanned: amavis at mykolab.com X-Spam-Flag: NO X-Spam-Score: -1 X-Spam-Level: X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] autolearn=ham autolearn_force=no Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out013.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id mxOUFGMIITGZ; Sun, 5 Jan 2025 19:20:14 +0100 (CET) Received: from int-mx011.mykolab.com (unknown [10.9.13.11]) by mx.kolabnow.com (Postfix) with ESMTPS id 06D4C3004C6B; Sun, 5 Jan 2025 19:20:11 +0100 (CET) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx011.mykolab.com (Postfix) with ESMTPS id BBA1B3141C4A; Sun, 5 Jan 2025 19:20:11 +0100 (CET) From: Theodor Thornhill To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86r05h6vsl.fsf@gnu.org> References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <86cyh18nee.fsf@gnu.org> <874j2dv3x4.fsf@thornhill.no> <86r05h6vsl.fsf@gnu.org> Date: Sun, 05 Jan 2025 19:20:10 +0100 Message-ID: <87y0zpt95x.fsf@thornhill.no> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca, juri@linkov.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> From: Theodor Thornhill >> Cc: juri@linkov.net, casouri@gmail.com, mickey@masteringemacs.org, >> 73404@debbugs.gnu.org, monnier@iro.umontreal.ca >> Date: Sun, 05 Jan 2025 13:30:31 +0100 >> >> > Is this goal achievable in practice, given the sometimes radical >> > differences between languages? Maybe we only can have examples and >> > some dwim-ish behavior in most cases? >> >> I'm not sure if it is achievable or not, but I think it'd be nice to >> avoid any "hand-waviness" to confuse us while we talk about >> sexps/lists/etc. I could try to document thoroughly how lisp-modes and >> cc modes implement these and see if we have some glaring discrepancies >> between just those two? > > They are not "discrepancies", and certainly not "glaring", but there > are differences. E.g., in c-ts-mode, we support "lists" enclosed in > (..), {..}, and <..> (but not [..], for some reason). You will not > find anything like that in Lisp-like languages. > I didn't mean to sound harsh, my apoligies. > So, if we want to have a broader picture, we will need to try more > modes. Can you or someone produce an exhaustive list of all TS modes > we have that implement these movement commands? Then we could try the > commands in each one of those modes and see if there are common > patterns that we can name and describe. > Sure! > For now, my conclusion was that (almost) every parenthetical grouping > supported by the language grammar can be moved across with these > commands. But I only tried Lisp and c-ts-mode. Yeah, absolutely. I think it is moving in the right direction. What I'm thinking of is more in the line of standarization as a conscious effort, not by consequence of convention. The reason being to help aid us when discussing these issues. I feel we often attribute bugs to our subjective meaning, perhaps because we are missing some clearer structure in the expected behavior. I'm absolutely open to being wrong here, though. Theo From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 13:24:43 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 18:24:43 +0000 Received: from localhost ([127.0.0.1]:35153 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUVIp-0001SC-69 for submit@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:43 -0500 Received: from relay2-d.mail.gandi.net ([2001:4b98:dc4:8::222]:38693) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUVIn-0001Rp-GN for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:42 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6988D40002; Sun, 5 Jan 2025 18:24:31 +0000 (UTC) From: Juri Linkov To: Theodor Thornhill Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87a5c5v5z8.fsf@thornhill.no> (Theodor Thornhill's message of "Sun, 05 Jan 2025 12:46:03 +0100") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> Date: Sun, 05 Jan 2025 19:59:31 +0200 Message-ID: <877c79qhcs.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , mickey@masteringemacs.org, casouri@gmail.com, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> The command is 'forward-list'. With 'list' it will do in ts-modes >> the same that 'forward-list' already does in non-ts modes. > > One issue I've had when exploring these things earlier is that I don't > know the "spec" for how navigation in programming languages should work > in Emacs. I mean, there are lots of examples in the source code, but no > true specification. Should we possibly try to define some sane defaults > here, so that it will be simpler to deal with idiosyncrasies of the tree > sitter parsers? Lisp isn't in my experience the best language to define > this, as it is just too simple. Very simple: I'm striving to provide compatibility between ts-modes with their predecessor non-ts modes. So the de-facto "spec" is the existing non-ts modes that provide canonical reference implementation. This helps the users to more smoothly switch to ts-modes. > Also, many of the functions are related to word processing, like > sentence, word, paragraph etc, which aren't really useful in > programming, imo. What is a sentence in java, lisp, or python, for > example? Also, maybe sexp is too lisp/ast-like of a term, so much so > that it is hard to reason about what a sexp is or isn't. Indeed, sentences and paragraphs are even more fuzzy concepts when applied to programming languages. But some modes make sense of these things nevertheless. For example, in both c-mode and c-ts-mode 'M-e' moves to the end of statement that usually end with a semicolon; so your intuition about these things looks correct. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 13:24:49 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 18:24:49 +0000 Received: from localhost ([127.0.0.1]:35156 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUVIu-0001SX-JS for submit@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:48 -0500 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:58997) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUVIs-0001S2-T4 for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:47 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id D82A0E0003; Sun, 5 Jan 2025 18:24:37 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <867c798lco.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 05 Jan 2025 15:02:31 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <867c798lco.fsf@gnu.org> Date: Sun, 05 Jan 2025 20:05:53 +0200 Message-ID: <871pxhp24e.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> The command is 'forward-list'. With 'list' it will do in ts-modes >> the same that 'forward-list' already does in non-ts modes. > > Thanks. > > I tried this now in c-ts-mode, and it seems to behave strangely in > some cases. > > First, AFAICT it supports "lists" enclosed in "(..)", "{..}" and > "<..>", but not "[..]". Is that expected? Why don't we support > "[..]"? This looks like a bug. It would be nice if you posted an example. > Next, it many times signals an error "No next group", which is less > useful than it could be. For example: > > w32_get_internal_run_time (void) > { > if (get_process_times_fn) > { > FILETIME create, exit, kernel, user; > HANDLE proc = GetCurrentProcess (); > if ((*get_process_times_fn) (proc, &create, &exit, &kernel, &user)) > { > return ltime (total.QuadPart); > } ^ > } > > return Fcurrent_time (); > } > > If you put point on the semi-colon marked by "^", C-M-n signals an > error. Can't we instead move to the next "list" after > "Fcurrent_time"? 'C-M-n' doesn't move to the next statement in c-mode, since we need to support backward-compatibility. The key that moves to the next statement ignoring nested lists in C modes is 'M-e'. > Also, it sometimes "misses" what looks like a "list". For example: > > s_pfn_Open_Process_Token = (OpenProcessToken_Proc) > get_proc_addr (hm_advapi32, "OpenProcessToken"); > > If you put point at the beginning of the line, C-M-n moves to the end > of the 2nd parenthesized group, not to the end of the first group. Welcome to the twisty maze of tree-sitter grammars. C-M-n is unable to move to the first group because it's inside the node in the AST: (cast_expression "(" type: (type_descriptor type: (type_identifier)) ")" value: (call_expression function: (identifier) arguments: (argument_list "(" (identifier) , (string_literal " (string_content) ") ")"))) Both the first and the 2nd parenthesized groups are inside the "cast_expression" node. > As for terminology: the Emacs user manual calls these "groupings > delimited by parentheses (or whatever else serves as delimiters in the > language you are working with)", or in short "parenthetical group". > Why cannot we keep this terminology? It sounds like it describes its > subject as accurate as we can come up with. So you prefer "group" over "list"? Then 'forward-list' should move over "group"? From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 13:24:55 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 18:24:55 +0000 Received: from localhost ([127.0.0.1]:35159 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUVJ0-0001St-Vy for submit@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:55 -0500 Received: from relay9-d.mail.gandi.net ([2001:4b98:dc4:8::229]:39179) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUVIz-0001SO-4u for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 13:24:53 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 89AE8FF802; Sun, 5 Jan 2025 18:24:42 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Sun, 05 Jan 2025 09:40:56 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> Date: Sun, 05 Jan 2025 20:10:41 +0200 Message-ID: <87ldvpnnby.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Yuan Fu , Mickey Petersen , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> I admit that hard-coding one symbol is not the right thing to do. >> So here is a better patch that checks for the symbol property: > > That's better but I still wonder why we don't want to use a keyword. The downside of a keyword is that changing the syntax will require updating all existing settings of treesit-thing-settings. But if Yuan agrees to this change, this would be fine. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 14:51:12 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 19:51:12 +0000 Received: from localhost ([127.0.0.1]:35346 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUWeW-0006F8-1p for submit@debbugs.gnu.org; Sun, 05 Jan 2025 14:51:12 -0500 Received: from mx.kolabnow.com ([212.103.80.154]:43990) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUWeR-0006E4-LO for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 14:51:10 -0500 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id AFA792092017; Sun, 5 Jan 2025 20:51:01 +0100 (CET) Authentication-Results: ext-mx-out011.mykolab.com (amavis); dkim=pass (2048-bit key) reason="pass (just generated, assumed good)" header.d=kolabnow.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kolabnow.com; h= content-type:content-type:mime-version:message-id:date:date :references:in-reply-to:subject:subject:from:from:received :received:received; s=dkim20240523; t=1736106660; x=1737921061; bh=qPjwjPSEQdCbUEQOfcw5TQlfd62K5aSQ+E5CByV7PwQ=; b=pCEqtyAFqxV5 uOUozmYTfSr+Fxc/miJz64kg2u5T6LkKwGjm7akvv8aRmVKKqHO0Y+2XHykK/kI4 zQr1m1LrzjjsncNkae9B9z+jG7E8pgbwT0cMq/r2hY3+bE2NXYTIFY5lwZ9n6wbp M6dp9zmXsT2rGhwB2EhH7HcbFykjv0DghOrtAOV0QkiH25W3uKr+KCf5iaQIzBXn RBsQcE4MoJDGwUj4mmpOEGZ07P013YHBmu9+DI/X7+lNavs8ny7mLKrH0W1PahWW EBxat5IS7GQVZwbLag5cLQ6YIzTzhFvbzZ/518QniQLQxLmNIDh4yf1Ho9b6RhdH MhOJv8yD7g== X-Virus-Scanned: amavis at mykolab.com X-Spam-Flag: NO X-Spam-Score: -1 X-Spam-Level: X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1] autolearn=ham autolearn_force=no Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out011.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id xB-q_1K6DiEb; Sun, 5 Jan 2025 20:51:00 +0100 (CET) Received: from int-mx011.mykolab.com (unknown [10.9.13.11]) by mx.kolabnow.com (Postfix) with ESMTPS id 736AE2092013; Sun, 5 Jan 2025 20:51:00 +0100 (CET) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx011.mykolab.com (Postfix) with ESMTPS id 1C8CE3141C4F; Sun, 5 Jan 2025 20:51:00 +0100 (CET) From: Theodor Thornhill To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <877c79qhcs.fsf@mail.linkov.net> References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> Date: Sun, 05 Jan 2025 20:50:58 +0100 Message-ID: <87sepxt4yl.fsf@thornhill.no> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , mickey@masteringemacs.org, casouri@gmail.com, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Juri Linkov writes: >>> The command is 'forward-list'. With 'list' it will do in ts-modes >>> the same that 'forward-list' already does in non-ts modes. >> >> One issue I've had when exploring these things earlier is that I don't >> know the "spec" for how navigation in programming languages should work >> in Emacs. I mean, there are lots of examples in the source code, but no >> true specification. Should we possibly try to define some sane defaults >> here, so that it will be simpler to deal with idiosyncrasies of the tree >> sitter parsers? Lisp isn't in my experience the best language to define >> this, as it is just too simple. > > Very simple: I'm striving to provide compatibility between ts-modes > with their predecessor non-ts modes. So the de-facto "spec" is the > existing non-ts modes that provide canonical reference implementation. > This helps the users to more smoothly switch to ts-modes. > Yes, I agree on this goal, absolutely. That was also my initial intent and attempt when we started doing this. There are still many places where either the canonical modes are inconsistent but has so much history that we're so used to it and forget. If we blindly try to copy everything one to one we might also adopt some stuff that doesn't really make sense. However, deciding on what makes sense is hard, which is why I'm suggesting we try to define a "spec". It needn't really be much more than reference suggestions for mode implementors at first, then maybe some intersection of features crystallize across all modes. >> Also, many of the functions are related to word processing, like >> sentence, word, paragraph etc, which aren't really useful in >> programming, imo. What is a sentence in java, lisp, or python, for >> example? Also, maybe sexp is too lisp/ast-like of a term, so much so >> that it is hard to reason about what a sexp is or isn't. > > Indeed, sentences and paragraphs are even more fuzzy concepts when applied > to programming languages. But some modes make sense of these things > nevertheless. For example, in both c-mode and c-ts-mode 'M-e' > moves to the end of statement that usually end with a semicolon; > so your intuition about these things looks correct. Yes, but imo more importantly, disregarding any confusing naming we have a very rich set of operations available to us that I think should be as distinct as possible: - C-M-f - M-f - M-a - M-e - C-f - C-a - C-e - more If we'll aim to get the absolute most out of the parse tree possibilities we should strive to make these distinct, at least down the line. Some of these are simpler to categorize than others, such as a sentence meaning "some expression that ends that isn't a function or a class definition". From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 14:54:26 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 19:54:26 +0000 Received: from localhost ([127.0.0.1]:35351 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUWhd-0006LZ-Od for submit@debbugs.gnu.org; Sun, 05 Jan 2025 14:54:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:48450) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUWhb-0006LH-Hk for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 14:54:24 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUWhU-0003Gt-AO; Sun, 05 Jan 2025 14:54:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=w3tu1DOTtjYdJJYoLZ+aIob9EdaBoAHQPkXwheIDhRI=; b=jEtRR2rRmSLC 1qX0gehvSQ2sd7R+OwgkkfX4TCyDFYRrnSrsIsGAYwEIqs44imC4RaFnk0RfibWb48xebdY1E/Zvj rHFOgy0y3tHnFdVJ0PZpDcKVNktE1m6r2NzGUu+1h2IZM7ZVxem2vFk2QBz5ACq1i2wysokA6vqXL vs2Pk+U7Zdote1xVt1cLILnEJGdliUK4rz6u5e5lGjZDi3CQULKqvPPsJsIymYTfNYDF9xlCgbGWl BwNhctUK0N+Ex1J7cYS+i3o8VIp3gF1qbpLBFsRrr2ulCkiecefC+UvrGjHRks6AkYWM1YbNLDZrP tcWeZujfvmpQcMZUsBwAWg==; Date: Sun, 05 Jan 2025 21:54:12 +0200 Message-Id: <86cyh16nq3.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <871pxhp24e.fsf@mail.linkov.net> (message from Juri Linkov on Sun, 05 Jan 2025 20:05:53 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <867c798lco.fsf@gnu.org> <871pxhp24e.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: casouri@gmail.com, theo@thornhill.no, mickey@masteringemacs.org, > 73404@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Sun, 05 Jan 2025 20:05:53 +0200 > > >> The command is 'forward-list'. With 'list' it will do in ts-modes > >> the same that 'forward-list' already does in non-ts modes. > > > > Thanks. > > > > I tried this now in c-ts-mode, and it seems to behave strangely in > > some cases. > > > > First, AFAICT it supports "lists" enclosed in "(..)", "{..}" and > > "<..>", but not "[..]". Is that expected? Why don't we support > > "[..]"? > > This looks like a bug. It would be nice if you posted an example. That's easy: C-f C-f src/dispnew.c RET M-x -c-ts-mode RET C-s [ RET This brings you to this function: static void check_rows (struct frame *f) { for (int y = 0; y < f->desired_matrix->nrows; ++y) if (MATRIX_ROW_ENABLED_P (f->desired_matrix, y)) { struct glyph_row *row = MATRIX_ROW (f->desired_matrix, y); for (int x = 0; x < row->used[TEXT_AREA]; ++x) eassert (row->glyphs[TEXT_AREA][x].frame != 0); } } Move point to any of the [ brackets there and type C-M-n -- you will be moved to the last ) of the function. Another example on line 152 in dispnew.c: struct redisplay_history { char trace[512 + 100]; }; Move point to the [ before 512: C-M-n will signal an error. Any bracket in the file will trigger either the first or the second behavior. > > Next, it many times signals an error "No next group", which is less > > useful than it could be. For example: > > > > w32_get_internal_run_time (void) > > { > > if (get_process_times_fn) > > { > > FILETIME create, exit, kernel, user; > > HANDLE proc = GetCurrentProcess (); > > if ((*get_process_times_fn) (proc, &create, &exit, &kernel, &user)) > > { > > return ltime (total.QuadPart); > > } ^ > > } > > > > return Fcurrent_time (); > > } > > > > If you put point on the semi-colon marked by "^", C-M-n signals an > > error. Can't we instead move to the next "list" after > > "Fcurrent_time"? > > 'C-M-n' doesn't move to the next statement in c-mode, > since we need to support backward-compatibility. The key > that moves to the next statement ignoring nested lists > in C modes is 'M-e'. Well, then maybe we could have the more useful behavior as an opt-in option? > > Also, it sometimes "misses" what looks like a "list". For example: > > > > s_pfn_Open_Process_Token = (OpenProcessToken_Proc) > > get_proc_addr (hm_advapi32, "OpenProcessToken"); > > > > If you put point at the beginning of the line, C-M-n moves to the end > > of the 2nd parenthesized group, not to the end of the first group. > > Welcome to the twisty maze of tree-sitter grammars. C-M-n is unable > to move to the first group because it's inside the node in the AST: > > (cast_expression "(" > type: (type_descriptor type: (type_identifier)) > ")" > value: > (call_expression function: (identifier) > arguments: > (argument_list "(" (identifier) , > (string_literal " (string_content) ") > ")"))) > > Both the first and the 2nd parenthesized groups are inside the > "cast_expression" node. Sorry, I don't follow: the parentheses of the argument list are also inside a node, and yet we do recognize them. What's the difference? > > As for terminology: the Emacs user manual calls these "groupings > > delimited by parentheses (or whatever else serves as delimiters in the > > language you are working with)", or in short "parenthetical group". > > Why cannot we keep this terminology? It sounds like it describes its > > subject as accurate as we can come up with. > > So you prefer "group" over "list"? As a shorthand; the better term is "parenthesized group". > Then 'forward-list' should move over "group"? I'm not sure we should rename the commands, given the legacy. But the doc string should use "group" or "parenthesized group", IMO. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 05 16:18:37 2025 Received: (at 73404) by debbugs.gnu.org; 5 Jan 2025 21:18:37 +0000 Received: from localhost ([127.0.0.1]:35535 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUY17-0002yM-Bh for submit@debbugs.gnu.org; Sun, 05 Jan 2025 16:18:37 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:43319) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUY15-0002y0-4t for 73404@debbugs.gnu.org; Sun, 05 Jan 2025 16:18:35 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 2C157100055; Sun, 5 Jan 2025 16:18:28 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736111907; bh=fDjxeE+U77Z+ML6qA2S4XT8nf0U10x0ogpwE3fSAb2A=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=CjKSgtd0lzrxxUBlaoRuIxC8gW6Whu0u01eWQ/CoLjPTU+vMfBzzMTb/780He7wyp gRbVrkQyvvpdUUQBb6oR3x1h25I+2oI+H4MMjhvOVjF7ui2TWq2comVCAaL7qT/6b3 C4P+WkuWWjtYozjqp0ozAhY5J6yHR92GTwaKmsFF1gGgVbQTRMOp50l4A0eKkdKM90 a+Lv+3/F9IMeXjQb6JC+JDu5XMNLKCRMEXu8cvAmmKzxhfKeQB/LtgpyZWKuCgsw42 KNcGWTa649VjKedKlGlBrjuG8ThSZuT0a9IOBeQvltkUqjSA7HJf05twhrm6dU2YaS C7ByDeM7bWATA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 77A10100035; Sun, 5 Jan 2025 16:18:27 -0500 (EST) Received: from alfajor (104-195-230-250.cpe.teksavvy.com [104.195.230.250]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 33709120588; Sun, 5 Jan 2025 16:18:27 -0500 (EST) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86r05h6vsl.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 05 Jan 2025 18:59:54 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <86cyh18nee.fsf@gnu.org> <874j2dv3x4.fsf@thornhill.no> <86r05h6vsl.fsf@gnu.org> Date: Sun, 05 Jan 2025 16:18:24 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.307 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, Theodor Thornhill , 73404@debbugs.gnu.org, juri@linkov.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > For now, my conclusion was that (almost) every parenthetical grouping > supported by the language grammar can be moved across with these > commands. But I only tried Lisp and c-ts-mode. Agreed, "list" should probably be anything that makes up an AST node and is delimited by an opening and a closing "keyword": (...), [...], if...fi, begin...end, ... Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 02:58:47 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 07:58:47 +0000 Received: from localhost ([127.0.0.1]:36556 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUi0c-0004Rb-KI for submit@debbugs.gnu.org; Mon, 06 Jan 2025 02:58:46 -0500 Received: from relay3-d.mail.gandi.net ([2001:4b98:dc4:8::223]:60809) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUi0a-0004RL-HX for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 02:58:45 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id E03AB60008; Mon, 6 Jan 2025 07:58:34 +0000 (UTC) From: Juri Linkov To: Theodor Thornhill Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87sepxt4yl.fsf@thornhill.no> (Theodor Thornhill's message of "Sun, 05 Jan 2025 20:50:58 +0100") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> Date: Mon, 06 Jan 2025 09:54:49 +0200 Message-ID: <87o70k9y2e.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Eli Zaretskii , mickey@masteringemacs.org, casouri@gmail.com, monnier@iro.umontreal.ca, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > However, deciding on what makes sense is hard, which is why I'm > suggesting we try to define a "spec". It needn't really be much more > than reference suggestions for mode implementors at first, then maybe > some intersection of features crystallize across all modes. I don't think it's possible to find a spec better than Stefan already defined as an AST node delimited by an opening and a closing "keyword": (...), [...], if...fi, begin...end, ... >> Indeed, sentences and paragraphs are even more fuzzy concepts when applied >> to programming languages. But some modes make sense of these things >> nevertheless. For example, in both c-mode and c-ts-mode 'M-e' >> moves to the end of statement that usually end with a semicolon; >> so your intuition about these things looks correct. > > Yes, but imo more importantly, disregarding any confusing naming we have > a very rich set of operations available to us that I think should be as > distinct as possible: > > - C-M-f > - M-f > - M-a > - M-e > - C-f > - C-a > - C-e > - more Or when looking at the commands with the 'forward-' prefix: forward-list (C-M-n) forward-page (C-x ]) forward-paragraph (M-}) forward-sentence (M-e) forward-sexp (C-M-f) forward-symbol forward-word (M-f) Interesting that a potentially useful 'forward-symbol' has no binding. > If we'll aim to get the absolute most out of the parse tree > possibilities we should strive to make these distinct, at least down the > line. Some of these are simpler to categorize than others, such as a > sentence meaning "some expression that ends that isn't a function or a > class definition". In languages with statements a sentence corresponds to a statement. Then a paragraph is a less useful concept since a group of statements compose a block that corresponds to a list, not a paragraph. Then probably the definition of a paragraph should remain as a group of lines separated by empty lines. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 03:56:57 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 08:56:57 +0000 Received: from localhost ([127.0.0.1]:36613 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUiuv-0007E0-8z for submit@debbugs.gnu.org; Mon, 06 Jan 2025 03:56:57 -0500 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:58536) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tUius-0007Ds-S0 for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 03:56:56 -0500 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-218c8aca5f1so1100615ad.0 for <73404@debbugs.gnu.org>; Mon, 06 Jan 2025 00:56:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736153814; x=1736758614; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=QWZQDX+LKDPJ0ZMkNiPSReL0pd048Qww455wsP/GB1E=; b=TH49qXhMn044NPF+xbwPvfaRY4ETc+Swtv2GPb0l0vbvGxOf7ead4bhXblNbjxEk4j jTSzfioEoIK3dVbwlYqNYW6XSVgp+n25LYb2hDEw8Z/yOH2HZbrqyyUDjfvGEPQn+9Qw tW3man3qMNi3cnbiOVBL3k0avtItAy4YyXXAA9AVS8wxmtcZq1Oh8JMFl8Eyqp5H7zPu 1NmL4r5nTQ1F/9fs5dj39dYF7G4vG/jILwWrC9box4xQ1qzfKpprLb/8S3kLD2uJBAUr zmQjadRjfywRQr4FLXJrBu+rEHSeaJww0oEes/SJ0u8hq3cLuCqseqGeDPqRO0i0RZOT P5XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736153814; x=1736758614; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QWZQDX+LKDPJ0ZMkNiPSReL0pd048Qww455wsP/GB1E=; b=aP985fZ3/3oFydbzXk8XpTNtss77qRsS7LrMSKJ+aDElLCnekNo2ngF6Km8Xv+G6W/ 5exiwxM0AhdLrBrdK0Xa6lag4tU9H2iQYSRsDAX/qsLdAAHQrWdDfPHgcZfdAz6TeMwc Taj64uBXDMXQe4wtvhB9TgdXYepHtdDjZValIz5eB6fCN0u2UKHL3QHQWly/bzz7dzNs OiyGilWqe7E/Ls2aipbq3Zqe5BevcWOwz9HlBvqDNrqm7i7Eer3OGlevZQeBNmao4MT6 YYLdGQLM/Vi7g2iE8PpKQs5tTeooEYrTeQpalqEajisHFS4jA7Dgiuh//PX5dqm1fmON bwpQ== X-Forwarded-Encrypted: i=1; AJvYcCW/98+d7xidhj9uPpSbAybpff50fXsZ9MKdyNqFjWCXLgxDxypEKYBQxzqkREl9QGsgv/KKzg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YzwPXfl30VHDgHsdH8yMJAmkQ1lbeJtBCj7PDqkaNont8QjONcS If0Oxj55PGmcPVISkSXQIMWBvgpg8kXAQZf+Oyx/s62YSoaOTd35 X-Gm-Gg: ASbGncu6346SYTPkflgsVGW5HTKy4had2r+7RViXUYnISEZ2C8CrIzyKvL9keeTclZN PGiB0UVwGNNtlyQe3bWLWIi/+r/LhKojaQ/NP9ehMWelU+L3vsl/JJ84xmf7S7NrFgF7iEagpec ONwrXvQt2MulS/4Vsz0oSZDggCgdqINrVUsYznRP+KB64NKN18GkxXykq4x60M/+qEK5YdyUl76 eyR58YUQQ0dgdApDN2XyN9r9HU4L9IhZQFdA1Jzb6jcqkrKP7Wol/I90m68GZW78ZF/iIOPJcFK YKS1 X-Google-Smtp-Source: AGHT+IGSU04G2oQDYCPdPwB7AFGK5AVRY8Nm+69o8pQu272USjiLFcEXrmkXlFs/MYZc38JozCgw+A== X-Received: by 2002:a17:902:fc84:b0:216:356b:2685 with SMTP id d9443c01a7336-219e6e8bb95mr901982905ad.11.1736153813619; Mon, 06 Jan 2025 00:56:53 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:85eb:71bd:58ac:56a8]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc962c75sm287871685ad.24.2025.01.06.00.56.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jan 2025 00:56:52 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: Date: Mon, 6 Jan 2025 00:56:41 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <87frpm20t7.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> To: Stefan Monnier X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Jan 5, 2025, at 6:40=E2=80=AFAM, Stefan Monnier = wrote: >=20 >> I admit that hard-coding one symbol is not the right thing to do. >> So here is a better patch that checks for the symbol property: >=20 > That's better but I still wonder why we don't want to use a keyword. >=20 >=20 > Stefan >=20 My impression is that Elisp has =E2=80=9Ctraditionally=E2=80=9D used = symbols? Like symbol properties, text properties, thing-at-point, etc. = Keywords are really only seen in plists, face attributes, and keyword = arguments, all of which involves a key-value relationship. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 09:15:12 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 14:15:12 +0000 Received: from localhost ([127.0.0.1]:37417 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUnsu-0006U4-09 for submit@debbugs.gnu.org; Mon, 06 Jan 2025 09:15:12 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:24808) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUnsr-0006Sd-Qc for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 09:15:10 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id AC7071000EF; Mon, 6 Jan 2025 09:15:02 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736172902; bh=/c2zCZ92ZIV44/zElbFY+XgWtx3FQGy57i9TLy9FyTw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=gR2X+SliHn6atJYouQAo2+FOjBYW3vW5K2qkWUhvPDR1fHOt89UCTxvtLpVjeaj6t FJoNqZAliSqYTymH2Z0QxtnVLFIHk64j/EENz+BEtAmt9rtCzhdRMl2mlSQOM1N7mz zJPwNmODmudWS9z4zR2yQW9C6RLfIzy03SpQ+wIQ8yEMLf6g5xKrjjL7cqDYrS1XDV cV2Lu7vqZi0gwpapxGrfLlzZUEcprT5nanDfjAbcFPnzmY4js4AqpgeGCcYxEi1zIO 5MpFYIpMhAp+jWAqjfsS0+vTNyHUKdvI/X0K8ZrHko1tK5z0Uw8CqRLXikn4+6LCzY VTf/3YJ/27PDg== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id F13FB100042; Mon, 6 Jan 2025 09:15:01 -0500 (EST) Received: from pastel (69-165-162-104.dsl.teksavvy.com [69.165.162.104]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B741D120511; Mon, 6 Jan 2025 09:15:01 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87o70k9y2e.fsf@mail.linkov.net> (Juri Linkov's message of "Mon, 06 Jan 2025 09:54:49 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> Date: Mon, 06 Jan 2025 09:14:58 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> However, deciding on what makes sense is hard, which is why I'm >> suggesting we try to define a "spec". It needn't really be much more >> than reference suggestions for mode implementors at first, then maybe >> some intersection of features crystallize across all modes. > > I don't think it's possible to find a spec better than Stefan already > defined as an AST node delimited by an opening and a closing "keyword": > (...), [...], if...fi, begin...end, ... IIUC, Theodor was talking more generally than just about "list" operations. I think "list" operations are among the clearest case (BTW, I wonder if someone has already tried to implement that functionality for Haskell or Python, where the opening and a closing "keywords" are basically newlines). > Or when looking at the commands with the 'forward-' prefix: > > forward-sexp (C-M-f) > forward-list (C-M-n) OK, we already talked about these two. We could probably start drafting a kind of guideline. > forward-symbol This one doesn't look too hard either. > forward-sentence (M-e) This one is trickier. You just provided a reasonably good start with "In languages with statements a sentence corresponds to a statement", but there are some non-trivial questions about language without statements, about use within comments, about how to handle nesting (e.g. statements within statements) and more generally the use of `M-e` from outside of statements. > forward-paragraph (M-}) This one is also tied to filling, so it's important for it to behave well w.r.t to comments to allow filling actual text paragraphs within comments. As for its behavior outside of comments ... > forward-page (C-x ]) > forward-word (M-f) I think these should be oblivious to the syntax (i.e. line `forward-line` and `forward-char`). Your list missed the "defun" unit of navigation. One other potentially useful unit of navigation that depends on the syntax would be "list element", which would do something like skip to the next/previous separator delimiting elements of the nearest enclosing paired delimiters. For {...;...;...} blocks, this would behave like `forward-sentence`, except that when used from within the args of a function call like a `foo (..., ..., ...)`, it would jump between arguments instead of skipping all the way to the next statement. Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 12:50:26 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 17:50:26 +0000 Received: from localhost ([127.0.0.1]:39954 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUrFC-0001Ii-0w for submit@debbugs.gnu.org; Mon, 06 Jan 2025 12:50:26 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:43979) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUrF9-0001I5-Q5 for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 12:50:24 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 5961EFF80A; Mon, 6 Jan 2025 17:50:13 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Mon, 06 Jan 2025 09:14:58 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> Date: Mon, 06 Jan 2025 19:40:39 +0200 Message-ID: <87plkzyh60.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >>> However, deciding on what makes sense is hard, which is why I'm >>> suggesting we try to define a "spec". It needn't really be much more >>> than reference suggestions for mode implementors at first, then maybe >>> some intersection of features crystallize across all modes. >> >> I don't think it's possible to find a spec better than Stefan already >> defined as an AST node delimited by an opening and a closing "keyword": >> (...), [...], if...fi, begin...end, ... > > IIUC, Theodor was talking more generally than just about "list" operations. > I think "list" operations are among the clearest case (BTW, I wonder if > someone has already tried to implement that functionality for Haskell or > Python, Recently I looked at python-ts-mode, but then discovered that it completely misses any use of treesit-thing-settings unlike many other ts-modes. Also there is no treesit-thing-settings yet in the new haskell-ts-mode. > where the opening and a closing "keywords" are basically newlines). The main problem with empty opening and closing keywords is that nodes from different levels all end up at the same position, and currently treesit prefers a higher-level node. This means that the sequence of `C-M-f C-M-b` is not idempotent anymore: `C-M-b` might jump to the start of a higher-level node. >> Or when looking at the commands with the 'forward-' prefix: >> >> forward-sexp (C-M-f) >> forward-list (C-M-n) > > OK, we already talked about these two. We could probably start drafting > a kind of guideline. Also in treesit.el, forward-sexp and forward-list could share the same implementation. >> forward-symbol > > This one doesn't look too hard either. > >> forward-sentence (M-e) > > This one is trickier. You just provided a reasonably good start with > "In languages with statements a sentence corresponds to a statement", > but there are some non-trivial questions about language without > statements, about use within comments, about how to handle nesting > (e.g. statements within statements) and more generally the use of `M-e` > from outside of statements. I've checked that `M-e` in text-mode currently doesn't support sentences in nested lists. For example, `M-e` jumps inside the list in: Top sentence. (First inner sentence. Second inner sentence.) >> forward-paragraph (M-}) > > This one is also tied to filling, so it's important for it to behave > well w.r.t to comments to allow filling actual text paragraphs > within comments. This provides an opportunity to move treesit-specific code from `prog-fill-reindent-defun` to treesit.el. > As for its behavior outside of comments ... > >> forward-page (C-x ]) >> forward-word (M-f) > > I think these should be oblivious to the syntax (i.e. line `forward-line` > and `forward-char`). > > Your list missed the "defun" unit of navigation. Because there is no `forward-defun` ;-) But strange that "defun" still is defined by `treesit-defun-type-regexp` instead of a "defun" thing in `treesit-thing-settings`. > One other potentially useful unit of navigation that depends on the > syntax would be "list element", which would do something like skip to > the next/previous separator delimiting elements of the nearest enclosing > paired delimiters. For {...;...;...} blocks, this would behave like > `forward-sentence`, except that when used from within the args of > a function call like a `foo (..., ..., ...)`, it would jump > between arguments instead of skipping all the way to the next statement. Indeed, there is a need to have an additional command to move between more ATS nodes, besides the list nodes navigated by 'C-M-n'. But the problem is to find free keys for more commands. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 13:04:46 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 18:04:46 +0000 Received: from localhost ([127.0.0.1]:39983 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUrT4-0001wA-Dn for submit@debbugs.gnu.org; Mon, 06 Jan 2025 13:04:46 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:60721) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tUrT1-0001vp-D3 for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 13:04:44 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-21654fdd5daso197066125ad.1 for <73404@debbugs.gnu.org>; Mon, 06 Jan 2025 10:04:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736186677; x=1736791477; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=efZLgDElrVvXzbpRd1K6AyqkHRYp8qoiDlTUyZFOHvk=; b=YGx7pCU+wZB8sQpUD3jEEja996F9GK00imGFjFGES+scfNkJiL95z5DHlfPqwaCvsA JAp2dWmlE9yjEl8NAhe5S0glpbfTygHHahUkA1GCFheIwEsayb21+qYwKY2JMbREePCo QiigAoiSBbffVQ7x+MHV9wEk10MCw7l6+xFVCI/UVW8S5pfiFMn9CfRu/NQ50x38dqNQ 5VU5Mj+bK7SMraShsA9LMnu6p2ZFJ36j/O9D1LSrjabABfb8q0v4g3u77INUWmp7UacN LmX1MlckJZwDdz5+fkwDVZQzrzbJ+W+HOhzVhOV4YnVnhfGKOy9LTxRIED6YE2zPj+Fn Th7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736186677; x=1736791477; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=efZLgDElrVvXzbpRd1K6AyqkHRYp8qoiDlTUyZFOHvk=; b=OH2GcL/Y7SARhd96NVwImtpI7Tx/6VCn86vQEXSTU3zy2fbBaEaO8SOzE1AF+9S//I 2QN4DayWYHBbyZeto6S5ft7v8o53X1p9rBYvGEhZioP2TaGyoZ2/v4YGq0CwDLwr7WOT QfQJQ53C3T9+SPRFNHp27PVYACJDQzMFfsEuY5YUBzhmtTsveehDaunU4mb/yqaiu9l+ ELJK2pglUM0EDZ9GCQ+779069q8EB9FVIb3QykZGtKPI6kNH+WFlFWOvy/wu6AZdbH2Z QS57Z2Heqfjckp2jw8qza+T9RKYkfotruBy4uYu0cmG/6RTcwiwrF2eSDE38ts6bvbww v+gg== X-Forwarded-Encrypted: i=1; AJvYcCVWyw9jN7xbekDMp/9yDg0CmZ+gFxT+db14hhhTBAuAWz6cTl7rwPNHFWebwCid7txCK75qAg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyymeHF2itFtb6j8Vsz94OTZ2fpWPBCkTmriVMA/4q+kDNPHOLo RlBsJDEOVkyI/WFPtuDRVAat8YnUomzkocSsALCKvsZUCmNXNBol X-Gm-Gg: ASbGncuhUTE1vC8BlKKOIg18mOr4FsRdsg1crXxnIbBdbbzDGoWx9pngn/bnMhHYY9T gvCvrSv7vlkILgJFyunb0K/NYIZhy8rkK53H4UGHmcrmrgWgIzgFjkx/dMmv9HAfAfiHUZ57KR+ 38ScE/qBuuYYUb92g6T8Nh4XCnoRnvNMe26ggBldX9cIqoGeKgD+doF7t1TwyvN0eODE8nnNV+E IB90/6BltlbmnucEgoxmzJAXzLMUsrEuISKqcRfsfygh0pQqogKzbHWjpp8rvOlVjtL5Y3PzZZ/ mIpr X-Google-Smtp-Source: AGHT+IHKlR4J6K4kkPUfA4f6J6aaw5tZjxw69bTIP6lDNVAlZySo20MWiCM+OkQMgNWDQNNZkz79kQ== X-Received: by 2002:a05:6a21:6da4:b0:1e1:c03c:b420 with SMTP id adf61e73a8af0-1e5e07f9731mr107184547637.31.1736186676941; Mon, 06 Jan 2025 10:04:36 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:c496:8cdf:4c5d:3617]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad830349sm31504571b3a.47.2025.01.06.10.04.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jan 2025 10:04:36 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87plkzyh60.fsf@mail.linkov.net> Date: Mon, 6 Jan 2025 10:04:24 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <41B5B189-0A51-49A5-B0B3-A745F7668BDC@gmail.com> References: <87plox4mtp.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>=20 >> I think these should be oblivious to the syntax (i.e. line = `forward-line` >> and `forward-char`). >>=20 >> Your list missed the "defun" unit of navigation. >=20 > Because there is no `forward-defun` ;-) >=20 > But strange that "defun" still is defined by = `treesit-defun-type-regexp` > instead of a "defun" thing in `treesit-thing-settings`. treesit-defun-type-regexp is shipped in Emacs 29, so we can=E2=80=99t = just remove it. Defun navigation functions in treesit.el does support = the defun thing, but I made treesit-defun-type-regexp an override of the = defun thing. OTOH treesit-sexp/sentence-type-regexp wasn=E2=80=99t = shipped in Emacs 29 so I removed them, preferring the thing definition. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 13:06:04 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 18:06:05 +0000 Received: from localhost ([127.0.0.1]:39991 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUrUJ-00025K-V2 for submit@debbugs.gnu.org; Mon, 06 Jan 2025 13:06:04 -0500 Received: from relay5-d.mail.gandi.net ([2001:4b98:dc4:8::225]:50897) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUrUG-00024i-08 for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 13:06:01 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 3F78B1C0006; Mon, 6 Jan 2025 18:05:51 +0000 (UTC) From: Juri Linkov To: Eli Zaretskii Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <86cyh16nq3.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 05 Jan 2025 21:54:12 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <867c798lco.fsf@gnu.org> <871pxhp24e.fsf@mail.linkov.net> <86cyh16nq3.fsf@gnu.org> Date: Mon, 06 Jan 2025 20:02:36 +0200 Message-ID: <87bjwjyg5f.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Another example on line 152 in dispnew.c: > > struct redisplay_history > { > char trace[512 + 100]; > }; > > Move point to the [ before 512: C-M-n will signal an error. > > Any bracket in the file will trigger either the first or the second > behavior. Sorry, I didn't foresee that you would expect C-M-n and C-M-f to behave the same way regarding list motion. This is fixed now by syncing treesit-forward-list with treesit-forward-sexp-list, so all your test cases work correctly now. >> > w32_get_internal_run_time (void) >> > { >> > if (get_process_times_fn) >> > { >> > FILETIME create, exit, kernel, user; >> > HANDLE proc = GetCurrentProcess (); >> > if ((*get_process_times_fn) (proc, &create, &exit, &kernel, &user)) >> > { >> > return ltime (total.QuadPart); >> > } ^ >> > } >> > >> > return Fcurrent_time (); >> > } >> > >> > If you put point on the semi-colon marked by "^", C-M-n signals an >> > error. Can't we instead move to the next "list" after >> > "Fcurrent_time"? >> >> 'C-M-n' doesn't move to the next statement in c-mode, >> since we need to support backward-compatibility. The key >> that moves to the next statement ignoring nested lists >> in C modes is 'M-e'. > > Well, then maybe we could have the more useful behavior as an opt-in > option? Do you also need an option to move in Lisp from 1 to 2 in: (compound_statement (if_statement (compound_statement (return_statement 1))) (return_statement 2)) >> C-M-n is unable to move to the first group because >> it's inside the node in the AST: >> >> (cast_expression "(" >> type: (type_descriptor type: (type_identifier)) >> ")" >> value: >> (call_expression function: (identifier) >> arguments: >> (argument_list "(" (identifier) , >> (string_literal " (string_content) ") >> ")"))) >> >> Both the first and the 2nd parenthesized groups are inside the >> "cast_expression" node. > > Sorry, I don't follow: the parentheses of the argument list are also > inside a node, and yet we do recognize them. What's the difference? The problem is that ")" is not the final child of 'cast_expression'. So C-M-f/C-M-n can't stop after ")". All these problems stem from the imperfection of ts grammars. It would be nice if someone will find a way to modify the grammars after importing them to Emacs core, to be able to insert more nodes. This will solve a whole class of problems. Otherwise, if vendoring grammars is not feasible, an alternative would be to define a new treesit-thing like e.g. "paren" that will match anonymous nodes like: (setq-local treesit-thing-settings `((c (paren ,(rx (or "{" "}" "[" "]" "(" ")"))) Then low-level treesit functions such as treesit-parent-until will have to check if there a preceding sibling that matches "paren" before using their real AST parent node. This means maintaining a parallel virtual tree. >> > As for terminology: the Emacs user manual calls these "groupings >> > delimited by parentheses (or whatever else serves as delimiters in the >> > language you are working with)", or in short "parenthetical group". >> > Why cannot we keep this terminology? It sounds like it describes its >> > subject as accurate as we can come up with. >> >> So you prefer "group" over "list"? > > As a shorthand; the better term is "parenthesized group". By definition a list is a parenthesized group. The term "list" is much shorter than its synonym "parenthesized group", so I see no reasons not to use "list". >> Then 'forward-list' should move over "group"? > > I'm not sure we should rename the commands, given the legacy. But > the doc string should use "group" or "parenthesized group", IMO. The doc string of 'forward-list' already uses this: (forward-list &optional ARG INTERACTIVE) Move forward across one balanced group of parentheses. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 13:45:41 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 18:45:42 +0000 Received: from localhost ([127.0.0.1]:40070 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUs6f-00046U-6j for submit@debbugs.gnu.org; Mon, 06 Jan 2025 13:45:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34692) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUs6c-000469-Bm for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 13:45:39 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUs6V-00047B-9U; Mon, 06 Jan 2025 13:45:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=RoEPcco1Vk7aNGqkKp8Yn1uBmHuJ9BtYs3txatQtEyM=; b=eUsDoAqEzWuw tOj6uSYvR/k+AgLo8K4IhOSGaRX5iKem7ZvVP9ifgTmhiLExPcaiab/45jYnUN5GchDhj51mZgeKK RChXZcE39BHzvEvnJltwUcPDQEfc/6MjVJUFi/I9VoYhZueFMQtOjuiCGlmp46of6BLv91ks6y5bM GTaXBcyh4cOZF+duuCceuEBXgU9UZ69z2idOzhQLaMctNxL05DV1rgAVRLCJnqwH2zNIH3uUldpoB 74Bfb7MNxnhO6nCMyUdLCVyZuTG8+hcWmO7fNL+83TWXl8ZvnuB9YtCuZ/AA8PNSLamCH5chlPq9w WGnf7p1KdTFYj183zWdr1A==; Date: Mon, 06 Jan 2025 20:45:14 +0200 Message-Id: <86a5c36ath.fsf@gnu.org> From: Eli Zaretskii To: Juri Linkov In-Reply-To: <87bjwjyg5f.fsf@mail.linkov.net> (message from Juri Linkov on Mon, 06 Jan 2025 20:02:36 +0200) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes References: <87plox4mtp.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <867c798lco.fsf@gnu.org> <871pxhp24e.fsf@mail.linkov.net> <86cyh16nq3.fsf@gnu.org> <87bjwjyg5f.fsf@mail.linkov.net> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, casouri@gmail.com, theo@thornhill.no, 73404@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Juri Linkov > Cc: casouri@gmail.com, theo@thornhill.no, mickey@masteringemacs.org, > 73404@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Mon, 06 Jan 2025 20:02:36 +0200 > > > Another example on line 152 in dispnew.c: > > > > struct redisplay_history > > { > > char trace[512 + 100]; > > }; > > > > Move point to the [ before 512: C-M-n will signal an error. > > > > Any bracket in the file will trigger either the first or the second > > behavior. > > Sorry, I didn't foresee that you would expect C-M-n and C-M-f to behave > the same way regarding list motion. This is fixed now by syncing > treesit-forward-list with treesit-forward-sexp-list, so all your > test cases work correctly now. Thanks. > >> > w32_get_internal_run_time (void) > >> > { > >> > if (get_process_times_fn) > >> > { > >> > FILETIME create, exit, kernel, user; > >> > HANDLE proc = GetCurrentProcess (); > >> > if ((*get_process_times_fn) (proc, &create, &exit, &kernel, &user)) > >> > { > >> > return ltime (total.QuadPart); > >> > } ^ > >> > } > >> > > >> > return Fcurrent_time (); > >> > } > >> > > >> > If you put point on the semi-colon marked by "^", C-M-n signals an > >> > error. Can't we instead move to the next "list" after > >> > "Fcurrent_time"? > >> > >> 'C-M-n' doesn't move to the next statement in c-mode, > >> since we need to support backward-compatibility. The key > >> that moves to the next statement ignoring nested lists > >> in C modes is 'M-e'. > > > > Well, then maybe we could have the more useful behavior as an opt-in > > option? > > Do you also need an option to move in Lisp from 1 to 2 in: > > (compound_statement > (if_statement > (compound_statement > (return_statement 1))) > (return_statement 2)) It sounds more useful to me than signaling an error, yes. > It would be nice if someone will find a way to modify the grammars > after importing them to Emacs core, to be able to insert more nodes. > This will solve a whole class of problems. Yes, an ability to modify the grammars and recompile them into the library would be great. > >> So you prefer "group" over "list"? > > > > As a shorthand; the better term is "parenthesized group". > > By definition a list is a parenthesized group. > The term "list" is much shorter than its synonym > "parenthesized group", so I see no reasons not to use "list". The problem with "list" is that it is not clear what it means in languages that are not Lisp-like. But we can keep that in the names of commands and variables, IMO. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 15:07:47 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 20:07:47 +0000 Received: from localhost ([127.0.0.1]:40218 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUtO7-0008Bp-Bx for submit@debbugs.gnu.org; Mon, 06 Jan 2025 15:07:47 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:53213) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUtO4-0008BO-8j for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 15:07:45 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 17AE180913; Mon, 6 Jan 2025 15:07:38 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736194057; bh=bwTTYJ8F3u3Izx9/WhavTWnkLw738sgxIX9otPwNUWY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=R+GkePrSDf2APKqqL4G95ccHnmzAyHiLHcO+NwUs5OdNlgwilDCuCSzE+lvnTMuTA zfovzFVXeGKSn5kMuBCFn05zOY1xUsY+2wXEJEv3c5j2Ab4rJqhV6V7AzXe6Wy8KK/ UfRdAGVqkfqJ1BYrYIvAWODYkppDmMPQljaQt/hD4t8POlOORDq2MBqYDqBN+4rZXw ypXVznajnrOVFW4qVLjT7mepHoWOXpGvR4opjAVkn+j8XLSHUysFwf+uFMPl6Hl+J0 VLiEnzqFVxe91yRkXdqhD5q/VuO0D0e+gCeBbOVEfCYDztpn8sb+x003OG/jmg5N2M nRsYAteAzdceA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 458B680394; Mon, 6 Jan 2025 15:07:37 -0500 (EST) Received: from pastel (69-165-162-104.dsl.teksavvy.com [69.165.162.104]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 0751E12049E; Mon, 6 Jan 2025 15:07:36 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87plkzyh60.fsf@mail.linkov.net> (Juri Linkov's message of "Mon, 06 Jan 2025 19:40:39 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> Date: Mon, 06 Jan 2025 15:07:28 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > The main problem with empty opening and closing keywords is that > nodes from different levels all end up at the same position, > and currently treesit prefers a higher-level node. This means > that the sequence of `C-M-f C-M-b` is not idempotent anymore: > `C-M-b` might jump to the start of a higher-level node. It's never been idempotent, tho, e.g. in cases like: ;; Starting from before this comment ;; `C-M-f C-M-b` will end up after it. (hello world) or + x But indeed for list operations in indentation-based languages like Haskell/Python/Coffeescript/YAML, we're faced again with the fact that a closing LF can close several nested lists, so when going backward there is some amount of ambiguity (and I'd argue that it's preferable (and closer to the historical behavior) to choose to jump less far, e.g. so that `C-M-n C-M-p` never moves backward). > Also in treesit.el, forward-sexp and forward-list could share the > same implementation. [ I assume you mean "share *some* of the implementation", rather than being literally aliases of each other. ] Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 06 15:19:52 2025 Received: (at 73404) by debbugs.gnu.org; 6 Jan 2025 20:19:52 +0000 Received: from localhost ([127.0.0.1]:40240 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUtZn-0000hV-Ks for submit@debbugs.gnu.org; Mon, 06 Jan 2025 15:19:52 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:28749) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUtZg-0000h3-FK for 73404@debbugs.gnu.org; Mon, 06 Jan 2025 15:19:49 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 749E380913; Mon, 6 Jan 2025 15:19:38 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736194777; bh=rChCcduHPl5aovbJAlApgkL/BlmIBK81VxNpW9v98y0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=aWvXzlKEzvveY+8h1fzAq2ghz+L8FsKVqWTmaXdCH8+wgKheyklqtYD0NRFlJk+iS kh63inujE1NWvZ8Zhkd0ErG7yoF/EpVKjmzvZ1xYn1PXZ2dpuHYwLZa+KhWXvXpH2y EZINI4Rv759s1QCLv2va0nq9EYKYPShsU6Jc7lp6vI/Rz/JqLRSUV3g4uorPuijcuy rlz36EdKsdd7FYmkPk1ZH3vYTwNUsjetdZYL8/EhnBt2xNYtWsnf6c71woe3BO3M+J rv4FzX2NAAM+NwXmdA45Zk4Pnfwl/3tTO7zZtCB1OMMCqmYwY2VI7rHfs62EW0WVGK UQy32TK5O7aKA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9FF9B806F8; Mon, 6 Jan 2025 15:19:37 -0500 (EST) Received: from pastel (69-165-162-104.dsl.teksavvy.com [69.165.162.104]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6529812029A; Mon, 6 Jan 2025 15:19:37 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87bjwjyg5f.fsf@mail.linkov.net> (Juri Linkov's message of "Mon, 06 Jan 2025 20:02:36 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <867c798lco.fsf@gnu.org> <871pxhp24e.fsf@mail.linkov.net> <86cyh16nq3.fsf@gnu.org> <87bjwjyg5f.fsf@mail.linkov.net> Date: Mon, 06 Jan 2025 15:19:35 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , theo@thornhill.no, casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > The problem is that ")" is not the final child of 'cast_expression'. > So C-M-f/C-M-n can't stop after ")". > > All these problems stem from the imperfection of ts grammars. I'm not sure if I'd call it "imperfection". There are tradeoffs. > It would be nice if someone will find a way to modify the grammars > after importing them to Emacs core, to be able to insert more nodes. > This will solve a whole class of problems. I think some way to massage the parse tree would be useful (maybe by massaging the grammar, but not necessarily). The `cast_expression` is one place where we could use it, but it would also help us deal with the fact that grammars can change outside of our control and we may want our major modes to support various versions of a grammar. > Otherwise, if vendoring grammars is not feasible, an alternative > would be to define a new treesit-thing like e.g. "paren" that > will match anonymous nodes like: > > (setq-local treesit-thing-settings > `((c > (paren ,(rx (or "{" "}" "[" "]" "(" ")"))) > > Then low-level treesit functions such as treesit-parent-until > will have to check if there a preceding sibling that matches "paren" > before using their real AST parent node. This means maintaining > a parallel virtual tree. I'd prefer something more targeted, i.e. some way for the major mode to add "extra rules" to explain that some part of `cast_expression` should also be treated as a `list`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 07 12:58:01 2025 Received: (at 73404) by debbugs.gnu.org; 7 Jan 2025 17:58:01 +0000 Received: from localhost ([127.0.0.1]:44505 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVDq4-0004VV-UT for submit@debbugs.gnu.org; Tue, 07 Jan 2025 12:58:01 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:45935) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVDq4-0004VD-2C for 73404@debbugs.gnu.org; Tue, 07 Jan 2025 12:58:00 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id C11541C0002; Tue, 7 Jan 2025 17:57:50 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> (Yuan Fu's message of "Mon, 6 Jan 2025 00:56:41 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <8634lmbs8t.fsf@gnu.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> Date: Tue, 07 Jan 2025 19:56:11 +0200 Message-ID: <87ldvmik3o.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >>> I admit that hard-coding one symbol is not the right thing to do. >>> So here is a better patch that checks for the symbol property: >> >> That's better but I still wonder why we don't want to use a keyword. > > My impression is that Elisp has “traditionally” used symbols? Like > symbol properties, text properties, thing-at-point, etc. Keywords are > really only seen in plists, face attributes, and keyword arguments, > all of which involves a key-value relationship. Indeed, symbols are more traditional. However, the problem arose only because 'treesit_traverse_validate_predicate' accepts a symbol for both a function and a thing, that caused ambiguity. So there are two variants to resolve this ambiguity: 1. use keywords like (setq-local treesit-thing-settings `((html (:sexp ,(regexp-opt '("element" "text" "attribute" "value"))) (:list ,(regexp-opt '("element""))) (:sentence "tag") (:text ,(regexp-opt '("comment" "text")))))) 2. use a symbol property like (put 'list 'treesit-thing t) You decide ;-) From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 07 13:07:28 2025 Received: (at 73404) by debbugs.gnu.org; 7 Jan 2025 18:07:28 +0000 Received: from localhost ([127.0.0.1]:44551 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVDzE-00054u-8D for submit@debbugs.gnu.org; Tue, 07 Jan 2025 13:07:28 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:34157) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVDzC-00054X-5J for 73404@debbugs.gnu.org; Tue, 07 Jan 2025 13:07:26 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 07DEB1C0004; Tue, 7 Jan 2025 18:07:18 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Mon, 06 Jan 2025 15:07:28 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> Date: Tue, 07 Jan 2025 20:05:40 +0200 Message-ID: <87bjwiijnv.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > But indeed for list operations in indentation-based languages like > Haskell/Python/Coffeescript/YAML, we're faced again with the fact that > a closing LF can close several nested lists, so when going backward > there is some amount of ambiguity (and I'd argue that it's preferable > (and closer to the historical behavior) to choose to jump less far, > e.g. so that `C-M-n C-M-p` never moves backward). I had an idea to keep the currently navigated level in an internal variable, so `C-M-n C-M-p` will always return back. But this doesn't look reliable. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 07 14:19:19 2025 Received: (at 73404) by debbugs.gnu.org; 7 Jan 2025 19:19:19 +0000 Received: from localhost ([127.0.0.1]:44708 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVF6l-0000nc-CF for submit@debbugs.gnu.org; Tue, 07 Jan 2025 14:19:19 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:26108) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVF6j-0000nH-NK for 73404@debbugs.gnu.org; Tue, 07 Jan 2025 14:19:18 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 8D1E380AA7; Tue, 7 Jan 2025 14:19:11 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736277550; bh=34ovqv7I7h9gNL+3IWaMacjkO0YYKdAq2WnnHtg6Pto=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=oVtPE1UgcLW0cmC15rTmdcg6bQ+5gqQXLzX0jxgDdBra7FTW5cXrAqWuKt1bavuaj MP1BOQIi1FWrUjkVGZCaG5TUPj7kuG/WX7+MkN1OoT2qBWwftUozheRecRrEZOOwVV QRpDCsl6MLx20sP+oF/AvMNjPKd34sfPKMcPa2aa+Y4mIfj0rwvPTzygbkkwZvFUbA 3TMXmtrD5PLn79EF3sAug10a5wcRPFnY1+AlGasahxXq1Q7LyxSJT0jEd/H6iQyJ+5 HUEclPFx4Dls1y3z4Sv776wmU5k6wBqXVtzCqc+V2LZUdwJhYjPstICaX8BGVUowsZ FA4/OxiZiiTNA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B798C80911; Tue, 7 Jan 2025 14:19:10 -0500 (EST) Received: from pastel (104-195-232-86.cpe.teksavvy.com [104.195.232.86]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 50695120642; Tue, 7 Jan 2025 14:19:10 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87bjwiijnv.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 07 Jan 2025 20:05:40 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> <87bjwiijnv.fsf@mail.linkov.net> Date: Tue, 07 Jan 2025 14:19:08 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.058 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I had an idea to keep the currently navigated level in an > internal variable, so `C-M-n C-M-p` will always return back. > But this doesn't look reliable. Not only it's not reliable, but it's not necessarily the behavior we want. After all, if you want to go back, you can say so explicitly by remembering the original position jumping back to it instead of hoping against all evidence that the navigation operations are mutual inverses. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 07 14:25:40 2025 Received: (at 73404) by debbugs.gnu.org; 7 Jan 2025 19:25:40 +0000 Received: from localhost ([127.0.0.1]:44721 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVFCt-0001FO-Ar for submit@debbugs.gnu.org; Tue, 07 Jan 2025 14:25:39 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:7308) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVFCr-0001Eu-Aw for 73404@debbugs.gnu.org; Tue, 07 Jan 2025 14:25:37 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 682EB441390; Tue, 7 Jan 2025 14:25:31 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736277926; bh=fUoBRtJyQGJrNGu7lfgNlg8AmmxBW83AsKxO9N3LBbg=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Pnc4rwhiJflRp9jbbwXpluORY2wRax/Dwqy1HPeCG40LMMTU0833RfWRQNRgblF7I jplFLG9Gyi6+IyTkWdnwm8TMJ9FEy4yrmxGxEljbaSmr/jk0zna1fhOZv7xJ+3A44B vwZRgU9uUFqXGuRMOTXygahxi7ppTVN+fWKUb7CrRR0GJ4xxbIRA2ikNiGH+2OL0Gl 7yoWAetPikp8u5K1dwlgu5L9XbLu8WIflxz8zTUM6A3YTNRzD5972ZygqQEndKhJXf WL8VtbeDYBxUZfHTvdts1oC6tYv3EOTlPC4OgIA7PZaSAkllrgaevr3lGK/mHYmFoD gdds3+iX/aliw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1B9A3441F5E; Tue, 7 Jan 2025 14:25:26 -0500 (EST) Received: from pastel (104-195-232-86.cpe.teksavvy.com [104.195.232.86]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id D36A2120399; Tue, 7 Jan 2025 14:25:25 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87ldvmik3o.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 07 Jan 2025 19:56:11 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> Date: Tue, 07 Jan 2025 14:25:24 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.023 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Yuan Fu , Mickey Petersen , Eli Zaretskii , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Indeed, symbols are more traditional. > > However, the problem arose only because > 'treesit_traverse_validate_predicate' accepts a symbol > for both a function and a thing, that caused ambiguity. > > So there are two variants to resolve this ambiguity: > > 1. use keywords like > > (setq-local treesit-thing-settings > `((html > (:sexp ,(regexp-opt '("element" "text" "attribute" "value"))) > (:list ,(regexp-opt '("element""))) > (:sentence "tag") > (:text ,(regexp-opt '("comment" "text")))))) > > 2. use a symbol property like (put 'list 'treesit-thing t) > > You decide ;-) Of course, there are more alternatives: - Use a symbol property like (put 'list 'treesit--this-is-a-function t) - Refuse functions represented as symbols (callers need to use an eta-wrapper). - Distinguish the two cases via a more verbose representation like `(thing list)` vs `(function list)`. - Use a string rather than a symbol. - Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 07 21:27:39 2025 Received: (at 73404) by debbugs.gnu.org; 8 Jan 2025 02:27:39 +0000 Received: from localhost ([127.0.0.1]:45386 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVLnH-0006Jf-2U for submit@debbugs.gnu.org; Tue, 07 Jan 2025 21:27:39 -0500 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]:46324) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tVLn9-0006JM-NC for 73404@debbugs.gnu.org; Tue, 07 Jan 2025 21:27:36 -0500 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-219f8263ae0so191211435ad.0 for <73404@debbugs.gnu.org>; Tue, 07 Jan 2025 18:27:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736303245; x=1736908045; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=D3XSK7TjEGX4nY/hjnHLidY9NcIMDPE6SHYqTjLTgdA=; b=Ttr7UUUnIqPji5hAl+3nigVdS8TPBimXQEaLezK/ThUSZhrA2h+FTyKfNL/qXNlqAA lR4h7+zLFztjP9cvr2Tp/c8tyDT9EYfSd60wciyrdXWmWv6w9O9EoYowsv8lWz5W+mJU iVGsnH4mzsqumocZsaxlPzCsMbJ/Ypwy5z3zdq3a2vklWf+aKt8c7dhXLui+yYFDgEiW 0405qN+XVHXJVUFNY7mDRUvz7UUusNfFzIcOADtkP69GeUkYDaC6eSQxPvuXid0Eo3hZ xcSPJnc5MGwVB6zj5Rr6nyoar711MtzTXcJhcsqtyHg29Y92MwVCM/kk5BGQ+AcuwIIi 37NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736303245; x=1736908045; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D3XSK7TjEGX4nY/hjnHLidY9NcIMDPE6SHYqTjLTgdA=; b=S4f2lDhx+D1CHwV3uuQVXgN9vDIIPRw26yyGljLSBxYbSry5lbPTGLQfqkCcN+mNa4 eEwwWERbVVbA0cNNE6IpVVHUEPWIaERRSHLqT3WmT7ii+aRqXx+tvi6DT++q4uvCsFbM qs/BmN6Z2/QObs2oy8/RpAnY6s5Za6gvBM56ZDiPe9vAON2UaaxbOZwI1csl1ZBzB0ap un92fJcu1EZvIJdLOMNCrs+HBiDpEw3NwGPLz4HIFcpG9TpLZFcYy73k4pcViW3Fd1U6 cSLtaGTv/lW9vLc14bJOeT/iWlvVvqER1jNenDe7S+kCeN/j92zaY8k6XafzaxLeAzWb si8w== X-Forwarded-Encrypted: i=1; AJvYcCXTAkx43P0pyiWXKOc9GP9QrrNZkpzVwdavMnkESGJTorb+Ih1+ekZVI3y2qkSFbFprJEe9ZQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0YwqnlTBTwtwtqx8rgBimY87r42Z2Pyal8JYwURG2zViZlqn8lhk F3k5kRAlyJFqJa6kO+uA5D49L2L/9w2at8Rdxgf6JgumDVq6CF9x X-Gm-Gg: ASbGncsA1UYZp7kl+z3eADR4eV5PAlRrP8FenHWpZuSpzxH3KoTOVPNF0zeQuYo1HhG NYdtyk0xjSIDVscSmU/yoW8nb0dRhv6msYXlTI3HbTUcATKxYBDeQAX4SLaQGsegDzM7DbWpjqx w/U3Qr8CdSKvz/+fLp21yBFHpH29pk9CNiVEfYNdjFj3mgE90OYEyuF9B0YO3UvwC9fsdhzaJ40 Dg/0b2N7XAi8NOnxhqvuaKWBURbBwIHCf2zXyvDbL/YVPhDFngNlVdovCoKxQ5rPQCH1+9XvM4v ZM91 X-Google-Smtp-Source: AGHT+IGET3HBupLGA8wKf6yZO06DVgDVTJqiQn19B3j+FF2ee2xZZW/VpOK+unanJUvfXCv0cCJzVA== X-Received: by 2002:a17:902:c951:b0:215:bc30:c952 with SMTP id d9443c01a7336-21a83f4298fmr17529315ad.6.1736303245534; Tue, 07 Jan 2025 18:27:25 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:715c:1de9:341d:f41d]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc96e933sm317782195ad.59.2025.01.07.18.27.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Jan 2025 18:27:24 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: Date: Tue, 7 Jan 2025 18:27:13 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <87bk0a1u0o.fsf@masteringemacs.org> <86tte2a5o3.fsf@gnu.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> To: Stefan Monnier X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , 73404@debbugs.gnu.org, Juri Linkov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Jan 7, 2025, at 11:25=E2=80=AFAM, Stefan Monnier = wrote: >=20 >> Indeed, symbols are more traditional. >>=20 >> However, the problem arose only because >> 'treesit_traverse_validate_predicate' accepts a symbol >> for both a function and a thing, that caused ambiguity. >>=20 >> So there are two variants to resolve this ambiguity: >>=20 >> 1. use keywords like >>=20 >> (setq-local treesit-thing-settings >> `((html >> (:sexp ,(regexp-opt '("element" "text" "attribute" = "value"))) >> (:list ,(regexp-opt '("element""))) >> (:sentence "tag") >> (:text ,(regexp-opt '("comment" "text")))))) >>=20 >> 2. use a symbol property like (put 'list 'treesit-thing t) >>=20 >> You decide ;-) >=20 > Of course, there are more alternatives: >=20 > - Use a symbol property like (put 'list 'treesit--this-is-a-function = t) > - Refuse functions represented as symbols (callers need to use an > eta-wrapper). > - Distinguish the two cases via a more verbose representation like > `(thing list)` vs `(function list)`. > - Use a string rather than a symbol. > - Thanks. IMHO it=E2=80=99s best to keep it simple and familiar, so = let=E2=80=99s keep using symbols and use a symbol property to solve this = edge case. I think this problem is rare enough that we don=E2=80=99t = need any fancy solutions for it. Juri, please feel free to apply your symbol property patch. For the = symbol property name, I feel that something like = treesit-symbol-predicate/treesit-use-as-symbol-predicate would be more = descriptive. Since we don=E2=80=99t have docstrings for symbol = properties (right?). Also it=E2=80=99d be nice to document this in the = manual, maybe in the manual section for treesit-thing-settings. The = docstring probably don=E2=80=99t need this niche detail. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 09 02:49:03 2025 Received: (at 73404) by debbugs.gnu.org; 9 Jan 2025 07:49:03 +0000 Received: from localhost ([127.0.0.1]:50279 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVnHq-0007zY-L3 for submit@debbugs.gnu.org; Thu, 09 Jan 2025 02:49:03 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:57827) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVnHo-0007yk-4t for 73404@debbugs.gnu.org; Thu, 09 Jan 2025 02:49:00 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id ED5131C0002; Thu, 9 Jan 2025 07:48:50 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Tue, 7 Jan 2025 18:27:13 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> Date: Thu, 09 Jan 2025 09:42:14 +0200 Message-ID: <87plkwwigx.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >>> 1. use keywords like >>> >>> (setq-local treesit-thing-settings >>> `((html >>> (:sexp ,(regexp-opt '("element" "text" "attribute" "value"))) >>> (:list ,(regexp-opt '("element""))) >>> (:sentence "tag") >>> (:text ,(regexp-opt '("comment" "text")))))) >>> >>> 2. use a symbol property like (put 'list 'treesit-thing t) >>> >>> You decide ;-) >> >> Of course, there are more alternatives: >> >> - Use a symbol property like (put 'list 'treesit--this-is-a-function t) >> - Refuse functions represented as symbols (callers need to use an >> eta-wrapper). >> - Distinguish the two cases via a more verbose representation like >> `(thing list)` vs `(function list)`. >> - Use a string rather than a symbol. >> - > > Thanks. IMHO it’s best to keep it simple and familiar, so let’s keep > using symbols and use a symbol property to solve this edge case. > I think this problem is rare enough that we don’t need any fancy > solutions for it. > > Juri, please feel free to apply your symbol property patch. For the symbol > property name, I feel that something like > treesit-symbol-predicate/treesit-use-as-symbol-predicate would be more > descriptive. If we settle on the variant that keeps the current format with symbols, wouldn't it better to keep a symbol property name shorter? First I proposed 'treesit-predicate', but then realized it's wrong since it's opposite to the predicate function. 'treesit-thing' was better since it indicates that it's used in 'treesit-thing-settings'. > Since we don’t have docstrings for symbol properties (right?). There are no docstrings for symbol properties indeed. But I think the important property for the name is that it should be unique enough to be easily greppable. But I tried to grep, and 'treesit-thing' finds too many matches. So probably the most suitable symbol name would be 'treesit-thing-symbol', that currently has no occurrences in source code. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 09 13:39:49 2025 Received: (at 73404) by debbugs.gnu.org; 9 Jan 2025 18:39:49 +0000 Received: from localhost ([127.0.0.1]:54228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVxRd-0006zO-FV for submit@debbugs.gnu.org; Thu, 09 Jan 2025 13:39:49 -0500 Received: from fout-b3-smtp.messagingengine.com ([202.12.124.146]:34103) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVxRb-0006z8-FS for 73404@debbugs.gnu.org; Thu, 09 Jan 2025 13:39:47 -0500 Received: from phl-compute-04.internal (phl-compute-04.phl.internal [10.202.2.44]) by mailfout.stl.internal (Postfix) with ESMTP id E23491140172; Thu, 9 Jan 2025 13:39:41 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Thu, 09 Jan 2025 13:39:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1736447981; x=1736534381; bh=FidFFS8yMttkfIOu/tl/wOIWAlIxhHo0ZFkgBmqDozU=; b= 3qLCI0+yPQ8WCDnBBhlFrADjnPk9FQlcTgI6Kj/dDVsSNTpMitVFqcKOi/cd2u+C gZ7M2Xs700s40XhtETSjciF8RgqdPsRQeRY6TTnX+SwsFWaszhZKfgiW3///wx6u oe+wPdjuvCKn6xGbrWMDAeFM6tnSZAIaCGr1l9NMdFmbCD51sCEOsL61/53D8ULi KXUbHY8zttWFl9jd2IzL4pXUrmQJ1P/sbypXziqtb31GKDNuvTOgh2EZqOj+99ZG XZngaXG2treySygQh41rdfio6Q4KrMG0ZU5WAIYNxN55t8YtpjSld+FiCGdccXVc 9ssiUQy/PpoN25Ex9PI+Fw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1736447981; x= 1736534381; bh=FidFFS8yMttkfIOu/tl/wOIWAlIxhHo0ZFkgBmqDozU=; b=l rZLFJcVOrs3eFBPOP9wuzKVDIbGH3UCdLVg3RIEsXGht22fozRItZJikeEOCfxzB 1P4hvUxRc1xJK2jqguGr6aSNVwonBFNgQRxv9T6TpzcI3iAJlPhS8EKz6qtu0o0j T2vdqiZoNM5jT/83gHipoN7GqvLmCnSJGhVNDqtAxXUTan86fk4KKGq0f5jW9ZMd RZbYJNj6JwQRksap1eGJXlGwvSgweVIDqItpU3UvN6Cx6j9F0585wXXMZA8WjMqU oXN1uvcStBy0/0yHtJL8HkM6pvveBmKpkCFgY/lGx0c1/0YrrKEyv/Fnbyhrnd8i 3PCmDcbSm7P75ehun6X6w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrudegiedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttddv jeenucfhrhhomhepffhmihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrd guvghvqeenucggtffrrghtthgvrhhnpeetudeljeegheetgfehgeejkeeuhedvveeikeeu fedtvddtveefhfdvveegudejheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpegumhhithhrhiesghhuthhovhdruggvvhdpnhgspghrtghpthht ohepjedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepjhhurhhisehlihhnkhhovh drnhgvthdprhgtphhtthhopegtrghsohhurhhisehgmhgrihhlrdgtohhmpdhrtghpthht ohepthhhvghosehthhhorhhnhhhilhhlrdhnohdprhgtphhtthhopegvlhhiiiesghhnuh drohhrghdprhgtphhtthhopehmihgtkhgvhiesmhgrshhtvghrihhnghgvmhgrtghsrdho rhhgpdhrtghpthhtohepmhhonhhnihgvrhesihhrohdruhhmohhnthhrvggrlhdrtggrpd hrtghpthhtohepjeefgedtgeesuggvsggsuhhgshdrghhnuhdrohhrgh X-ME-Proxy: Feedback-ID: i07de48aa:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 9 Jan 2025 13:39:38 -0500 (EST) Message-ID: <5aed86a5-9e54-4e63-a568-cbbad577e39b@gutov.dev> Date: Thu, 9 Jan 2025 20:39:35 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes To: Juri Linkov , Yuan Fu References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> <87plkwwigx.fsf@mail.linkov.net> Content-Language: en-US From: Dmitry Gutov In-Reply-To: <87plkwwigx.fsf@mail.linkov.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Mickey Petersen , Eli Zaretskii , Theodor Thornhill , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 09/01/2025 09:42, Juri Linkov wrote: >> Juri, please feel free to apply your symbol property patch. For the symbol >> property name, I feel that something like >> treesit-symbol-predicate/treesit-use-as-symbol-predicate would be more >> descriptive. > If we settle on the variant that keeps the current format with symbols, > wouldn't it better to keep a symbol property name shorter? > > First I proposed 'treesit-predicate', but then realized it's wrong > since it's opposite to the predicate function. > 'treesit-thing' was better since it indicates that it's used in > 'treesit-thing-settings'. Is it too late to keep the current name, 'sexp-list'? It it of course somewhat redundant, but not by much, and not needing an extra indirection is a plus on its own. Just my 2c. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 09 13:46:02 2025 Received: (at 73404) by debbugs.gnu.org; 9 Jan 2025 18:46:02 +0000 Received: from localhost ([127.0.0.1]:54244 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tVxXd-0007LI-Li for submit@debbugs.gnu.org; Thu, 09 Jan 2025 13:46:01 -0500 Received: from relay1-d.mail.gandi.net ([2001:4b98:dc4:8::221]:41869) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tVxXc-0007Kw-Hg for 73404@debbugs.gnu.org; Thu, 09 Jan 2025 13:46:00 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id D0FAA240002; Thu, 9 Jan 2025 18:45:48 +0000 (UTC) From: Juri Linkov To: Dmitry Gutov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <5aed86a5-9e54-4e63-a568-cbbad577e39b@gutov.dev> (Dmitry Gutov's message of "Thu, 9 Jan 2025 20:39:35 +0200") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> <87plkwwigx.fsf@mail.linkov.net> <5aed86a5-9e54-4e63-a568-cbbad577e39b@gutov.dev> Date: Thu, 09 Jan 2025 20:44:12 +0200 Message-ID: <87ldvju8sj.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Yuan Fu , Theodor Thornhill , Mickey Petersen , 73404@debbugs.gnu.org, Eli Zaretskii , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >>> Juri, please feel free to apply your symbol property patch. For the symbol >>> property name, I feel that something like >>> treesit-symbol-predicate/treesit-use-as-symbol-predicate would be more >>> descriptive. >> If we settle on the variant that keeps the current format with symbols, >> wouldn't it better to keep a symbol property name shorter? >> First I proposed 'treesit-predicate', but then realized it's wrong >> since it's opposite to the predicate function. >> 'treesit-thing' was better since it indicates that it's used in >> 'treesit-thing-settings'. > > Is it too late to keep the current name, 'sexp-list'? No, not late. And it's too ugly. > It it of course somewhat redundant, but not by much, and not needing an > extra indirection is a plus on its own. An extra indirection is needed in any case to avoid such ambiguity in the future. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 09 22:14:29 2025 Received: (at 73404) by debbugs.gnu.org; 10 Jan 2025 03:14:29 +0000 Received: from localhost ([127.0.0.1]:55937 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tW5Tg-0006jm-SG for submit@debbugs.gnu.org; Thu, 09 Jan 2025 22:14:29 -0500 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:53249) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tW5Te-0006jV-9H for 73404@debbugs.gnu.org; Thu, 09 Jan 2025 22:14:26 -0500 Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-2ef28f07dbaso2207608a91.2 for <73404@debbugs.gnu.org>; Thu, 09 Jan 2025 19:14:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736478860; x=1737083660; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=bvhzzL6T98F9CyVVpvtPPDs1cW5yP++GviAEmmgqpBc=; b=adNo4vzS82OMonReOqHpRE3NgnY8YOSmCEFZ/5ZaM74o2giimFJs8zaqwJklqGz82D SK19D2RFg+Jz4r8V7BKlb2qwxrsE/OZQkJ64AMOrEk8gcFNP7Sl3cw0LNQTl4+wzmmtp S3Ne/OgOYsCHnUzhdLaWXF6Fzc9/D1Y5GST1adt2vpjVhbBoDGxnXjy8fDM3f8DkI7sJ 6X8mKbcRp5Kque5KtX2QfHGmeKkE/ney5fKyui4ELZCfvL47s/2AqUOX5YrhIdntCIv7 IUVmhGcLNtHC8zqHawtdWe2sFJ1jP7BeKGVR69w7JkswOalZV0CDc3FLYsPFFC/qtU7W cbfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736478860; x=1737083660; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bvhzzL6T98F9CyVVpvtPPDs1cW5yP++GviAEmmgqpBc=; b=xGiJLrlybelErviDfQ1EkJISUeZCsg+8BcXRo792+8rr0G35hiNSNNZJ+LycoXeNML ZIaJ9y5D0uee0Fq5IlnIIoEjAq5hMmj7xn8Lc286yIJQXT2xwRe3bNiaIYv5c5MJqVK7 8r/h0Ir8moWQsXLYrFkmvh93flQ1ZiEEcKGmOW7E2uiPJUs/bQh4NBmEK3Fs+x0wj7wr TrFqYVEKVatzgb4ArNYcy48rhldiBm7wCoJ6swAhnIEEr+YT7farJcHvBiVafd5oHp/w lWUstiM7wyn7T/wtQQZjytb154xvJhKXwNP9nyc916L/s0OYiaQRUByVpRQ3Bw3LiKR2 DJKQ== X-Forwarded-Encrypted: i=1; AJvYcCWh38Koiuce4kzDWSU+ARa4f+UPjfq614PWaImLA/mGJGmeFcOJ74wzYXZ6/J6J5VKfJVidpg==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yzj2VaaiV3ary36imF80Ee2ptH8IzU1Of3rtjPOzCpSiatG9xRf wSlJwLMt8yWQApuE6xvPsCkRAHU2Py5K1QbOAypRgteDPxEuDDq4 X-Gm-Gg: ASbGnctkM53B6BVIZdQYSNl35NLLMGOsbF+tHBfSdAmXMU2FDqGCnRRJ+/OctNECSi0 nIBXw8JyLdcbw1dLzMV/gaxUk5RKolzqNkMcVQPcbdOpJzlCWU5Bhp237Uh8Ke4W1AKESslIuar unbA6nN7w+CTE6KJMt5FXksEqNIERoKCTH5YJKtYiJxxLeO6gomynnPgAaY6JiZZHqtjS3/2zV9 VWpxQamjg51RHSi2QWPeMa1GFxo2+qAkBEn4SovBw9WS7ACT26xEBt3SZq33/lhYxV0CEEB8EiM 73sm X-Google-Smtp-Source: AGHT+IEgvb4pFNF4h5PjQKSzNw8AXPwK2m/GnfeaUpXOge0ta2gjJ3lQDsRskxuSW2uwWyGRgctSWA== X-Received: by 2002:a17:90b:270d:b0:2ea:3f34:f194 with SMTP id 98e67ed59e1d1-2f548eae587mr14071370a91.10.1736478860213; Thu, 09 Jan 2025 19:14:20 -0800 (PST) Received: from smtpclient.apple ([2601:646:8f81:6120:d801:ab3e:9baf:5ef1]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f559454f73sm2242058a91.38.2025.01.09.19.14.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Jan 2025 19:14:19 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes From: Yuan Fu In-Reply-To: <87plkwwigx.fsf@mail.linkov.net> Date: Thu, 9 Jan 2025 19:14:08 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87plox4mtp.fsf@masteringemacs.org> <877cay1lqt.fsf@masteringemacs.org> <86frpma06f.fsf@gnu.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> <87plkwwigx.fsf@mail.linkov.net> To: Juri Linkov X-Mailer: Apple Mail (2.3776.700.51) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On Jan 8, 2025, at 11:42=E2=80=AFPM, Juri Linkov = wrote: >=20 >>>> 1. use keywords like >>>>=20 >>>> (setq-local treesit-thing-settings >>>> `((html >>>> (:sexp ,(regexp-opt '("element" "text" "attribute" = "value"))) >>>> (:list ,(regexp-opt '("element""))) >>>> (:sentence "tag") >>>> (:text ,(regexp-opt '("comment" "text")))))) >>>>=20 >>>> 2. use a symbol property like (put 'list 'treesit-thing t) >>>>=20 >>>> You decide ;-) >>>=20 >>> Of course, there are more alternatives: >>>=20 >>> - Use a symbol property like (put 'list 'treesit--this-is-a-function = t) >>> - Refuse functions represented as symbols (callers need to use an >>> eta-wrapper). >>> - Distinguish the two cases via a more verbose representation like >>> `(thing list)` vs `(function list)`. >>> - Use a string rather than a symbol. >>> - >>=20 >> Thanks. IMHO it=E2=80=99s best to keep it simple and familiar, so = let=E2=80=99s keep >> using symbols and use a symbol property to solve this edge case. >> I think this problem is rare enough that we don=E2=80=99t need any = fancy >> solutions for it. >>=20 >> Juri, please feel free to apply your symbol property patch. For the = symbol >> property name, I feel that something like >> treesit-symbol-predicate/treesit-use-as-symbol-predicate would be = more >> descriptive. >=20 > If we settle on the variant that keeps the current format with = symbols, > wouldn't it better to keep a symbol property name shorter? >=20 > First I proposed 'treesit-predicate', but then realized it's wrong > since it's opposite to the predicate function. > 'treesit-thing' was better since it indicates that it's used in > 'treesit-thing-settings'. >=20 >> Since we don=E2=80=99t have docstrings for symbol properties = (right?). >=20 > There are no docstrings for symbol properties indeed. But I think > the important property for the name is that it should be unique enough > to be easily greppable. But I tried to grep, and 'treesit-thing' = finds > too many matches. >=20 > So probably the most suitable symbol name would be = 'treesit-thing-symbol', > that currently has no occurrences in source code. Sounds good. Let=E2=80=99s go with it then. Yuan= From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 10 02:35:15 2025 Received: (at 73404) by debbugs.gnu.org; 10 Jan 2025 07:35:15 +0000 Received: from localhost ([127.0.0.1]:56251 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tW9Y3-0001pf-0e for submit@debbugs.gnu.org; Fri, 10 Jan 2025 02:35:15 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:59611) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tW9Y1-0001mg-Rt; Fri, 10 Jan 2025 02:35:14 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id D770060003; Fri, 10 Jan 2025 07:35:04 +0000 (UTC) From: Juri Linkov To: Yuan Fu Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Yuan Fu's message of "Thu, 9 Jan 2025 19:14:08 -0800") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <87pll190ui.fsf@mail.linkov.net> <674226FE-597C-44DC-823A-667E5C55397A@gmail.com> <87ldvmik3o.fsf@mail.linkov.net> <87plkwwigx.fsf@mail.linkov.net> Date: Fri, 10 Jan 2025 09:34:08 +0200 Message-ID: <87ed1bm8b3.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: Theodor Thornhill , Eli Zaretskii , Mickey Petersen , Stefan Monnier , 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) close 73404 31.0.50 thanks >>> Juri, please feel free to apply your symbol property patch. For the symbol >>> property name, I feel that something like >>> treesit-symbol-predicate/treesit-use-as-symbol-predicate would be more >>> descriptive. >> >> If we settle on the variant that keeps the current format with symbols, >> wouldn't it better to keep a symbol property name shorter? >> >> First I proposed 'treesit-predicate', but then realized it's wrong >> since it's opposite to the predicate function. >> 'treesit-thing' was better since it indicates that it's used in >> 'treesit-thing-settings'. >> >>> Since we don’t have docstrings for symbol properties (right?). >> >> There are no docstrings for symbol properties indeed. But I think >> the important property for the name is that it should be unique enough >> to be easily greppable. But I tried to grep, and 'treesit-thing' finds >> too many matches. >> >> So probably the most suitable symbol name would be 'treesit-thing-symbol', >> that currently has no occurrences in source code. > > Sounds good. Let’s go with it then. So now pushed together with updates in the manual. From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 11 13:21:08 2025 Received: (at 73404) by debbugs.gnu.org; 11 Jan 2025 18:21:08 +0000 Received: from localhost ([127.0.0.1]:44756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tWg6d-00049D-K3 for submit@debbugs.gnu.org; Sat, 11 Jan 2025 13:21:08 -0500 Received: from relay5-d.mail.gandi.net ([2001:4b98:dc4:8::225]:52473) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tWg6b-00048C-1f for 73404@debbugs.gnu.org; Sat, 11 Jan 2025 13:21:05 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id E63C71C0002; Sat, 11 Jan 2025 18:20:55 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Mon, 06 Jan 2025 15:07:28 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86ikueiekp.fsf@mail.linkov.net> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> Date: Sat, 11 Jan 2025 20:19:47 +0200 Message-ID: <87plktkybg.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > But indeed for list operations in indentation-based languages like > Haskell/Python/Coffeescript/YAML, we're faced again with the fact that > a closing LF can close several nested lists, so when going backward > there is some amount of ambiguity (and I'd argue that it's preferable > (and closer to the historical behavior) to choose to jump less far, > e.g. so that `C-M-n C-M-p` never moves backward). I tried to implement list operations in yaml-ts-mode on the nodes "block_mapping_pair" and "flow_sequence". While testing everything looked so nice when `C-M-f` and `C-M-b` moved on the whole key-value pairs. But in actual use, it turned out to be completely unusable. It feels this is because `C-M-f` is expected to move to the next symbol unless it's at the beginning of (...), [...], if...fi, begin...end. Perhaps we should still support list navigation only for these 4 keys: 'C-M-n', 'C-M-p', 'C-M-d', 'C-M-y', but not for `C-M-f` and `C-M-b`. From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 11 14:14:33 2025 Received: (at 73404) by debbugs.gnu.org; 11 Jan 2025 19:14:33 +0000 Received: from localhost ([127.0.0.1]:44950 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tWgwL-00075D-9R for submit@debbugs.gnu.org; Sat, 11 Jan 2025 14:14:33 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:36988) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tWgwI-00074y-RT for 73404@debbugs.gnu.org; Sat, 11 Jan 2025 14:14:31 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 60B5610004C; Sat, 11 Jan 2025 14:14:24 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736622859; bh=uPAndEGdCbmeg1jFODCvtUg6zXCoYlKPzsFE4viVrnY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=hIx2dvFA5cR5at8/dfFe6NVnOyUAn7Wat6UjhVqcX1DrpiPpMrfp0GFuSzeFDnE9J +4maHqdd3LSWtN2fWDBLrm2uI85HRY48VvbqJmzsjsx/B7NpLqlncIMSIy8E4r/Xk6 FHBeCP9cGXjUvf1exRjvohJl0efoSwAJZhbGAKO+xlWDHFGWBMhcI3KQ1QHmDHt9vE pgt3vu+ut0/K8bMvEp+vpodtF15zGjnmIbNldvUGWq9TrkK5ykR7CGwXG4F6Rxvrl4 QsUpEYwMZ/yWmzwwMdiQRFfSyBo1yjW+hM3pmQsayOTr/JtvnUzCcEcp0jf6FM0pmD /oFD+A0Mr/1sg== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 80CBA10002E; Sat, 11 Jan 2025 14:14:19 -0500 (EST) Received: from alfajor (104-195-232-86.cpe.teksavvy.com [104.195.232.86]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 2766912038D; Sat, 11 Jan 2025 14:14:19 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87plktkybg.fsf@mail.linkov.net> (Juri Linkov's message of "Sat, 11 Jan 2025 20:19:47 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> <87plktkybg.fsf@mail.linkov.net> Date: Sat, 11 Jan 2025 14:14:23 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.121 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > While testing everything looked so nice when `C-M-f` > and `C-M-b` moved on the whole key-value pairs. > > But in actual use, it turned out to be completely unusable. > > It feels this is because `C-M-f` is expected to move > to the next symbol unless it's at the beginning > of (...), [...], if...fi, begin...end. Maybe this can be tuned? Let's take a piece of Haskell code like: foo x y = do a <- readsomething x (bar y) b <- readsomethingelse x (baz y) return (a, b) A) If the cursor is right before `a <- ...`, I definitely expect `C-M-f` to move just over the identifier. B) If the cursor is right after `do` (i.e. looking-at EOL), I could see `C-M-f` moving either to the end of the `do` block (i.e. the end of the `foo` function, here), or to right after the closing paren of `(bar y)`. C) If the cursor is right after the closing paren of `(bar y)`, then I'd expect `C-M-f` to move to the closing paren of `(baz y)`. If we make the block/list syntax more explicit (which Haskell allows), the above code is: foo x y = do{ a <- readsomething x (bar y); b <- readsomethingelse x (baz y); return (a, b) } In (A) it's clear we're already inside the block. In (B), we have an ambiguity where the cursor can be considered to be looking either at the whole {...} block, or just at the first statement within it, but I think it's clearly not looking at just the `a` identifier. In (C) what I propose is the behavior that SMIE uses in such cases. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 12 02:36:21 2025 Received: (at 73404) by debbugs.gnu.org; 12 Jan 2025 07:36:21 +0000 Received: from localhost ([127.0.0.1]:45859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tWsWC-0003Et-SQ for submit@debbugs.gnu.org; Sun, 12 Jan 2025 02:36:21 -0500 Received: from relay2-d.mail.gandi.net ([217.70.183.194]:54687) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tWsWB-0003Ee-9o for 73404@debbugs.gnu.org; Sun, 12 Jan 2025 02:36:19 -0500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 59CB640004; Sun, 12 Jan 2025 07:36:08 +0000 (UTC) From: Juri Linkov To: Stefan Monnier Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: (Stefan Monnier's message of "Sat, 11 Jan 2025 14:14:23 -0500") Organization: LINKOV.NET References: <87plox4mtp.fsf@masteringemacs.org> <86ed4zg1cc.fsf@mail.linkov.net> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> <87plktkybg.fsf@mail.linkov.net> Date: Sun, 12 Jan 2025 09:34:52 +0200 Message-ID: <87frlo5w0j.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/31.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-GND-Sasl: juri@linkov.net X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Let's take a piece of Haskell code like: > > foo x y = do > a <- readsomething > x (bar y) > b <- readsomethingelse > x (baz y) > return (a, b) > > A) If the cursor is right before `a <- ...`, I definitely expect `C-M-f` to > move just over the identifier. Is the cursor at the beginning of the line or after indentation? It seems this scheme is whitespace-sensitive since moving the cursor back over the newline to the end of the previous line with `do` will change whether `C-M-f` should move over the next `bind` node or just over the next symbol. > B) If the cursor is right after `do` (i.e. looking-at EOL), I could see > `C-M-f` moving either to the end of the `do` block (i.e. the end of the > `foo` function, here), or to right after the closing paren of `(bar y)`. Usually `do` designates the beginning of the node, so I'd expect the cursor to be before `do` to be able to move to the end of the `do` block. Unless explicit curly braces are used in which case the cursor after `do` is expected to move after the closing curly brace. > C) If the cursor is right after the closing paren of `(bar y)`, then I'd > expect `C-M-f` to move to the closing paren of `(baz y)`. This heuristics looks whitespace-sensitive too: if the cursor was after the newline at the beginning of the next line then `C-M-f` would move over the next symbol. > If we make the block/list syntax more explicit (which Haskell allows), > the above code is: > > foo x y = do{ > a <- readsomething > x (bar y); > b <- readsomethingelse > x (baz y); > return (a, b) > } > > In (A) it's clear we're already inside the block. Explicit syntax removes a lot of ambiguity indeed. > In (B), we have an ambiguity where the cursor can be considered to be > looking either at the whole {...} block, or just at the first statement > within it, but I think it's clearly not looking at just the `a` identifier. When the cursor is after `{` then it's inside the {...} block. > In (C) what I propose is the behavior that SMIE uses in such cases. Is SMIE sensitive to the position of the cursor inside whitespace region including newlines? From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 12 10:09:08 2025 Received: (at 73404) by debbugs.gnu.org; 12 Jan 2025 15:09:08 +0000 Received: from localhost ([127.0.0.1]:48735 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tWzaN-0001Z1-Np for submit@debbugs.gnu.org; Sun, 12 Jan 2025 10:09:08 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:57808) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tWzaL-0001YW-Vv for 73404@debbugs.gnu.org; Sun, 12 Jan 2025 10:09:06 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 49B7810004C; Sun, 12 Jan 2025 10:08:59 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1736694538; bh=pjPI2xnEo6U+h6b9BB/4YPA/AyGWAefZeMj1ICPp6cc=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=PUSmlWuBIZyEFj+q/0UemC7epRYJHBUwOSMn2RmfY6aRf7pYZiJEzmaPENgLQa3ct Hh0x2d9CCp7wIkqw08ElHZ19EpZp+Oh28h26kfZvJfKgKBhADPmH3bHI1D5nzfbJFH KSF32y8mYYvCb1Kz8uaEy2A9VZQLu2DNpp0Y1W3hzXTkMbateYMIi49+Ppf2Y5LmJ2 96RriN4jXcDwuN2XzuMoG3navZ6LvKOsI/brIpRly+GB33EVUIWggpI8W64/VZTjRb y7uEZRGqvOyCrDJ1FjvfbFveluedqEOqIC2rc9CxPvfTHSU9ufvHwjwPKVkBv8Ik5r v9xWO1eyV32ww== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 9C5F1100040; Sun, 12 Jan 2025 10:08:58 -0500 (EST) Received: from alfajor (104-195-232-86.cpe.teksavvy.com [104.195.232.86]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 56863120804; Sun, 12 Jan 2025 10:08:58 -0500 (EST) From: Stefan Monnier To: Juri Linkov Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes In-Reply-To: <87frlo5w0j.fsf@mail.linkov.net> (Juri Linkov's message of "Sun, 12 Jan 2025 09:34:52 +0200") Message-ID: References: <87plox4mtp.fsf@masteringemacs.org> <87zflac68t.fsf@mail.linkov.net> <87jzcdlxdp.fsf@mail.linkov.net> <87o71jocgs.fsf@mail.linkov.net> <87wmfwqg7e.fsf@mail.linkov.net> <8734i5fyv1.fsf@mail.linkov.net> <875xmumpzv.fsf@mail.linkov.net> <86ikqubdsd.fsf@gnu.org> <87wmf9912l.fsf@mail.linkov.net> <87a5c5v5z8.fsf@thornhill.no> <877c79qhcs.fsf@mail.linkov.net> <87sepxt4yl.fsf@thornhill.no> <87o70k9y2e.fsf@mail.linkov.net> <87plkzyh60.fsf@mail.linkov.net> <87plktkybg.fsf@mail.linkov.net> <87frlo5w0j.fsf@mail.linkov.net> Date: Sun, 12 Jan 2025 10:08:57 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.115 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 73404 Cc: mickey@masteringemacs.org, Eli Zaretskii , Theodor Thornhill , casouri@gmail.com, 73404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> A) If the cursor is right before `a <- ...`, I definitely expect `C-M-f` to >> move just over the identifier. > Is the cursor at the beginning of the line or after indentation? I was thinking "after indentation". I don't know what behavior I'd want if it's at the beginning of the line. > This heuristics looks whitespace-sensitive too: if the cursor was > after the newline at the beginning of the next line then `C-M-f` > would move over the next symbol. Of course: in such languages, LF is very much significant. >> In (C) what I propose is the behavior that SMIE uses in such cases. > Is SMIE sensitive to the position of the cursor inside whitespace region > including newlines? In those languages where LF is a statement separator (e.g. `octave-mode`), yes. Stefan From unknown Sun Aug 10 16:49:02 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 10 Feb 2025 12:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator