From unknown Sun Jun 15 07:58:05 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#47799 <47799@debbugs.gnu.org> To: bug#47799 <47799@debbugs.gnu.org> Subject: Status: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames Reply-To: bug#47799 <47799@debbugs.gnu.org> Date: Sun, 15 Jun 2025 14:58:05 +0000 retitle 47799 28.0.50; Default `project-files' implementation doesn't work = with quoted filenames reassign 47799 emacs submitter 47799 Philipp Stephani severity 47799 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 15 09:44:03 2021 Received: (at submit) by debbugs.gnu.org; 15 Apr 2021 13:44:03 +0000 Received: from localhost ([127.0.0.1]:36880 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX2Hr-00014z-4k for submit@debbugs.gnu.org; Thu, 15 Apr 2021 09:44:03 -0400 Received: from lists.gnu.org ([209.51.188.17]:57478) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX2Ho-00014S-00 for submit@debbugs.gnu.org; Thu, 15 Apr 2021 09:44:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45784) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lX2Hn-00056N-J7 for bug-gnu-emacs@gnu.org; Thu, 15 Apr 2021 09:43:59 -0400 Received: from mail-ed1-x52a.google.com ([2a00:1450:4864:20::52a]:39879) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lX2Hg-0000jO-Gd for bug-gnu-emacs@gnu.org; Thu, 15 Apr 2021 09:43:58 -0400 Received: by mail-ed1-x52a.google.com with SMTP id g17so27404983edm.6 for ; Thu, 15 Apr 2021 06:43:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=+p5TRMOueURhJrpaJNi1WQdxx0a78PVIRvrGhp4woP0=; b=gka0maP8GexCvDzYO6ILGlrpOYit0NwTLmrbvBNAm2ufvMT/3pHcWL8aIJ/baxjuZI p2g2Pxl1jUEI/JuEo+QM3GCipLmR1hTrXP8TDewplZbVA9A8kRPquRHUV6UbizG6tjCx GTnoP++qMjOhtzUDQpGpv57ByeGc/kQwQ0LsLR/xK7R8d/RJVdvq4M4HyaihmaZgDarL 1N1Eru/yA7kj8+igQIqJrLWyka87HhjVUemLoiWrAL2B6OUIqBDKa9SFL1X6UPbEoXAk G8MJrnN4wcTQ8TYSK9w/QqHcjQPYPZBc/MjCBQIVGGlaFI0EyfJci7NRA81wXNOhmDyy fNnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=+p5TRMOueURhJrpaJNi1WQdxx0a78PVIRvrGhp4woP0=; b=f6WYIwWxNCBFjH4HQDNQPeSK2DV2LxlUOHw/rVXBuLCw5LKB8kb4uGrz0fu2twCqRZ +5XKGv7HC+uSHtAaRGFE8b+W7Y5kkcEIdpvLNcGC12D4kZBd4DJPUvg9KmjoEwW0eN/C eAVW7LGIt8M00ZrNfxP2ihEQ4MuNu9H99P+dJdn8W7/upiW9RI5ivdsZdlz+fZlpEQnq IJ9fjGeV3mwsplR2XlPo/vvByaDY0oyc+n15ifIInByKjMOdEl9WIdMln2CkGCbjbQSH 5eE8kDWRaz6wrNnlPZNl4rVEFfnQQ1D3Y1lLJJHuhzkk92YvvjMisyR9gFAXaz5raGHx ar7g== X-Gm-Message-State: AOAM530w/PYj7v3IsgLU8V+E5iRwcsQIYNVwswpyXGOPy/xTW9ZMhgS5 a8l5sv114JjBKxTjyD94NXN51cxbHtOTjg== X-Google-Smtp-Source: ABdhPJx5Zf8XTD5YlX0miMj9RjC8oHYCl9gjqSuwqkNwev63RfrDOEDMADikWH43Ul4C7OT/XJqRdw== X-Received: by 2002:a05:6402:cbb:: with SMTP id cn27mr4301505edb.296.1618494228730; Thu, 15 Apr 2021 06:43:48 -0700 (PDT) Received: from phst (p57997ce8.dip0.t-ipconnect.de. [87.153.124.232]) by smtp.gmail.com with ESMTPSA id j6sm2508210edw.73.2021.04.15.06.43.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Apr 2021 06:43:48 -0700 (PDT) From: Philipp Stephani To: bug-gnu-emacs@gnu.org Subject: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames Date: Thu, 15 Apr 2021 15:43:47 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::52a; envelope-from=p.stephani2@gmail.com; helo=mail-ed1-x52a.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) emacs -Q -batch -l project -eval '(print (project-files (quote (transient .= "/:/"))))' ("find: =E2=80=98/:/=E2=80=99: No such file or directory ") Note that the error message is listed as a file. 1. `project-files' should unquote local filenames before passing them to `find'. 2. `project-files' should check for errors returned from `find'. In GNU Emacs 28.0.50 (build 74, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, = cairo version 1.16.0) of 2021-04-15 Repository revision: 31f8ae53beb9bada58750160c1bf7f867ecd442e Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12010000 System Description: Debian GNU/Linux rodete Configured using: 'configure --enable-gcc-warnings=3Dwarn-only --enable-gtk-deprecation-warnings --without-pop --with-mailutils --enable-checking=3Dall --enable-check-lisp-object-type --with-modules 'CFLAGS=3D-O0 -ggdb3'' Configured features: CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON LIBSELINUX LIBSYSTEMD MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB Important settings: value of $LC_TIME: en_DK.utf8 value of $LANG: en_US.utf8 value of $XMODIFIERS: @im=3Dibus locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc dired dired-loaddefs rfc822 mml mml-sec epa epg epg-config gnus-util rmail rmail-loaddefs time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils phst skeleton derived edmacro kmacro pcase ffap thingatpt url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json map url-vars mailcap rx gnutls puny dbus xml subr-x seq byte-opt gv bytecomp byte-compile cconv compile text-property-search comint ansi-color ring cl-loaddefs cl-lib iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 68419 8560) (symbols 48 8313 1) (strings 32 24255 1847) (string-bytes 1 785382) (vectors 16 14967) (vector-slots 8 194817 4757) (floats 8 26 32) (intervals 56 223 0) (buffers 992 11)) --=20 Google Germany GmbH Erika-Mann-Stra=C3=9Fe 33 80636 M=C3=BCnchen Gesch=C3=A4ftsf=C3=BChrer: Paul Manicle, Halimah DeLaine Prado Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Diese E-Mail ist vertraulich. Falls Sie diese f=C3=A4lschlicherweise erhal= ten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, l=C3=B6sche= n Sie alle Kopien und Anh=C3=A4nge davon und lassen Sie mich bitte wissen, dass d= ie E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don=E2=80=99t forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 15 12:15:35 2021 Received: (at 47799) by debbugs.gnu.org; 15 Apr 2021 16:15:36 +0000 Received: from localhost ([127.0.0.1]:38957 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX4eV-0005Lh-KR for submit@debbugs.gnu.org; Thu, 15 Apr 2021 12:15:35 -0400 Received: from mail-wm1-f48.google.com ([209.85.128.48]:54054) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX4eT-0005LU-Ti for 47799@debbugs.gnu.org; Thu, 15 Apr 2021 12:15:34 -0400 Received: by mail-wm1-f48.google.com with SMTP id w186so8239441wmg.3 for <47799@debbugs.gnu.org>; Thu, 15 Apr 2021 09:15:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=3P4pOXUc73v3nEtGcJm3spcpyfX18oJJVp9qIQI0+Jw=; b=OaZ/5UH3EoAN8eCtHi65AI7sfK8v474HejNfp3SeuRfwgqpe4Px6fuMEiG56OopSl5 mARu4T21Tw4VXkI8mc4+SUbKnmdI5GeX0YkppMQwJS5AS3CqINv/F2KghKhqGSOeLVdk FjAfb9U1WBpMIBwICyE5LFiQFSPUaEQTRCLU0yPIilkTdTBpOMKNiZYDGoU3WjuD/ECA zbBxZ/eKn7A61bzgTAuNty01Fd2HZAvhxjgUvOubb/UsLB7dYl6iFIwXI2O1+9/7LfGB QL/88SWJ7lprT4/QktfcN7/g1Ta+WwAhPDu4rZPAld3O6vGNSbkCfq5AKm43JJI8K4Le vXTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=3P4pOXUc73v3nEtGcJm3spcpyfX18oJJVp9qIQI0+Jw=; b=FjUfhlyewBqdHgDUB/q9sbbEk9zS6L1RjcfmAG0p1u6pwWV0+YIV1DQ2Ez4HS1Faqo uitGKsIWlzyaGUD1inFh3Qc7zARS12jFlDsTMclROlHVI1Imwc1dGWBsMlobezBbe9Te py+FxPItzvWUf9RAzG3MqzljtLk1G6oXLDvBgYPcq1n7vnE/hQvZLS5ibMuDW6D8w2Ed N6AGX1MhD6v8/sKrbPaAoXhZHdpxzJbjg0/x1jmWlw5zBC35oFMgU+fyFaUIPWbAYbzt fl7zHmNdj5KTNQk94YKVh6twkR34uqnQ2O0qIsXxcwd6pP4jgvVY1FQgPZuUdDwARIO2 YmOQ== X-Gm-Message-State: AOAM5327Fa/ItNtxm3Z1TeBirgcpb+P0iJQH6bogEQUL5n+EeIUoboz0 wJr1onJmUdTOa6dP24UiXrb0UmMJQwA= X-Google-Smtp-Source: ABdhPJxamCeB0iPYU/ZkbZrsUiWhWBbc7O2olAxinE1GCkUr0waErRM9iwjrogt6GWcIJQBtaEf3MA== X-Received: by 2002:a1c:2985:: with SMTP id p127mr3939966wmp.165.1618503327869; Thu, 15 Apr 2021 09:15:27 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id b206sm3657895wmc.15.2021.04.15.09.15.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Apr 2021 09:15:27 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp Stephani , 47799@debbugs.gnu.org References: From: Dmitry Gutov Message-ID: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> Date: Thu, 15 Apr 2021 19:15:25 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Hi Philipp, On 15.04.2021 16:43, Philipp Stephani wrote: > emacs -Q -batch -l project -eval '(print (project-files (quote (transient . "/:/"))))' > > ("find: ‘/:/’: No such file or directory > ") > > Note that the error message is listed as a file. > > 1. `project-files' should unquote local filenames before passing them to > `find'. > > 2. `project-files' should check for errors returned from `find'. Would you like to propose a patch? I don't really understand the file quoting feature. Is project--files-in-directory supposed to unquote? Should project--vc-list-files do that as well? Does read-directory-name return quoted names when needed? Can locate-dominating-file return one? From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 15 12:26:43 2021 Received: (at 47799) by debbugs.gnu.org; 15 Apr 2021 16:26:43 +0000 Received: from localhost ([127.0.0.1]:38970 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX4pG-0005bw-Oy for submit@debbugs.gnu.org; Thu, 15 Apr 2021 12:26:42 -0400 Received: from mail-oi1-f181.google.com ([209.85.167.181]:40698) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX4pE-0005bj-G6 for 47799@debbugs.gnu.org; Thu, 15 Apr 2021 12:26:41 -0400 Received: by mail-oi1-f181.google.com with SMTP id u16so7532909oiu.7 for <47799@debbugs.gnu.org>; Thu, 15 Apr 2021 09:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TVC/EQcCW9ueIzdOrYs6XTjYywWo56tds5iA0UujIxg=; b=KxEgLaU1WBut9HIM5FiN74ZvnfpMVjVa7UiJjDkXPqGQ66IDoDUKSxvpwHOAr0p+jP QIarvOX3nzmGMLOhRcAHdZs9AFQnKij0qZyaKtRMB+pgrY38pcNs9+eHvViDcTb2q1Si bRGvnWjG/5MMJd0mDsCw1YiJOv5b7nEH4pvc6JPNgUIXgVgMYmhqIEUkCLIqPeoUjCbU idxYLF6mbYaneSnGhyjgrP03vnj8cZ2Hdl+TqSucODwwGAbqeLgSTq0OIt91K0mIARgd 5EpFPfLOIjWcDv0dA5oIVCKjxVvtRYjLnWRT4wy3rdk9v8DJlu3O5ZmtQ6yCQaqKSLwN wuKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TVC/EQcCW9ueIzdOrYs6XTjYywWo56tds5iA0UujIxg=; b=Umxsn/jQlVJeI+B+kBJZAfCOmQJHHajj9BceSfkN1Bo0JTKS8JgEo1Txp2bWI2Kv70 +xnjcl6cq3RlW0tLOUH91+1FjErpgKp7w3XTvGOhkbUneSkFN/FO+NaR/Ucz7IvDnFWA ny2v5UhiLr7cXQRaPMA9uLfq0hsjbvmeP//qS5qU5LiG6CWCrR4flRT+Tjdf4h98w3op LdAZDwekPOSICmBabhpz8A1a4aqxq0AcpOTMtfqgKmdFFMyylgiJ3DBPjdk1L1GnYyrA jmNFgKLrOqPF15W6OEOhPF85pyds/sQ9GNNE9uz3iC5ltJOeFtFZWRaznwi4dg3fSx1H s6jg== X-Gm-Message-State: AOAM533trcxAy4c14TRt/gLwpTwm2WkfdHLyaAz3fermHT83ztC0iK2D CVL63zFGs7qauJBvItWtuzIJLLcZVjTBD0Dz8Qg= X-Google-Smtp-Source: ABdhPJwQxpjkaqjd4+Do9S8+TiWgPojcHigGeMGEICgkTbMI/AdnASQIN5O0GFk4u1Yzfasi+gL+h60ggh3/z0cSa2w= X-Received: by 2002:aca:1814:: with SMTP id h20mr3116225oih.150.1618503993371; Thu, 15 Apr 2021 09:26:33 -0700 (PDT) MIME-Version: 1.0 References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> In-Reply-To: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> From: Philipp Stephani Date: Thu, 15 Apr 2021 18:26:21 +0200 Message-ID: Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Dmitry Gutov Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am Do., 15. Apr. 2021 um 18:15 Uhr schrieb Dmitry Gutov : > > Hi Philipp, > > On 15.04.2021 16:43, Philipp Stephani wrote: > > emacs -Q -batch -l project -eval '(print (project-files (quote (transie= nt . "/:/"))))' > > > > ("find: =E2=80=98/:/=E2=80=99: No such file or directory > > ") > > > > Note that the error message is listed as a file. > > > > 1. `project-files' should unquote local filenames before passing them t= o > > `find'. > > > > 2. `project-files' should check for errors returned from `find'. > > Would you like to propose a patch? > > I don't really understand the file quoting feature. > > Is project--files-in-directory supposed to unquote? Should > project--vc-list-files do that as well? I think only functions that pass filenames to external programs (that don't know about Emacs filename handlers) should unquote. Unquoting can change the meaning of a filename. >From what I can see, project--vs-list-files shouldn't unquote, because it doesn't pass filenames to external programs. > > Does read-directory-name return quoted names when needed? Can > locate-dominating-file return one? Yes, both of these can return quoted names. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 15 12:44:37 2021 Received: (at 47799) by debbugs.gnu.org; 15 Apr 2021 16:44:37 +0000 Received: from localhost ([127.0.0.1]:38988 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX56b-00062d-9E for submit@debbugs.gnu.org; Thu, 15 Apr 2021 12:44:37 -0400 Received: from mail-oi1-f173.google.com ([209.85.167.173]:38537) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lX56Z-00062O-OG for 47799@debbugs.gnu.org; Thu, 15 Apr 2021 12:44:36 -0400 Received: by mail-oi1-f173.google.com with SMTP id b3so10121602oie.5 for <47799@debbugs.gnu.org>; Thu, 15 Apr 2021 09:44:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=OdPl3zecFy44hNy2aYMQqnTMJXn+Y26k1f9+3KzOb8w=; b=S6cq0e7vgwqXjRtzDYmuq3JdVLXxXaJnHAH3ulxKSD4p8f6LqLznZ02RonLrk4XJVX T5hpXgcawD+dD/XH7jL0kfIQHjlJB4BNCP0k/rlIUdP5o6wK9q+VWkg7Kk9lknF8fRdV 6ZzxZ8mvYUWcoFX147VYWq5Pqo9ZGCY0YX6CkUPnJ6G8WV309fzgNJNkL3rWHfofOMHB fEzlaxj+3p2DResBFzzB+9R/rMhvi2EooB9NfnuFE1CgSIwMUXWW4XdOaJAZ4Z35GDxz 5jb+BnW5N43bDty0WztnZIAWWUXhQixRnaAdbOoO68xG2YOIuUWcQjgl3JzgNY3ejTW1 jCYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=OdPl3zecFy44hNy2aYMQqnTMJXn+Y26k1f9+3KzOb8w=; b=gHnGHFlLITjMl7XmHRlI+1wAsopKmp/0Av9OM0aJgNfVI2ShfJ44ckEx+iFC/joEsn F9LfGtxn7V4WBnDNeg0RdwKDMxB/O4JIdEAUpMYSnM76znYPJyoE5fBNt7Qy1dvpBqvf zr+0V41ooM0+xDwBcEpYrvnkC6t5t0FnaD7p+uIo+td8TKe99MF5tk7JxhqTKRZ19lG6 fPppTqI7eFnVmlPd5dXUTcS/Aj4NRDg30pXfXtsBplAVEAOvGf35LP0g0L3vodTa9Q+R VozpRYExVjowjXsSOuvA+AaEiIs6z6Nkltah6bU4cnPKnZfHAPJbwKStd09BYJ3bjZmb +8KQ== X-Gm-Message-State: AOAM530kOhFjZVPTuuGnSRkaoYSFwfwTjtWOimJKc2NGt9N9RQ40AQyD vmQBM6deGPUUo5zhhj4SztfUhpb5z9m+bAwSvFk= X-Google-Smtp-Source: ABdhPJxj/mZh4SVAvg24skhmvoJS++EVFLoEFbJdZHEffbpUtQjFh+Yj76MBa2q7+5MAyuYTmY4lIie7BiPSX9hN0Qw= X-Received: by 2002:a54:4582:: with SMTP id z2mr3228156oib.158.1618505069954; Thu, 15 Apr 2021 09:44:29 -0700 (PDT) MIME-Version: 1.0 References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> In-Reply-To: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> From: Philipp Stephani Date: Thu, 15 Apr 2021 18:44:18 +0200 Message-ID: Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Dmitry Gutov Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am Do., 15. Apr. 2021 um 18:15 Uhr schrieb Dmitry Gutov : > > Hi Philipp, > > On 15.04.2021 16:43, Philipp Stephani wrote: > > emacs -Q -batch -l project -eval '(print (project-files (quote (transie= nt . "/:/"))))' > > > > ("find: =E2=80=98/:/=E2=80=99: No such file or directory > > ") > > > > Note that the error message is listed as a file. > > > > 1. `project-files' should unquote local filenames before passing them t= o > > `find'. > > > > 2. `project-files' should check for errors returned from `find'. > > Would you like to propose a patch? I've now pushed a minimal fix to this specific problem (commit 157bfc1812c51a0a48162c71eadf7959f7de9ac6), but there are probably more places that should get fixed, e.g. xref--find-ignores-arguments. project--files-in-directory also still ignores any errors from the find binary. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 15 21:08:51 2021 Received: (at 47799) by debbugs.gnu.org; 16 Apr 2021 01:08:51 +0000 Received: from localhost ([127.0.0.1]:39607 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lXCyZ-0007zW-6J for submit@debbugs.gnu.org; Thu, 15 Apr 2021 21:08:51 -0400 Received: from mail-wm1-f50.google.com ([209.85.128.50]:33718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lXCyV-0007zH-2s for 47799@debbugs.gnu.org; Thu, 15 Apr 2021 21:08:49 -0400 Received: by mail-wm1-f50.google.com with SMTP id q123-20020a1c43810000b029012c7d852459so4738142wma.0 for <47799@debbugs.gnu.org>; Thu, 15 Apr 2021 18:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=4/BC5C8S+Uko3Py1v9tcGkwyRf/iM6IX3N2Od+A9M7U=; b=I6/CULbnTn1IuWSsh+ld9Ouo6wE5YFZoPEAvTXwqWWv/KwypZn1qHpYPa0ZG8mup8z iNZBOhLMOEpB8n3qDkZsmEkp4RedObYMkSU8XXa0jHoTTqRzQcFU32FBf+RdHdaHWKL2 /sVuW4mq3Zl+4Q/07KdryVtx/DYKqtmokWKIyc/bLdnBLbAFekVBrgw+YWDlIK0jveC2 jw70thhDX8pX1yHqOWM1T+F4aakV7eeRRJQJ9+2mNU4bQsIAZgtxFLmGLeW3MqjFjand Dy+S2ofT36CBMZBSSb09GfmBEZrCKvPGoRY9+tQuGNIqBNUmQBJGpgTxBYktE5gvlZ4Q 1jQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=4/BC5C8S+Uko3Py1v9tcGkwyRf/iM6IX3N2Od+A9M7U=; b=EwwESXoM22Sha9W8oQ0M2burLnZR2O9v2z5LqBLdzTI+wBdOB+pVhmrkGX/68rqdKn FezvwUie3iZYgIrssRmpxwaGHw3Ji/35T5RaWLuF86kXOYdA+bZGqzA4GMVtlAIfPIE6 QgxoSL5mmiTj3Vvb3r3RkqsowcgS6W7BWHMviDg0jy0djchUZwjGyh/yrzrj+UQtwosm 9scHtt7xRcek9OG2E5FZBLzD0GOn8e/LptnkeBVCxFP52TVISmNR5IIiMG/sx+JlW0sK cjEJFXtEfZyO/zM+7jpqKc9qIhDzYn4Z3SnYFw+oGd9IzmPvaLEkQUVkT4Jq6I5HuLjI rW9A== X-Gm-Message-State: AOAM533n0O70B4DmDmNUBXzzaKezBNLX7Y7g7ynjRHg6zGz7owulTtWc n1RANwXgEYCVSIlosE9AXo0Y9XGYyjo= X-Google-Smtp-Source: ABdhPJybooZv6S+q0tqD274LYtKYGp5DGgzj5ELGiBSv76uuUeOfLQoKfqsJAARG1c17yX6/TgNwEA== X-Received: by 2002:a1c:771a:: with SMTP id t26mr5642357wmi.96.1618535321199; Thu, 15 Apr 2021 18:08:41 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id v8sm6835554wrt.71.2021.04.15.18.08.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Apr 2021 18:08:40 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp Stephani References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> From: Dmitry Gutov Message-ID: Date: Fri, 16 Apr 2021 04:08:38 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 15.04.2021 19:44, Philipp Stephani wrote: > Am Do., 15. Apr. 2021 um 18:15 Uhr schrieb Dmitry Gutov : >> >> Hi Philipp, >> >> On 15.04.2021 16:43, Philipp Stephani wrote: >>> emacs -Q -batch -l project -eval '(print (project-files (quote (transient . "/:/"))))' >>> >>> ("find: ‘/:/’: No such file or directory >>> ") >>> >>> Note that the error message is listed as a file. >>> >>> 1. `project-files' should unquote local filenames before passing them to >>> `find'. >>> >>> 2. `project-files' should check for errors returned from `find'. >> >> Would you like to propose a patch? > > I've now pushed a minimal fix to this specific problem (commit > 157bfc1812c51a0a48162c71eadf7959f7de9ac6), but there are probably more > places that should get fixed, e.g. xref--find-ignores-arguments. > project--files-in-directory also still ignores any errors from the > find binary. Thank you. I've added error handling to project--files-in-directory in a follow-up commit. Regarding xref--find-ignores-arguments, it seems it would be economical to do the quoting on the value passed to it. See f955df1. Regarding your change, though, have you tried project-find-regexp in a "transient" project with a quoted root directory name? You've made project--files-in-directory quote the returned file names, but that list gets passed to xref-matches-in-files, which pipes them to find-grep in the end. I suppose xref-matches-in-files could use a step similar to (when remote-id ...) that is already there. A bit unfortunate for the users of large projects with quoted names, but not sure what else we could do. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 18 16:07:01 2021 Received: (at 47799) by debbugs.gnu.org; 18 Apr 2021 20:07:01 +0000 Received: from localhost ([127.0.0.1]:47915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYDh6-0002L2-US for submit@debbugs.gnu.org; Sun, 18 Apr 2021 16:07:01 -0400 Received: from mail-oi1-f173.google.com ([209.85.167.173]:40499) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYDh5-0002Kq-QN for 47799@debbugs.gnu.org; Sun, 18 Apr 2021 16:07:00 -0400 Received: by mail-oi1-f173.google.com with SMTP id u16so16182971oiu.7 for <47799@debbugs.gnu.org>; Sun, 18 Apr 2021 13:06:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aZkVLmVhnj7mgW6sa7Zvmf7xw7pDhHHddKWhdGkg60A=; b=Ya9nzu+jcS05f1bRKOKPY8TdcU2H7RX7v0cZ8mAIuF3yGl4Bc9S5vW3fXTNWxqIBKI LQGB4fEkMCUAQW90FDoAYKCP7MCPxsGyyC4rKceCjGH3wUy0LtD/ti5xtoxKqN0G0KYi xzkoGW57vlkuysSFtoOzf7PO+00uRJUjJH6IhnMXJUW/9i/u37h+pCCjnVaVZFhayHN2 3aTFrsuzD5zB2x1swqzHU4GsMlKCc5X642jy6AhZefo316RU409tVelG4hIi0IrCU3tf nBqh3srIO2xWPLpjtjCH+FZhwt7kUPsxqr0M79MpXsfIsSpMF7O+UJ0OFT+4HnGpq6bh DK6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aZkVLmVhnj7mgW6sa7Zvmf7xw7pDhHHddKWhdGkg60A=; b=K/0CU+UmMMhKUkw9SLmTygz035gepjbgtuAoH47+pHyTqRp5mJrOaKKUdW/RMpwvEw 2OxI6Fu5D7Bn+E54fhxRSF1XuwKQObVnRNyMODT5m0VCWr/rOindtixnHW86wDo4Ar91 ndHM6gjLS0IuIhOFcwW75p7KG/ZOAbQG+LjyLu09jddnvcGzBnNAweEShojHIptkfyVr ZqV+wlv8YaoGP2juKJKzkCY5KaoSXlCkXVam7aCedu+jh52eSiulc/yqsgp/2oQfHUeT h3i+7SIvqc8rv9FNwVLABZkfF9GO98nxGbofzreyrQUVfW6x2kxEVzYzUXtsXuwigsLV 7SIQ== X-Gm-Message-State: AOAM5322OJZxpcl70/H/3+bq/Hn3sYFXo5dN09gaUXQiZHXy1co0RmBy 57Q1o40Nu/c0otboBjxGaaVgdxLmDMrpKS2IstY= X-Google-Smtp-Source: ABdhPJxi9QQVydENe37oKKOuA9bTIgmcDvyG2DGxDLp6ZB+ezcO6FkJM+L/t9WhQfZm735z8gAkYRnNxIbvyyDQdk+Y= X-Received: by 2002:aca:1814:: with SMTP id h20mr13390314oih.150.1618776414156; Sun, 18 Apr 2021 13:06:54 -0700 (PDT) MIME-Version: 1.0 References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> In-Reply-To: From: Philipp Stephani Date: Sun, 18 Apr 2021 22:06:43 +0200 Message-ID: Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Dmitry Gutov Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am Fr., 16. Apr. 2021 um 03:08 Uhr schrieb Dmitry Gutov : > > On 15.04.2021 19:44, Philipp Stephani wrote: > > Am Do., 15. Apr. 2021 um 18:15 Uhr schrieb Dmitry Gutov : > >> > >> Hi Philipp, > >> > >> On 15.04.2021 16:43, Philipp Stephani wrote: > >>> emacs -Q -batch -l project -eval '(print (project-files (quote (trans= ient . "/:/"))))' > >>> > >>> ("find: =E2=80=98/:/=E2=80=99: No such file or directory > >>> ") > >>> > >>> Note that the error message is listed as a file. > >>> > >>> 1. `project-files' should unquote local filenames before passing them= to > >>> `find'. > >>> > >>> 2. `project-files' should check for errors returned from `find'. > >> > >> Would you like to propose a patch? > > > > I've now pushed a minimal fix to this specific problem (commit > > 157bfc1812c51a0a48162c71eadf7959f7de9ac6), but there are probably more > > places that should get fixed, e.g. xref--find-ignores-arguments. > > project--files-in-directory also still ignores any errors from the > > find binary. > > Thank you. > > I've added error handling to project--files-in-directory in a follow-up > commit. Thanks. > > Regarding xref--find-ignores-arguments, it seems it would be economical > to do the quoting on the value passed to it. See f955df1. Since it's an internal function, this is mostly a matter of style. I prefer using Emacs filenames as function arguments as much as possible; it's the expected behavior for functions dealing with files, and Emacs doesn't have a strong enough type system to distinguish "Emacs filename" (which can be quoted or remote) from "filename for external programs" (which must be unquoted and local). > > Regarding your change, though, have you tried project-find-regexp in a > "transient" project with a quoted root directory name? > > You've made project--files-in-directory quote the returned file names, > but that list gets passed to xref-matches-in-files, which pipes them to > find-grep in the end. I suppose xref-matches-in-files could use a step > similar to (when remote-id ...) that is already there. Good point, I've pushed 6ebc6e12cf to fix. But xref.el should also be fixed. Maybe I'll find some time to take a look at it. > > A bit unfortunate for the users of large projects with quoted names, but > not sure what else we could do. Quoting is a purely lexical operation which should be reasonably fast even for a large list of files. For truly enormous projects, something like "list of all project files" is infeasible anyway. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 18 16:21:10 2021 Received: (at 47799) by debbugs.gnu.org; 18 Apr 2021 20:21:10 +0000 Received: from localhost ([127.0.0.1]:47952 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYDuo-0002iE-JK for submit@debbugs.gnu.org; Sun, 18 Apr 2021 16:21:10 -0400 Received: from mail-wm1-f50.google.com ([209.85.128.50]:41876) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYDun-0002i2-3f for 47799@debbugs.gnu.org; Sun, 18 Apr 2021 16:21:09 -0400 Received: by mail-wm1-f50.google.com with SMTP id o21-20020a1c4d150000b029012e52898006so7307034wmh.0 for <47799@debbugs.gnu.org>; Sun, 18 Apr 2021 13:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=qkcclDzjjj14vNCE6S5TzCbpgTGRWF2NISCj+vP64t8=; b=hLApbK2w4NhQIwJK9QeFDTiMi8lWtDiyiaOKR4QDtBu6HqRpQi4tgeO23tf369aCxS iC5ASLOmpDVfo1kIRaKoxny1uwUOt9ehZfOoEseaA6emNng9JtbziaN3S95pToDZ95ii DUtMqPnE4+Ek0Hh1cLWSblHz/OF0Ksca7hQGJu6xKymfiV6tA0PT3hUPPipYkYeqvd7+ nzDZvXENsE/Uy3aoDo0vZLp8iqGxpvtU76cHT2778gdhNJp18ywy+lALBLAi/B8DFKa+ oCNXBXMjalX0TCHglb4wX1EoF5J3kp5ryowCBb20Vd6oapD9ftf+i+0mRmgjxZmKf1gf baPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=qkcclDzjjj14vNCE6S5TzCbpgTGRWF2NISCj+vP64t8=; b=H1IZNc0hsaj27GUvb/1wDNC4ANN2b12bJEz/82ddpEyWFNppThdSxef4cnONL9MaRs EUQJPEcvM2UxotnWL6wyJJMKzgxWok/LbqKWCiNvP9bQTBikpmMGd9ZwR89fqEEYEZTh LlXKM8oMSvr+TUM1n79AYjPY5kchi7VmZXqhEmuRSgFXYDiAQkKaI/iByRwIvlHLThoN Oq94IA5enRAwt2LsLSRS40xPcXNNgS2Q9hkdUUfWqiFhEuPEw1T2RVePFZ8NY8Gnvqw7 zPVUEJZTfSIPlSKBClIexC7JgpQjviXa6lTWEeFUeNu+qvwepZlAhnkRR4Bw/oiZJ4N6 6Jow== X-Gm-Message-State: AOAM531xBcMkT0+PHqoElSDjjf2RXn7lq9lOynBq9eS6k4LXuIgIR7bh x+gofm36VyahLs1Jo0QYuaAB5CCGnPA= X-Google-Smtp-Source: ABdhPJz04HJdzeUSlhSaNhGXqTJG1lK6X1lRUAzDwRe5bbx7vw/OXRJq1ulroD8R30k7v4oirqEmvw== X-Received: by 2002:a7b:c401:: with SMTP id k1mr18490839wmi.48.1618777263287; Sun, 18 Apr 2021 13:21:03 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id v3sm17502202wmj.25.2021.04.18.13.21.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 18 Apr 2021 13:21:02 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp Stephani References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> From: Dmitry Gutov Message-ID: Date: Sun, 18 Apr 2021 23:21:00 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 18.04.2021 23:06, Philipp Stephani wrote: >> Regarding xref--find-ignores-arguments, it seems it would be economical >> to do the quoting on the value passed to it. See f955df1. > > Since it's an internal function, this is mostly a matter of style. I > prefer using Emacs filenames as function arguments as much as > possible; it's the expected behavior for functions dealing with files, > and Emacs doesn't have a strong enough type system to distinguish > "Emacs filename" (which can be quoted or remote) from "filename for > external programs" (which must be unquoted and local). Matter of style, yes, but the way I fixed it requires one unquoting instead of two, which seems like a win. Your point is also valid, of course. >> Regarding your change, though, have you tried project-find-regexp in a >> "transient" project with a quoted root directory name? >> >> You've made project--files-in-directory quote the returned file names, >> but that list gets passed to xref-matches-in-files, which pipes them to >> find-grep in the end. I suppose xref-matches-in-files could use a step >> similar to (when remote-id ...) that is already there. > > Good point, I've pushed 6ebc6e12cf to fix. But xref.el should also be > fixed. Maybe I'll find some time to take a look at it. Yes, I think the fix needs to be in xref-matches-in-files, and it also needs to use file-name-quoted-p similarly to (file-remote-p default-directory, to avoid the mapping overhead when the list of file names is large. It's a realistic use case, and the impact is significant (e.g. 1s to list files, 2s to unquote them all), so please look into this soon. The fix in xref should be simple enough, I'd just like for someone to realistically test it before installing (I can send a patch, if you want). >> A bit unfortunate for the users of large projects with quoted names, but >> not sure what else we could do. > > Quoting is a purely lexical operation which should be reasonably fast > even for a large list of files. > For truly enormous projects, something like "list of all project > files" is infeasible anyway. File listing in projects backed by Git (at least) is quite optimized. For example, in a gecko-dev checkout: (benchmark 1 '(project-files (project-current))) => 0.97s (benchmark 1 '(mapcar #'file-name-unquote (project-files (project-current)))) => 2.97s From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 19 10:49:04 2021 Received: (at 47799) by debbugs.gnu.org; 19 Apr 2021 14:49:04 +0000 Received: from localhost ([127.0.0.1]:51585 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYVCy-0008W3-8I for submit@debbugs.gnu.org; Mon, 19 Apr 2021 10:49:04 -0400 Received: from mail-ot1-f53.google.com ([209.85.210.53]:33750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYVCw-0008VC-7P for 47799@debbugs.gnu.org; Mon, 19 Apr 2021 10:49:02 -0400 Received: by mail-ot1-f53.google.com with SMTP id 92-20020a9d02e50000b029028fcc3d2c9eso10305489otl.0 for <47799@debbugs.gnu.org>; Mon, 19 Apr 2021 07:49:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Huyknc7hxw59fqP+7MQvfUhVmIcdptjmtI+E2WP4wlo=; b=IfhdWyBlgj4G0tyMn5FsndCilsKGXfbYKBqSMGzoJQ2wDb+XXkWlBeWww+8DUIeBXe HXQInmFmnXFthPLefISmwBnPnDZCzjck65NHCePy4PQgYWXtm5NNbO7ZduNsCsgPz1YL ZIwe/c5kngMKzweyce72AL+wzXZNYGcJNiXEcSWef2j7EcZ4fJO+DjVr5ll4C9HtG0Ys uJBd6cj/4+BMKVyYh9d7MRQzmxWk18ogROJQyngj5wjlIH8qUhcH4ATu4lRYzFjT8xgh 0AUC9XJjO9oXkrKslG0pzYzPSKppn9okbbT2U+T9/zLL13ZNcOxwfY216mHmy3Ou2T7u jw8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Huyknc7hxw59fqP+7MQvfUhVmIcdptjmtI+E2WP4wlo=; b=LuiAZ1/NnLUeEqvurTji4lt274ZJd8XXlcvV3YBlH/Jo9WVyu7Lt31dRNMyTxGrtTJ H039mwT66vrOuVLCLqH4fHRkvfNMoBxB/0qyxezPjo0Xo/B4oeV1MyFsoyAUWmFXbyXX oijkHyLvxbYtEBeZWchazHPlSwSo9u5mqdN+AAuSp7TmANtaCydEhTOHBZ4m3Ow0tgCp WwX3/GC/CT8cPa60p+6EpfRbBN8EPpaf/5/h5FQRMLFi+mJsJ4qpM/VB0+8LXDmQANp6 vlxKN0olsjBnN+zmT15bOgizkm/X8WFFUgQTE4LiAgnoVM5z02Pu2rH61u7bT9Qi/q+V 5+AA== X-Gm-Message-State: AOAM531m0SpR0LNuvE6DrJNejqUuQhHkXLJGYjyRLCUu8Z32qMdzA0V1 jDwPK2W4go4zcwzrmhsWFTWzRWElUglTqWlPN9I= X-Google-Smtp-Source: ABdhPJw9t4oKfI84V4vpJ7rYsz0p3+NTFjs879eHpKzga9Rlnn1E/xUAZbmAOQFniPysuApvUSTTst+DykeD92K4nio= X-Received: by 2002:a05:6830:150d:: with SMTP id k13mr12425918otp.156.1618843736465; Mon, 19 Apr 2021 07:48:56 -0700 (PDT) MIME-Version: 1.0 References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> In-Reply-To: From: Philipp Stephani Date: Mon, 19 Apr 2021 16:48:45 +0200 Message-ID: Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Dmitry Gutov Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Am So., 18. Apr. 2021 um 22:21 Uhr schrieb Dmitry Gutov : > >> Regarding your change, though, have you tried project-find-regexp in a > >> "transient" project with a quoted root directory name? > >> > >> You've made project--files-in-directory quote the returned file names, > >> but that list gets passed to xref-matches-in-files, which pipes them to > >> find-grep in the end. I suppose xref-matches-in-files could use a step > >> similar to (when remote-id ...) that is already there. > > > > Good point, I've pushed 6ebc6e12cf to fix. But xref.el should also be > > fixed. Maybe I'll find some time to take a look at it. > > Yes, I think the fix needs to be in xref-matches-in-files, and it also > needs to use file-name-quoted-p similarly to (file-remote-p > default-directory, to avoid the mapping overhead when the list of file > names is large. > > It's a realistic use case, and the impact is significant (e.g. 1s to > list files, 2s to unquote them all), so please look into this soon. The > fix in xref should be simple enough, I'd just like for someone to > realistically test it before installing (I can send a patch, if you want). Hah, I wasn't aware that quoting/unquoting is so slow. Rather than making assumptions in xref-matches-in-files, maybe we could work more with relative filenames. For example: 1. Add another project method "project-relative-files" that returns filenames relative to the root. By default, this would call project-files and make the filenames relative, but project implementations can provide an optimized implementation. 2. Give xref-matches-in-files an optional root directory argument and allow users to pass names relative to that root. Then I think both project and xref could leave these relative filenames alone. WDYT? From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 19 16:49:02 2021 Received: (at 47799) by debbugs.gnu.org; 19 Apr 2021 20:49:02 +0000 Received: from localhost ([127.0.0.1]:52224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYapK-0005HM-0t for submit@debbugs.gnu.org; Mon, 19 Apr 2021 16:49:02 -0400 Received: from mail-wr1-f53.google.com ([209.85.221.53]:33576) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYapH-0005Gu-8Z for 47799@debbugs.gnu.org; Mon, 19 Apr 2021 16:49:00 -0400 Received: by mail-wr1-f53.google.com with SMTP id g9so19370675wrx.0 for <47799@debbugs.gnu.org>; Mon, 19 Apr 2021 13:48:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=VR3x5kLk0tc+3uoKDtJKLD8yyA+QxUdJHcFENBkJAv8=; b=fb4GK31kId2M8yUpjiip7oxwQo0yItCnDXbZ4/IkmFlTQlG+qWQEQFQVJr0Nb0I6ij DRxqeHVJwanUzHMywZoFYXB1tn8UssraRyeJiJUzacZ8+APOv+cg7Gu/E66pcFNXQI/P YkSAmRunSICjVsnyUCH46u04xhku0d+/ASDeXQOktUHAhuqAM8tbSVSJ9IMXlyQfINes UCHwbiQ6oLZ4DcpCBR4dPMt6siYCA0LY+BwrCPAJltBGrYdMpZ/6qbhYWXHI9OyhohxK DA4RJVyaObIXNfCAOmbgjPtUj2UVo0B3UIkLzMdPR8y7ehrQpBc/KexmUfpWn2LZGhxn 48Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=VR3x5kLk0tc+3uoKDtJKLD8yyA+QxUdJHcFENBkJAv8=; b=s9AgJvp5qWVEWZdtGN828qM5s1Hg8P1yvnWpV5MhC23463QEjF7AoVi8DDAp1y0RTg 63Rug5DLfRfA27u11zcAS3UzDY5dqO7bcQdLnaO3czBmM8Q4gflKEHkzLWOfwKvCTETb Xs/6C7Xu+dcOf7kVEFnIJ0EZvt/msMXrOcYtrpVQdVLpTT3ZiDL8k/Sj9MKaeNjvga9s a1Jwo1jtBSkFlyQiFl79NDfVdu1waSilqk7o7sFakTMHYqSIuVgWj59zpPI5IWDt5AtH mPhQCXyehqX+4nTYOSYUUCrhGWQIy63LnTIOq+PTcV0+CMBIkdVaVgzhSnDc2Q1fUkKd Pa6A== X-Gm-Message-State: AOAM533znSLjZkGbCKHBHAMDIbUqBArl43snwdG+n+vbQvfgRFELibtj ATlNhK/jbZJ/HKU5N8r8qNPeQIz8rp0= X-Google-Smtp-Source: ABdhPJwzYFvqot5fW1TM5O5Ywdgt335Yyff1J7h65eD3WA7i+rzTjeaZGmOFUBFA1yJlhUEOoRVPIw== X-Received: by 2002:a5d:4948:: with SMTP id r8mr16399283wrs.35.1618865333329; Mon, 19 Apr 2021 13:48:53 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id f23sm818116wmf.37.2021.04.19.13.48.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 19 Apr 2021 13:48:52 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp Stephani References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> From: Dmitry Gutov Message-ID: <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> Date: Mon, 19 Apr 2021 23:48:51 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 19.04.2021 17:48, Philipp Stephani wrote: >> It's a realistic use case, and the impact is significant (e.g. 1s to >> list files, 2s to unquote them all), so please look into this soon. The >> fix in xref should be simple enough, I'd just like for someone to >> realistically test it before installing (I can send a patch, if you want). > > Hah, I wasn't aware that quoting/unquoting is so slow. It's file listing that is fast, rather. ;-) The project in question has 200'000 files. > Rather than making assumptions in xref-matches-in-files, maybe we > could work more with relative filenames. For example: > 1. Add another project method "project-relative-files" that returns > filenames relative to the root. By default, this would call > project-files and make the filenames relative, but project > implementations can provide an optimized implementation. > 2. Give xref-matches-in-files an optional root directory argument and > allow users to pass names relative to that root. > Then I think both project and xref could leave these relative > filenames alone. WDYT? We've discussed this before, but it's a change in the API, a +1 method for a very minor feature. And how will we explain anyway that xref-matches-in-files, when called without the new ROOT argument, doesn't handle remote or quoted file names? So if you can fix this to avoid performance loss in the general case, that would be a good improvement for now. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 21 20:46:29 2021 Received: (at 47799) by debbugs.gnu.org; 22 Apr 2021 00:46:29 +0000 Received: from localhost ([127.0.0.1]:60119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZNUD-0005zj-2f for submit@debbugs.gnu.org; Wed, 21 Apr 2021 20:46:29 -0400 Received: from mail-ej1-f51.google.com ([209.85.218.51]:41600) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZNUB-0005zT-4Y for 47799@debbugs.gnu.org; Wed, 21 Apr 2021 20:46:28 -0400 Received: by mail-ej1-f51.google.com with SMTP id mh2so44565543ejb.8 for <47799@debbugs.gnu.org>; Wed, 21 Apr 2021 17:46:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=aZrxETZhy2QkPq5d3aM/ToTN0U1Xn4cN/va8nqffDTs=; b=HDy8S1w/M6UZ8bwkDHH4SKFupZsmHTFSUpxgX+py0OBkge/jB43HLtwNK0CuR0cjjh U/q1W4O1klfZOW1wFbOY/kA/rabHbagiWzISI8Z+tsXk1ONPK/KQnTb6Xmx7dcMNdiOV TTQ+L2Ugybx6RADMKD+grL4gYzvxkdJRJdReDSEGlbL/X+7D7PrVevdHwpFsjztuqwQX vPTcY5ChWQfWOwuNzGQeoKy/BujzAovask5OliV/JJul/8Q8CAWkuywwAR9FnhaCkNjk RdkvSioXA6iOcoPxPdcamIsTwt2qy+tDVMKy3ChvQThpy+UmuXtUjkFqQV1z9W221uQj +IMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:references:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=aZrxETZhy2QkPq5d3aM/ToTN0U1Xn4cN/va8nqffDTs=; b=ZiWIG1ifNzMJZPPAz7TgW0pldzP0fJbGlx8+uTmMbtshdrc7lC+IfacRV6dIYlMWOM swlENPaw6sSEGdSUWqok3dM4rTMWjwunP3s1LujDKkUbSzyDvK2IJar5vMWD8MN/GK9B d2zAY//zdLwlQ/a3AWzf34dda5XeyFDqY2b8ou8mwfhzqqCVAi70xutr0kvl6ySvmXUI mtQAHxEaIAj6NyBZnRI0Ksuhf4CXwPqRcNdS8ikPX3OmTcMToJelX5BO9+FIMJtvaz5F pe09ZIRHNZDb5CUK5OVGqlnZlCFKgHORtnElkpF4Qr52yp5/uO/jAuehzIeX1QGfaYH0 zuFg== X-Gm-Message-State: AOAM532LBBwlGfxM4RomMyrOseoswQQooxJueFvI69HULdCQZEMPKqD9 NbHxJSSkihJsqZldPtoFW4paci68cLU= X-Google-Smtp-Source: ABdhPJxdlQZLKlwKibkt0Bem4kFO2xcEnIEECuqHjUtfM1ILWOIVDdBWcNbZU4Il2edS6qJ42DZVQQ== X-Received: by 2002:a17:906:dc90:: with SMTP id cs16mr563369ejc.210.1619052381223; Wed, 21 Apr 2021 17:46:21 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id f19sm729099ejc.54.2021.04.21.17.46.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 21 Apr 2021 17:46:19 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames From: Dmitry Gutov To: Philipp Stephani References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> Message-ID: <40e4cb97-c0e5-9ec1-123d-0f73a39e34c2@yandex.ru> Date: Thu, 22 Apr 2021 03:46:17 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 19.04.2021 23:48, Dmitry Gutov wrote: > And how will we explain anyway that xref-matches-in-files, when called > without the new ROOT argument, doesn't handle remote or quoted file names? The above is probably the only real blocker I see. Else we would need to document it to explicitly only take relative file names, I think. Do we have a precedent in the core library for this? Other than that, an exploration into an API working with relative files names sounds good, actually. It could further speed up file listing, eliminating some concatenations in project--vc-list-files (in the big project I referred to, that takes it from 1s down to 0.75s). If you have the time, patches welcome, even rough ones. > So if you can fix this to avoid performance loss in the general case, > that would be a good improvement for now. In the meantime, I've changed the fix to use the plan explained previously. Downsides: either all files should be quoted, or none (is that a reasonable assumption?), and, of course, users which which do have directories making use of quoting still pay the performance overhead. From debbugs-submit-bounces@debbugs.gnu.org Sun May 16 09:37:16 2021 Received: (at 47799) by debbugs.gnu.org; 16 May 2021 13:37:16 +0000 Received: from localhost ([127.0.0.1]:49875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liGxI-0005Cy-B3 for submit@debbugs.gnu.org; Sun, 16 May 2021 09:37:16 -0400 Received: from mail-ed1-f43.google.com ([209.85.208.43]:33574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liGxG-0005Cm-NO for 47799@debbugs.gnu.org; Sun, 16 May 2021 09:37:15 -0400 Received: by mail-ed1-f43.google.com with SMTP id b17so3865161ede.0 for <47799@debbugs.gnu.org>; Sun, 16 May 2021 06:37:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=JwF7KDPvhUIxjuQ9gPIL+yuwhg0gBiRr40vq2o9aj08=; b=BcCtCnvpZd8VIH9sAKG/j5EGflUXsCngPr14bVwdFoYBn74rz79+Q6KwJ0ebob9LGD 0fevOy4cZNcgFnkGlwq2NoSSmJz6tFUtkXrfLju2SRPGd/wtnoElYlRKPv18U2CMvMmL lkndtAkuatf9dDwkFgCJ1VwliOUQA8QAcFBLabpt6MRBqFZU4/POGk85XywCrsrithXM saA99hGCWh/dSpi8Kb1BYmSQIxeZoQMx4Djq+4qaVl/3kNVk6Sy7ZkXKabN6djhSeEVK /WkDCEENj+IzVyzVkw45WKqm71CHWZ8W5rOLz4g8ilxWoRMWHhk535jElQ9sW0VIoNhs kvtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=JwF7KDPvhUIxjuQ9gPIL+yuwhg0gBiRr40vq2o9aj08=; b=fNKYA5dFD1R0pVCdcbTB34FN73FcGhdPg3RYOEqxhEuYD4sdzjaewZNysGSuDy2euk 2XQAF8kYlwnN2mewDdYuprypw2gnPTtJ7VCvIhb2MpMI/LL5hyZeFU0TQJHZCXIjx30v g3axd+SqtbKMA7lI/oHSkciLXYBoz2XAlJB1q0k99C6kh8CAkSGpOCH2mNdfH/DiTxeh i6oTB26i29/lDmTrWzjE5GUh2BU0POP5skNvz5YVqpfefk+IZsZVpXTzag8NQiyx3/wj bww7LJTzR+pFBpWu8ObZMjnJYUFQgAwrG7nrK00HjWYOct9TCvuijhOF1phuvoLi2UlL 3zFQ== X-Gm-Message-State: AOAM5339KPfi95Cd4gcqKX7DzurnAtXyb8uTXo7mJE7Jm7oFQVFN0o/7 F44K0P1oVOqkLCtex/vKA5Q= X-Google-Smtp-Source: ABdhPJxbSCQ6PjRiYit/WV+3G3XSqpSwWppzze80CycM4E2Vu2OJMwTsbW27wfc9lF7k73rvZIeE3A== X-Received: by 2002:aa7:d90d:: with SMTP id a13mr64584667edr.76.1621172228793; Sun, 16 May 2021 06:37:08 -0700 (PDT) Received: from smtpclient.apple ([46.128.198.100]) by smtp.gmail.com with ESMTPSA id i8sm7362313ejj.68.2021.05.16.06.37.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 May 2021 06:37:08 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames From: Philipp In-Reply-To: <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> Date: Sun, 16 May 2021 15:37:07 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> To: Dmitry Gutov X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) > Am 19.04.2021 um 22:48 schrieb Dmitry Gutov : >=20 >=20 >> Rather than making assumptions in xref-matches-in-files, maybe we >> could work more with relative filenames. For example: >> 1. Add another project method "project-relative-files" that returns >> filenames relative to the root. By default, this would call >> project-files and make the filenames relative, but project >> implementations can provide an optimized implementation. >> 2. Give xref-matches-in-files an optional root directory argument and >> allow users to pass names relative to that root. >> Then I think both project and xref could leave these relative >> filenames alone. WDYT? >=20 > We've discussed this before, but it's a change in the API, a +1 method = for a very minor feature. >=20 > And how will we explain anyway that xref-matches-in-files, when called = without the new ROOT argument, doesn't handle remote or quoted file = names? >=20 > So if you can fix this to avoid performance loss in the general case, = that would be a good improvement for now. Yeah, I think you're right, we shouldn't complicate the API = unnecessarily for optimization purposes. One thing that came to my mind is: in general, in Elisp (not just XRef), = we spend lots of time parsing filenames to support remote and quoted = filenames. Other languages probably solve this by introducing proper = types for filenames (e.g. the Java Path class), which can then hold = preprocessed information about the underlying filesystem (or special = file name handler, in the case of Elisp). How about doing similar for = Elisp? For example, introduce a `parsed-file-name' class or structure = holding the remote/quoting state, or attach it to string properties? I = haven't tried out that idea, but I think it could significantly speed up = the parsing (since we'd only have to do it once and don't have to search = for filename handlers all the time), as well as remain = backward-compatible to "plain" unparsed filenames by allowing both = strings and this new object type. WDYT?= From debbugs-submit-bounces@debbugs.gnu.org Sun May 16 19:23:07 2021 Received: (at 47799) by debbugs.gnu.org; 16 May 2021 23:23:07 +0000 Received: from localhost ([127.0.0.1]:51664 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liQ6E-0003wj-UB for submit@debbugs.gnu.org; Sun, 16 May 2021 19:23:07 -0400 Received: from mail-wm1-f53.google.com ([209.85.128.53]:51145) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liQ69-0003vx-0s for 47799@debbugs.gnu.org; Sun, 16 May 2021 19:23:05 -0400 Received: by mail-wm1-f53.google.com with SMTP id t206so2587583wmf.0 for <47799@debbugs.gnu.org>; Sun, 16 May 2021 16:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=W4gyun+mD1bBV3/7yIOpmKsG6BCZb4Vc4RqB1hSgmDY=; b=nhYQt8xAnHqlwSifk4flSSao1jsSC2+/n/JdGNUAEeLgqaJmaw2ZcnkyG1zwwDSC7R GdJhXgLGl7lNg03ptowoegzyFoY0WZNbgdJNzeKPn2u29ahfNp3bmZyId/v+wgNq6tGc aCYCCfSLhqIZL5Yz+Eg6x7bkOQPaGFGAVppfqeawbtrZCgnMklWp5AjuX4qIMO1RcW4D Bld6/TVtRnOtHYFhDOHqgkBqK7tnLfqQPz5KtpGUXzyq9woXMFVfk36Cp0JodghoL1HE 3Tcup8nY9psaoKOMSPPoehKmRnWKvQJJG/J2O8o+v0P8vaWeq9rOotB7AquCjIF2oRp0 FL2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=W4gyun+mD1bBV3/7yIOpmKsG6BCZb4Vc4RqB1hSgmDY=; b=CuH537Bb5ZeFnic8SIkX5Chy3FKT4m/m7fkqthCed7R2ZoSidcOnLfu2dNu9yF2CpJ RiO83PciTogEGyVNGNzUBCiEI6zMj/71xk4X4q+x1OETMyWB33tgVe0TPytAVdlAQu5I wVfMmJJ269ODrw6kSq/usyttH04WEGbkw9WSxXYgJU64g3in3fmOybXZ1/36PDiQ0xvG S/gXzMo2TLdXVFb4/2QZ7keqGmJQWWSvKh1lTd7hHxFFwqwaGFlHsz2dT84562AmEwWs 6QkbLjZPs8BB0mazCkp9ymf1y7TIu/EkEK1ISp3HimmoIYSs8bd9ipLc4M2Bhdq+mlmz 3aOg== X-Gm-Message-State: AOAM53251FD+Q8whOfk2nIW6TVoShjI3Am8XA7VMZ/GUu8JExkBkKBIm snXgEcJ5djh9JKjHgfgeGBfN5pJJdN8= X-Google-Smtp-Source: ABdhPJwpPYnaalXWi1hIwgWYciPx/1hLjT1S/rzosuDzBHHqo3qul4E1hvCo2AeedD7zR6YGdHw3Sg== X-Received: by 2002:a7b:cc15:: with SMTP id f21mr20367501wmh.86.1621207375283; Sun, 16 May 2021 16:22:55 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id r5sm14221174wmh.23.2021.05.16.16.22.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 16 May 2021 16:22:54 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> From: Dmitry Gutov Message-ID: Date: Mon, 17 May 2021 02:22:52 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 16.05.2021 16:37, Philipp wrote: > One thing that came to my mind is: in general, in Elisp (not just XRef), we spend lots of time parsing filenames to support remote and quoted filenames. Other languages probably solve this by introducing proper types for filenames (e.g. the Java Path class), which can then hold preprocessed information about the underlying filesystem (or special file name handler, in the case of Elisp). How about doing similar for Elisp? For example, introduce a `parsed-file-name' class or structure holding the remote/quoting state, or attach it to string properties? I haven't tried out that idea, but I think it could significantly speed up the parsing (since we'd only have to do it once and don't have to search for filename handlers all the time), as well as remain backward-compatible to "plain" unparsed filenames by allowing both strings and this new object type. WDYT? That sounds like an interesting idea to explore. We create/concatenate those file names inside project-files, and then "parse" them again to convert to local names inside xref-matches-in-files. Creating such structures might indeed save us on some parsing and garbage generation. Experiments and patches welcome. What I was also thinking of previously, is some "fileset" data structure which could contain a list of local file names and their connection in a separate slot. Maybe even separating the parent/root directory into a separate slot when feasible, to minimize GC further, though that might complicate applications. A more structured "file" value format might make this stuff easier to use indeed, and perhaps the performance difference will be negligible. The difficulty is having a method like project-files return one format for some users, and another for users who want to take advantage of this performance improvement. Or we break the compatibility and/or introduce a new method with this new behavior. There is a one in the works already in the 'scratch/etags-regen' branch after all. Or another, more simplistic approach would be to have the method project-files-filtered return file names relative to the root (always, or when called with a certain argument). And then pass the root (and the connection/host) in the default-directory var. Then change xref-matches-in-files to use default-directory if the values in FILES are not absolute. The last approach would only work if we decide that a search across multiple roots (e.g. project roots together with external roots) can be done efficiently enough through multiple calls to xref-matches-in-files (and thus using multiple consecutive process calls). Someone should benchmark this in a real-world scenario; it might or might not show worse performance: OT1H, the potential for parallelism is more limited, and there is more overhead on process calls, OTOH, the practical parallelism is not infinite anyway, and the process soon bottlenecks on CPU and/or disk access throughput. From debbugs-submit-bounces@debbugs.gnu.org Sun May 16 19:32:06 2021 Received: (at 47799) by debbugs.gnu.org; 16 May 2021 23:32:06 +0000 Received: from localhost ([127.0.0.1]:51669 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liQEw-0004AL-1B for submit@debbugs.gnu.org; Sun, 16 May 2021 19:32:06 -0400 Received: from mail-wm1-f51.google.com ([209.85.128.51]:43767) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1liQEu-00049s-L4 for 47799@debbugs.gnu.org; Sun, 16 May 2021 19:32:05 -0400 Received: by mail-wm1-f51.google.com with SMTP id b19-20020a05600c06d3b029014258a636e8so2512514wmn.2 for <47799@debbugs.gnu.org>; Sun, 16 May 2021 16:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=K8BpH6XHWg22etE50PSaK227zYjhKeo+C3f5iiDM3uQ=; b=QOfB5h59faGBqN5lYvwD4zkGsjx1eqhN0PtZ7tQcOivkGUtmhc4bMfkdUeGGtgbca+ YIl5o1ank8kGKgGF6zBSE4vb7qIV7+Jno+3f/FTZuD20WFnW/0QbmlQnlRm47eNiNs0G hJ01g7Q6z3Vh4zs0QKvX7JbjG5ZLtoBJVAd7OMXc0ustgOMLBiEq8VA0OPPZtMLZj9iv FF8+j9aGqhobcZqEWE8hMTQGOtKQ2zXA0kem5q6Xy3Oms43c81J+dtBkbOM9CYX6fU2I pSuQuXALqwTc2Sr04I7A8lmhz+2S9YnEPSHg/l1o+oueEGOX5nceMGVU09lTiITvGXLc GUhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:references:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=K8BpH6XHWg22etE50PSaK227zYjhKeo+C3f5iiDM3uQ=; b=WDULzQVOVNjZF0QTVXO4mIUvUQQfpR80+hsXEcpsyGjfSrKGdVs2hDI19jQLmLCOSR kQ7cinIPKc2vN58KykwkUheCMQx4L1Qbtcjd8F6YeMeRou8U7H6uEGZlvV0obRZiKC0e yl0cmFYc6adaSrY9I3mVoWL+RZCHZDlIyzKXs8Ozaw6xrfLYnAs5TajaVcxQqeb2PI6D P+jNqhsFbKv6j/olGZX4ZnQUBEuJz/UFE8rVpIoNdkkGWD/YEsRHo0wBeWqCa8jjPcK9 CQx0Z5M6GSxlF4TQIQi6xJOnAJcKRmVYCGtV+3hCwiJqaRt4qPiRoS/23J+PgA3HzO5k vl/g== X-Gm-Message-State: AOAM532GmDAqFRAT5xKv9iHi1iUwejBRAuHHS1pzi3139BjqdQRpGHn6 6Nu/KUyrCVg7U2JcPZJGHZLndCk+6eE= X-Google-Smtp-Source: ABdhPJwTnQ2+N19Z9VZO9HbcBR0Sfu/U3fj34n+ybMXPr1NFoGBTAtxy/sQItIJNsNT1XI5sO2yTNw== X-Received: by 2002:a05:600c:22d0:: with SMTP id 16mr59717015wmg.117.1621207918861; Sun, 16 May 2021 16:31:58 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id k9sm12330463wmk.5.2021.05.16.16.31.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 16 May 2021 16:31:58 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames From: Dmitry Gutov To: Philipp References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> Message-ID: <2f2b5a9e-d46c-6a8b-4c0d-da9de68b54f3@yandex.ru> Date: Mon, 17 May 2021 02:31:57 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 17.05.2021 02:22, Dmitry Gutov wrote: > Or another, more simplistic approach would be to have the method > project-files-filtered return file names relative to the root (always, > or when called with a certain argument). And then pass the root (and the > connection/host) in the default-directory var. Then change > xref-matches-in-files to use default-directory if the values in FILES > are not absolute. Looking at the previous discussion, this actually seems very close to what you, Philipp, suggested on 19.04.2021. So above are my subsequent thoughts on how it can be implemented without requiring much change to the API (though complicating the implementations a bit). From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 05 15:05:10 2021 Received: (at 47799) by debbugs.gnu.org; 5 Jul 2021 19:05:10 +0000 Received: from localhost ([127.0.0.1]:46662 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0Tu1-0003Yw-RE for submit@debbugs.gnu.org; Mon, 05 Jul 2021 15:05:10 -0400 Received: from mail-wm1-f46.google.com ([209.85.128.46]:34461) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0Tu0-0003Yc-6L for 47799@debbugs.gnu.org; Mon, 05 Jul 2021 15:05:08 -0400 Received: by mail-wm1-f46.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso400627wmc.1 for <47799@debbugs.gnu.org>; Mon, 05 Jul 2021 12:05:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6m6wM+6Y3TYnLNfjB4pYB6Q98RWXvC93Zns07Z7eGYU=; b=ni1efZf2nvkYIguQtG3JyTTJbUwy3RmuyfJ1ikiLLDEu5/jWU/Ia1JjdQqntUGGNiL M+nygPFaL78gP1mSAcDeOldo7OEwL3fNr2pIxHQnmqDhzx+WyrkYy8FKBpvu1YlpV3Og 0wz6yB3Jx59jng7FF4p0on2uUwM0vVA44JZyf71VzvZP/ZtLkFaI8dTmf72GeyYR2Ec4 utAbePz7x5cVqC/h5cchGF/rwK7mM/braYActSxSsvU0LLHB5Csd8Mz4PwrrbvqrOJ9D Vq5nzR2tGovzFxtUL0RF0eGMYu+/c3S/aBhupyKrcegMNtWF/Q3TT43QCc8I18A5Ix/g CJwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6m6wM+6Y3TYnLNfjB4pYB6Q98RWXvC93Zns07Z7eGYU=; b=AblqysGpjTA63ZSW2Fp1IZI0TBPWUmWreXAoxD5Wf/Fh3aRnCrGBmBNVsqF723ZLhL x0UQE0zCciJgW5dUw26x0KhifdeEHkLKoGdq4L92RaaZwzXISZKYtnvVGrOLivyFnXEy TlyODFpLbvX+SUULaR6y/mzrPoKTzSxNYuvYj7LvNO2bYNNEd8DdGkFuDxFsukZeOuAd eCOzqoKYqRQNVy8e8tcsxqZeMyO2Tjd2+Z1gx99a8CkL02C93WeNR9JHrS5bTWsbyQjW rm+y/snLAVPeWeUWAxDRd3GeJ6SRILG6SqM32O1JcD1bq6WybNYcclbHEVWPWNo+u8mW f6EQ== X-Gm-Message-State: AOAM531IMX9++hCD8EKlCeHUa2nn9ke4WgQK2RiTjhBhseaTQU+W+PKt oB606uZKPQCRONdyfPZ65Rg= X-Google-Smtp-Source: ABdhPJzv5t7o1e0/qfZ/emw37F7DTYgVcVXFppdj5sHmUCf47STdCyAj6nYTw7Fdtfc+4VjTwA4ANw== X-Received: by 2002:a05:600c:4fc1:: with SMTP id o1mr2691965wmq.158.1625511902395; Mon, 05 Jul 2021 12:05:02 -0700 (PDT) Received: from smtpclient.apple ([46.128.198.100]) by smtp.gmail.com with ESMTPSA id r9sm314675wmh.40.2021.07.05.12.05.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 12:05:01 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames From: Philipp In-Reply-To: Date: Mon, 5 Jul 2021 21:05:01 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <5196C041-589E-4876-8254-3A6974D6DA53@gmail.com> References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> To: Dmitry Gutov X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) > Am 17.05.2021 um 01:22 schrieb Dmitry Gutov : >=20 > On 16.05.2021 16:37, Philipp wrote: >=20 >> One thing that came to my mind is: in general, in Elisp (not just = XRef), we spend lots of time parsing filenames to support remote and = quoted filenames. Other languages probably solve this by introducing = proper types for filenames (e.g. the Java Path class), which can then = hold preprocessed information about the underlying filesystem (or = special file name handler, in the case of Elisp). How about doing = similar for Elisp? For example, introduce a `parsed-file-name' class or = structure holding the remote/quoting state, or attach it to string = properties? I haven't tried out that idea, but I think it could = significantly speed up the parsing (since we'd only have to do it once = and don't have to search for filename handlers all the time), as well as = remain backward-compatible to "plain" unparsed filenames by allowing = both strings and this new object type. WDYT? >=20 > That sounds like an interesting idea to explore. >=20 > We create/concatenate those file names inside project-files, and then = "parse" them again to convert to local names inside = xref-matches-in-files. Creating such structures might indeed save us on = some parsing and garbage generation. >=20 > Experiments and patches welcome. >=20 > What I was also thinking of previously, is some "fileset" data = structure which could contain a list of local file names and their = connection in a separate slot. Maybe even separating the parent/root = directory into a separate slot when feasible, to minimize GC further, = though that might complicate applications. >=20 > A more structured "file" value format might make this stuff easier to = use indeed, and perhaps the performance difference will be negligible. I think those are very good ideas. The "fileset" structure sounds like = a pretty good abstraction. >=20 > The difficulty is having a method like project-files return one format = for some users, and another for users who want to take advantage of this = performance improvement. Or we break the compatibility and/or introduce = a new method with this new behavior. A general design approach in OOP is to not treat abstract virtual = functions (generic functions in ELisp terminology) as part of the public = interface of a type; i.e., abstract functions can be implemented, but = shouldn't be called outside of the module that defines them (project.el = in this case). That allows for changes like this: implementers could = freely return the new fileset structure because only project.el would = call project-files. Not sure how much ELisp code adheres to this = principle, though. If there's too much code (outside of project.el) = that relies on project-files returning a list, we need to indeed fall = back to some of the other options. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 17 20:54:06 2021 Received: (at 47799) by debbugs.gnu.org; 18 Jul 2021 00:54:06 +0000 Received: from localhost ([127.0.0.1]:54893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4v4I-000715-H3 for submit@debbugs.gnu.org; Sat, 17 Jul 2021 20:54:06 -0400 Received: from mail-ed1-f48.google.com ([209.85.208.48]:44605) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4v4G-00070a-C4 for 47799@debbugs.gnu.org; Sat, 17 Jul 2021 20:54:05 -0400 Received: by mail-ed1-f48.google.com with SMTP id l1so18107102edr.11 for <47799@debbugs.gnu.org>; Sat, 17 Jul 2021 17:54:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=M66UMmKJ83F8OfbMRleiCeBlHytp8Rju/rI6mSmqT0c=; b=AevKDn7fsdIdSKL58G/FQPER4DGp2K+6p5Y0ULKbTerCg5I/peufwQVKlJ3W6Pkn3l Qtdk3TMCy+SEHrToDMkf9jnFo2QclWYXUR+f7xYdcl66c/LR108//dFin1LutiM8+hxl Yfg0HsJxbHUV6OOS00+zpzDI0czyA4wZgULRT3BFa51o2G6QKrWqcY728p8DyJr1n9ey R77kWaxLK8ib9rP4bmHhDUQLr9Tn9bTua7YHZumMpfJASv2QVRyF9+S/lPYSHqLHJ0d1 RJuHOvYmKmY84+SW/6nGlLlCUDLFHyfKEFO24605EUDzdInuFnALi8vXKyINJejbtPmO /yeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=M66UMmKJ83F8OfbMRleiCeBlHytp8Rju/rI6mSmqT0c=; b=ChUUknjfmVdslZoy8/Ymb1vfwMrmC7AEprnugIbqT4rmsy7NsUYSVFdkcCiCopb1Uv tn5qSePhGMZPcQJvBy2uOfr9OCOk9EcSfoOVBC9heNPi7o1izl/bBYFiNEkDfd10LzTW LRdYca9l2h2N/lkcpA8AxCbFGno5Dsl5IQcUN5wXHTs+fl6k6KuZxxDQVANaNwRxXce+ ApwTF83uooiuqNxH9Bl/9JScHRQ4FprSkiLnMbPaxiWk668eLLT2lSUCslCWHC+0Jprc fZIY3nmQbtGMkU71/uhy2YvsIoAw+LLYzJHt8nOIBqC9ysQ8SHxcp2TqTrNey1OSrWYb KRpQ== X-Gm-Message-State: AOAM5305ATXtil5s8TwiThYME0zU+j2S4pBi/JV0W4gXwYrwpaayWYpX 72KznsqqguwhPs6uuIvcFC+WVR1Z41I= X-Google-Smtp-Source: ABdhPJyE+OLih3sieCBpX6xO/UYala44eh5c5ahimV0lAy95jVkQC6dmEC4MnwOhU2s3wCfURCWnbA== X-Received: by 2002:aa7:d4c2:: with SMTP id t2mr16023803edr.241.1626569638524; Sat, 17 Jul 2021 17:53:58 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id n10sm5696192edw.70.2021.07.17.17.53.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 17 Jul 2021 17:53:57 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> <5196C041-589E-4876-8254-3A6974D6DA53@gmail.com> From: Dmitry Gutov Message-ID: Date: Sun, 18 Jul 2021 03:53:55 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <5196C041-589E-4876-8254-3A6974D6DA53@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05.07.2021 22:05, Philipp wrote: >> What I was also thinking of previously, is some "fileset" data structure which could contain a list of local file names and their connection in a separate slot. Maybe even separating the parent/root directory into a separate slot when feasible, to minimize GC further, though that might complicate applications. >> >> A more structured "file" value format might make this stuff easier to use indeed, and perhaps the performance difference will be negligible. > > I think those are very good ideas. The "fileset" structure sounds like a pretty good abstraction. I've yet to express that in terms of code, though. >> The difficulty is having a method like project-files return one format for some users, and another for users who want to take advantage of this performance improvement. Or we break the compatibility and/or introduce a new method with this new behavior. > > A general design approach in OOP is to not treat abstract virtual functions (generic functions in ELisp terminology) as part of the public interface of a type; i.e., abstract functions can be implemented, but shouldn't be called outside of the module that defines them (project.el in this case). That allows for changes like this: implementers could freely return the new fileset structure because only project.el would call project-files. Not sure how much ELisp code adheres to this principle, though. When you say "abstract virtual functions", do you mean OOP as in C++ OOP? I'm not sure about standard practices there, but this sounds more like C++ and less like OOP in general. I'm looking as generic functions here as part of an interface signature (like Java or Go interface). They are programmed against (which is the case with project.el) and are supposed to be stable. > If there's too much code (outside of project.el) that relies on project-files returning a list, we need to indeed fall back to some of the other options. A new method seems to be the way forward. Or, say, an ad-hoc argument which determines whether file names should be relative. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 05 13:14:38 2021 Received: (at 47799) by debbugs.gnu.org; 5 Sep 2021 17:14:38 +0000 Received: from localhost ([127.0.0.1]:50921 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mMvj4-0000YY-E9 for submit@debbugs.gnu.org; Sun, 05 Sep 2021 13:14:38 -0400 Received: from mail-wm1-f43.google.com ([209.85.128.43]:56229) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mMvj2-0000YL-JG for 47799@debbugs.gnu.org; Sun, 05 Sep 2021 13:14:37 -0400 Received: by mail-wm1-f43.google.com with SMTP id g135so2915116wme.5 for <47799@debbugs.gnu.org>; Sun, 05 Sep 2021 10:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=MJ44jntjSlmldBloQ6xEHm2EjgjBbAxykmhN+H1DXy0=; b=bLWVWVvq4c3gE0nCyR0h/dfzVz78n6pcs7e15WfuwRZj57IF5j5cI7cOTGbqjpclux P6SJrJ9HJGYORAOOa+EZptxt1a96WeFTrwlcNyLHRFRsiFZCB6gaLyCxACEc/wEoDZxo o8LIRUUKFqeoMzVle+OovPS991UbLqRIba6XrDHuvTkrch68cVv6ZPISpydxj/jIssL3 T6RE1yvByc4TiXmLnseP9b4m+GU1WWg1Put32Q1R0U/YEON6wuWLgc/U+RU5EvtXXllk iZ3IO2Hz3UmXN5f6pYeiiBKcJ/ahaEmaL4r8VURR+nS3Ch7O6Q9tUT9zPWwGBKGTNHtU xodA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=MJ44jntjSlmldBloQ6xEHm2EjgjBbAxykmhN+H1DXy0=; b=B47GCQQZGdvP9rNG51e1h/KajjyYhDZCIWovWUlc6hdMIGoxS5tBsHk4dBudldfpxA 1Vt/wiZDc2hmFxv/OiNrUKDNrNE2AvuP+cUw30ovvE7LWRdiHvzCYsfF/1fxrLb5zLRM xT0KrPeTPkYmr4aZ++qk67l/RPihszfNAcxzNTiAaKodHafcSMsF3P78rTPb3WsLQEO6 IJjCt+cbUg2fMDW09mskyP5+QB3vSEtH5sss5VO6M61ZBgwG47/goeWCSc7FZqm96o1R k0bdTwEs+MMAhQnwUdA8TnCm8SZcCSv65ytoqlfrZGk2ZDujYSxC9ykcPDWV7sPqJmvr UpzQ== X-Gm-Message-State: AOAM5310DrvTXLGFPTb2X1hTmrD2dx/NSWww5z5pfHyJQvZQej09Ys5y d6ZEs+Xf/lsKXgbnrI7Gq0A= X-Google-Smtp-Source: ABdhPJy6IIJiLke4I5ngHJOULOrgEqqHrPHYXS1S/CtfjjzxORxxu9bNglNA06NNvKBIs3OTr6dIgg== X-Received: by 2002:a05:600c:1d29:: with SMTP id l41mr7707608wms.177.1630862070597; Sun, 05 Sep 2021 10:14:30 -0700 (PDT) Received: from smtpclient.apple ([46.128.209.111]) by smtp.gmail.com with ESMTPSA id c3sm5439451wrd.34.2021.09.05.10.14.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Sep 2021 10:14:30 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames From: Philipp In-Reply-To: Date: Sun, 5 Sep 2021 19:14:29 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> <5196C041-589E-4876-8254-3A6974D6DA53@gmail.com> To: Dmitry Gutov X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) > Am 18.07.2021 um 02:53 schrieb Dmitry Gutov : >=20 > On 05.07.2021 22:05, Philipp wrote: >=20 >>> The difficulty is having a method like project-files return one = format for some users, and another for users who want to take advantage = of this performance improvement. Or we break the compatibility and/or = introduce a new method with this new behavior. >> A general design approach in OOP is to not treat abstract virtual = functions (generic functions in ELisp terminology) as part of the public = interface of a type; i.e., abstract functions can be implemented, but = shouldn't be called outside of the module that defines them (project.el = in this case). That allows for changes like this: implementers could = freely return the new fileset structure because only project.el would = call project-files. Not sure how much ELisp code adheres to this = principle, though. =20 >=20 > When you say "abstract virtual functions", do you mean OOP as in C++ = OOP? I'm not sure about standard practices there, but this sounds more = like C++ and less like OOP in general. >=20 > I'm looking as generic functions here as part of an interface = signature (like Java or Go interface). They are programmed against = (which is the case with project.el) and are supposed to be stable. I think the idea is applicable to most programming languages that have = some form of subtype polymorphism. Basically, for a normal = (monomorphic) function, you can make the parameter types more general or = the return type over time more specific over time without breaking = compatibility. For a polymorphic function that's only specialized but = not called outside the defining entity, e.g. a private virtual function = in C++ or a method marked as @ForOverride = (https://github.com/google/error-prone/blob/master/annotations/src/main/ja= va/com/google/errorprone/annotations/ForOverride.java) in Java, it's the = other way round: you can make the parameter types more specific and the = return type more generic over time. That implies that for a polymorphic = function that's also called outside the defining entity, you can't = change any of the types without breaking compatibility. Thus the = suggestion to separate the interface for callers from the interface for = subclasses/specializers. >=20 > > If there's too much code (outside of project.el) that relies on = project-files returning a list, we need to indeed fall back to some of = the other options. >=20 > A new method seems to be the way forward. Or, say, an ad-hoc argument = which determines whether file names should be relative. I guess you also can't introduce new parameters without breaking = compatibility either. That would only leave the new method possibility. = We could then say that nothing outside project.el should call it to = avoid the above problem. Ideally, the byte compiler would support a = declaration form similar to @ForOverride to warn about such invocations. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 20 12:05:14 2021 Received: (at 47799) by debbugs.gnu.org; 20 Sep 2021 16:05:14 +0000 Received: from localhost ([127.0.0.1]:43912 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSLn8-0003Gd-A0 for submit@debbugs.gnu.org; Mon, 20 Sep 2021 12:05:14 -0400 Received: from mail-wr1-f52.google.com ([209.85.221.52]:33688) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSLn6-0003G4-32 for 47799@debbugs.gnu.org; Mon, 20 Sep 2021 12:05:12 -0400 Received: by mail-wr1-f52.google.com with SMTP id t18so31250612wrb.0 for <47799@debbugs.gnu.org>; Mon, 20 Sep 2021 09:05:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=f7L9T5N3+d6/GZN0M7h1P8yKlOU3blOZyP8ujvbcyhc=; b=Vr/KBhJfclAO+t90+z6yNJlYD6L/qaDQmSO3sWTzuiX2wRs5QPr4H/DVpDAoXXE7ZG H56xXa3pcijgv4BO/EmYj9Tkz4IYfWR+IXkxopt5FqtrqCfkKjjBiwH69/5YZ2drDUgN TKt2CX6p7UcdRWLfCEzWvVPFSq1vHJzM1NU59L2gB4JvNI3Jb0raw4yhPTMK0D5GQmMC cZ3SmBfE1pjOH9JWh4SmXzzj8RwfEbcTYXM7XfS6KsbxVbBPJJrhb6WM+F2FeQmLy0b3 7Q24dVxKtpQzFjozZDXt52DP/nGVuRYaOh2jHYlYY/mNZMm4HlamkxVnZMwmwyfdSw++ I7iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=f7L9T5N3+d6/GZN0M7h1P8yKlOU3blOZyP8ujvbcyhc=; b=wYsDquGdLBcXqL/ZgBWpJZA20YWkj3Hywz2TOerZ7GJYMrAzhAOZUprQNH5mLeYVGb I6URfKPiRCrZZRnN0LpSwXYAB7JcPYH9dINmVMSUIULTbF08zTTatHSRJwqqm08GLq6j TQ0Z3FIvKgh5rX/ke1LUYUNkWr0bTRKIxBDgikts51jxf371CMaHEJWzo6zZHtFZ0vTk v7yDhJX08i9DBIukdPVk6a1HXXMAkgcWy8eCKrK6cQiv9cQitko8xvqs5tv8l7QeK6DO jYuRtzyK8rYOSL9qkMiFnOfCVOWKKeaUPeeaHLgb9kIy1U5fdAgTcvWVwZGbgy+VVdrJ wb5Q== X-Gm-Message-State: AOAM532yChpjZYAjkNAxV2ZYfBuMAP+6z9M8dtlZdHMKFKmWqFqJp+Qe GmX3OGlZmjzDqTFLJYpOlXz6uqKV00s= X-Google-Smtp-Source: ABdhPJxqzVey6VVqRgMBoVZwLTx0gxilYqjPjhPYDIrnbdD/DFbFiJ9neW1Tge3wiWf6flRguUW4jg== X-Received: by 2002:adf:e406:: with SMTP id g6mr29744498wrm.172.1632153906284; Mon, 20 Sep 2021 09:05:06 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id k17sm16231444wrq.7.2021.09.20.09.05.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 20 Sep 2021 09:05:05 -0700 (PDT) Subject: Re: bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames To: Philipp References: <658a3e61-9511-5502-43de-8f591cec7387@yandex.ru> <91dd2467-f64e-eede-8098-14fc8ccd7ae7@yandex.ru> <429484E1-DDFA-4050-B5BF-E43477441C84@gmail.com> <5196C041-589E-4876-8254-3A6974D6DA53@gmail.com> From: Dmitry Gutov Message-ID: Date: Mon, 20 Sep 2021 19:05:03 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.4 (/) X-Debbugs-Envelope-To: 47799 Cc: 47799@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) Hi Philipp, Sorry for the long pause. On 05.09.2021 20:14, Philipp wrote: > > >> Am 18.07.2021 um 02:53 schrieb Dmitry Gutov : >> >> On 05.07.2021 22:05, Philipp wrote: >> >>>> The difficulty is having a method like project-files return one format for some users, and another for users who want to take advantage of this performance improvement. Or we break the compatibility and/or introduce a new method with this new behavior. >>> A general design approach in OOP is to not treat abstract virtual functions (generic functions in ELisp terminology) as part of the public interface of a type; i.e., abstract functions can be implemented, but shouldn't be called outside of the module that defines them (project.el in this case). That allows for changes like this: implementers could freely return the new fileset structure because only project.el would call project-files. Not sure how much ELisp code adheres to this principle, though. >> >> When you say "abstract virtual functions", do you mean OOP as in C++ OOP? I'm not sure about standard practices there, but this sounds more like C++ and less like OOP in general. >> >> I'm looking as generic functions here as part of an interface signature (like Java or Go interface). They are programmed against (which is the case with project.el) and are supposed to be stable. > > I think the idea is applicable to most programming languages that have some form of subtype polymorphism. Basically, for a normal (monomorphic) function, you can make the parameter types more general or the return type over time more specific over time without breaking compatibility. For a polymorphic function that's only specialized but not called outside the defining entity, e.g. a private virtual function in C++ or a method marked as @ForOverride (https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/ForOverride.java) in Java, it's the other way round: you can make the parameter types more specific and the return type more generic over time. That implies that for a polymorphic function that's also called outside the defining entity, you can't change any of the types without breaking compatibility. Thus the suggestion to separate the interface for callers from the interface for subclasses/specializers. I'm not quite familiar with the practice, and since we don't do type parameterization here, the justification probably doesn't apply. >>> If there's too much code (outside of project.el) that relies on project-files returning a list, we need to indeed fall back to some of the other options. >> >> A new method seems to be the way forward. Or, say, an ad-hoc argument which determines whether file names should be relative. > > I guess you also can't introduce new parameters without breaking compatibility either. That is also true. So we'll probably go this way: I'm already experimenting with a method called project-files-filtered in the scratch/etags-regen branch. > That would only leave the new method possibility. We could then say that nothing outside project.el should call it to avoid the above problem. Ideally, the byte compiler would support a declaration form similar to @ForOverride to warn about such invocations. I get the idea of having a Template Method pattern where the public method is something else (this approach seems to be described in ForOverride.java), but I'm not sure how to apply it to our case. I.e., what kind of data the "public" function would accept and return. If you mean it would return a list of absolute file names, that would actually limit its usefulness. And/or perhaps you imagined that our projects could have a "private" method which would not be "exported" to outside code, but which project-find-regexp would be able to use? I don't really like that approach, outside code should be able to reimplement the same features.