GNU bug report logs -
#47799
28.0.50; Default `project-files' implementation doesn't work with quoted filenames
Previous Next
Full log
View this message in rfc822 format
On 16.05.2021 16:37, Philipp wrote:
> One thing that came to my mind is: in general, in Elisp (not just XRef), we spend lots of time parsing filenames to support remote and quoted filenames. Other languages probably solve this by introducing proper types for filenames (e.g. the Java Path class), which can then hold preprocessed information about the underlying filesystem (or special file name handler, in the case of Elisp). How about doing similar for Elisp? For example, introduce a `parsed-file-name' class or structure holding the remote/quoting state, or attach it to string properties? I haven't tried out that idea, but I think it could significantly speed up the parsing (since we'd only have to do it once and don't have to search for filename handlers all the time), as well as remain backward-compatible to "plain" unparsed filenames by allowing both strings and this new object type. WDYT?
That sounds like an interesting idea to explore.
We create/concatenate those file names inside project-files, and then
"parse" them again to convert to local names inside
xref-matches-in-files. Creating such structures might indeed save us on
some parsing and garbage generation.
Experiments and patches welcome.
What I was also thinking of previously, is some "fileset" data structure
which could contain a list of local file names and their connection in a
separate slot. Maybe even separating the parent/root directory into a
separate slot when feasible, to minimize GC further, though that might
complicate applications.
A more structured "file" value format might make this stuff easier to
use indeed, and perhaps the performance difference will be negligible.
The difficulty is having a method like project-files return one format
for some users, and another for users who want to take advantage of this
performance improvement. Or we break the compatibility and/or introduce
a new method with this new behavior.
There is a one in the works already in the 'scratch/etags-regen' branch
after all.
Or another, more simplistic approach would be to have the method
project-files-filtered return file names relative to the root (always,
or when called with a certain argument). And then pass the root (and the
connection/host) in the default-directory var. Then change
xref-matches-in-files to use default-directory if the values in FILES
are not absolute.
The last approach would only work if we decide that a search across
multiple roots (e.g. project roots together with external roots) can be
done efficiently enough through multiple calls to xref-matches-in-files
(and thus using multiple consecutive process calls). Someone should
benchmark this in a real-world scenario; it might or might not show
worse performance: OT1H, the potential for parallelism is more limited,
and there is more overhead on process calls, OTOH, the practical
parallelism is not infinite anyway, and the process soon bottlenecks on
CPU and/or disk access throughput.
This bug report was last modified 3 years and 269 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.