GNU bug report logs - #55636
27.2; etags performance fix when working with very big TAGS files

Previous Next

Full log

View this message in rfc822 format

From: VAN VLIERBERGHE Stef <stef.van-vlierberghe <at> eurocontrol.int>
To: Eli Zaretskii <eliz <at> gnu.org>, "DE BACKER Jurgen (EXT)" <jurgen.de-backer.ext <at> eurocontrol.int>
Cc: "55636 <at> debbugs.gnu.org" <55636 <at> debbugs.gnu.org>, WAROQUIERS Philippe <philippe.waroquiers <at> eurocontrol.int>
Subject: bug#55636: 27.2; etags performance fix when working with very big TAGS files
Date: Wed, 25 May 2022 20:42:16 +0000

Hi Eli,

For us the 10 sec is reduced to below 1 sec, loading the tags file is no longer noticed after this change.

I assume the reason is a huge amount of files all accessed over NFS, and expand-file-name does a lot of system calls that translate into network packets.

An alternative approach is to add some switch that allows a customization that simply never calls the expand-file-name, we generate tags files that already contain absolute paths so don't need any of this logic and disabling it would also be ok for us.

All the best,
Stef

-----Original Message-----
From: Eli Zaretskii <eliz <at> gnu.org>
Sent: 25 May 2022 19:16
To: DE BACKER Jurgen (EXT) <jurgen.de-backer.ext <at> eurocontrol.int>
Cc: 55636 <at> debbugs.gnu.org; VAN VLIERBERGHE Stef <stef.van-vlierberghe <at> eurocontrol.int>; WAROQUIERS Philippe <philippe.waroquiers <at> eurocontrol.int>
Subject: Re: bug#55636: 27.2; etags performance fix when working with very big TAGS files

> Cc: jurgen.de-backer.ext <at> eurocontrol.int,
> stef.van-vlierberghe <at> eurocontrol.int,
>  philippe.waroquiers <at> eurocontrol.int
> Date: Wed, 25 May 2022 16:04:04 +0000
> From:  Jurgen De Backer via "Bug reports for GNU Emacs,  the Swiss
> army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>
> We implemented a fix for making etags searching faster in our custom
> emacs build.
> In our project, sometimes a search could take more than 10 seconds.
> The fix is to expand filenames only if they are relative:
>
> (defun expand-if-relative (a-file)
> (if (file-name-absolute-p a-file)
>     a-file
>   (expand-file-name a-file)
>   )
> )
>
> and then use this function in the mapcar statement inside (defun
> tags-table-including ...), replacing expand-file-name:
>   ....
>   (if (member this-file (mapcar #'expand-if-relative
>                                           (tags-table-files)))
>                 ;; Found it.
>                 (setq found tables)
>                 )
>   ....

Thanks.  How much speedup does this bring in your use cases?  From 10 sec down to how long?

> Looking at expand-file-name doc, it does 2 things:
>   * make file name absolute
>   * and canonicalize it (removes the xxxx/.. dir components,
>        the . dir components, the double slashes).

It actually does more, like resolve the ~/ etc.

> We are wondering if the standard C function in emacs could do a fast
> "do nothing" when the file name is absolute and has nothing to
> cannonicalize.

I think testing for "nothing to canonicalize" is as complex as canonicalizing the file name.
____

This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.

Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.

Any views expressed in this message are those of the sender.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.