From unknown Thu Aug 14 21:49:00 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#23223 <23223@debbugs.gnu.org> To: bug#23223 <23223@debbugs.gnu.org> Subject: Status: 25.0.92; Can xref-find-references be sped up? Reply-To: bug#23223 <23223@debbugs.gnu.org> Date: Fri, 15 Aug 2025 04:49:00 +0000 retitle 23223 25.0.92; Can xref-find-references be sped up? reassign 23223 emacs submitter 23223 Eli Zaretskii severity 23223 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 05 11:16:46 2016 Received: (at submit) by debbugs.gnu.org; 5 Apr 2016 15:16:46 +0000 Received: from localhost ([127.0.0.1]:50495 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anSij-0000DA-TC for submit@debbugs.gnu.org; Tue, 05 Apr 2016 11:16:46 -0400 Received: from eggs.gnu.org ([208.118.235.92]:58700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anSih-0000Cw-Hy for submit@debbugs.gnu.org; Tue, 05 Apr 2016 11:16:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anSib-0002Va-9K for submit@debbugs.gnu.org; Tue, 05 Apr 2016 11:16:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:53135) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anSib-0002VW-6C for submit@debbugs.gnu.org; Tue, 05 Apr 2016 11:16:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49668) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anSiZ-0007es-TY for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2016 11:16:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anSiV-0002Rt-PX for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2016 11:16:35 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50474) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anSiV-0002Rm-LJ for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2016 11:16:31 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4281 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1anSiU-0004Zg-KU for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2016 11:16:31 -0400 Date: Tue, 05 Apr 2016 18:16:04 +0300 Message-Id: <83pou4m6h7.fsf@gnu.org> From: Eli Zaretskii To: bug-gnu-emacs@gnu.org Subject: 25.0.92; Can xref-find-references be sped up? X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) With an ID database present in the Emacs top-level directory, do this: emacs -Q C-x C-f src/buffer.c RET C-u 1316 M-x goto-char RET You should now see point on "current_buffer". Now: M-x xref-find-references RET RET When prompted for the project directory, remove the "src/" part, to specify the root of the Emacs tree, and type RET. This takes around 13 sec here, although the corresponding 'lid' command: lid --result=grep -l -w current_buffer takes only 0.07 sec. The important part of the Lisp-level profile appears below. It looks like we are visiting each match of each file in the list of matches, and that takes most of the time. Can this be avoided somehow? The 'lid' command already includes all the matches in Grep format, so why are we visiting each match, when the information should be already available? - command-execute 833 92% - call-interactively 833 92% - funcall-interactively 830 92% - execute-extended-command 830 92% - command-execute 826 91% - call-interactively 815 90% - funcall-interactively 813 90% - xref-find-references 813 90% - xref--find-xrefs 813 90% - xref-backend-references 800 88% - cl-mapcan 788 87% - apply 788 87% - cl-mapcar 788 87% - mapcar 788 87% - # 788 87% - xref-collect-references 788 87% - cl-mapcan 770 85% - apply 770 85% - cl-mapcar 770 85% - mapcar 770 85% - # 770 85% - xref--collect-matches 728 80% - semantic-find-file-noselect 556 61% - find-file-noselect 556 61% - find-file-noselect-1 519 57% - after-find-file 503 55% - normal-mode 499 55% - set-auto-mode 493 54% - set-auto-mode-0 488 54% - c-mode 488 54% - c-common-init 480 53% - mapc 473 52% - # 472 52% - c-neutralize-syntax-in-and-mark-CPP 466 51% - c-extend-font-lock-region-for-macros 205 22% - c-state-literal-at 205 22% - c-state-safe-place 199 22% - c-beginning-of-macro 3 0% back-to-indentation 3 0% c-syntactic-end-of-macro 1 0% c-syntactic-end-of-macro 9 1% c-neutralize-CPP-line 7 0% - c-beginning-of-macro 4 0% back-to-indentation 2 0% # 1 0% - c-basic-common-init 7 0% - c-set-style 4 0% - mapc 4 0% - # 4 0% - c-set-style-1 2 0% - mapcar 2 0% # 1 0% - run-mode-hooks 3 0% In GNU Emacs 25.0.92.75 (i686-pc-mingw32) of 2016-04-04 built on HOME-C4E4A596F7 Repository revision: f501116ea896b20f195f5c841e8770d7fe0418b9 Windowing system distributor 'Microsoft Corp.', version 5.1.2600 Configured using: 'configure --prefix=/d/usr --enable-checking=yes,glyphs --with-wide-int --with-modules 'CFLAGS=-O0 -gdwarf-4 -g3'' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES Important settings: value of $LANG: ENU locale-coding-system: cp1255 Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message dired format-spec rfc822 mml mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote w32notify w32 multi-tty make-network-process emacs) Memory information: ((conses 16 93253 6020) (symbols 56 20721 0) (miscs 48 36 107) (strings 16 17749 6443) (string-bytes 1 439871) (vectors 16 12404) (vector-slots 8 429720 3894) (floats 8 162 70) (intervals 40 261 118) (buffers 856 11)) From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 05 20:37:49 2016 Received: (at 23223) by debbugs.gnu.org; 6 Apr 2016 00:37:49 +0000 Received: from localhost ([127.0.0.1]:50829 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anbTg-00059k-U0 for submit@debbugs.gnu.org; Tue, 05 Apr 2016 20:37:49 -0400 Received: from mail-wm0-f51.google.com ([74.125.82.51]:36570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anbTf-00059X-4g for 23223@debbugs.gnu.org; Tue, 05 Apr 2016 20:37:47 -0400 Received: by mail-wm0-f51.google.com with SMTP id v188so5817382wme.1 for <23223@debbugs.gnu.org>; Tue, 05 Apr 2016 17:37:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=2eLYy1km8lHv0cCQ315TFiLi8ABHOexmlQxYcqPq0hk=; b=Y0ZfLAgks20RkCYPcIGb44twsWDpEOIIQIvqhS2c3GXpi+mh+jC14vOX1uACGSU1w/ ikI/bi4wiYVXqPHURkziQD6cf1cJWl4OzLM4k5VKOOslaUR0x/lPii71nm2aFbg7B8tu Vw/U0n4Y2P/CNcymQjjdUDNe4zvpBXk58phZZz5j1MQ6oOr5akGC3bmW/d+x9lS/IvN0 kvE7bbkncZoXvc9jq3uCtodb4lUkMQobEbPecwQHD9HQbIPQZEFFiiOeMdxHgK7hm82a 26EJSNYRuxmnb/8Em2Bes5gNiI0L21k2g9+NTeIUHVJK0QwDiyDsoPttT6HepPiioClk sdLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=2eLYy1km8lHv0cCQ315TFiLi8ABHOexmlQxYcqPq0hk=; b=md1zWmrykHMFvCxVVdLShFnD7fBSVq5VMlkRVbq8VJz2ox6QrrrywT82wJ3vcfd7HG 6EVNMrjtdKFJ/saCDlYh8dOVKAYMtAtV1YW7EvgE8l/vJ5mAi5tIpElA4eDM+CJ96Rsw gr2UC0FYo1al5RDdZGeizHTjxErA1BynwJuvsO+FrkpauXWpiOXw8+gefhugeFgg/8vF N+mFALONJ7mkSN3KnVUCTgzF6PFNxX1cyEyJN4AAihjkqw+BLhSKcmijmza+bNmbaS4P SWwvfiwktvLQ0apIps17ctY41pg0Rts4KeGQYvTzp57kxhV4878IdyZ3r3Vx84Rz4al+ LipQ== X-Gm-Message-State: AD7BkJI3i5cA3dIfYSsG3NpgOL9mUVuLLtbKmD2s+IA2WQx7hIh+PqvQkbggMUVn4QxGEg== X-Received: by 10.194.92.237 with SMTP id cp13mr21881951wjb.168.1459903061617; Tue, 05 Apr 2016 17:37:41 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id u3sm21933360wmg.15.2016.04.05.17.37.40 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 05 Apr 2016 17:37:40 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii , 23223@debbugs.gnu.org References: <83pou4m6h7.fsf@gnu.org> From: Dmitry Gutov Message-ID: <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> Date: Wed, 6 Apr 2016 03:37:39 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83pou4m6h7.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/05/2016 06:16 PM, Eli Zaretskii wrote: > The important part of the Lisp-level profile appears below. It looks > like we are visiting each match of each file in the list of matches, > and that takes most of the time. Can this be avoided somehow? The > 'lid' command already includes all the matches in Grep format, so why > are we visiting each match, when the information should be already > available? Huh, yes, that's the area of optimization I've forgotten about (mostly because of using plain Grep myself anyway). If id-utils (and some other tools we delegate to) know symbol boundaries, maybe we don't need to double-check. However, Grep output, which 'lid --result=grep' also uses, only outputs the line number, but not the starting column. Take this example: I'm searching for 'buffer'. The last line of the output looks like this: xmenu.c:832: set_buffer_internal_1 (XBUFFER (buffer)); How do I find the appropriate 'buffer' match in this line? Or can we ask 'lid' (and, ideally, Grep too) to include the column of a match in the output? And when there are several matches on the same line, output that line multiple times? Grep has colorized output, so I should be able to work with that. But not 'lid', it seems. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 11:10:05 2016 Received: (at 23223) by debbugs.gnu.org; 6 Apr 2016 15:10:05 +0000 Received: from localhost ([127.0.0.1]:51894 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anp5p-0002N1-Lg for submit@debbugs.gnu.org; Wed, 06 Apr 2016 11:10:05 -0400 Received: from mail-wm0-f49.google.com ([74.125.82.49]:38212) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anp5n-0002MT-Qk for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 11:10:04 -0400 Received: by mail-wm0-f49.google.com with SMTP id u206so50110252wme.1 for <23223@debbugs.gnu.org>; Wed, 06 Apr 2016 08:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=JIU12CrT9D9hjW4cV/KWELmaOQczIFrB/SzpcfjBdZg=; b=hYekqM1IQvo2NVyKVDpjn5e7d12T+7ICcwhFaLxHpcZdMKJYlGeBUoHvGBmcRR0oBG /gXUq8xlbpAOFrY3CIaFN2ffW0VT1ovx8cxVwTHNT4X03PEW5I5AFVk72HwNoq76G46n JkjBeKS5MVxi/wOQ2cVcRirsRFHp1KPfm1MHglaSgpb1WLXOhe5gamOcTPG4ApOip0Ox LMrpuvmktfq2qUh82VcuFnXyIMNMRXNmfqP+94VhJm4kgsnoqbottt+Fg5BvPlA8YFUU RQ7tXF3+GgRrVYis1EiuhJ/u2Z3+bFiU19HURK/vKteGXwQJsHxqc4O1bi8Gpg/E/enN cHCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=JIU12CrT9D9hjW4cV/KWELmaOQczIFrB/SzpcfjBdZg=; b=cStLz30b9Tls2+tf2dbVl2RjFX5P2hkWYzggeU0Yyna7KxpmCN5ovXQ9b3K16ShJSM OdaVuk899r6o/79OavRitTmO4Lq5aroFmm0qMd3DQAppSag93R66Gc7nrAGYprAhbdhe 6Oc2OXagopqRWSA8NpiX6JCMo1fzkBdBgdYp696Nk/SxTv+mHqGc/3fCDMXkJCeEDeo8 hlZIfEbxC0QKv/LumN8t/JKED5V8xXNkhL40SOmO4dMnURrpO8JRZQM/On1wq6fAREud 0JHaIuriLThkHL55d88DkPdv3TLzoMjPOfuLmGc64VlAMvBR2qcYOZAKC7RpvoVg4imq D0fg== X-Gm-Message-State: AD7BkJK8wvS2UOFLNZ3KSaEOkL6vgluu7RBfGdgdGFJznwh7IhW3oKvRAwi0aizGpsOH2w== X-Received: by 10.28.141.141 with SMTP id p135mr24284181wmd.8.1459955398275; Wed, 06 Apr 2016 08:09:58 -0700 (PDT) Received: from [192.168.0.185] (static-nbl2-118.cytanet.com.cy. [212.31.107.118]) by smtp.googlemail.com with ESMTPSA id e190sm4193567wma.15.2016.04.06.08.09.56 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 06 Apr 2016 08:09:57 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii , 23223@debbugs.gnu.org References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> From: Dmitry Gutov Message-ID: <8351f540-d999-125c-9ad2-2e1b221f9bf5@yandex.ru> Date: Wed, 6 Apr 2016 18:09:55 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/06/2016 03:37 AM, Dmitry Gutov wrote: > If id-utils (and some other > tools we delegate to) know symbol boundaries Is that actually the case, though? At most, it's using some wonky heuristics. For instance, if I run 'mkid' in the top directory of the Emacs sources, I can search for 'defmacro', and get a few lines with 'cl-defmacro' instead among the results. Or search for 'eval-when', and get some lines with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. id-utils version 4.6 here. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 13:12:58 2016 Received: (at 23223) by debbugs.gnu.org; 6 Apr 2016 17:12:58 +0000 Received: from localhost ([127.0.0.1]:51984 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anr0k-0005O3-Kx for submit@debbugs.gnu.org; Wed, 06 Apr 2016 13:12:58 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46335) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anr0j-0005Nr-M0 for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 13:12:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anr0Z-0000Om-GG for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 13:12:52 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:44998) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anr0Z-0000Oi-Db; Wed, 06 Apr 2016 13:12:47 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1908 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1anr0Y-0002lD-LZ; Wed, 06 Apr 2016 13:12:47 -0400 Date: Wed, 06 Apr 2016 20:12:27 +0300 Message-Id: <8360vumzk4.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> (message from Dmitry Gutov on Wed, 6 Apr 2016 03:37:39 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Dmitry Gutov > Date: Wed, 6 Apr 2016 03:37:39 +0300 > > On 04/05/2016 06:16 PM, Eli Zaretskii wrote: > > > The important part of the Lisp-level profile appears below. It looks > > like we are visiting each match of each file in the list of matches, > > and that takes most of the time. Can this be avoided somehow? The > > 'lid' command already includes all the matches in Grep format, so why > > are we visiting each match, when the information should be already > > available? > > Huh, yes, that's the area of optimization I've forgotten about (mostly > because of using plain Grep myself anyway). But using plain Grep has the same problem: Grep reports the hits, and we then go ahead and visit each hit, right? > If id-utils (and some other tools we delegate to) know symbol > boundaries, maybe we don't need to double-check. It should know. > However, Grep output, which 'lid --result=grep' also uses, only outputs > the line number, but not the starting column. > > Take this example: > > I'm searching for 'buffer'. The last line of the output looks like this: > > xmenu.c:832: set_buffer_internal_1 (XBUFFER (buffer)); > > How do I find the appropriate 'buffer' match in this line? You already have the code that looks for the match in each line. What I had in mind is looking in the output of Grep or lid, instead of visiting the file and looking in that file's buffer. Does this make sense? > Or can we ask 'lid' (and, ideally, Grep too) to include the column of a > match in the output? No, I don't think such an option exists. It could be a great enhancement request, though ;-) > Grep has colorized output, so I should be able to work with that. But > not 'lid', it seems. Another enhancement to request, I guess. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 13:20:44 2016 Received: (at 23223) by debbugs.gnu.org; 6 Apr 2016 17:20:44 +0000 Received: from localhost ([127.0.0.1]:52004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anr8G-0005bT-2r for submit@debbugs.gnu.org; Wed, 06 Apr 2016 13:20:44 -0400 Received: from eggs.gnu.org ([208.118.235.92]:48024) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anr8E-0005bA-5a for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 13:20:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anr83-000346-TN for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 13:20:36 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45105) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anr83-000340-N2; Wed, 06 Apr 2016 13:20:31 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1913 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1anr83-0003c9-3g; Wed, 06 Apr 2016 13:20:31 -0400 Date: Wed, 06 Apr 2016 20:20:11 +0300 Message-Id: <834mbemz78.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <8351f540-d999-125c-9ad2-2e1b221f9bf5@yandex.ru> (message from Dmitry Gutov on Wed, 6 Apr 2016 18:09:55 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8351f540-d999-125c-9ad2-2e1b221f9bf5@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Dmitry Gutov > Date: Wed, 6 Apr 2016 18:09:55 +0300 > > On 04/06/2016 03:37 AM, Dmitry Gutov wrote: > > > If id-utils (and some other > > tools we delegate to) know symbol boundaries > > Is that actually the case, though? At most, it's using some wonky > heuristics. > > For instance, if I run 'mkid' in the top directory of the Emacs sources, > I can search for 'defmacro', and get a few lines with 'cl-defmacro' > instead among the results. Or search for 'eval-when', and get some lines > with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. > > id-utils version 4.6 here. Sounds like a bug in reporting matches. We invoke lid with the -w switch, which should have reported only complete symbols, AFAIU (although the documentation confusingly talks about "words"). Perhaps report this to the developers. Anyway, if you search in the hits instead of in the file's buffer, this problem should not affect us, I think. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 19:11:48 2016 Received: (at 23223) by debbugs.gnu.org; 6 Apr 2016 23:11:48 +0000 Received: from localhost ([127.0.0.1]:52237 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anwc0-0007Zv-Mj for submit@debbugs.gnu.org; Wed, 06 Apr 2016 19:11:48 -0400 Received: from mail-wm0-f47.google.com ([74.125.82.47]:33826) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anwby-0007Zi-UC for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 19:11:47 -0400 Received: by mail-wm0-f47.google.com with SMTP id l6so2666851wml.1 for <23223@debbugs.gnu.org>; Wed, 06 Apr 2016 16:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=NnnAXO1b51RYP1dFIKfnTWHMUwCHKxAlxybwqagS0wE=; b=hzLFHJ3U30Yzf/AQ+j/HwapHkT5Y37fLE6D59g9h92NbxukjoLL6s95PVtQCuLzcWQ FNW8BsHz1M1E7jVthBqBLNqk24PtFx92vxWeF3giot9pNgn6CS5YeLAG/HgCHDvG6eVv NNQeiV4U2QLODvOYd2Bv0MOXXde3Kl5BGIWyoeYSLhAxjkTalitl1pk+6ym8PHSLyHhy fdw8vx52ldhhBTrKO/jNd42OCN81FfjLA4hq+EA/aRqW9hZ+yp5M48kKKCmMVfhe8shu a4nRGiEdKKvYTOckg2fSEbXsqolAIW3uq7gjHlbDWKi9W+geMmCfxfcNyftPqPu3QwK4 n5og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=NnnAXO1b51RYP1dFIKfnTWHMUwCHKxAlxybwqagS0wE=; b=ZYd934/DaLaNm6TqCEs3lcBdY6S2B+vGqKpLq1hFuLsdnzrg7my1RH6w8ne4o6k4fY n6WMIKwaMu6XbHNnFmi4K1xHE53fBybPurDycm7q8rvaxCN73BKkxZJDE39RWWBXDN/H SZFUIb0H1vhiu2N4+cO0jMuiLNYlt6U5N/Z45iCyr6X8On0gdDnzaVfwTLTFiu8+nDGd YBr8Qs7vZlXmg8GdzdnuBOfwIqP6LqNv6TBjHsx22JMss/LayWd4ha1A0kDgg2gnAbqG uYcnow08mQpvkjmQekOcaKpbe1MivCsp2t+bOoAQbeen/5WpwiLvVEA/mmbnVtBiH+P1 hgIw== X-Gm-Message-State: AD7BkJKv4EAxZCiie4jB0w6i6tEMACV6il9FB88qD0846A8mDomrWrdQdGSVFpd+Znq8GQ== X-Received: by 10.194.188.20 with SMTP id fw20mr34289540wjc.80.1459984301494; Wed, 06 Apr 2016 16:11:41 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id 138sm27281934wmf.13.2016.04.06.16.11.40 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 06 Apr 2016 16:11:40 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8351f540-d999-125c-9ad2-2e1b221f9bf5@yandex.ru> <834mbemz78.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Thu, 7 Apr 2016 02:11:39 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <834mbemz78.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/06/2016 08:20 PM, Eli Zaretskii wrote: > Sounds like a bug in reporting matches. A bug in the matcher, maybe? > We invoke lid with the -w > switch, which should have reported only complete symbols, AFAIU > (although the documentation confusingly talks about "words"). Perhaps > report this to the developers. I suspected the wording would mean that it just invokes Grep and uses the "boundary" anchors (\b), but no, doing a Grep search for "\beval-when\b" yields a bigger (and more consistent) set of matches. So it's most likely a bug. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 20:11:35 2016 Received: (at 23223) by debbugs.gnu.org; 7 Apr 2016 00:11:35 +0000 Received: from localhost ([127.0.0.1]:52264 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anxXr-0000cQ-A4 for submit@debbugs.gnu.org; Wed, 06 Apr 2016 20:11:35 -0400 Received: from mail-wm0-f54.google.com ([74.125.82.54]:34262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1anxXp-0000c9-Ed for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 20:11:33 -0400 Received: by mail-wm0-f54.google.com with SMTP id l6so3767817wml.1 for <23223@debbugs.gnu.org>; Wed, 06 Apr 2016 17:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=EXWlu7eTPwu7NenwDyF/0MnI52/Lofd5WZGrSDFM1e0=; b=zAZKMhZI6CboYY0/OKZQsUN+7EvPwTOqr5uTQPM/hRBZbJMmJp96+S1bZTOH+P0PEv 1KG7iKzPlfnQiE9cE8G6vFFZ4EIzV8ZGhCYbumlxQ8Rb7x3WwCUM9nGBDOediAUahN9Z n4FHmDgkudeFHsG1VuVzXK10O843qruBi3M9rbcIKJyaYl70vRwWn2v2/xi3ugKysYgb QcsLzgPMUs93EuO6Uf+wrGWoV6ZfRvZUIMfqBX6JeqslSf3MUZS9q16gczIix38k363U AoD8Eejsy3szaVPPmf8ZbduQNb/WPeKIWRPVM/3CAs/Ujxe9EeDinXlkomwbApF1cUHt p1fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=EXWlu7eTPwu7NenwDyF/0MnI52/Lofd5WZGrSDFM1e0=; b=TRxBiWOF2ST/ZFbm2zoIu5ZusDWVm8zWwHQjdyLcgPYWOIIOII93pUyEb3Qn17D8QI igBEZzlxWuByoMJxoho1dPbNkwFFcWaGBYhFBEeV3Q4ycgkHf6GgN6PZgpFWsUz81MCa 5vYNjTSazwNAkaazvzQxGU6tMfXUp+NJsmdKQv0g2PH/ffoC4WoH2zn8AiXod0SGnyQX stcx0YEMetq+H/EQyhxRDbrxv5bdSE+JDF03yT3sWbKwiWqN3r0musfzYRgZls2/bYhi Rl8tv1SZmtkywbFnKPhjusqDnsPJCh9JLHyDJnDKUY8Crps8WW0Oib+hY2p/iBEu2mlG 0OaA== X-Gm-Message-State: AD7BkJI4EYd/SJ5Gz9hxDZMxhIUX9My6T1VAZ/8xcX/i+mKT190GgA2JsIxo51nm4fU7FQ== X-Received: by 10.194.84.66 with SMTP id w2mr143761wjy.6.1459987887729; Wed, 06 Apr 2016 17:11:27 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id gk4sm5631608wjd.7.2016.04.06.17.11.26 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 06 Apr 2016 17:11:26 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Thu, 7 Apr 2016 03:11:25 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <8360vumzk4.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/06/2016 08:12 PM, Eli Zaretskii wrote: > But using plain Grep has the same problem: Grep reports the hits, and > we then go ahead and visit each hit, right? Using Grep has the slowness problem, but it doesn't have this opportunity for optimization, since Grep definitely doesn't know what characters constitute a symbol in different languages. But your proposal may speed up Grep searches too, so that's a plus. >> How do I find the appropriate 'buffer' match in this line? > > You already have the code that looks for the match in each line. What > I had in mind is looking in the output of Grep or lid, instead of > visiting the file and looking in that file's buffer. Does this make > sense? Yes, but the search for \_ needs a properly set syntax table, syntax-propertize-function, and possibly other variables that the latter might depend on. Basically, it needs a buffer in the right major mode. And switching major modes is actually not cheap: (benchmark 230 '(with-temp-buffer (c-mode))) => 1.20s in my working Emacs instance (much faster in a pristine one, admittedly). But I'll have a try at doing this while reusing the temporary buffer. Some major modes could also trip over incomplete source code, but we'll get to that when we see that. Syntax highlighting seems out of the question, though, with high likelihood of getting it wrong. We also have the possibility of syntax-propertize-function not applying the correct syntax classes to the code fragment because it doesn't see the whole file, but I guess it's a small-ish price to pay for the performance improvement, and could be fixed on a case-by-case basis. >> Or can we ask 'lid' (and, ideally, Grep too) to include the column of a >> match in the output? > > No, I don't think such an option exists. It could be a great > enhancement request, though ;-) I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. But anyway, if there is no such feature now, we have to choose another solution in Emacs 25.1. Even if id-utils were to add this today, it would take time to trickle to the majority of our users. The bar is actually higher: to make use of the enhancement, we'd need Global and CScope to support it, too, or at least to be able to detect the feature at runtime. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 06 22:50:04 2016 Received: (at 23223) by debbugs.gnu.org; 7 Apr 2016 02:50:04 +0000 Received: from localhost ([127.0.0.1]:52304 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ao01E-0004Rp-4j for submit@debbugs.gnu.org; Wed, 06 Apr 2016 22:50:04 -0400 Received: from eggs.gnu.org ([208.118.235.92]:55881) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ao01B-0004RH-IN for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 22:50:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ao011-0002hN-Mk for 23223@debbugs.gnu.org; Wed, 06 Apr 2016 22:49:56 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:53740) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ao011-0002hJ-JE; Wed, 06 Apr 2016 22:49:51 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2586 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ao010-0005YC-VW; Wed, 06 Apr 2016 22:49:51 -0400 Date: Thu, 07 Apr 2016 05:49:33 +0300 Message-Id: <83oa9mku9u.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Thu, 7 Apr 2016 02:11:39 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8351f540-d999-125c-9ad2-2e1b221f9bf5@yandex.ru> <834mbemz78.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 7 Apr 2016 02:11:39 +0300 > > On 04/06/2016 08:20 PM, Eli Zaretskii wrote: > > > Sounds like a bug in reporting matches. > > A bug in the matcher, maybe? Could be, but less likely. I looked at the Lisp scanner code and didn't say anything that could explain this. Moreover, the fid command correctly shows both defmacro and cl-defmacro being present in, say, lisp/ibuf-macs.el. > > We invoke lid with the -w > > switch, which should have reported only complete symbols, AFAIU > > (although the documentation confusingly talks about "words"). Perhaps > > report this to the developers. > > I suspected the wording would mean that it just invokes Grep and uses > the "boundary" anchors (\b), but no, doing a Grep search for > "\beval-when\b" yields a bigger (and more consistent) set of matches. No, lid doesn't invoke Grep, AFAIR. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 07 11:04:14 2016 Received: (at 23223) by debbugs.gnu.org; 7 Apr 2016 15:04:14 +0000 Received: from localhost ([127.0.0.1]:53030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aoBTe-0001Up-Ho for submit@debbugs.gnu.org; Thu, 07 Apr 2016 11:04:14 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aoBTa-0001UH-08 for 23223@debbugs.gnu.org; Thu, 07 Apr 2016 11:04:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aoBTP-00041f-1g for 23223@debbugs.gnu.org; Thu, 07 Apr 2016 11:04:00 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36391) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aoBTO-00041b-VQ; Thu, 07 Apr 2016 11:03:54 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2905 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aoBTO-0001Nh-CA; Thu, 07 Apr 2016 11:03:54 -0400 Date: Thu, 07 Apr 2016 18:03:37 +0300 Message-Id: <83mvp5lauu.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Thu, 7 Apr 2016 03:11:25 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 7 Apr 2016 03:11:25 +0300 > > We invoke lid with the -w > switch, which should have reported only complete symbols, AFAIU > (although the documentation confusingly talks about "words"). Perhaps > report this to the developers. > > I suspected the wording would mean that it just invokes Grep and uses the "boundary" anchors (\b), but no, doing a Grep search for "\beval-when\b" yields a bigger (and more consistent) set of matches. > > So it's most likely a bug. Heh, I completely forgot how IDutils work. The ID database doesn't store the locations where the identifiers were found, all it stores is the list of identifiers found in each file. When lid is invoked, it queries the database about the files that reference the identifier; then it opens each file in the query results, and searches for the identifier on each line of that file. When lid is invoked to produce a Grep-like output and with the -w option (which is what we do), it searches each line for the moral equivalent of [^[:alnum:]_]IDENT[^[:alnum:]_] where IDENT is the identifier we asked about. So therefore, your observation: > For instance, if I run 'mkid' in the top directory of the Emacs sources, I can search for 'defmacro', and get a few lines with 'cl-defmacro' instead among the results. Or search for 'eval-when', and get some lines with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. can now be perfectly explained, because none of the files we have in Emacs that reference cl-eval-when also reference eval-when, but some files that reference defmacro have matches for cl-defmacro, and some files that reference eval-when have matches for eval-when-compile. IOW, the ID database is used to minimize the number of files to search, i.e. avoid searching files that cannot possibly result in a match, that's all. > Or can we ask 'lid' (and, ideally, Grep too) to include the column of a > match in the output? > > No, I don't think such an option exists. It could be a great > enhancement request, though ;-) > > I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. I could do that, but given the explanation above of what -w means, it will hardly help us, right? > But anyway, if there is no such feature now, we have to choose another solution in Emacs 25.1. Even if id-utils were to add this today, it would take time to trickle to the majority of our users. > > The bar is actually higher: to make use of the enhancement, we'd need Global and CScope to support it, too, or at least to be able to detect the feature at runtime. Right. So I think we need to assume that the program we invoke to bring the matches is not smart enough anyway, and Emacs will have to search in the matches for those we want to present, filtering out the false positives. This assumption seems to fit well all the programs that can support the queries required by xref-find-references. I'm not sure I understand what you have in mind for that, though. Are you going to switch the major mode of the temporary buffer as determined by the file name of each match? Maybe we should take a hint from the major mode of the buffer in which the command was invoked? Also, how about just using the syntax table of the major mode, instead of actually turning the mode in the buffer -- could that be significantly cheaper? From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 08 23:12:38 2016 Received: (at 23223) by debbugs.gnu.org; 9 Apr 2016 03:12:38 +0000 Received: from localhost ([127.0.0.1]:55076 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aojKA-0005c5-CH for submit@debbugs.gnu.org; Fri, 08 Apr 2016 23:12:38 -0400 Received: from mail-wm0-f47.google.com ([74.125.82.47]:37080) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aojK9-0005bt-Aw for 23223@debbugs.gnu.org; Fri, 08 Apr 2016 23:12:37 -0400 Received: by mail-wm0-f47.google.com with SMTP id n3so42388096wmn.0 for <23223@debbugs.gnu.org>; Fri, 08 Apr 2016 20:12:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=9aZrTIIdGAqwkvdT62Kj7Wt4P4O4EiK3DCBKZQqu7Og=; b=xH80f+NzcAGzskXB+dHVAG65VfD0aQmyCDGklqbS2YqhZXBjbvdC3DBaOgDCbJKDQA KcgTEuu+ZEOG5d4Ry7FBxYQsvkuU04HIX32qSKvOH8A9miR96AENdmbnTbFcsxVpThgP OlVrPmkZKRs0EG/ybWbvbSf/bKohVXJ++XIAXJApAJSQV+7h7uRTCFmVHFX2ls9zq49t 3hHsLKgoRrur/pcFijTzKXupqtZeYfvKjK9QSfPW/lvT/DGDKpSpVinKcIoM+fypB1/V CwN8giDXNMcmfSgrUURR/Th8WhLdOya5JJCDl2PCZxSz2k95nt9Zenr9lnykTv998yLZ Hx/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=9aZrTIIdGAqwkvdT62Kj7Wt4P4O4EiK3DCBKZQqu7Og=; b=BIszRnP+enJvVx362DpXsNKOifpDFZDU5riQaBMU5xbHbreC2M8ev7P2FE3SI0gXRp TUFe7Hk2YmykvEkne3G1BRsK0pA9zhp1PXJ9kM2X+6A3V50mFN0W+uZFflM5J+tDUnL+ EgeYZKkg2Z8/vQfM19/bfmW03i7lM7MC4zX+UeHLgQGOp5k7cAuf9tEALmhAEU5RQCwd ogALNOz9V+2dNyiIloYgVqtacPGfD9zRRMrYpAL2bismpDVDsyJmyIilLbnDFxjx7zUs hU8uGGLB0O74HLL9T9Iqkky1EypH1HCq+2NEF2jIfVeaze8zKZRtDjYs+6+PniRpSJyc FwTg== X-Gm-Message-State: AD7BkJL5REHotXEspd2ny0/hDHROnONeAc2YCOhctURMIAXIVwvLZXPwPZispSvS9O5Msg== X-Received: by 10.28.97.197 with SMTP id v188mr7189961wmb.67.1460171551653; Fri, 08 Apr 2016 20:12:31 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id m6sm16069239wje.21.2016.04.08.20.12.30 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 08 Apr 2016 20:12:30 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> From: Dmitry Gutov Message-ID: <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> Date: Sat, 9 Apr 2016 06:12:29 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83mvp5lauu.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/07/2016 06:03 PM, Eli Zaretskii wrote: > Heh, I completely forgot how IDutils work. The ID database doesn't > store the locations where the identifiers were found, all it stores is > the list of identifiers found in each file. When lid is invoked, it > queries the database about the files that reference the identifier; > then it opens each file in the query results, and searches for the > identifier on each line of that file. When lid is invoked to produce > a Grep-like output and with the -w option (which is what we do), it > searches each line for the moral equivalent of > > [^[:alnum:]_]IDENT[^[:alnum:]_] > > where IDENT is the identifier we asked about. That explains everything, thanks. >> I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. > > I could do that, but given the explanation above of what -w means, it > will hardly help us, right? It might if -w were to be improved, though. We can request for it to be smarter and do the post-filtering itself, using a language-aware scanner. Even if no other feature is added, the accuracy of results will improve for all users of id-utils, and Emacs will automatically end up having to do less work for this feature. That's not pressing, though. > Right. So I think we need to assume that the program we invoke to > bring the matches is not smart enough anyway, and Emacs will have to > search in the matches for those we want to present, filtering out the > false positives. This assumption seems to fit well all the programs > that can support the queries required by xref-find-references. Probably. > I'm not sure I understand what you have in mind for that, though. Are > you going to switch the major mode of the temporary buffer as > determined by the file name of each match? Yes. We can avoid switching if the major mode of the previous match was the same as the current one. Having too many major modes involved in one search is highly unlikely, so the switching overhead shouldn't be too much of a problem, actually. > Maybe we should take a > hint from the major mode of the buffer in which the command was > invoked? That could only work for the output of xref-find-references, but not project-find-regexp, which is also in need of speeding up. Anyway, whether we "take a hint" or not, we're going to end up with imperfect results: if we don't visit the target buffers, we're going to have to ignore all the ways Emacs allows specifying the major mode, aside from auto-mode-alist ("mode: " specification, interpreter-mode-alist, magic-mode-alist, magic-fallback-mode-alist). Hmm. Maybe we can still support most of those using (insert-file-contents "file-name" nil 0 200), at the cost of some extra overhead. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 09 03:26:32 2016 Received: (at 23223) by debbugs.gnu.org; 9 Apr 2016 07:26:32 +0000 Received: from localhost ([127.0.0.1]:55100 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aonHr-0003MA-S9 for submit@debbugs.gnu.org; Sat, 09 Apr 2016 03:26:32 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47858) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aonHq-0003Ly-JE for 23223@debbugs.gnu.org; Sat, 09 Apr 2016 03:26:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aonHi-0001sz-8n for 23223@debbugs.gnu.org; Sat, 09 Apr 2016 03:26:25 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36271) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aonHi-0001sv-5W; Sat, 09 Apr 2016 03:26:22 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1115 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aonHh-0005il-G2; Sat, 09 Apr 2016 03:26:21 -0400 Date: Sat, 09 Apr 2016 10:25:43 +0300 Message-Id: <83vb3ri6q0.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> (message from Dmitry Gutov on Sat, 9 Apr 2016 06:12:29 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Sat, 9 Apr 2016 06:12:29 +0300 > > >> I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. > > > > I could do that, but given the explanation above of what -w means, it > > will hardly help us, right? > > It might if -w were to be improved, though. We can request for it to be > smarter and do the post-filtering itself, using a language-aware > scanner. Even if no other feature is added, the accuracy of results will > improve for all users of id-utils, and Emacs will automatically end up > having to do less work for this feature. > > That's not pressing, though. I agree. > > I'm not sure I understand what you have in mind for that, though. Are > > you going to switch the major mode of the temporary buffer as > > determined by the file name of each match? > > Yes. We can avoid switching if the major mode of the previous match was > the same as the current one. Having too many major modes involved in one > search is highly unlikely, so the switching overhead shouldn't be too > much of a problem, actually. Would it help to only use the mode's syntax table, and avoid switching on the major mode as a whole? > Anyway, whether we "take a hint" or not, we're going to end up with > imperfect results: if we don't visit the target buffers, we're going to > have to ignore all the ways Emacs allows specifying the major mode, > aside from auto-mode-alist ("mode: " specification, > interpreter-mode-alist, magic-mode-alist, magic-fallback-mode-alist). That problem is relevant for IDutils as well (the scanner is determined by the file's extension only), so we already have a certain (hopefully, small) number of misses and false positives. I think this cannot be entirely avoided. So maybe we shouldn't try so hard avoiding false positives. E.g., the "M-x gid" command, which comes with IDutils and is a trivial wrapper around lid invocation, simply shows the output in a Grep-like buffer, through which you can step with next-error. It is lightning-fast: what takes 13 sec with xref-find-references, takes less than 2 sec with "M-x gid". > Hmm. Maybe we can still support most of those using > (insert-file-contents "file-name" nil 0 200), at the cost of some extra > overhead. Perhaps use insert-file-contents-literally, as decoding could slow down things considerably. You are only looking for ASCII text. Also note that a mode can be specified in file-local variables at the file's end, though. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 10 19:27:44 2016 Received: (at 23223) by debbugs.gnu.org; 10 Apr 2016 23:27:44 +0000 Received: from localhost ([127.0.0.1]:57693 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apOlb-0002GF-V4 for submit@debbugs.gnu.org; Sun, 10 Apr 2016 19:27:44 -0400 Received: from mail-wm0-f41.google.com ([74.125.82.41]:34281) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apOla-0002G0-QB for 23223@debbugs.gnu.org; Sun, 10 Apr 2016 19:27:43 -0400 Received: by mail-wm0-f41.google.com with SMTP id l6so124512106wml.1 for <23223@debbugs.gnu.org>; Sun, 10 Apr 2016 16:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=eMSIQYRQWCGvNSZldVAvuXki+b19QcRtNWEAh9azv4M=; b=DRgKXru4+b1xoX8typFZ1F0HO8yQcbtoL0t6LirdAnIPrgWOI9c5TsO8w9f+TF5NLN HJlv+KzXVGiKcRDT4gWalxUrjS6FPkqFHCY1ODLgEBASjBDZk0w2602iiWDTZJ4iPvAW 2PJuGAW7K5K9Nbr7/VBVh5LmDwRGTTxViaoPPYePkEbAk4BUxx9DULtCcNQnEOsEJfFR VjF562708XLrta8xp4ixwBUf0wmocvC9VGSklB0ZUMfk4EXzYYc1RmuPkNtX9eRvCXQD mkkWEvE21s8THXtZ0TKuCwv2dj4oCYmsGVBQwaxxp8188lYbJp/RFBtIvIgbkne5gv6x 14Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to; bh=eMSIQYRQWCGvNSZldVAvuXki+b19QcRtNWEAh9azv4M=; b=kDgYTAGyzJ9fM5ta9I9X3l1lyEJCkyjG9N35ucDsecyuQijk9D/Y60/JAexQGDtSQO VFjeTEonWB8GSmsyyDf6ZPT5saaD/8bnhKrBs16q3sn5E4XuLk1zYIonczFpFtyXROdy YXnmTYwbeTG8HUPBZ17x8EFrRf/ALHG3UvNDvh8nySolG5cELe5YYGxguwD6jtKoHp/5 02H/q2tTnoz50TNr7dWoVDwa5l6NOEBhWYgpDKpNSqxOtFf1DlyR6IUCfAH+KSNFv0au hEah9d8zciaMNK63lMh+odCzTop5mbitHGz17kTR/6ta5HQHz4sBq7ib4op/OHXeJR2w dCMg== X-Gm-Message-State: AD7BkJKuA7XAisFAriP7Ai69vRYNOWZ6UbFf82zrTw5hHoZZ5wFIWIW7XjpgMsceWRfuvg== X-Received: by 10.194.57.236 with SMTP id l12mr20197528wjq.126.1460330857237; Sun, 10 Apr 2016 16:27:37 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id a73sm14397625wme.2.2016.04.10.16.27.35 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 10 Apr 2016 16:27:36 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> From: Dmitry Gutov Message-ID: Date: Mon, 11 Apr 2016 02:27:34 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83vb3ri6q0.fsf@gnu.org> Content-Type: multipart/mixed; boundary="------------000DF9DF9ACD1293B1D3637F" X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) This is a multi-part message in MIME format. --------------000DF9DF9ACD1293B1D3637F Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Please try the attached patch. It cuts the time to search for 'current_buffer' from 4.5s to 0.75s here. You can comment out different parts of xref--collect-matches, to test if there are any easy bottlenecks left. The only one I see is find-buffer-visiting; replacing it with get-file-buffer yields some extra boost, but it would come with corresponding downsides in those exact cases for which find-buffer-visiting was designed (although I can add another local variable instead that would cache the previous file name and buffer used for it; but that's scraping the bottom). To measure, open src/xdisp.c and evaluate (benchmark 1 '(xref-collect-references "current_buffer" default-directory)). On 04/09/2016 10:25 AM, Eli Zaretskii wrote: > Would it help to only use the mode's syntax table, and avoid switching > on the major mode as a whole? Not for performance anymore (see above). You can't always determine the syntax table from the major mode name (there is a convention, but it's not iron-clad), it might be in an autoloaded file, etc... And like I mentioned before, we also need syntax-propertize-function, and potentially any buffer-local variables that it could use. > That problem is relevant for IDutils as well (the scanner is > determined by the file's extension only), so we already have a certain > (hopefully, small) number of misses and false positives. That affects xref-find-references (as long as you use id-utils, which is not mandatory) but not project-find-regexp. The currently discussed tuneup should improve both. > I think this > cannot be entirely avoided. So maybe we shouldn't try so hard > avoiding false positives. It seems we can't avoid ignoring the mode specification at the bottom, but that's about it. And nothing's stopping id-utils (or other tools) from using a better language detection scheme in the future. > E.g., the "M-x gid" command, which comes > with IDutils and is a trivial wrapper around lid invocation, simply > shows the output in a Grep-like buffer, through which you can step > with next-error. Do you actually want xref-find-regexp, and not xref-find-references? > It is lightning-fast: what takes 13 sec with > xref-find-references, takes less than 2 sec with "M-x gid". What's the new time you get from the former? > Perhaps use insert-file-contents-literally, as decoding could slow > down things considerably. No significant difference here, on the given example. By the way, the "insert-file-contents + set-auto-mode" dance comes with a new minor downside: extra chatter from the major modes. E.g. try project-file-regexp with "should have received a copy". We can avoid saving it to the message log, but it appears in the echo area either way. --------------000DF9DF9ACD1293B1D3637F Content-Type: text/x-patch; name="xref-with-temp-buffer.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xref-with-temp-buffer.diff" diff --git a/lisp/cedet/semantic/symref/cscope.el b/lisp/cedet/semantic/symref/cscope.el index 4890b5b..3abd8b3 100644 --- a/lisp/cedet/semantic/symref/cscope.el +++ b/lisp/cedet/semantic/symref/cscope.el @@ -60,6 +60,9 @@ semantic-symref-tool-cscope (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-cscope--line-re + "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-cscope)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -78,8 +81,13 @@ semantic-symref-tool-cscope ;; We have to return something at this point. subtxt))) ) - (t - (when (re-search-forward "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) " nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-cscope--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1)) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; :resulttype is 'line + (when (re-search-forward semantic-symref-cscope--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1))) )))) diff --git a/lisp/cedet/semantic/symref/global.el b/lisp/cedet/semantic/symref/global.el index e4c114e..a33427e 100644 --- a/lisp/cedet/semantic/symref/global.el +++ b/lisp/cedet/semantic/symref/global.el @@ -49,6 +49,9 @@ semantic-symref-tool-global (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-global--line-re + "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-global)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -57,8 +60,13 @@ semantic-symref-tool-global ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-global--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 3) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) " nil t) + (when (re-search-forward semantic-symref-global--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 3)) )))) diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el index 5d1fea8..868e6c3 100644 --- a/lisp/cedet/semantic/symref/grep.el +++ b/lisp/cedet/semantic/symref/grep.el @@ -188,6 +188,9 @@ semantic-symref-grep-shell ;; Return the answer ans)) +(defconst semantic-symref-grep--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-grep)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -195,8 +198,13 @@ semantic-symref-grep-shell ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-grep--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + (when (re-search-forward semantic-symref-grep--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 1)) )))) diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el index 4127d7a..db3e9a0 100644 --- a/lisp/cedet/semantic/symref/idutils.el +++ b/lisp/cedet/semantic/symref/idutils.el @@ -49,6 +49,9 @@ semantic-symref-tool-idutils (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-idutils--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-idutils)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -59,8 +62,13 @@ semantic-symref-tool-idutils ((eq (oref tool :searchtype) 'tagcompletions) (when (re-search-forward "^\\([^ ]+\\) " nil t) (match-string 1))) - (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-idutils--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1) default-directory) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; resulttype is line + (when (re-search-forward semantic-symref-idutils--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1) default-directory)) )))) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index feed0fb..7354f8b 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -859,19 +859,25 @@ xref-collect-references ;; to force the backend to use `default-directory'. (let* ((ede-minor-mode nil) (default-directory dir) + ;; FIXME: Remove CScope and Global from the recognized tools? + ;; The current implementations interpret the symbol search as + ;; "find all calls to the given function", but not function + ;; definition. And they return nothing when passed a variable + ;; name, even a global one. (semantic-symref-tool 'detect) (case-fold-search nil) - (res (semantic-symref-find-references-by-name symbol 'subdirs)) - (hits (and res (oref res hit-lines))) - (orig-buffers (buffer-list))) + (inst (semantic-symref-instantiate :searchfor symbol + :searchtype 'symbol + :searchscope 'subdirs + :resulttype 'line-and-text)) + (hits (semantic-symref-perform-search inst)) + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (unwind-protect (cl-mapcan (lambda (hit) (xref--collect-matches - hit (format "\\_<%s\\_>" (regexp-quote symbol)))) + hit (format "\\_<%s\\_>" (regexp-quote symbol)) + tmp-buffer)) hits) - ;; TODO: Implement "lightweight" buffer visiting, so that we - ;; don't have to kill them. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) ;;;###autoload (defun xref-collect-matches (regexp files dir ignores) @@ -890,34 +896,24 @@ xref-collect-matches files (expand-file-name dir) ignores)) - (orig-buffers (buffer-list)) (buf (get-buffer-create " *xref-grep*")) (grep-re (caar grep-regexp-alist)) - (counter 0) - reporter - hits) + hits + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (with-current-buffer buf (erase-buffer) (call-process-shell-command command nil t) (goto-char (point-min)) (while (re-search-forward grep-re nil t) - (push (cons (string-to-number (match-string 2)) - (match-string 1)) + (push (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))) hits))) - (setq reporter (make-progress-reporter - (format "Collecting search results...") - 0 (length hits))) (unwind-protect (cl-mapcan (lambda (hit) - (prog1 - (progress-reporter-update reporter counter) - (cl-incf counter)) - (xref--collect-matches hit regexp)) + (xref--collect-matches hit regexp tmp-buffer)) (nreverse hits)) - (progress-reporter-done reporter) - ;; TODO: Same as above. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) (defun xref--rgrep-command (regexp files dir ignores) (require 'find-dired) ; for `find-name-arg' @@ -980,30 +976,49 @@ xref--regexp-to-extended (match-string 1 str))))) str t t)) -(defun xref--collect-matches (hit regexp) - (pcase-let* ((`(,line . ,file) hit) - (buf (or (find-buffer-visiting file) - (semantic-find-file-noselect file)))) - (with-current-buffer buf - (save-excursion +(defvar xref--temp-buffer-file-name nil) + +(defun xref--collect-matches (hit regexp tmp-buffer) + (pcase-let* ((`(,line ,file ,text) hit) + (buf (find-buffer-visiting file))) + (if buf + (with-current-buffer buf + (save-excursion + (goto-char (point-min)) + (forward-line (1- line)) + (xref--collect-matches-1 regexp file line + (line-beginning-position) + (line-end-position)))) + (with-current-buffer tmp-buffer + (erase-buffer) + (unless (equal file xref--temp-buffer-file-name) + (insert-file-contents file nil 0 200) + (setq-local delay-mode-hooks t) + (set-auto-mode t) + (setq-local xref--temp-buffer-file-name file) + (setq-local inhibit-read-only t) + (erase-buffer)) + (insert text) (goto-char (point-min)) - (forward-line (1- line)) - (let ((line-end (line-end-position)) - (line-beg (line-beginning-position)) - matches) - (syntax-propertize line-end) - ;; FIXME: This results in several lines with the same - ;; summary. Solve with composite pattern? - (while (re-search-forward regexp line-end t) - (let* ((beg-column (- (match-beginning 0) line-beg)) - (end-column (- (match-end 0) line-beg)) - (loc (xref-make-file-location file line beg-column)) - (summary (buffer-substring line-beg line-end))) - (add-face-text-property beg-column end-column 'highlight - t summary) - (push (xref-make-match summary loc (- end-column beg-column)) - matches))) - (nreverse matches)))))) + (xref--collect-matches-1 regexp file line + (point) + (point-max)))))) + +(defun xref--collect-matches-1 (regexp file line line-beg line-end) + (let (matches) + (syntax-propertize line-end) + ;; FIXME: This results in several lines with the same + ;; summary. Solve with composite pattern? + (while (re-search-forward regexp line-end t) + (let* ((beg-column (- (match-beginning 0) line-beg)) + (end-column (- (match-end 0) line-beg)) + (loc (xref-make-file-location file line beg-column)) + (summary (buffer-substring line-beg line-end))) + (add-face-text-property beg-column end-column 'highlight + t summary) + (push (xref-make-match summary loc (- end-column beg-column)) + matches))) + (nreverse matches))) (provide 'xref) --------------000DF9DF9ACD1293B1D3637F-- From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 11 11:57:40 2016 Received: (at 23223) by debbugs.gnu.org; 11 Apr 2016 15:57:40 +0000 Received: from localhost ([127.0.0.1]:59625 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apeDc-0001V6-Hj for submit@debbugs.gnu.org; Mon, 11 Apr 2016 11:57:40 -0400 Received: from eggs.gnu.org ([208.118.235.92]:55839) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apeDa-0001Us-3z for 23223@debbugs.gnu.org; Mon, 11 Apr 2016 11:57:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1apeDR-0000yn-4r for 23223@debbugs.gnu.org; Mon, 11 Apr 2016 11:57:33 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35811) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apeDR-0000yj-1Q; Mon, 11 Apr 2016 11:57:29 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1030 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1apeDN-00041h-Tc; Mon, 11 Apr 2016 11:57:28 -0400 Date: Mon, 11 Apr 2016 18:56:54 +0300 Message-Id: <83zit0f8ah.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: (message from Dmitry Gutov on Mon, 11 Apr 2016 02:27:34 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Mon, 11 Apr 2016 02:27:34 +0300 > > Please try the attached patch. It cuts the time to search for > 'current_buffer' from 4.5s to 0.75s here. Thanks, I see a similar speedup (from 13 sec to just 3). Very impressive. Unfortunately, it seems to miss matches: out of 1127 matches of current_buffer with the original version, the new one only shows 963. It sounds like some conditions on what exactly is a symbol need adjustment, because the new code finds this: 122: (buf == current_buffer ? BEGV \ but not this: 42: #define BEGV (current_buffer->begv) IOW, if the symbol is followed by a "->", it is not recognized. As an aside, if I invoke xref-find-references without an ID file, which AFAIU means Emacs will invoke find+grep, I get this error: semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode unless I invoke xref-find-references from a buffer in C mode. Curiously, this doesn't happen when there's an ID file and IDutils is invoked. Is this expected? > > E.g., the "M-x gid" command, which comes > > with IDutils and is a trivial wrapper around lid invocation, simply > > shows the output in a Grep-like buffer, through which you can step > > with next-error. > > Do you actually want xref-find-regexp, and not xref-find-references? No, of course not. "gid" is just a short for "lid -R grep", so the contents I get is the same as xref-find-references does, it's just formatted differently. And that is what I want: symbol hits, not just regular expressions that ignore the source language syntax.1 > > It is lightning-fast: what takes 13 sec with > > xref-find-references, takes less than 2 sec with "M-x gid". > > What's the new time you get from the former? 3 sec (in an unoptimized build, I'd expect this to become 1 sec in an optimized build). So we are OK speed-wise, we just need to fix the misses mentioned above. > By the way, the "insert-file-contents + set-auto-mode" dance comes with > a new minor downside: extra chatter from the major modes. E.g. try > project-file-regexp with "should have received a copy". I don't see this in xref-find-references. Should I? > We can avoid saving it to the message log, but it appears in the > echo area either way. Can't you bind inhibit-message to a non-nil value? Thanks again for working on this. From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 11 19:26:05 2016 Received: (at 23223) by debbugs.gnu.org; 11 Apr 2016 23:26:05 +0000 Received: from localhost ([127.0.0.1]:59837 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aplDZ-0006vz-6f for submit@debbugs.gnu.org; Mon, 11 Apr 2016 19:26:05 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:34555) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aplDX-0006vV-2j for 23223@debbugs.gnu.org; Mon, 11 Apr 2016 19:26:03 -0400 Received: by mail-wm0-f44.google.com with SMTP id l6so165696581wml.1 for <23223@debbugs.gnu.org>; Mon, 11 Apr 2016 16:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=zmxyY9CjyC6uDTV0c/CkqcCOs5kCzqZj+k86d/GJiV0=; b=q5semnjKveEsqU7R3m/1qyD9eO1B54gxrGn51sw90gmFTvjkTY/VoeZnI2wgufcCKl mQkwetBJfq1NsfZk3W8IQzqO93B9Y53YbXSlK7jQHzsekaBVK4nm8NPYktkhiXiBOtNF J1kF7PymvqvfoIKgFyOitWtKB3aLdDYJOBayy03PELvUwUBupoH2kxgbhabLpTCvVUzz MJuhrcVaPktPATVrTzmaBY2ahu84orKBB++exetn51JaF2JHRXFHKsxzA+PYAX7onfht FkHcq2sfPgk9JhMCTHIoFAdcU9CU6e55KGFC1hE8ZB/K8rJMkMzEYnqNPOND3sRHMkj8 rZ0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to; bh=zmxyY9CjyC6uDTV0c/CkqcCOs5kCzqZj+k86d/GJiV0=; b=OaeyG0Pp17F65R8MoJk35YeP5qpgYrZ2/VWT/Zgo0+FRFxdwMCQTr4/waVVEuXXBpa KMT4W+XcX9K7goxLbR1A02Ywah/6rXKZn6718J0DCFN4RjvoDuY2UOFn07kg3qzn9Zny WYwSZp7NJidTsRggTKXK3L/KsA11C+YTct7iM6FATldESGGrO29tGLD/x1Y8KUNqoo5G tWNjeOk5Smc3+58+gOkkWAYtJyyMS5LkXxBLW8WfTkMY2EuftF4Qm95F3D9o+0L+628R WgnhcDse6MDPjPxsrQEvnz2+mg7dL9ZisU4so4fm5VqnKDi0kyMLFMqBXuCqljwue2GX M3Xw== X-Gm-Message-State: AOPr4FWTOBQhxcIAJ+l7uDOvjEjCdFSx4pB8kgsNmpZl/WUrsixomwTG2FsNm8FGg9E4sA== X-Received: by 10.28.111.77 with SMTP id k74mr504651wmc.37.1460417157542; Mon, 11 Apr 2016 16:25:57 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id 202sm510541wmw.5.2016.04.11.16.25.55 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 11 Apr 2016 16:25:56 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> From: Dmitry Gutov Message-ID: <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> Date: Tue, 12 Apr 2016 02:25:54 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83zit0f8ah.fsf@gnu.org> Content-Type: multipart/mixed; boundary="------------D5E3E6A1CE927668B030A8E0" X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) This is a multi-part message in MIME format. --------------D5E3E6A1CE927668B030A8E0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit On 04/11/2016 06:56 PM, Eli Zaretskii wrote: > Unfortunately, it seems to miss matches: out of 1127 matches of > current_buffer with the original version, the new one only shows 963. > It sounds like some conditions on what exactly is a symbol need > adjustment, Ooh, that's a great catch, thanks! Turns out, auto-mode-alist wasn't getting applied because buffer-file-name wasn't set. Fixing that added ~25% performance hit, so I also added xref--find-buffer-visiting to cache the expensive lookup mentioned previously. See the new patch attached. That seems to about exhaust the optimization opportunity here. > As an aside, if I invoke xref-find-references without an ID file, > which AFAIU means Emacs will invoke find+grep, I get this error: > > semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode > > unless I invoke xref-find-references from a buffer in C mode. ...or the current major mode is one of the currently supported ones, via the above variable. > Curiously, this doesn't happen when there's an ID file and IDutils is > invoked. Is this expected? Yes. semantic-symref-filepattern-alist is defined in and used by semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. > "gid" is just a short for "lid -R grep", so the > contents I get is the same as xref-find-references does, it's just > formatted differently. Not exactly: you get more false positives because it doesn't apply the language-aware filtering. >> What's the new time you get from the former? > > 3 sec (in an unoptimized build, I'd expect this to become 1 sec in an > optimized build). So we are OK speed-wise, we just need to fix the > misses mentioned above. Cool. >> By the way, the "insert-file-contents + set-auto-mode" dance comes with >> a new minor downside: extra chatter from the major modes. E.g. try >> project-file-regexp with "should have received a copy". > > I don't see this in xref-find-references. Should I? Not at all. xref-find-references only searches in the files related to the current one by type (or in a predefined database like id-utils's one). The chatter I got was from e.g. sh-mode (about the indentation variable being guessed), nxml-mode, and so on, with matches in ~4000 files. >> We can avoid saving it to the message log, but it appears in the >> echo area either way. > > Can't you bind inhibit-message to a non-nil value? That works! > Thanks again for working on this. Thanks for the thoughtful bug report. --------------D5E3E6A1CE927668B030A8E0 Content-Type: text/x-patch; name="xref-with-temp-buffer.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xref-with-temp-buffer.diff" diff --git a/lisp/cedet/semantic/symref/cscope.el b/lisp/cedet/semantic/symref/cscope.el index 4890b5b..3abd8b3 100644 --- a/lisp/cedet/semantic/symref/cscope.el +++ b/lisp/cedet/semantic/symref/cscope.el @@ -60,6 +60,9 @@ semantic-symref-tool-cscope (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-cscope--line-re + "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-cscope)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -78,8 +81,13 @@ semantic-symref-tool-cscope ;; We have to return something at this point. subtxt))) ) - (t - (when (re-search-forward "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) " nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-cscope--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1)) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; :resulttype is 'line + (when (re-search-forward semantic-symref-cscope--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1))) )))) diff --git a/lisp/cedet/semantic/symref/global.el b/lisp/cedet/semantic/symref/global.el index e4c114e..a33427e 100644 --- a/lisp/cedet/semantic/symref/global.el +++ b/lisp/cedet/semantic/symref/global.el @@ -49,6 +49,9 @@ semantic-symref-tool-global (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-global--line-re + "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-global)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -57,8 +60,13 @@ semantic-symref-tool-global ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-global--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 3) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) " nil t) + (when (re-search-forward semantic-symref-global--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 3)) )))) diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el index 5d1fea8..868e6c3 100644 --- a/lisp/cedet/semantic/symref/grep.el +++ b/lisp/cedet/semantic/symref/grep.el @@ -188,6 +188,9 @@ semantic-symref-grep-shell ;; Return the answer ans)) +(defconst semantic-symref-grep--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-grep)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -195,8 +198,13 @@ semantic-symref-grep-shell ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-grep--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + (when (re-search-forward semantic-symref-grep--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 1)) )))) diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el index 4127d7a..db3e9a0 100644 --- a/lisp/cedet/semantic/symref/idutils.el +++ b/lisp/cedet/semantic/symref/idutils.el @@ -49,6 +49,9 @@ semantic-symref-tool-idutils (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-idutils--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-idutils)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -59,8 +62,13 @@ semantic-symref-tool-idutils ((eq (oref tool :searchtype) 'tagcompletions) (when (re-search-forward "^\\([^ ]+\\) " nil t) (match-string 1))) - (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-idutils--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1) default-directory) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; resulttype is line + (when (re-search-forward semantic-symref-idutils--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1) default-directory)) )))) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index feed0fb..68021c5 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -839,16 +839,18 @@ xref-etags-mode (kill-local-variable 'xref-backend-functions)) (setq-local xref-backend-functions xref-etags-mode--saved))) -(declare-function semantic-symref-find-references-by-name "semantic/symref") -(declare-function semantic-find-file-noselect "semantic/fw") +(declare-function semantic-symref-instantiate "semantic/symref") +(declare-function semantic-symref-perform-search "semantic/symref") (declare-function grep-expand-template "grep") (defvar ede-minor-mode) ;; ede.el +(defvar xref--last-visiting-buffer nil) + (defun xref-collect-references (symbol dir) "Collect references to SYMBOL inside DIR. This function uses the Semantic Symbol Reference API, see -`semantic-symref-find-references-by-name' for details on which -tools are used, and when." +`semantic-symref-tool-alist' for details on which tools are used, +and when." (cl-assert (directory-name-p dir)) (require 'semantic/symref) (defvar semantic-symref-tool) @@ -859,19 +861,26 @@ xref-collect-references ;; to force the backend to use `default-directory'. (let* ((ede-minor-mode nil) (default-directory dir) + ;; FIXME: Remove CScope and Global from the recognized tools? + ;; The current implementations interpret the symbol search as + ;; "find all calls to the given function", but not function + ;; definition. And they return nothing when passed a variable + ;; name, even a global one. (semantic-symref-tool 'detect) (case-fold-search nil) - (res (semantic-symref-find-references-by-name symbol 'subdirs)) - (hits (and res (oref res hit-lines))) - (orig-buffers (buffer-list))) + (inst (semantic-symref-instantiate :searchfor symbol + :searchtype 'symbol + :searchscope 'subdirs + :resulttype 'line-and-text)) + (hits (semantic-symref-perform-search inst)) + xref--last-visiting-buffer + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (unwind-protect (cl-mapcan (lambda (hit) (xref--collect-matches - hit (format "\\_<%s\\_>" (regexp-quote symbol)))) + hit (format "\\_<%s\\_>" (regexp-quote symbol)) + tmp-buffer)) hits) - ;; TODO: Implement "lightweight" buffer visiting, so that we - ;; don't have to kill them. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) ;;;###autoload (defun xref-collect-matches (regexp files dir ignores) @@ -890,34 +899,25 @@ xref-collect-matches files (expand-file-name dir) ignores)) - (orig-buffers (buffer-list)) (buf (get-buffer-create " *xref-grep*")) (grep-re (caar grep-regexp-alist)) - (counter 0) - reporter - hits) + hits + xref--last-visiting-buffer + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (with-current-buffer buf (erase-buffer) (call-process-shell-command command nil t) (goto-char (point-min)) (while (re-search-forward grep-re nil t) - (push (cons (string-to-number (match-string 2)) - (match-string 1)) + (push (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))) hits))) - (setq reporter (make-progress-reporter - (format "Collecting search results...") - 0 (length hits))) (unwind-protect (cl-mapcan (lambda (hit) - (prog1 - (progress-reporter-update reporter counter) - (cl-incf counter)) - (xref--collect-matches hit regexp)) + (xref--collect-matches hit regexp tmp-buffer)) (nreverse hits)) - (progress-reporter-done reporter) - ;; TODO: Same as above. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) (defun xref--rgrep-command (regexp files dir ignores) (require 'find-dired) ; for `find-name-arg' @@ -980,30 +980,60 @@ xref--regexp-to-extended (match-string 1 str))))) str t t)) -(defun xref--collect-matches (hit regexp) - (pcase-let* ((`(,line . ,file) hit) - (buf (or (find-buffer-visiting file) - (semantic-find-file-noselect file)))) - (with-current-buffer buf - (save-excursion +(defvar xref--temp-buffer-file-name nil) + +(defun xref--collect-matches (hit regexp tmp-buffer) + (pcase-let* ((`(,line ,file ,text) hit) + (buf (xref--find-buffer-visiting file))) + (if buf + (with-current-buffer buf + (save-excursion + (goto-char (point-min)) + (forward-line (1- line)) + (xref--collect-matches-1 regexp file line + (line-beginning-position) + (line-end-position)))) + (with-current-buffer tmp-buffer + (erase-buffer) + (unless (equal file xref--temp-buffer-file-name) + (insert-file-contents file nil 0 200) + ;; Can't (setq-local delay-mode-hooks t) because of + ;; bug#23272, but the performance penalty seems minimal. + (let ((buffer-file-name file) + (inhibit-message t) + message-log-max) + (ignore-errors + (set-auto-mode t))) + (setq-local xref--temp-buffer-file-name file) + (setq-local inhibit-read-only t) + (erase-buffer)) + (insert text) (goto-char (point-min)) - (forward-line (1- line)) - (let ((line-end (line-end-position)) - (line-beg (line-beginning-position)) - matches) - (syntax-propertize line-end) - ;; FIXME: This results in several lines with the same - ;; summary. Solve with composite pattern? - (while (re-search-forward regexp line-end t) - (let* ((beg-column (- (match-beginning 0) line-beg)) - (end-column (- (match-end 0) line-beg)) - (loc (xref-make-file-location file line beg-column)) - (summary (buffer-substring line-beg line-end))) - (add-face-text-property beg-column end-column 'highlight - t summary) - (push (xref-make-match summary loc (- end-column beg-column)) - matches))) - (nreverse matches)))))) + (xref--collect-matches-1 regexp file line + (point) + (point-max)))))) + +(defun xref--collect-matches-1 (regexp file line line-beg line-end) + (let (matches) + (syntax-propertize line-end) + ;; FIXME: This results in several lines with the same + ;; summary. Solve with composite pattern? + (while (re-search-forward regexp line-end t) + (let* ((beg-column (- (match-beginning 0) line-beg)) + (end-column (- (match-end 0) line-beg)) + (loc (xref-make-file-location file line beg-column)) + (summary (buffer-substring line-beg line-end))) + (add-face-text-property beg-column end-column 'highlight + t summary) + (push (xref-make-match summary loc (- end-column beg-column)) + matches))) + (nreverse matches))) + +(defun xref--find-buffer-visiting (file) + (unless (equal (car xref--last-visiting-buffer) file) + (setq xref--last-visiting-buffer + (cons file (find-buffer-visiting file)))) + (cdr xref--last-visiting-buffer)) (provide 'xref) --------------D5E3E6A1CE927668B030A8E0-- From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 12 11:51:29 2016 Received: (at 23223) by debbugs.gnu.org; 12 Apr 2016 15:51:29 +0000 Received: from localhost ([127.0.0.1]:32976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq0bB-0002H9-MT for submit@debbugs.gnu.org; Tue, 12 Apr 2016 11:51:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33905) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq0bA-0002Gw-UF for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 11:51:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aq0b2-0000dZ-KU for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 11:51:23 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57021) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aq0b2-0000dV-H1; Tue, 12 Apr 2016 11:51:20 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1950 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aq0b0-0002WQ-VM; Tue, 12 Apr 2016 11:51:19 -0400 Date: Tue, 12 Apr 2016 18:50:49 +0300 Message-Id: <83h9f6g71i.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> (message from Dmitry Gutov on Tue, 12 Apr 2016 02:25:54 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> MIME-version: 1.0 Content-type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 12 Apr 2016 02:25:54 +0300 > > > Unfortunately, it seems to miss matches: out of 1127 matches of > > current_buffer with the original version, the new one only shows 963. > > It sounds like some conditions on what exactly is a symbol need > > adjustment, > > Ooh, that's a great catch, thanks! Turns out, auto-mode-alist wasn't > getting applied because buffer-file-name wasn't set. Fixing that added > ~25% performance hit, so I also added xref--find-buffer-visiting to > cache the expensive lookup mentioned previously. See the new patch attached. > > That seems to about exhaust the optimization opportunity here. LGTM, please push. Btw, I noticed another strange thing, but I was only able to reproduce it with the current version, before your patch: the set of results returned by xref-find-references, when it uses IDUtils, is not entirely repeatable. Sometimes, a few matches, sometimes a few dozen of them, are missing. I have no idea why; the lid command consistently returns the same number of lines each time I invoke it. > > semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode > > > > unless I invoke xref-find-references from a buffer in C mode. > > ...or the current major mode is one of the currently supported ones, via > the above variable. > > > Curiously, this doesn't happen when there's an ID file and IDutils is > > invoked. Is this expected? > > Yes. semantic-symref-filepattern-alist is defined in and used by > semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. I think we should do that, yes. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 12 14:49:31 2016 Received: (at 23223-done) by debbugs.gnu.org; 12 Apr 2016 18:49:31 +0000 Received: from localhost ([127.0.0.1]:33059 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq3NT-0006ea-2i for submit@debbugs.gnu.org; Tue, 12 Apr 2016 14:49:31 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:34248) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq3NR-0006eH-0l for 23223-done@debbugs.gnu.org; Tue, 12 Apr 2016 14:49:29 -0400 Received: by mail-wm0-f43.google.com with SMTP id l6so199928396wml.1 for <23223-done@debbugs.gnu.org>; Tue, 12 Apr 2016 11:49:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=dAUrArOmw/+kxc6Pp4t+bTM6WDUiwTPgA46pIdrlU9g=; b=VpkGnbUBUBDNp0wlz4ifV+ZZBcoNC9ziqWerBa7Nlqkm3jed8sBdd+3DRl306zqb9j RLBohyYolmLu60yAqpHX177qf7GGzguTYMEe/3tc8EYVGwzQ0vzFC+y6TpiZjM0JyAxg 9iBgDTH3cd0x5pB85t2xehokVeKcfeSEzvwiPI78I90boem5oAgQbFkrcSvznuiuEE7h 74Vn0B1xpZbgWshVRb61yPLNK6+VS5Jmt6X6rbn1WGl+21/cZD9B3yrbjyaQef3QcHIQ tx6fYpz2rWorw2sWJw1Q9KqjIquHXSd63LuRjRJ0G2rM5eMcekxvnKrPEHSlmkK4PkjI ztrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=dAUrArOmw/+kxc6Pp4t+bTM6WDUiwTPgA46pIdrlU9g=; b=eqmgIOei2jkQ126iMiNytGficuDooTgeLpJbOqohRqRJ5Vk+AGQ0cs6TXgLbzuXN5C XhHl6UatBOSIQ0h8LW5EGwRbrV1TjB2kch4tTFqR/0fYuKgl7B5RaN8ydwsVxSo/q9WP aJ3oDHrZfTfXmH9YgvBtvnJn0MoGICI0XpvViVRJ125Mp8c+sNijgIkF9JmwhP7U7YOD +as0yRiVPvOMTieJ2HwAYa2fqSMUBskJDRAJpuDo1vj/Ht8dJaHwUQPRDBqp0pbmWdu0 ZbPjUJSNz4F0/XALP3uw5NL2CpOJVxQji9SKLXgcGZHBQLBieWdOwpy/8DTL03ML662A n7oQ== X-Gm-Message-State: AOPr4FWwdI+/90Kiz5MglYsj6Ap0h1Lwiga49gYlwf/gewZI+C+ipIVZgYjHWIccGU3/BQ== X-Received: by 10.28.23.70 with SMTP id 67mr5775735wmx.70.1460486963397; Tue, 12 Apr 2016 11:49:23 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id w8sm34780056wjf.19.2016.04.12.11.49.19 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 12 Apr 2016 11:49:22 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> From: Dmitry Gutov Message-ID: <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> Date: Tue, 12 Apr 2016 21:49:14 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83h9f6g71i.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223-done Cc: 23223-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Version: 25.1 On 04/12/2016 06:50 PM, Eli Zaretskii wrote: > LGTM, please push. Done, and closing. > Btw, I noticed another strange thing, but I was only able to reproduce > it with the current version, before your patch: the set of results > returned by xref-find-references, when it uses IDUtils, is not > entirely repeatable. Sometimes, a few matches, sometimes a few dozen > of them, are missing. I have no idea why; the lid command > consistently returns the same number of lines each time I invoke it. That sounds troubling. Please report if you encounter this again, together with: do you get different results when immediately repeating the same search? Or do you search from different files, in different directories? Getting different results between searches from c-mode and emacs-lisp-mode is to be expected, because the latter searches in all load-path as well (and it ends up using Grep in most of those directories anyway). >semantic-symref-filepattern-alist is defined in and used by >> semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. > > I think we should do that, yes. Also done (with a slightly longer value). From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 12 15:17:03 2016 Received: (at 23223) by debbugs.gnu.org; 12 Apr 2016 19:17:03 +0000 Received: from localhost ([127.0.0.1]:33066 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq3o7-0007JE-Bt for submit@debbugs.gnu.org; Tue, 12 Apr 2016 15:17:03 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37678) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq3o6-0007Ii-3I for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 15:17:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aq3nw-0003Do-8F for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 15:16:56 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60264) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aq3nw-0003Dk-4u; Tue, 12 Apr 2016 15:16:52 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3606 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aq3nv-0002FF-GO; Tue, 12 Apr 2016 15:16:51 -0400 Date: Tue, 12 Apr 2016 22:16:22 +0300 Message-Id: <83d1pufxix.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> (message from Dmitry Gutov on Tue, 12 Apr 2016 21:49:14 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223-done@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 12 Apr 2016 21:49:14 +0300 > > > Btw, I noticed another strange thing, but I was only able to reproduce > > it with the current version, before your patch: the set of results > > returned by xref-find-references, when it uses IDUtils, is not > > entirely repeatable. Sometimes, a few matches, sometimes a few dozen > > of them, are missing. I have no idea why; the lid command > > consistently returns the same number of lines each time I invoke it. > > That sounds troubling. Please report if you encounter this again, I just did, with the current emacs-25 branch. > together with: do you get different results when immediately repeating > the same search? Or do you search from different files, in different > directories? The same search, immediately repeated from the same directory. More accurately, the first time I invoke xref-find-references in the top-level directory of the Emacs repository, I get 980 matches; thereafter, all the subsequent searches from the same Emacs session yield 1128 matches. This is with the ID database present in the top-level directory. > Getting different results between searches from c-mode and > emacs-lisp-mode is to be expected, because the latter searches in all > load-path as well (and it ends up using Grep in most of those > directories anyway). All the searches are done from *scratch* in "emacs -Q". Let me know if I can provide more info. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 12 16:26:24 2016 Received: (at 23223) by debbugs.gnu.org; 12 Apr 2016 20:26:24 +0000 Received: from localhost ([127.0.0.1]:33092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq4tD-0000X5-QC for submit@debbugs.gnu.org; Tue, 12 Apr 2016 16:26:23 -0400 Received: from mail-wm0-f46.google.com ([74.125.82.46]:38813) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aq4tB-0000Wr-Mn for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 16:26:22 -0400 Received: by mail-wm0-f46.google.com with SMTP id u206so42977406wme.1 for <23223@debbugs.gnu.org>; Tue, 12 Apr 2016 13:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=WWvwDzz9mHsgzeRtwRZIjV+CZRV8Cw5u6zorwpPjB4M=; b=nNH/FAjd6UvRAtSeG+pFg+1cZrlbZxDLsfNOTB/Ya5qlbqvEQVJBmQsXm6Gshgymmy 3VGAo+z/T0BmxfsrK+zZRhPj63eP/uCYmXMCOx/fhx5YjG3NwJf9AgN4WLf+SKi30Ur6 X53/hi2jH7GYEFFLzXn+CYhcSuQL1u5fj72sAaZH28RfyyQqXHP0Om7tB/QV0wyTDfpV g/yUrIDbYSbKD7jgn9aG/HfhwChK5vuFM4JEWp3FdcdJ5UT1P11+9n7TbGvaTHlrLQ7h Ye0MHQQxcWf0eqfeFeVKjmVjpsAbLYO8xa/7nhfxhyDDaPobrThL2QNbejvACOcUJwfx 5iDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=WWvwDzz9mHsgzeRtwRZIjV+CZRV8Cw5u6zorwpPjB4M=; b=TTSUoGs4DKluRCD6xphCzbmNyPk1MY0vA+wNvMEAaqoSUX0XpkkvaO1G5qpbeggLrM ctMTEBoBuBYoBwPGAxurLvKXS1c49kGU8fhF0UT0zfQmg7uDsBQKSdbEW+BUpeHpNfCA QU2fRO7c+CZgoVV1aiuGvdvKuug6OBeRkHpuGuhCGuKrHmK0QMTpH51kYDaTBTanRMaq 7RHqB4jej9mcUGPDHwD6f8t1S2vD8XYkgD0qDWHuRkxmXMCWUu/9M/CaOBrBMOvthekT kNFiihP9e6s7pJrn6kBsDzt+Yl+Vq3VdA/veCY/8Y1hIu6Ws+vcnkHAXxI2DyujZuF3k Zw+w== X-Gm-Message-State: AD7BkJJnx6e+wJ+DibmtGgj7jePCv/w+rZZiexK3/CBpUIRE+EISQYVV3tkJdydSnmry1g== X-Received: by 10.28.99.6 with SMTP id x6mr25551405wmb.25.1460492776091; Tue, 12 Apr 2016 13:26:16 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id l124sm24493264wmf.11.2016.04.12.13.26.14 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 12 Apr 2016 13:26:15 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> <83d1pufxix.fsf@gnu.org> From: Dmitry Gutov Message-ID: <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> Date: Tue, 12 Apr 2016 23:26:13 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <83d1pufxix.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/12/2016 10:16 PM, Eli Zaretskii wrote: > The same search, immediately repeated from the same directory. More > accurately, the first time I invoke xref-find-references in the > top-level directory of the Emacs repository, I get 980 matches; > thereafter, all the subsequent searches from the same Emacs session > yield 1128 matches. This is with the ID database present in the > top-level directory. In the same scenario, I consistently get 1076 matches (on a GNU/Linux system): https://asciinema.org/a/69fdwo7os38fxu01e4fcwftwv > All the searches are done from *scratch* in "emacs -Q". The screencast above starts from Dired, but it's the same if I start from *scratch*. > Let me know if I can provide more info. Could you compare the lists of matches you get? E.g. C-x C-s on the xref buffer in both cases, and then diff the results. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 12 22:46:46 2016 Received: (at 23223) by debbugs.gnu.org; 13 Apr 2016 02:46:46 +0000 Received: from localhost ([127.0.0.1]:33162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqApJ-0001EW-SS for submit@debbugs.gnu.org; Tue, 12 Apr 2016 22:46:46 -0400 Received: from eggs.gnu.org ([208.118.235.92]:44106) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqApI-0001EJ-Up for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 22:46:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aqApA-0002P9-N4 for 23223@debbugs.gnu.org; Tue, 12 Apr 2016 22:46:39 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:39396) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aqApA-0002P1-Js; Tue, 12 Apr 2016 22:46:36 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3931 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aqAnc-0006Mz-72; Tue, 12 Apr 2016 22:46:36 -0400 Date: Wed, 13 Apr 2016 05:44:32 +0300 Message-Id: <837fg2fcrz.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> (message from Dmitry Gutov on Tue, 12 Apr 2016 23:26:13 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> <83d1pufxix.fsf@gnu.org> <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 12 Apr 2016 23:26:13 +0300 > > Could you compare the lists of matches you get? E.g. C-x C-s on the xref > buffer in both cases, and then diff the results. Attached. --- x00 2016-04-13 05:42:17.702125000 +0300 +++ x01 2016-04-13 05:42:35.092750000 +0300 @@ -692,14 +692,162 @@ 2174: i = find_interval (buffer_intervals (current_buffer), pos); 2465: INTERVAL i = buffer_intervals (current_buffer); D:/gnu/git/emacs/branch/src/keyboard.c - 736: && current_buffer != XBUFFER (XWINDOW (selected_window)->contents)) -1410: prev_buffer = current_buffer; -1469: buffer_before_last_command_or_undo = current_buffer; -1516: if (!NILP (BVAR (current_buffer, mark_active)) -1538: && XMARKER (BVAR (current_buffer, mark))->buffer -1553: if (current_buffer != prev_buffer || MODIFF != prev_modiff) -1562: if (current_buffer == prev_buffer + 736: && current_buffer != XBUFFER (XWINDOW (selected_window)->contents)) + 1410: prev_buffer = current_buffer; + 1469: buffer_before_last_command_or_undo = current_buffer; + 1516: if (!NILP (BVAR (current_buffer, mark_active)) + 1538: && XMARKER (BVAR (current_buffer, mark))->buffer + 1553: if (current_buffer != prev_buffer || MODIFF != prev_modiff) + 1562: if (current_buffer == prev_buffer + 1563: && XBUFFER (XWINDOW (selected_window)->contents) == current_buffer + 1633: eassert (XBUFFER (XWINDOW (selected_window)->contents) == current_buffer); + 2833: struct buffer *prev_buffer = current_buffer; + 2844: if (current_buffer != prev_buffer) + 3905: XSETBUFFER (obj, current_buffer); + 7392: if (tem = get_local_map (PT, current_buffer, Qkeymap), !NILP (tem)) + 7399: maps[nmaps++] = get_local_map (PT, current_buffer, Qlocal_map); + 7945: if (tem = get_local_map (PT, current_buffer, Qkeymap), !NILP (tem)) + 7952: maps[nmaps++] = get_local_map (PT, current_buffer, Qlocal_map); + 8808: If FIX_CURRENT_BUFFER, we restore current_buffer + 8935: starting_buffer = current_buffer; + 9151: != current_buffer) + 9163: && current_buffer != starting_buffer) + 9201: != current_buffer)) + 9252: && XBUFFER (XWINDOW (window)->contents) != current_buffer) + 9642: < XCHAR_TABLE (BVAR (current_buffer, downcase_table))->header.size) +11759: * confused because it doesn't realize that the current_buffer was +D:/gnu/git/emacs/branch/src/keymap.c +1486: && XBUFFER (XWINDOW (window)->contents) != current_buffer) +1515: Lisp_Object local_map = get_local_map (pt, current_buffer, Qlocal_map); +1517: Lisp_Object keymap = get_local_map (pt, current_buffer, Qkeymap); +1536: current_buffer, Qlocal_map); +1539: current_buffer, Qkeymap); +1677: map = BVAR (current_buffer, keymap); +1777: bset_keymap (current_buffer, keymap); +1787: return BVAR (current_buffer, keymap); +D:/gnu/git/emacs/branch/src/lread.c +1470: filename = Fexpand_file_name (filename, BVAR (current_buffer, directory)); +D:/gnu/git/emacs/branch/src/marker.c +715: for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next) +750: BUF_INC_POS (current_buffer, below_byte); +D:/gnu/git/emacs/branch/src/minibuf.c +422: ambient_dir = BVAR (current_buffer, directory); +538: enable_multibyte = BVAR (current_buffer, enable_multibyte_characters); +548: bset_truncate_lines (current_buffer, Qnil); +552: bset_enable_multibyte_characters (current_buffer, enable_multibyte); +557: the current_buffer, so reset_buffer leaves buf's default +563: bset_directory (current_buffer, ambient_dir); +571: bset_directory (current_buffer, +619: if (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +649: bset_keymap (current_buffer, map); +658: bset_undo_list (current_buffer, Qnil); +D:/gnu/git/emacs/branch/src/print.c + 91: struct buffer *old = current_buffer; \ + 97: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); \ +102: if (XBUFFER (printcharfun) != current_buffer) \ +111: if (XMARKER (printcharfun)->buffer != current_buffer) \ +128: if (NILP (BVAR (current_buffer, enable_multibyte_characters)) \ +131: if (! NILP (BVAR (current_buffer, enable_multibyte_characters)) \ +158: && NILP (BVAR (current_buffer, enable_multibyte_characters)))\ +302: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +359: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +429: : ! NILP (BVAR (current_buffer, enable_multibyte_characters)))) +543: register struct buffer *old = current_buffer; +551: delete_all_overlays (current_buffer); +552: bset_directory (current_buffer, BVAR (old, directory)); +553: bset_read_only (current_buffer, Qnil); +554: bset_filename (current_buffer, Qnil); +555: bset_undo_list (current_buffer, Qt); +556: eassert (current_buffer->overlays_before == NULL); +557: eassert (current_buffer->overlays_after == NULL); +559: (current_buffer, BVAR (&buffer_defaults, enable_multibyte_characters)); +563: XSETBUFFER (buf, current_buffer); +672: struct buffer *previous = current_buffer; +856: struct buffer *old = current_buffer; +D:/gnu/git/emacs/branch/src/process.c +2194: else if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +3721: else if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +4604: struct buffer *old_buffer = current_buffer; +4612: && (old_buffer != current_buffer +4959: struct buffer *old_buffer = current_buffer; +4974: && (old_buffer != current_buffer +5377: XSETBUFFER (obuffer, current_buffer); +5378: okeymap = BVAR (current_buffer, keymap); +5462: || ! EQ (current_buffer->keymap, okeymap)) +5497: old_read_only = BVAR (current_buffer, read_only); +5503: bset_read_only (current_buffer, Qnil); +5520: if (NILP (BVAR (current_buffer, enable_multibyte_characters)) +5534: && (b = XBUFFER (p->buffer), b != current_buffer)) +5563: bset_read_only (current_buffer, old_read_only); +5723: struct buffer *cur = current_buffer; +6559: XSETBUFFER (obuffer, current_buffer); +6560: okeymap = BVAR (current_buffer, keymap); +6607: || ! EQ (current_buffer->keymap, okeymap)) +6720: struct buffer *old = current_buffer; +6730: if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +6747: tem = BVAR (current_buffer, read_only); +6748: bset_read_only (current_buffer, Qnil); +6754: bset_read_only (current_buffer, tem); +D:/gnu/git/emacs/branch/src/search.c + 147: cp->syntax_table = cp->buf.used_syntax ? BVAR (current_buffer, syntax_table) : Qt; + 226: || EQ (cp->syntax_table, BVAR (current_buffer, syntax_table))) + 275: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, + 276: BVAR (current_buffer, case_eqv_table)); + 282: (!NILP (BVAR (current_buffer, case_fold_search)) + 283: ? BVAR (current_buffer, case_canon_table) : Qnil), + 285: !NILP (BVAR (current_buffer, enable_multibyte_characters))); + 333: XSETBUFFER (last_thing_searched, current_buffer); + 391: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, + 392: BVAR (current_buffer, case_eqv_table)); + 397: (!NILP (BVAR (current_buffer, case_fold_search)) + 398: ? BVAR (current_buffer, case_canon_table) : Qnil), + 559: multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); + 670: newline_cache = newline_cache_on_off (current_buffer); + 671: if (current_buffer->base_buffer) + 672: cache_buffer = current_buffer->base_buffer; + 674: cache_buffer = current_buffer; +1055: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, +1056: BVAR (current_buffer, case_eqv_table)); +1059: (!NILP (BVAR (current_buffer, case_fold_search)) +1060: ? BVAR (current_buffer, case_canon_table) +1062: (!NILP (BVAR (current_buffer, case_fold_search)) +1063: ? BVAR (current_buffer, case_eqv_table) +1189: !NILP (BVAR (current_buffer, enable_multibyte_characters))); +1243: XSETBUFFER (last_thing_searched, current_buffer); +1287: XSETBUFFER (last_thing_searched, current_buffer); +1312: bool multibyte = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +1528: bool multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); +1748: bool multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); +2159: XSETBUFFER (last_thing_searched, current_buffer); +2574: bool buf_multibyte = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +3220: buf = current_buffer; +3225: old = current_buffer; +D:/gnu/git/emacs/branch/src/syntax.c + 423: gl_state.current_syntax_table = BVAR (current_buffer, syntax_table); + 563: if (NILP (BVAR (current_buffer, enable_multibyte_characters))) + 590: if (current_buffer == find_start_buffer + 640: find_start_buffer = current_buffer; + 981: return BVAR (current_buffer, syntax_table); +1026: bset_syntax_table (current_buffer, table); +1029: SET_PER_BUFFER_VALUE_P (current_buffer, idx, 1); +1233: syntax_table = BVAR (current_buffer, syntax_table); +1654: multibyte = (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +2137: multibyte = (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +2895: if (! NILP (BVAR (current_buffer, enable_multibyte_characters))) +D:/gnu/git/emacs/branch/src/term.c +3692: make_save_ptr_ptr (menu, current_buffer)); D:/gnu/git/emacs/branch/src/textprop.c + 83: struct buffer *buf = XBUFFER (buffer), *old = current_buffer; + 94: bset_point_before_scroll (current_buffer, Qnil); + 525: XSETBUFFER (object, current_buffer); + 566: XSETBUFFER (object, current_buffer); + 610: XSETBUFFER (object, current_buffer); + 622: struct buffer *obuf = current_buffer; + 793: if (BUFFERP (object) && current_buffer != XBUFFER (object)) + 877: if (BUFFERP (object) && current_buffer != XBUFFER (object)) + 948: XSETBUFFER (object, current_buffer); +1015: XSETBUFFER (object, current_buffer); +1062: XSETBUFFER (object, current_buffer); 1112: XSETBUFFER (object, current_buffer); 1160: XSETBUFFER (object, current_buffer); 1360: XSETBUFFER (object, current_buffer); From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 13 06:18:24 2016 Received: (at 23223) by debbugs.gnu.org; 13 Apr 2016 10:18:24 +0000 Received: from localhost ([127.0.0.1]:33334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqHsO-0003Za-Ek for submit@debbugs.gnu.org; Wed, 13 Apr 2016 06:18:24 -0400 Received: from mail-wm0-f46.google.com ([74.125.82.46]:36613) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqHsM-0003ZN-Gz for 23223@debbugs.gnu.org; Wed, 13 Apr 2016 06:18:22 -0400 Received: by mail-wm0-f46.google.com with SMTP id v188so165721278wme.1 for <23223@debbugs.gnu.org>; Wed, 13 Apr 2016 03:18:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=m8o4VHL5i3i/Ab01+z4Y+Zav1afYflD1EVslUhpbwOc=; b=R8vBS6dGuijKB3z7wNKb2z1xg6XXw25enF/6FcX5awt9YAVMEQ2Qy7qT2+GdHL/0xw H4PtCbJP6UACm6vbavjn51DifiJz/JI64uFnqF1mcNcIAFVI2rINfVwPuWiPXQR0Wkn0 OcdSp/eb5jY7gdIxA2MPeCR0JhV9K7S5KZaouaX6ZqDOZft0Uj/LotDgmIREIcfcwJ0x OLLLu04EewV9wuTSNaI2W9LmlVmzBphbW1WXyAitKP9+Rh+pBdz+0QxnOX2fsD4p6kkS 38sMFvKROYaxQ78dWLl/9QmzmjAyFd2XndkYsWPKN5GP4JXNGTA09tfQCNHpwL9g3wnC KtHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=m8o4VHL5i3i/Ab01+z4Y+Zav1afYflD1EVslUhpbwOc=; b=glWTRYVUf0fVoj+aMzzQOSl3W8HHDUUOb2E2001/Ft3WMNCfkj5++I4EsYbOJTom3d gD1Td0kWxr2Jksg4nfnwVjDloyos4Bw38Ezu1fI4ctCQj2ufHcY6i2jUMN6hPwrlDiLt PYCGvHwpRXsqP2csG08Issx1UOl9YcXX9kKGwy0/NpsBJNXUyl6RRPG2yBcyNNLJjfKk T6hzwYzhqPo0UvCCowDnJItJur6/qF4QaIQJvbFA0FPQgyI9Zz5Pypj/9w3JgCJWAc7d NgchUJ8sOQ96sA5wr6Vk/niUDCGkHtcgJIkZgMU9XFHrJjiROn99reWF9P/aWnyRbKMJ odyA== X-Gm-Message-State: AOPr4FXjlC2X9GDMWM+BgXEq4GzM4pRLncHRC5F2FPhSzLcmDRvjE8R6t6jyCap7978ofg== X-Received: by 10.194.82.199 with SMTP id k7mr8672450wjy.63.1460542696945; Wed, 13 Apr 2016 03:18:16 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id u3sm27258009wmg.15.2016.04.13.03.18.15 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 13 Apr 2016 03:18:16 -0700 (PDT) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? To: Eli Zaretskii References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> <83d1pufxix.fsf@gnu.org> <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> <837fg2fcrz.fsf@gnu.org> From: Dmitry Gutov Message-ID: <8cf0c462-6210-e876-1314-0374247d221a@yandex.ru> Date: Wed, 13 Apr 2016 13:18:14 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <837fg2fcrz.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 04/13/2016 05:44 AM, Eli Zaretskii wrote: > Attached. Thank you. Unfortunately, nothing jumps out. Could you try instrumenting a couple of functions, to see where the matches get lost? First, semantic-symref-parse-tool-output, see whether the buffer contents are different between invocations (press `e' and evaluate (write-region (point-min) (point-max) "lid-output-x.txt")). Next, xref--convert-hits. See that its gets the expected number of hits as input (plus maybe a few that we legitimately need to filter out). If both are true, I guess there's nothing better to do than step through xref--convert-hits and see where it loses the matches. But I'm guessing the problem is somewhere before it. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 13 11:17:30 2016 Received: (at 23223) by debbugs.gnu.org; 13 Apr 2016 15:17:30 +0000 Received: from localhost ([127.0.0.1]:33837 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqMXq-00043g-GD for submit@debbugs.gnu.org; Wed, 13 Apr 2016 11:17:30 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33707) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aqMXp-00043U-GJ for 23223@debbugs.gnu.org; Wed, 13 Apr 2016 11:17:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aqMXh-00057H-6H for 23223@debbugs.gnu.org; Wed, 13 Apr 2016 11:17:24 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52979) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aqMXh-00057D-2f; Wed, 13 Apr 2016 11:17:21 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4437 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aqMXg-0007YP-DR; Wed, 13 Apr 2016 11:17:20 -0400 Date: Wed, 13 Apr 2016 18:16:53 +0300 Message-Id: <8337qpfsii.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <8cf0c462-6210-e876-1314-0374247d221a@yandex.ru> (message from Dmitry Gutov on Wed, 13 Apr 2016 13:18:14 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> <83d1pufxix.fsf@gnu.org> <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> <837fg2fcrz.fsf@gnu.org> <8cf0c462-6210-e876-1314-0374247d221a@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov > Date: Wed, 13 Apr 2016 13:18:14 +0300 > > First, semantic-symref-parse-tool-output, see whether the buffer > contents are different between invocations They are different: the first one has a long run of binary nulls right in the middle of it. So this is unrelated to xref-find-references, it's some basic problem with buffer memory management. Could be the same problem that causes this: http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00308.html I will debug it, now that I have a reproducible recipe. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 15 10:30:31 2016 Received: (at 23223) by debbugs.gnu.org; 15 Apr 2016 14:30:31 +0000 Received: from localhost ([127.0.0.1]:36868 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ar4lS-0002eV-TD for submit@debbugs.gnu.org; Fri, 15 Apr 2016 10:30:31 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59881) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ar4lR-0002eJ-T1 for 23223@debbugs.gnu.org; Fri, 15 Apr 2016 10:30:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ar4lH-0000zG-U2 for 23223@debbugs.gnu.org; Fri, 15 Apr 2016 10:30:24 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56237) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar4lH-0000zB-Qj; Fri, 15 Apr 2016 10:30:19 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2133 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1ar4lH-0003Xu-11; Fri, 15 Apr 2016 10:30:19 -0400 Date: Fri, 15 Apr 2016 17:29:58 +0300 Message-Id: <83shync5cp.fsf@gnu.org> From: Eli Zaretskii To: dgutov@yandex.ru In-reply-to: <8337qpfsii.fsf@gnu.org> (message from Eli Zaretskii on Wed, 13 Apr 2016 18:16:53 +0300) Subject: Re: bug#23223: 25.0.92; Can xref-find-references be sped up? References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> <83zit0f8ah.fsf@gnu.org> <90b81ad1-c3ab-272b-724b-ba63f0451d5a@yandex.ru> <83h9f6g71i.fsf@gnu.org> <1d635548-1fe3-ea72-4c98-6f787d7438b0@yandex.ru> <83d1pufxix.fsf@gnu.org> <06dcdbf7-4d1d-8e24-955f-40a9458096fd@yandex.ru> <837fg2fcrz.fsf@gnu.org> <8cf0c462-6210-e876-1314-0374247d221a@yandex.ru> <8337qpfsii.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 23223 Cc: 23223@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Wed, 13 Apr 2016 18:16:53 +0300 > From: Eli Zaretskii > Cc: 23223@debbugs.gnu.org > > > Cc: 23223@debbugs.gnu.org > > From: Dmitry Gutov > > Date: Wed, 13 Apr 2016 13:18:14 +0300 > > > > First, semantic-symref-parse-tool-output, see whether the buffer > > contents are different between invocations > > They are different: the first one has a long run of binary nulls right > in the middle of it. > > So this is unrelated to xref-find-references, it's some basic problem > with buffer memory management. Could be the same problem that causes > this: > > http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00308.html > > I will debug it, now that I have a reproducible recipe. Done, see commit ab849b7 on the emacs-25 branch. From unknown Thu Aug 14 21:49:00 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 14 May 2016 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator