From unknown Sat Aug 09 04:55:26 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#32462 <32462@debbugs.gnu.org> To: bug#32462 <32462@debbugs.gnu.org> Subject: Status: 26.1; Can `count-lines' be rewritten to use the newline cache? Reply-To: bug#32462 <32462@debbugs.gnu.org> Date: Sat, 09 Aug 2025 11:55:26 +0000 retitle 32462 26.1; Can `count-lines' be rewritten to use the newline cache? reassign 32462 emacs submitter 32462 Phil Sainty severity 32462 wishlist thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 16 21:01:58 2018 Received: (at submit) by debbugs.gnu.org; 17 Aug 2018 01:01:58 +0000 Received: from localhost ([127.0.0.1]:53047 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fqT9J-0001vN-St for submit@debbugs.gnu.org; Thu, 16 Aug 2018 21:01:58 -0400 Received: from eggs.gnu.org ([208.118.235.92]:36973) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fqT9H-0001ux-5s for submit@debbugs.gnu.org; Thu, 16 Aug 2018 21:01:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqT9B-0005tx-4n for submit@debbugs.gnu.org; Thu, 16 Aug 2018 21:01:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:36635) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqT9B-0005tj-1Y for submit@debbugs.gnu.org; Thu, 16 Aug 2018 21:01:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqT99-0005HD-Si for bug-gnu-emacs@gnu.org; Thu, 16 Aug 2018 21:01:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqT96-0005qw-Mp for bug-gnu-emacs@gnu.org; Thu, 16 Aug 2018 21:01:47 -0400 Received: from smtp-4.orcon.net.nz ([60.234.4.59]:59958) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fqT96-0005cW-Bw for bug-gnu-emacs@gnu.org; Thu, 16 Aug 2018 21:01:44 -0400 Received: from [10.253.37.70] (port=31316 helo=webmail.orcon.net.nz) by smtp-4.orcon.net.nz with esmtpa (Exim 4.86_2) (envelope-from ) id 1fqT90-00077s-Dj for bug-gnu-emacs@gnu.org; Fri, 17 Aug 2018 13:01:39 +1200 Received: from wlgwil-nat-office.catalyst.net.nz ([202.78.240.7]) via [10.253.37.253] by webmail.orcon.net.nz with HTTP (HTTP/1.1 POST); Fri, 17 Aug 2018 13:01:38 +1200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 17 Aug 2018 13:01:38 +1200 From: Phil Sainty To: bug-gnu-emacs@gnu.org Subject: 26.1; Can `count-lines' be rewritten to use the newline =?UTF-8?Q?cache=3F?= Message-ID: X-Sender: psainty@orcon.net.nz User-Agent: Orcon Webmail X-GeoIP: -- X-Spam_score: -2.9 X-Spam_score_int: -28 X-Spam_bar: -- X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.3 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.3 (-----) I saw this story the other day: https://fuco1.github.io/2018-08-12-WAR-STORY:-When-turning-to-the-profiler-turns-out-to-be-a-good-call.html The summary is that some very slow code turned out to be spending the vast bulk of its time inside `line-number-at-pos' (which was used frequently), and once the author discovered what that function actually entailed they were able to reduce their processing time from 42 seconds down to 5 seconds (processing a file of ~10,000 lines) by finding an alternative approach which did not involve calling `count-lines'. `count-lines' uses a regexp search to find all the newlines (and/or carriage returns -- I don't know if that's a problem) and I recall that internally Emacs uses a newline cache to make certain line-oriented functionality performant. I know nothing about the cache other than that it exists, but I wondered whether `count-lines' might be able to use it to avoid most of the work that it currently does? -Phil In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars) of 2018-04-10 built on shodan Windowing system distributor 'The X.Org Foundation', version 11.0.11501000 System Description: Ubuntu 14.04.5 LTS Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --prefix=/home/phil/emacs/26/26.1rc1/usr/local --with-x-toolkit=lucid --without-sound' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK GPM DBUS GSETTINGS NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 THREADS LCMS2 Important settings: value of $LANG: en_NZ.UTF-8 value of $XMODIFIERS: @im=ibus locale-coding-system: utf-8-unix Major mode: Dired by name Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t buffer-read-only: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils dired dired-loaddefs advice elec-pair time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 99206 10028) (symbols 48 20474 0) (miscs 40 101 169) (strings 32 29605 992) (string-bytes 1 777027) (vectors 16 14293) (vector-slots 8 495220 11440) (floats 8 55 100) (intervals 56 315 0) (buffers 992 14) (heap 1024 30317 1420)) From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 17 10:40:58 2018 Received: (at 32462) by debbugs.gnu.org; 17 Aug 2018 14:40:58 +0000 Received: from localhost ([127.0.0.1]:53683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fqfvu-0002s7-KZ for submit@debbugs.gnu.org; Fri, 17 Aug 2018 10:40:58 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43345) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fqfvs-0002rs-P7 for 32462@debbugs.gnu.org; Fri, 17 Aug 2018 10:40:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqfvi-0006KS-2x for 32462@debbugs.gnu.org; Fri, 17 Aug 2018 10:40:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40823) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqfvh-0006KA-Ly; Fri, 17 Aug 2018 10:40:45 -0400 Received: from [176.228.60.248] (port=3361 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fqfvg-0006QT-Un; Fri, 17 Aug 2018 10:40:45 -0400 Date: Fri, 17 Aug 2018 17:40:35 +0300 Message-Id: <834lftgios.fsf@gnu.org> From: Eli Zaretskii To: Phil Sainty In-reply-to: (message from Phil Sainty on Fri, 17 Aug 2018 13:01:38 +1200) Subject: Re: bug#32462: 26.1; Can `count-lines' be rewritten to use the newline cache? References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32462 Cc: 32462@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Fri, 17 Aug 2018 13:01:38 +1200 > From: Phil Sainty > > `count-lines' uses a regexp search to find all the newlines (and/or > carriage returns It uses regexp search only when selective-display is in effect, which means almost never. Otherwise, it uses forward-line, which uses scan_newline_from_point, which already uses the newline cache (unless the cache is disabled). (And based on my experience, the newline cache doesn't actually speed up things all that much, except in buffers with unusually long lines.) From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 18 08:10:32 2018 Received: (at 32462-done) by debbugs.gnu.org; 18 Aug 2018 12:10:33 +0000 Received: from localhost ([127.0.0.1]:53976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fr03s-0007BW-No for submit@debbugs.gnu.org; Sat, 18 Aug 2018 08:10:32 -0400 Received: from smtp-3.orcon.net.nz ([60.234.4.44]:37079) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fr03q-0007BF-Hb for 32462-done@debbugs.gnu.org; Sat, 18 Aug 2018 08:10:31 -0400 Received: from [150.107.172.63] (port=59375 helo=[192.168.20.103]) by smtp-3.orcon.net.nz with esmtpa (Exim 4.86_2) (envelope-from ) id 1fr03o-0004RE-2K; Sun, 19 Aug 2018 00:10:28 +1200 Subject: Re: bug#32462: 26.1; Can `count-lines' be rewritten to use the newline cache? To: Eli Zaretskii References: <834lftgios.fsf@gnu.org> From: Phil Sainty Message-ID: <4940f256-d5fa-fcd3-f6b7-5be72399de11@orcon.net.nz> Date: Sun, 19 Aug 2018 00:10:27 +1200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <834lftgios.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-GeoIP: NZ X-Spam_score: -2.9 X-Spam_score_int: -28 X-Spam_bar: -- X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 32462-done Cc: 32462-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On 18/08/18 02:40, Eli Zaretskii wrote: >> From: Phil Sainty >> `count-lines' uses a regexp search to find all the newlines (and/or >> carriage returns > > It uses regexp search only when selective-display is in effect, which > means almost never. Otherwise, it uses forward-line, which uses > scan_newline_from_point, which already uses the newline cache (unless > the cache is disabled). Ah, thanks Eli; I see that now. A look at the C code suggests that the newline cache is also a more complicated arrangement than I'd imagined, so I don't think my original thoughts about this were actually viable. I'm closing this bug. From unknown Sat Aug 09 04:55:26 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 16 Sep 2018 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator