From unknown Fri Aug 15 16:17:53 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#45200 <45200@debbugs.gnu.org> To: bug#45200 <45200@debbugs.gnu.org> Subject: Status: Wishlist: There should be a `malloc-trim' function Reply-To: bug#45200 <45200@debbugs.gnu.org> Date: Fri, 15 Aug 2025 23:17:53 +0000 retitle 45200 Wishlist: There should be a `malloc-trim' function reassign 45200 emacs submitter 45200 Konstantin Kharlamov severity 45200 wishlist thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 12 13:43:23 2020 Received: (at submit) by debbugs.gnu.org; 12 Dec 2020 18:43:23 +0000 Received: from localhost ([127.0.0.1]:46507 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ko9rX-0000u4-73 for submit@debbugs.gnu.org; Sat, 12 Dec 2020 13:43:23 -0500 Received: from lists.gnu.org ([209.51.188.17]:38118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ko9rV-0000tw-UO for submit@debbugs.gnu.org; Sat, 12 Dec 2020 13:43:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41584) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ko9rV-0003wd-I9 for bug-gnu-emacs@gnu.org; Sat, 12 Dec 2020 13:43:21 -0500 Received: from forward106p.mail.yandex.net ([77.88.28.109]:57409) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ko9rS-0002Tr-D0 for bug-gnu-emacs@gnu.org; Sat, 12 Dec 2020 13:43:20 -0500 Received: from mxback9g.mail.yandex.net (mxback9g.mail.yandex.net [IPv6:2a02:6b8:c03:7aa:0:640:d10:d80f]) by forward106p.mail.yandex.net (Yandex) with ESMTP id 73FF11C816EE for ; Sat, 12 Dec 2020 21:43:11 +0300 (MSK) Received: from iva8-174eb672ffa9.qloud-c.yandex.net (iva8-174eb672ffa9.qloud-c.yandex.net [2a02:6b8:c0c:b995:0:640:174e:b672]) by mxback9g.mail.yandex.net (mxback/Yandex) with ESMTP id ZktHbZDohB-hBlKKG4K; Sat, 12 Dec 2020 21:43:11 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1607798591; bh=oxv1AY/uOWizwjT7CWACUwRcNqxVlyMGswP6MDIAXkE=; h=To:From:Subject:Message-ID:Date; b=fvJ37UX4cPhGpjoZHd1jmR4+Q/RJTiOLQLEnkgzTqS2kWOgqFYZuXlEPQb/74VlAw JBWnzEARCupotd5Pjt0VPnpVIAPZKSetO1H1ChSmbkkb4uJVgFdW+8G2cKYGoDsgGb R2E1n3mZhyriGoDP/rYPHKORu07AjMCIZ+aozkeY= Authentication-Results: mxback9g.mail.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva8-174eb672ffa9.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id XHIvb3LCoV-hAI0l86g; Sat, 12 Dec 2020 21:43:10 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: bug-gnu-emacs@gnu.org Date: Sat, 12 Dec 2020 21:43:10 +0300 Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=77.88.28.109; envelope-from=hi-angel@yandex.ru; helo=forward106p.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) The problem is basically: (garbage-collect) refuses to collect some of the memory. It is more visible if you delay garbage collection, which is what I do for for performance-related reasons. My config has the following snippet: ;; only run garbage collection on idle (setq gc-cons-threshold most-positive-fixnum) (run-with-idle-timer 2 t (lambda () (garbage-collect))) How much memory gets lost depends on configuration. For me right after start the difference is 40 MB: ≈60 MB is Emacs PSS size without above code, and ≈100 MB it is when garbage-collection is delayed, *after* I run explicitly (garbage-collect). It is less visible without any other configs, nonetheless it's visible. # Steps to reproduce: 1. Run `mkdir /tmp/.emacs.d` 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS 3. Create a file /tmp/.emacs.d/early-init.el with content: ;; only run garbage collection on idle (setq gc-cons-threshold most-positive-fixnum) (run-with-idle-timer 2 t (lambda () (garbage-collect))) 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS ## Expected Size has no statistically-significant difference, because in both cases we garbage-collected memory. ## Actual Size without calling explicit garbage-collect, from 3 runs, varied around 41.2..41.7 MB. Size afterwards, also 3 runs, varied around 45.4..45.5 MB. This is 4 MB lost. While not much, but as I mentioned it grows as much as to 40MB. The Emacs whose emacslient I'm using to write this email has size 218 MB, and now I wouldn't be surprised if much of that is actually a leaked memory. # Version Emacs-git 28.0.50, build from commit abd15e088e99 From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 12 15:16:15 2020 Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 20:16:15 +0000 Received: from localhost ([127.0.0.1]:46762 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koBJP-0001K0-39 for submit@debbugs.gnu.org; Sat, 12 Dec 2020 15:16:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koBJO-0001Jc-42 for 45200@debbugs.gnu.org; Sat, 12 Dec 2020 15:16:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51314) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1koBJH-0002dH-DS; Sat, 12 Dec 2020 15:16:08 -0500 Received: from [176.228.60.248] (port=1537 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1koBJG-000875-PR; Sat, 12 Dec 2020 15:16:07 -0500 Date: Sat, 12 Dec 2020 22:15:52 +0200 Message-Id: <83k0tmeq6f.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: (message from Konstantin Kharlamov on Sat, 12 Dec 2020 21:43:10 +0300) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Date: Sat, 12 Dec 2020 21:43:10 +0300 > > # Steps to reproduce: > > 1. Run `mkdir /tmp/.emacs.d` > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS > 3. Create a file /tmp/.emacs.d/early-init.el with content: > > ;; only run garbage collection on idle > (setq gc-cons-threshold most-positive-fixnum) > (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS > > ## Expected > > Size has no statistically-significant difference, because in both cases we garbage-collected memory. > > ## Actual > > Size without calling explicit garbage-collect, from 3 runs, varied around 41.2..41.7 MB. > > Size afterwards, also 3 runs, varied around 45.4..45.5 MB. Could be simply the effect of different stack size, since Emacs's GC is conservative, and when there's doubt whether something is a live object, it won't GC it. I think more specific and detailed evidence is needed to prove your case: which objects were not GC'ed and why. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 12 17:44:23 2020 Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 22:44:23 +0000 Received: from localhost ([127.0.0.1]:47081 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koDcl-0001Ao-Jb for submit@debbugs.gnu.org; Sat, 12 Dec 2020 17:44:23 -0500 Received: from forward100j.mail.yandex.net ([5.45.198.240]:48976) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koDcj-0001AX-V7 for 45200@debbugs.gnu.org; Sat, 12 Dec 2020 17:44:22 -0500 Received: from mxback30g.mail.yandex.net (mxback30g.mail.yandex.net [IPv6:2a02:6b8:c03:7b3:0:640:11da:816c]) by forward100j.mail.yandex.net (Yandex) with ESMTP id 7115050E05EA; Sun, 13 Dec 2020 01:44:15 +0300 (MSK) Received: from sas1-f4dc5f2fc86f.qloud-c.yandex.net (sas1-f4dc5f2fc86f.qloud-c.yandex.net [2a02:6b8:c08:cb28:0:640:f4dc:5f2f]) by mxback30g.mail.yandex.net (mxback/Yandex) with ESMTP id lJ20sGCHhX-iF68mw13; Sun, 13 Dec 2020 01:44:15 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1607813055; bh=CsczHDEKLHpHQTtgc48u6y+nG1g/IH3PtkJUT8dU74k=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=j8O+CCChqjnDxzp8DcAh1XHwHEH9wg9KYOmmv/raicWSXyf9urLD9INgcvdbPCmT7 j9ZmM1GxmR2gEY88hrlEXeVLk3ykiORHmedr6w8UuI/gdDgZ6OSX/wPUzH1ul17aBJ T36N+O5qrC5pS/FtTXcOLZCZbpBF8FADI18LC+S8= Authentication-Results: mxback30g.mail.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-f4dc5f2fc86f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id xeQDxoCdlM-iEI4txB7; Sun, 13 Dec 2020 01:44:14 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Eli Zaretskii Date: Sun, 13 Dec 2020 01:44:13 +0300 In-Reply-To: <83k0tmeq6f.fsf@gnu.org> References: <83k0tmeq6f.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Sat, 2020-12-12 at 22:15 +0200, Eli Zaretskii wrote: > > From: Konstantin Kharlamov > > Date: Sat, 12 Dec 2020 21:43:10 +0300 > > > > # Steps to reproduce: > > > > 1. Run `mkdir /tmp/.emacs.d` > > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS > > 3. Create a file /tmp/.emacs.d/early-init.el with content: > > > >     ;; only run garbage collection on idle > >     (setq gc-cons-threshold most-positive-fixnum) > >     (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure > > its PSS > > > > ## Expected > > > > Size has no statistically-significant difference, because in both cases we > > garbage-collected memory. > > > > ## Actual > > > > Size without calling explicit garbage-collect, from 3 runs, varied around > > 41.2..41.7 MB. > > > > Size afterwards, also 3 runs, varied around 45.4..45.5 MB. > > Could be simply the effect of different stack size, since Emacs's GC > is conservative, and when there's doubt whether something is a live > object, it won't GC it. > > I think more specific and detailed evidence is needed to prove your > case: which objects were not GC'ed and why. Alright, fair enough. I crafted up another testcase, it may be better. The following code first temporarily disables GC, then it prints "hello" 1000000 times, and finally it calls GC manually. I call `emacs -Q`, then measure PSS, then evaluate the code below, then again measure PSS. (let ((i 1000000)) (setq gc-cons-threshold most-positive-fixnum) (while (> i 0) (print "hello") (setq i (- i 1))) (garbage-collect)) The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS after is 266.3M. That is ≈200M of memory just vanished. Regarding, whether it is stack size: λ grep VmStk /proc/283047/status VmStk: 132 kB Apparently, it is not stack size. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 12 17:59:43 2020 Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 22:59:44 +0000 Received: from localhost ([127.0.0.1]:47101 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koDrb-0001Zr-Nv for submit@debbugs.gnu.org; Sat, 12 Dec 2020 17:59:43 -0500 Received: from quimby.gnus.org ([95.216.78.240]:52088) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koDra-0001Zc-95 for 45200@debbugs.gnu.org; Sat, 12 Dec 2020 17:59:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=TDcnhUf7LkLJWm2aKW38GCv2UT1WRgLFFfgVVA9rfOk=; b=d+6r8CB6ij0YlLq1Y6ktF0EQ3p ehqpRsdvx7yHc26xczuhlikWBxBlwcasrfwIDwLkwyNLXdJD9wtd4kfPw3isXwGE+SMOsHJUHedoh ZPSx4HW1qxS4FgiwSMTJ7Seat96xEER8KxdWFE+zYp9j2A6MEd4aUIbxOYCCp6tqe75s=; Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1koDrR-0003ZT-Gn; Sat, 12 Dec 2020 23:59:35 +0100 From: Lars Ingebrigtsen To: Konstantin Kharlamov Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <83k0tmeq6f.fsf@gnu.org> <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAADFBMVEWXaIHevcLBv03/ //8lKWoeAAAAAWJLR0QDEQxM8gAAAAd0SU1FB+QMDBYpNyp3RKsAAAGOSURBVCjPTdDNatwwEAfw v428mD2ZIoekp01oSqqncEoDOSpgBbwn92DD+ilcQ8Pik09huyfXZI09T9kZd/shjNDPMxqNhMDi 3wiW2RCPHIpoPlbPpzHLLFS2LV6z41js5k5QvrrjWGYCJ+jG8scZz8XhY7HsKYtyVxxuaVt95Woz URv+PkXx7FJcJ/FCD4H1WsV/zh141fdU/cXdKZU051ziOTMlZ1jnaLYLnpiOSNIQPxUucYbSIBGc OM9Q7Tgx3t0u+OkcoB8EFZ0E8f8oXuLUui3Ngp3SNno445PWXOY9keBKR4x3ZAShjvhon4zcLOwE IPoGKGU7rBltz42Gtsaav6GfoC454neJGfoKKso1g19+aqD8/AC/NgNNn6Gu8wkRBBy5T4jrmGmp fd+YidET8QFBvV1ghhd5OJ8Ed/24gCP0Fm5a4MMeggkbrqb33OaGavgDd8DrVS+38LGqvQQXNa9v WqjmApGWAAasGnA3OW/b9wgEaz4pbG8gz70A/RrBF0mTMfkIHv+Aml/OxZANo//3HgAAACV0RVh0 ZGF0ZTpjcmVhdGUAMjAyMC0xMi0xMlQyMjo0MTo1NCswMDowMH/P9kgAAAAldEVYdGRhdGU6bW9k aWZ5ADIwMjAtMTItMTJUMjI6NDE6NTQrMDA6MDAOkk70AAAAAElFTkSuQmCC X-Now-Playing: JD Twitch, Optimo's _Mutant Disco Vol 4_: "Contort Yourself Optimo (Re Edit Mix)" Date: Sat, 12 Dec 2020 23:59:32 +0100 In-Reply-To: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> (Konstantin Kharlamov's message of "Sun, 13 Dec 2020 01:44:13 +0300") Message-ID: <87czze7hrf.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Konstantin Kharlamov writes: > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and > PSS after is 266.3M. That is ≈200M of memory just vanished. I get similar results. `M-x memory-report' shows that *Messages* and *Echo Area* each group to 9MB, and almost all of that is in `gap-size'. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Konstantin Kharlamov writes: > The loop takes 20-30 seconds for me, I think. PSS before is =E2=89=8841M,= and > PSS after is 266.3M. That is =E2=89=88200M of memory just vanished. I get similar results. `M-x memory-report' shows that *Messages* and *Echo Area* each group to 9MB, and almost all of that is in `gap-size'. But that's still a long way from 200MB. Perhaps Emacs is bouncing the buffer area around a lot and isn't returning the data to the OS? (Does Emacs ever return memory used by buffers to the OS?) --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 13 00:54:05 2020 Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 05:54:05 +0000 Received: from localhost ([127.0.0.1]:47299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koKKb-0004PU-Bi for submit@debbugs.gnu.org; Sun, 13 Dec 2020 00:54:05 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54838) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koKKY-0004Ov-Se for 45200@debbugs.gnu.org; Sun, 13 Dec 2020 00:54:03 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59427) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1koKKT-0002Hx-FX; Sun, 13 Dec 2020 00:53:57 -0500 Received: from eliz by fencepost.gnu.org with local (Exim 4.82) (envelope-from ) id 1koKKT-0007bp-7N; Sun, 13 Dec 2020 00:53:57 -0500 From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> (message from Konstantin Kharlamov on Sun, 13 Dec 2020 01:44:13 +0300) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <83k0tmeq6f.fsf@gnu.org> <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-Id: Date: Sun, 13 Dec 2020 00:53:57 -0500 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Cc: 45200@debbugs.gnu.org > Date: Sun, 13 Dec 2020 01:44:13 +0300 > > Alright, fair enough. I crafted up another testcase, it may be better. The following code first temporarily disables GC, then it prints "hello" 1000000 times, and finally it calls GC manually. > > I call `emacs -Q`, then measure PSS, then evaluate the code below, then again measure PSS. > > (let ((i 1000000)) > (setq gc-cons-threshold most-positive-fixnum) > (while (> i 0) > (print "hello") > (setq i (- i 1))) > (garbage-collect)) > > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS after is 266.3M. That is ≈200M of memory just vanished. That memory hasn't vanished, it is in your libc's malloc arena, available for future allocations. When and if it will be given back to the OS is up to the specifics of the malloc implementation. E.g., when I do the above on MS-Windows, where malloc is more eager to return memory to the OS, I end up with just 40 MB footprint, and if I then invoke GC manually, the memory goes down almost to the original value: 14 MB vs 12 MB after startup. There are many places on the Internet which explain why the memory footprint of a program doesn't go back to the original value even though the program frees all the heap memory it allocated. I suggest to read some of those explanations. > Regarding, whether it is stack size: > > λ grep VmStk /proc/283047/status > VmStk: 132 kB > > Apparently, it is not stack size. This is a misunderstanding. The space allocated for the stack doesn't need to grow. Values are pushed and popped there depending on the callstack depth, and Emacs regards anything on the stack that looks like a Lisp value or a pointer to a Lisp value as an indication that this Lisp value is in use, and shouldn't be GC'd. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 13 01:08:47 2020 Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 06:08:47 +0000 Received: from localhost ([127.0.0.1]:47313 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koKYp-0004zn-82 for submit@debbugs.gnu.org; Sun, 13 Dec 2020 01:08:47 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56344) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koKYn-0004zW-Bd for 45200@debbugs.gnu.org; Sun, 13 Dec 2020 01:08:45 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59664) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1koKYh-0000h9-R1; Sun, 13 Dec 2020 01:08:39 -0500 Received: from eliz by fencepost.gnu.org with local (Exim 4.82) (envelope-from ) id 1koKYh-0004uX-IT; Sun, 13 Dec 2020 01:08:39 -0500 From: Eli Zaretskii To: Lars Ingebrigtsen In-Reply-To: <87czze7hrf.fsf@gnus.org> (message from Lars Ingebrigtsen on Sat, 12 Dec 2020 23:59:32 +0100) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <83k0tmeq6f.fsf@gnu.org> <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> <87czze7hrf.fsf@gnus.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-Id: Date: Sun, 13 Dec 2020 01:08:39 -0500 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org, hi-angel@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Lars Ingebrigtsen > Cc: Eli Zaretskii , 45200@debbugs.gnu.org > Date: Sat, 12 Dec 2020 23:59:32 +0100 > > Konstantin Kharlamov writes: > > > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and > > PSS after is 266.3M. That is ≈200M of memory just vanished. > > I get similar results. `M-x memory-report' shows that *Messages* and > *Echo Area* each group to 9MB, and almost all of that is in `gap-size'. > > But that's still a long way from 200MB. Perhaps Emacs is bouncing the > buffer area around a lot and isn't returning the data to the OS? (Does > Emacs ever return memory used by buffers to the OS?) Yes, Emacs returns buffer memory to the OS when a buffer is killed. Buffer text memory is usually obtained via mmap, and is returned when no longer needed. Buffer text memory is also returned to the OS when buffers have their gap resized. I think it isn't buffer memory that takes up most of the space here, it's memory allocated for all the strings consed by this snippet. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 13 07:07:56 2020 Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 12:07:56 +0000 Received: from localhost ([127.0.0.1]:47585 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koQAO-0008SB-Ip for submit@debbugs.gnu.org; Sun, 13 Dec 2020 07:07:56 -0500 Received: from forward104p.mail.yandex.net ([77.88.28.107]:55659) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1koQAL-0008Rc-S3 for 45200@debbugs.gnu.org; Sun, 13 Dec 2020 07:07:55 -0500 Received: from mxback14o.mail.yandex.net (mxback14o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::65]) by forward104p.mail.yandex.net (Yandex) with ESMTP id 7E8244B00D54; Sun, 13 Dec 2020 15:07:47 +0300 (MSK) Received: from myt5-95c1fb78270f.qloud-c.yandex.net (myt5-95c1fb78270f.qloud-c.yandex.net [2a02:6b8:c12:1725:0:640:95c1:fb78]) by mxback14o.mail.yandex.net (mxback/Yandex) with ESMTP id E1pNK9OF5a-7lVq638b; Sun, 13 Dec 2020 15:07:47 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1607861267; bh=izcAo6NbdFJ4KXQv4oPCQvEna4EUceJEw9aT8DRatFA=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=aXjqOMH/g5MdYjPTxajlU5Cd8nI6w58lXd83bMJE33i93QDtxD4hRoJWnM6mpGmhl 18h+eqUN/WazlPlRWl2NgJWAEfcn5OryiRdkwAbRmcCVFkW8CJ2nhuwAoLCnOGKanH cwNEYldz8RXR8HNzLZG2XL7kIBtnGXm0WW1BElig= Authentication-Results: mxback14o.mail.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt5-95c1fb78270f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id Rpbu72hy8I-7kn0dX9E; Sun, 13 Dec 2020 15:07:46 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <72e40cc93232dd0054977bb7037a5ac38e2165ba.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Eli Zaretskii Date: Sun, 13 Dec 2020 15:07:46 +0300 In-Reply-To: References: <83k0tmeq6f.fsf@gnu.org> <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, 2020-12-13 at 00:53 -0500, Eli Zaretskii wrote: > > From: Konstantin Kharlamov > > Cc: 45200@debbugs.gnu.org > > Date: Sun, 13 Dec 2020 01:44:13 +0300 > > > > Alright, fair enough. I crafted up another testcase, it may be better. The > > following code first temporarily disables GC, then it prints "hello" 1000000 > > times, and finally it calls GC manually. > > > > I call `emacs -Q`, then measure PSS, then evaluate the code below, then > > again measure PSS. > > > >     (let ((i 1000000)) > >       (setq gc-cons-threshold most-positive-fixnum) > >       (while (> i 0) > >         (print "hello") > >         (setq i (- i 1))) > >       (garbage-collect)) > > > > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS > > after is 266.3M. That is ≈200M of memory just vanished. > > That memory hasn't vanished, it is in your libc's malloc arena, > available for future allocations.  When and if it will be given back > to the OS is up to the specifics of the malloc implementation.  E.g., > when I do the above on MS-Windows, where malloc is more eager to return > memory to the OS, I end up with just 40 MB footprint, and if I then > invoke GC manually, the memory goes down almost to the original value: > 14 MB vs 12 MB after startup. Your explanation seems likely, because such memory is reused by glibc on later mallocs, and if I repeat the testcase, I can see that memory does not increase further, meaning it got reused. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 10:24:22 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 15:24:22 +0000 Received: from localhost ([127.0.0.1]:36987 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3hFV-0000h0-Vp for submit@debbugs.gnu.org; Sun, 24 Jan 2021 10:24:22 -0500 Received: from forward100p.mail.yandex.net ([77.88.28.100]:59195) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3hFQ-0000gi-DY for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 10:24:20 -0500 Received: from myt3-a3d1bb5f3f13.qloud-c.yandex.net (myt3-a3d1bb5f3f13.qloud-c.yandex.net [IPv6:2a02:6b8:c12:28:0:640:a3d1:bb5f]) by forward100p.mail.yandex.net (Yandex) with ESMTP id 2C5195980937 for <45200@debbugs.gnu.org>; Sun, 24 Jan 2021 18:24:09 +0300 (MSK) Received: from myt6-efff10c3476a.qloud-c.yandex.net (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3]) by myt3-a3d1bb5f3f13.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 7DLljfbBP1-O8G81gaj; Sun, 24 Jan 2021 18:24:09 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611501849; bh=WpgoFbZUg3vTl+wC5eMn1D+d9Rt7AY/rmlsl2+v1W50=; h=Date:Subject:To:From:Message-Id; b=pDXxk1V+TA1QQrRZxxWK3QaKeSJaHOCRocszIgAPpoF9dhNbRw29NrUTM7eKXR232 +kfVShAV2sMAqEscN9dE9I1IVtn+FH7IU8ni9K26UXf3bNX+z6TUZIyjyWMSV2bBfY O6LamhIcgvL84toSmobC6NzsJwUQeny1BhCxSxZg= Authentication-Results: myt3-a3d1bb5f3f13.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id MkwfNY9RdE-O8H4ZoA6; Sun, 24 Jan 2021 18:24:08 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) From: Konstantin Kharlamov To: 45200@debbugs.gnu.org Subject: [PATCH] Force Glibc to free the memory freed Date: Sun, 24 Jan 2021 18:24:02 +0300 Message-Id: <20210124152402.40270-1-Hi-Angel@yandex.ru> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) configure.ac: check whether malloc_trim is suported src/alloc.c (lisp_free): call malloc_trim() if possible (bug#45200) --- configure.ac | 3 +++ src/alloc.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/configure.ac b/configure.ac index bcc0be7de0..3e0459a0e2 100644 --- a/configure.ac +++ b/configure.ac @@ -4544,6 +4544,9 @@ AC_DEFUN dnl the current CFLAGS etc. AC_CHECK_FUNCS(snprintf) + +AC_CHECK_FUNCS(malloc_trim) + dnl Check for glib. This differs from other library checks in that dnl Emacs need not link to glib unless some other library is already dnl linking to glib. Although glib provides no facilities that Emacs diff --git a/src/alloc.c b/src/alloc.c index c0a55e61b9..97e3ceb52c 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -1047,6 +1047,9 @@ lisp_free (void *block) MALLOC_BLOCK_INPUT; free (block); +#ifdef HAVE_MALLOC_TRIM + malloc_trim(0); /* work around for high memory consumption, see bug 45200 */ +#endif /* HAVE_MALLOC_TRIM */ #ifndef GC_MALLOC_CHECK mem_delete (mem_find (block)); #endif -- 2.30.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 10:40:37 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 15:40:38 +0000 Received: from localhost ([127.0.0.1]:36992 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3hVF-00017e-KV for submit@debbugs.gnu.org; Sun, 24 Jan 2021 10:40:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33922) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3hVB-00017O-GM for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 10:40:37 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60069) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3hV5-0005z8-4l; Sun, 24 Jan 2021 10:40:27 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2771 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l3hV2-000592-NG; Sun, 24 Jan 2021 10:40:26 -0500 Date: Sun, 24 Jan 2021 17:40:29 +0200 Message-Id: <834kj64a36.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <20210124152402.40270-1-Hi-Angel@yandex.ru> (message from Konstantin Kharlamov on Sun, 24 Jan 2021 18:24:02 +0300) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: <20210124152402.40270-1-Hi-Angel@yandex.ru> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Date: Sun, 24 Jan 2021 18:24:02 +0300 > > configure.ac: check whether malloc_trim is suported > src/alloc.c (lisp_free): call malloc_trim() if possible > (bug#45200) > --- > configure.ac | 3 +++ > src/alloc.c | 3 +++ > 2 files changed, 6 insertions(+) > > diff --git a/configure.ac b/configure.ac > index bcc0be7de0..3e0459a0e2 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -4544,6 +4544,9 @@ AC_DEFUN > dnl the current CFLAGS etc. > AC_CHECK_FUNCS(snprintf) > > + > +AC_CHECK_FUNCS(malloc_trim) > + > dnl Check for glib. This differs from other library checks in that > dnl Emacs need not link to glib unless some other library is already > dnl linking to glib. Although glib provides no facilities that Emacs > diff --git a/src/alloc.c b/src/alloc.c > index c0a55e61b9..97e3ceb52c 100644 > --- a/src/alloc.c > +++ b/src/alloc.c > @@ -1047,6 +1047,9 @@ lisp_free (void *block) > > MALLOC_BLOCK_INPUT; > free (block); > +#ifdef HAVE_MALLOC_TRIM > + malloc_trim(0); /* work around for high memory consumption, see bug 45200 */ > +#endif /* HAVE_MALLOC_TRIM */ > #ifndef GC_MALLOC_CHECK > mem_delete (mem_find (block)); > #endif Thanks, but is it really a good idea to call malloc_trim each time we free some chunk of memory? Carlos, Florian, DJ: any thoughts or comments on this proposal? (Let me know if you need some background about the code involved in this.) From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 13:51:58 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 18:51:58 +0000 Received: from localhost ([127.0.0.1]:37260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kUP-0001kF-SL for submit@debbugs.gnu.org; Sun, 24 Jan 2021 13:51:58 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:34091) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kUO-0001k0-5o for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 13:51:56 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7DB0D80B69; Sun, 24 Jan 2021 13:51:50 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id EEDD280057; Sun, 24 Jan 2021 13:51:48 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1611514309; bh=tiaED2ZbqzCrNjpdzBUTKfq/fzgmO6rpkQQf+lPUOSc=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=T65lOMKDRlIcdDVJPKfMx/IAgzUMYq9/iHIrmzfg6ye3UvaPrBBqsXn1O9L5i+pK4 1SxeoXJEcbv7EaomNTtToRcoR6jirgL5sAfu3RAubfB4qyPqsKce+cR3am1XvQh0IC zi5L8pW5CQpgHuKubTuNGuc9Wy1gPJEm1zEivJ5GGL1RQCB4HlOTJgmV/uYbQtAR3y +KCUde6jjtjGc94RchG8mipe2YiF9Km9QSDO/NF+oAmyG1orQs844xFoifu6B2FjTz WvGVnpvf4RNiZbPAN5pXgp7rgJeg4KMyMIhuiBcYt41T4j48txufh48x39p/W9sXjd EpMnJ7OeK4V9Q== Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BA4801201CC; Sun, 24 Jan 2021 13:51:48 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Message-ID: References: Date: Sun, 24 Jan 2021 13:51:41 -0500 In-Reply-To: (Konstantin Kharlamov's message of "Sat, 12 Dec 2020 21:43:10 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.010 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > # Steps to reproduce: > > 1. Run `mkdir /tmp/.emacs.d` > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS > 3. Create a file /tmp/.emacs.d/early-init.el with content: > > ;; only run garbage collection on idle > (setq gc-cons-threshold most-positive-fixnum) > (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS > > ## Expected > > Size has no statistically-significant difference, because in both > cases we garbage-collected memory. I disagree with this expectation: it is perfectly normal for the amount of memory allocated to the Emacs process to be left higher if you delay the GC. There are various reasons for that: - fragmentation, of course. Not much we can do about it short of using a moving collector. - the desire to keep memory around rather than return it to the OS, under the assumption that we'll need it again soon. And it's not considered as a memory leak as long as that memory has indeed been needed in the past and that future allocations can still make use of it. Eli wrote: > Thanks, but is it really a good idea to call malloc_trim each time we > free some chunk of memory? I think if we want to call `malloc_trim`, the obvious place would be to do it at the end of `garbage_collect`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 14:00:42 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:00:42 +0000 Received: from localhost ([127.0.0.1]:37275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kcs-0001zQ-9E for submit@debbugs.gnu.org; Sun, 24 Jan 2021 14:00:42 -0500 Received: from forward106j.mail.yandex.net ([5.45.198.249]:35443) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kcq-0001zA-42 for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 14:00:41 -0500 Received: from iva3-3aa7bbf53bdf.qloud-c.yandex.net (iva3-3aa7bbf53bdf.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:4983:0:640:3aa7:bbf5]) by forward106j.mail.yandex.net (Yandex) with ESMTP id E1EA711A0D7F; Sun, 24 Jan 2021 22:00:32 +0300 (MSK) Received: from iva8-174eb672ffa9.qloud-c.yandex.net (iva8-174eb672ffa9.qloud-c.yandex.net [2a02:6b8:c0c:b995:0:640:174e:b672]) by iva3-3aa7bbf53bdf.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 4G3T6JmRog-0WG0OMjd; Sun, 24 Jan 2021 22:00:32 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611514832; bh=bHettI+ZzAnUuZGNz0T0xIOWtjvdJd8nsWDkhcqJBME=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=DVYwPw1989BZeEJfI33lXl3+PR2T18S59740qPN4+9m4Yd/5RinU/jKibkPtteTaD iy2fhR15/HGSHMGuBVL/MRy0ODPsVhJzFbakWZDX+Pe57Wqg4O4Ey3sIMeixXwpJaF le8/KoGt8aezRBXkvJLoBQXeH0vurZRz/f2HsKaI= Authentication-Results: iva3-3aa7bbf53bdf.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva8-174eb672ffa9.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id r3NgoBo7Lp-0WImEXdo; Sun, 24 Jan 2021 22:00:32 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Stefan Monnier Date: Sun, 24 Jan 2021 22:00:31 +0300 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Sun, 2021-01-24 at 13:51 -0500, Stefan Monnier wrote: > > # Steps to reproduce: > > > > 1. Run `mkdir /tmp/.emacs.d` > > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS > > 3. Create a file /tmp/.emacs.d/early-init.el with content: > > > >     ;; only run garbage collection on idle > >     (setq gc-cons-threshold most-positive-fixnum) > >     (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure > > its PSS > > > > ## Expected > > > > Size has no statistically-significant difference, because in both > > cases we garbage-collected memory. > > I disagree with this expectation: it is perfectly normal for the amount > of memory allocated to the Emacs process to be left higher if you delay > the GC.  There are various reasons for that: > - fragmentation, of course.  Not much we can do about it short of using >   a moving collector. > - the desire to keep memory around rather than return it to the OS, >   under the assumption that we'll need it again soon. > > And it's not considered as a memory leak as long as that memory has > indeed been needed in the past and that future allocations can still > make use of it. I'm fine with Emacs possibly keeping a dozen of megabytes for personal use. The problem though is that the issue easily manifests itself in hundreds of megabytes, as can be seen with the testcase marked as "Message #11" in web-ui. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 14:06:32 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:06:32 +0000 Received: from localhost ([127.0.0.1]:37287 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kiW-00028G-G9 for submit@debbugs.gnu.org; Sun, 24 Jan 2021 14:06:32 -0500 Received: from forward103o.mail.yandex.net ([37.140.190.177]:53439) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3kiT-00027y-G3 for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 14:06:30 -0500 Received: from sas1-37a648709e20.qloud-c.yandex.net (sas1-37a648709e20.qloud-c.yandex.net [IPv6:2a02:6b8:c08:fe03:0:640:37a6:4870]) by forward103o.mail.yandex.net (Yandex) with ESMTP id B88AC5F80D98; Sun, 24 Jan 2021 22:06:22 +0300 (MSK) Received: from sas1-e20a8b944cac.qloud-c.yandex.net (sas1-e20a8b944cac.qloud-c.yandex.net [2a02:6b8:c14:6696:0:640:e20a:8b94]) by sas1-37a648709e20.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 1Z1ZMHGEDe-6MGO7SHw; Sun, 24 Jan 2021 22:06:22 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611515182; bh=uMD8zf8vnMHzN04u2mb17Tq3FpEpsyUiU1ecPRJhjsE=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=kB6Iv0QnGDJwAJKIhWCIv0Gxf6ICzSTnkR0lS14EB4VfBymMg8LXCBIQIqgnB244g crDMNckmKXOSErArpdJhPOoHPuwvhbuTucujFVBX/KAnbsg86Exj/T6GiQ6hEnF2UX z2GU3x6ODA+mQVRMB0E4grJrUh+qumPPUklhHqFg= Authentication-Results: sas1-37a648709e20.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-e20a8b944cac.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id nl5csKbbqS-6MIaVelL; Sun, 24 Jan 2021 22:06:22 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <0457437ff45b535a61f7c35011de1d133cdd3496.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Stefan Monnier Date: Sun, 24 Jan 2021 22:06:21 +0300 In-Reply-To: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, 2021-01-24 at 22:00 +0300, Konstantin Kharlamov wrote: > On Sun, 2021-01-24 at 13:51 -0500, Stefan Monnier wrote: > > > # Steps to reproduce: > > > > > > 1. Run `mkdir /tmp/.emacs.d` > > > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS > > > 3. Create a file /tmp/.emacs.d/early-init.el with content: > > > > > >     ;; only run garbage collection on idle > > >     (setq gc-cons-threshold most-positive-fixnum) > > >     (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > > > > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then > > > measure > > > its PSS > > > > > > ## Expected > > > > > > Size has no statistically-significant difference, because in both > > > cases we garbage-collected memory. > > > > I disagree with this expectation: it is perfectly normal for the amount > > of memory allocated to the Emacs process to be left higher if you delay > > the GC.  There are various reasons for that: > > - fragmentation, of course.  Not much we can do about it short of using > >   a moving collector. > > - the desire to keep memory around rather than return it to the OS, > >   under the assumption that we'll need it again soon. > > > > And it's not considered as a memory leak as long as that memory has > > indeed been needed in the past and that future allocations can still > > make use of it. > > I'm fine with Emacs possibly keeping a dozen of megabytes for personal use. > The problem though is that the issue easily manifests itself in hundreds of > megabytes, as can be seen with the testcase marked as "Message #11" in web-ui. Case in point btw: after the previous weekend I found my Emacs instance that I did not touch during the weekend to bloat to as much as 6xxMB. I think that happened because the on-idle hook that I hooked garbage collector onto was not run for 2 days, whilst Emacs might have been doing something like communication to LSP server from time to time, so this is why it ended up taking so much memory. I think it's needless to say that I had to restart Emacs for the half-gigabyte of memory to get returned to my system. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 14:12:38 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:12:38 +0000 Received: from localhost ([127.0.0.1]:37308 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3koP-0002IV-Q0 for submit@debbugs.gnu.org; Sun, 24 Jan 2021 14:12:37 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61613) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3koO-0002IF-Fe for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 14:12:36 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 86AD044061A; Sun, 24 Jan 2021 14:12:30 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DBB8A44059C; Sun, 24 Jan 2021 14:12:21 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1611515541; bh=hyRyxFPMOW0abSrju6coe8g1J4SJN+WSdbhe1fOKa+E=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=iZMVQbuzPvcpqu1NbhlQzQDjAsPEsepHnTkzerXDDMun5vuQbQkFItoKRVA+5jY9k iKTQ8tuV/kk38NAbt476fyXq093JiCYEAiWBSpEQv8I87tADHv03dKmhvsGKHd8hSd X/y0doBWbxXxfsT2SqQWpXzSDPrJCNSBZQe/PSRCL7MW87XKiMQu57NB5YkYPcECeH gFAkfrbmBqPsCSgZGvnGaW23EAGfrFKY8PKP3nTHJPZzaEwKByRFL4Us1FQoY/a13x xAonBlacVRueoRAscDppisS7yqngXvg4RDvQB9em3zv2+OWeGHxgpG7ghfUc3JdlJ7 iNKrzAKyMavHw== Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id AC7CA1202B6; Sun, 24 Jan 2021 14:12:21 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Message-ID: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> Date: Sun, 24 Jan 2021 14:12:20 -0500 In-Reply-To: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> (Konstantin Kharlamov's message of "Sun, 24 Jan 2021 22:00:31 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.085 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I'm fine with Emacs possibly keeping a dozen of megabytes for personal > use. The problem though is that the issue easily manifests itself in > hundreds of megabytes, as can be seen with the testcase marked as "Message > #11" in web-ui. I don't think the exact number matters: you were willing to make use of *any* amount of extra memory by delaying all GC until idle time. The fact that the GC you finally do after that long wait doesn't recover that memory is not a failure in my book: there's no reason Emacs or glibc should presume that you won't be reusing just as much heap space before the next GC. IOW in my book, you got what you asked for ;-) BTW, I wish `malloc_trim` could be passed an argument that says something like "release about half of the free memory", which would provide a better balance between "hoard free memory indefinitely" and "constantly free&reallocate memory from the OS". Also, the manpage of mallopt suggests that glibc should automatically do the trim on its own every once in a while anyway, so maybe the problem won't be solved by `malloc_trim`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 14:55:04 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:55:04 +0000 Received: from localhost ([127.0.0.1]:37353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lTU-0003J1-3U for submit@debbugs.gnu.org; Sun, 24 Jan 2021 14:55:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42284) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lTS-0003IT-7A for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 14:55:02 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35861) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3lTM-00022h-N7; Sun, 24 Jan 2021 14:54:56 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2791 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l3lTL-0008Ob-MY; Sun, 24 Jan 2021 14:54:56 -0500 Date: Sun, 24 Jan 2021 21:55:00 +0200 Message-Id: <83y2gi2jqj.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <0457437ff45b535a61f7c35011de1d133cdd3496.camel@yandex.ru> (message from Konstantin Kharlamov on Sun, 24 Jan 2021 22:06:21 +0300) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <0457437ff45b535a61f7c35011de1d133cdd3496.camel@yandex.ru> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Date: Sun, 24 Jan 2021 22:06:21 +0300 > Cc: 45200@debbugs.gnu.org > > Case in point btw: after the previous weekend I found my Emacs instance that I did not touch during the weekend to bloat to as much as 6xxMB. I think that happened because the on-idle hook that I hooked garbage collector onto was not run for 2 days, whilst Emacs might have been doing something like communication to LSP server from time to time, so this is why it ended up taking so much memory. I think it's needless to say that I had to restart Emacs for the half-gigabyte of memory to get returned to my system. That bug (bug#43389) was recently tracked down and fixed. I don't think it has anything to do with malloc_trim and friends. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 15:01:01 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:01:01 +0000 Received: from localhost ([127.0.0.1]:37369 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lZF-0003Te-Ie for submit@debbugs.gnu.org; Sun, 24 Jan 2021 15:01:01 -0500 Received: from forward101j.mail.yandex.net ([5.45.198.241]:49730) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lZD-0003TG-Gb for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 15:01:01 -0500 Received: from myt2-4597fff41cb2.qloud-c.yandex.net (myt2-4597fff41cb2.qloud-c.yandex.net [IPv6:2a02:6b8:c00:3b2c:0:640:4597:fff4]) by forward101j.mail.yandex.net (Yandex) with ESMTP id 587741BE022B; Sun, 24 Jan 2021 23:00:52 +0300 (MSK) Received: from myt6-efff10c3476a.qloud-c.yandex.net (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3]) by myt2-4597fff41cb2.qloud-c.yandex.net (mxback/Yandex) with ESMTP id YPeZnxVA9D-0qH8wQt8; Sun, 24 Jan 2021 23:00:52 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611518452; bh=DGHB7wTmROhuQJyXdW2kGLCrWOr9FKnXoWpERvnKdyo=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=p8evXnbDVwjP27fx+KfRrHA0PUuZqm2caDNYaOyqdXfJGqJ5kqKxm5CaJYuVlCQbh 10p2shh2hBVMMcm6Xiz1A3vwMdvSG7rBog0XDJPurLdvsOm6wxdPTrHJ5XF9mJae60 1H/HtZ4FClOfNWnRRaOr0w0nv0M+UieaX0b1p5e0= Authentication-Results: myt2-4597fff41cb2.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id wghFCbz68h-0pHiXOJg; Sun, 24 Jan 2021 23:00:51 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Stefan Monnier Date: Sun, 24 Jan 2021 23:00:50 +0300 In-Reply-To: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Sun, 2021-01-24 at 14:12 -0500, Stefan Monnier wrote: > > I'm fine with Emacs possibly keeping a dozen of megabytes for personal > > use. The problem though is that the issue easily manifests itself in > > hundreds of megabytes, as can be seen with the testcase marked as "Message > > #11" in web-ui. > > I don't think the exact number matters: you were willing to make use of > *any* amount of extra memory by delaying all GC until idle time. > The fact that the GC you finally do after that long wait doesn't recover > that memory is not a failure in my book: there's no reason Emacs or > glibc should presume that you won't be reusing just as much heap space > before the next GC. > > IOW in my book, you got what you asked for ;-) This doesn't look right. I mean, sure, I might need the heap again. Or I might not. I don't know. Neither you can. Nobody knows. If glibc can predict the future then fine. But my experience as part of this report suggests glibc can't. Glibc could for example use statistics of previous allocation patterns to see if it's needed to release memory and how much, but this is not what happens. And I am not the only dissatisfied user. Ruby for example too¹. Also, many people experience strange memory usage grow with Qutebrowser that nobody knows where it's coming from; and after today's research I suspect Glibc is the culprit (I don't have yet enough evidence because my Qutebrowser instance doesn't have much uptime ATM, but attaching to it with gdb and calling `malloc_trim` inside it already freed ≈40M of memory to me today). Glib also seems affected². With so many unhappy users, do you still think nothing wrong with the glibc allocator? It seems to me, we're doing perfectly valid usecase: we used memory, we don't need it anymore, we free it. If somebody ever notes that malloc/free cycle takes too much time for them, then they know they need to optimize the specific codepath where this happens. It's a nice bonus if Glibc could do malloc/free faster, but not so much if this causes other problems, especially given it's a core system library. > BTW, I wish `malloc_trim` could be passed an argument that says > something like "release about half of the free memory", which would > provide a better balance between "hoard free memory indefinitely" and > "constantly free&reallocate memory from the OS". > > Also, the manpage of mallopt suggests that glibc should automatically > do the trim on its own every once in a while anyway, so maybe the > problem won't be solved by `malloc_trim`. Yeah, I've seen that too. Idk why it doesn't work, but that `malloc_trim()` does I know because I tested it with the testcase where previously Emacs was never returning ≈200M of memory. 1: https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#investigating-fragmentation-at-the-memory-allocator-level 2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057 From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 15:11:51 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:11:51 +0000 Received: from localhost ([127.0.0.1]:37377 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lji-0003jc-Vl for submit@debbugs.gnu.org; Sun, 24 Jan 2021 15:11:51 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45674) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3ljh-0003jN-6e for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 15:11:49 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36236) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l3ljb-00089A-T7; Sun, 24 Jan 2021 15:11:43 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3853 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l3ljY-0004te-Fh; Sun, 24 Jan 2021 15:11:41 -0500 Date: Sun, 24 Jan 2021 22:11:46 +0200 Message-Id: <83tur62iyl.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> (message from Konstantin Kharlamov on Sun, 24 Jan 2021 23:00:50 +0300) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Date: Sun, 24 Jan 2021 23:00:50 +0300 > Cc: 45200@debbugs.gnu.org > > With so many unhappy users, do you still think nothing wrong with the glibc > allocator? Please be aware that this is not the right place to send bug reports about glibc's memory allocation algorithms and implementation. They have their own bug tracker. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 15:21:18 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:21:18 +0000 Received: from localhost ([127.0.0.1]:37391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lss-000414-C1 for submit@debbugs.gnu.org; Sun, 24 Jan 2021 15:21:18 -0500 Received: from forward103j.mail.yandex.net ([5.45.198.246]:54285) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3lsq-00040p-Hb for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 15:21:17 -0500 Received: from forward101q.mail.yandex.net (forward101q.mail.yandex.net [IPv6:2a02:6b8:c0e:4b:0:640:4012:bb98]) by forward103j.mail.yandex.net (Yandex) with ESMTP id AAC0F6740FB7; Sun, 24 Jan 2021 23:21:10 +0300 (MSK) Received: from vla1-5c8ce23a551e.qloud-c.yandex.net (vla1-5c8ce23a551e.qloud-c.yandex.net [IPv6:2a02:6b8:c0d:400d:0:640:5c8c:e23a]) by forward101q.mail.yandex.net (Yandex) with ESMTP id A585ACF40002; Sun, 24 Jan 2021 23:21:10 +0300 (MSK) Received: from vla4-d1b041059520.qloud-c.yandex.net (vla4-d1b041059520.qloud-c.yandex.net [2a02:6b8:c17:914:0:640:d1b0:4105]) by vla1-5c8ce23a551e.qloud-c.yandex.net (mxback/Yandex) with ESMTP id BqfOFRLV27-LAGGdOui; Sun, 24 Jan 2021 23:21:10 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611519670; bh=fejtMT4drRKGDP0WOrNBh0AjQrQReMLp0PtRt/p6Dl8=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=BhizMNPx20KZHlWTNRX9ARuFWmcwnUSr9HJ3wszrFnvzZSSwqMGmrROAUWMWdoK9N 0UEjaFK21gR9bbMOoK4gyrTLu2dN+i0h+74fNJVNUfI7eFzyJWzOs/7T8N20GsbZTj 8hG5QyqMzWo4z5YQj+tPx0/2BBfq8U7NUbMdOQmI= Authentication-Results: vla1-5c8ce23a551e.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by vla4-d1b041059520.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id 9wHMxPrxjX-LAmqTha2; Sun, 24 Jan 2021 23:21:10 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Eli Zaretskii Date: Sun, 24 Jan 2021 23:21:09 +0300 In-Reply-To: <83tur62iyl.fsf@gnu.org> References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> <83tur62iyl.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Sun, 2021-01-24 at 22:11 +0200, Eli Zaretskii wrote: > > From: Konstantin Kharlamov > > Date: Sun, 24 Jan 2021 23:00:50 +0300 > > Cc: 45200@debbugs.gnu.org > > > > With so many unhappy users, do you still think nothing wrong with the glibc > > allocator? > > Please be aware that this is not the right place to send bug reports > about glibc's memory allocation algorithms and implementation.  They > have their own bug tracker. Sure, they have my report on the problem. In my text I was just trying to convince Stefan that this is not correct behaviour, whereas he was implying otherwise. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 16:20:32 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:20:32 +0000 Received: from localhost ([127.0.0.1]:37410 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3moC-0005Uz-Iv for submit@debbugs.gnu.org; Sun, 24 Jan 2021 16:20:32 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:49489) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3moA-0005Uk-S6 for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 16:20:31 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 3E15A101136; Sun, 24 Jan 2021 16:20:25 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id CF83A100707; Sun, 24 Jan 2021 16:20:23 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1611523223; bh=aFRM3icbGp6HZvFLVY4AGVL1sfwS1MI5QvyfPniiiEg=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=VBedqCDazqQx0squT5QF/ARGO0kQUYDs4nEg2EGobk9iDQozl8Cn2yBkqwzwHcAeO OfE8zE41+QM1sE6jD+SXWwYn7QzmmrqaMG11GTyD/Z0tTQtoxCGC1/FgH9Dk0Saa6p ypfFrPpDLHes1aXTzL7DWEmZBp526MraiJmSAgUn3B0Ww20q/V4IRpXOjEFbgdGq46 5cB/LqmVf/mqAdtpJfZGWdRa496wT66YDnpNbDsZpJBXELh8QlLiyGAgk6mEVkTGYU Ftj0vN7rQqrvLFb3OSvHFSbIpfLxxGikHzl4gA2edd+fw9W02lNfiI+zE1A1bZb7gF GJ4SqpBTg/WqA== Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 851AE12014C; Sun, 24 Jan 2021 16:20:23 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Message-ID: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> <83tur62iyl.fsf@gnu.org> <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> Date: Sun, 24 Jan 2021 16:20:22 -0500 In-Reply-To: <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> (Konstantin Kharlamov's message of "Sun, 24 Jan 2021 23:21:09 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.006 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Sure, they have my report on the problem. In my text I was just trying to > convince Stefan that this is not correct behaviour, whereas he was > implying otherwise. I'm not saying it's necessarily correct behavior. What I'm saying is that the expectation that the temporary use of 200MB should not affect the memory use later is unrealistic. And also that if you (setq gc-cons-threshold most-positive-fixnum) then you're simply asking for trouble unless you *really* know what you're doing (in which case you'd understand that the above expectation is unrealistic). > Glibc could for example use statistics of previous allocation patterns > to see if it's needed to release memory and how much, but this is not > what happens. And I am not the only dissatisfied user. Ruby for > example too=C2=B9. Also, many people experience strange memory usage grow > with Qutebrowser that nobody knows where it's coming from; and after > today's research I suspect Glibc is the culprit (I don't have yet > enough evidence because my Qutebrowser instance doesn't have much > uptime ATM, but attaching to it with gdb and calling `malloc_trim` > inside it already freed =E2=89=8840M of memory to me today). Glib also se= ems > affected=C2=B2. Memory management is hard, and lots and lots of things can go wrong. Maybe you're right and they're all due to some underlying problem in glibc. I suspect that instead the only thing in common is that they're all problems with the memory management. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 16:26:53 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:26:53 +0000 Received: from localhost ([127.0.0.1]:37418 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3muL-0005ex-KC for submit@debbugs.gnu.org; Sun, 24 Jan 2021 16:26:53 -0500 Received: from forward100j.mail.yandex.net ([5.45.198.240]:57839) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3muK-0005eg-5W for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 16:26:52 -0500 Received: from myt5-36442628f0be.qloud-c.yandex.net (myt5-36442628f0be.qloud-c.yandex.net [IPv6:2a02:6b8:c12:1c0c:0:640:3644:2628]) by forward100j.mail.yandex.net (Yandex) with ESMTP id 2153150E0597; Mon, 25 Jan 2021 00:26:46 +0300 (MSK) Received: from myt6-efff10c3476a.qloud-c.yandex.net (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3]) by myt5-36442628f0be.qloud-c.yandex.net (mxback/Yandex) with ESMTP id YJIA7gGm9j-QkG4XOMu; Mon, 25 Jan 2021 00:26:46 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611523606; bh=Qycj0hjBfBNbcdVa43YeYyb+TjO/g5CXDmU8CLFTuWs=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=EUijUzxWeYCrB/0/hqnNQUC9k3k1hNntMHHywmE4rU2mkfdiy78wTimm6Rr8jJi8/ EMxxnfqNs0ivL4zzRRaFmBr5/oln58zpyT5UwYVLBnyMpn+chCfGR0p1MMTZ/7bFrT GQcCGOglpZql64k/y7sgidkNxVSpZti/Wo14PAqY= Authentication-Results: myt5-36442628f0be.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id JWczKzgOJq-QjHuoa55; Mon, 25 Jan 2021 00:26:45 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Stefan Monnier Date: Mon, 25 Jan 2021 00:26:45 +0300 In-Reply-To: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> <83tur62iyl.fsf@gnu.org> <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Sun, 2021-01-24 at 16:20 -0500, Stefan Monnier wrote: > > Sure, they have my report on the problem. In my text I was just trying to > > convince Stefan that this is not correct behaviour, whereas he was > > implying otherwise. > > I'm not saying it's necessarily correct behavior. > What I'm saying is that the expectation that the temporary use of 200MB > should not affect the memory use later is unrealistic. > > And also that if you (setq gc-cons-threshold most-positive-fixnum) > then you're simply asking for trouble unless you *really* know what > you're doing (in which case you'd understand that the above expectation > is unrealistic). Why unrealistic? To me the situation is pretty clear: I needed 200M, but I don't need them anymore, so I want them released. What's unrealistic about that? If that was your point about that "maybe the 200M of memory will be needed again", then as I said, yeah, maybe, maybe not — unless Glibc has a prophet module built-in, it can't know, so it should not behave as if it does. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 16:41:19 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:41:19 +0000 Received: from localhost ([127.0.0.1]:37434 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3n8J-00064k-GZ for submit@debbugs.gnu.org; Sun, 24 Jan 2021 16:41:19 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20566) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3n8I-00064W-GG for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 16:41:18 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 2E1F7101136; Sun, 24 Jan 2021 16:41:13 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 79C0D10022E; Sun, 24 Jan 2021 16:41:05 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1611524465; bh=Jop/6kD6RyxWduTAk6dUXwT3KlnMtUp0Zl4rU36QqTA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Txz/b+WXnDs5bcR139OpTPLsFjvGHv1uHfDqrSnH3QAvSU5rqVdzvBm8A650e1H+l rRsjdy3IwSGuwmCMamPEoGFRQN/7f1K1kQaJCtC6JGpAJFMFIL26N2tQpqkU6WP1XH DWoRHfLueeSOBfg589ZhnTgQxlDw/YeDEVHDsb2344tKYqcTqvMrjoZZveKXftpLJ+ c3WMS744wkGZrw8pM4OZb1oMP5eXn+uyiFZ6qlA42BrslnBd8Yc+etEiurAIXZgPsx WgRhwISc0nZgRbBhb+4U9opiJsqB9lME75KZNyxzXiFfwv7g8XXLmzsy1zHblNlkzg /kG5s0T2DPgwQ== Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 4367E120155; Sun, 24 Jan 2021 16:41:05 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Message-ID: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> <83tur62iyl.fsf@gnu.org> <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@yandex.ru> Date: Sun, 24 Jan 2021 16:41:04 -0500 In-Reply-To: <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@yandex.ru> (Konstantin Kharlamov's message of "Mon, 25 Jan 2021 00:26:45 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.006 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Why unrealistic? To me the situation is pretty clear: I needed 200M, but = I don't > need them anymore, so I want them released. What's unrealistic about that? What can I say: it's not realistic and I already gave you two technical reasons why that is. > If that was your point about that "maybe the 200M of memory will be > needed again", then as I said, yeah, maybe, maybe not =E2=80=94 unless Gl= ibc > has a prophet module built-in, it can't know, so it should not behave > as if it does. Maybe so. But to the extent that you consciously decided it's OK for Emacs to use 200MB at time-step A, then you basically lost the "moral authority" to say that it's not OK for Emacs to still uses up 200MB at time-step B. If you can show a similar excessive use of memory without doing something like (setq gc-cons-threshold most-positive-fixnum), the question becomes quite different. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 24 16:55:45 2021 Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:55:46 +0000 Received: from localhost ([127.0.0.1]:37444 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3nMH-00006c-LU for submit@debbugs.gnu.org; Sun, 24 Jan 2021 16:55:45 -0500 Received: from forward103o.mail.yandex.net ([37.140.190.177]:54828) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l3nMG-00006N-28 for 45200@debbugs.gnu.org; Sun, 24 Jan 2021 16:55:44 -0500 Received: from iva8-e3673018e436.qloud-c.yandex.net (iva8-e3673018e436.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:da5:0:640:e367:3018]) by forward103o.mail.yandex.net (Yandex) with ESMTP id CCFFC5F80DEF; Mon, 25 Jan 2021 00:55:37 +0300 (MSK) Received: from iva3-dd2bb2ff2b5f.qloud-c.yandex.net (iva3-dd2bb2ff2b5f.qloud-c.yandex.net [2a02:6b8:c0c:7611:0:640:dd2b:b2ff]) by iva8-e3673018e436.qloud-c.yandex.net (mxback/Yandex) with ESMTP id UzEkRZSlG8-tbGaAQQt; Mon, 25 Jan 2021 00:55:37 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611525337; bh=szkGapR15KDm5BeDQE0N53iL4mi61X/TpzVUm8WfK5U=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=DuURmxZvOgelfi/SiJNYxhOLOSrWOX2bLohJVhwcMq8zwSx09CPeuSVPE8/5KYj2V hg/8duUkXNc15CvKPi6qUSbaOn0uQxiJbQxbIFCVLyqUKNlo2x7cqpe6pQcBnCxJfU /5Dwmszz530wAuyfFao9wD6FMjfsO505zTxn7I20= Authentication-Results: iva8-e3673018e436.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva3-dd2bb2ff2b5f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id dAbuR4a4Ic-taIKAAnm; Mon, 25 Jan 2021 00:55:36 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <6d4755efea07ca4f8f133e25f587d72abc1dcc63.camel@yandex.ru> Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory From: Konstantin Kharlamov To: Stefan Monnier Date: Mon, 25 Jan 2021 00:55:36 +0300 In-Reply-To: References: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@yandex.ru> <5c5696670dea31a4647c901be1f87ddb1474f820.camel@yandex.ru> <83tur62iyl.fsf@gnu.org> <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@yandex.ru> <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, 2021-01-24 at 16:41 -0500, Stefan Monnier wrote: > > Why unrealistic? To me the situation is pretty clear: I needed 200M, but I > > don't > > need them anymore, so I want them released. What's unrealistic about that? > > What can I say: it's not realistic and I already gave you two technical > reasons why that is. What 2 reasons? The reason I had the paragraph below is that I was sure I answered to everything you said, so I have no idea what you mean. > > If that was your point about that "maybe the 200M of memory will be > > needed again", then as I said, yeah, maybe, maybe not — unless Glibc > > has a prophet module built-in, it can't know, so it should not behave > > as if it does. > > Maybe so.  But to the extent that you consciously decided it's OK for > Emacs to use 200MB at time-step A, then you basically lost the "moral > authority" to say that it's not OK for Emacs to still uses up 200MB at > time-step B. Okay, so, my point you replied to was that I allocated 200M when I needed them, and I am releasing them because I no longer need them. You are opposing that with I "lost the moral authority to say that it's not OK for Emacs to still uses up 200MB". "moral"…? Seriously? ☺ Was that some code of conduct I violated by allocating 200M, or what does mean "I lost moral authority"? Is that a technical reason you referred to? ☺ From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 25 17:17:37 2021 Received: (at 45200) by debbugs.gnu.org; 25 Jan 2021 22:17:37 +0000 Received: from localhost ([127.0.0.1]:40253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4AAz-0002mY-IR for submit@debbugs.gnu.org; Mon, 25 Jan 2021 17:17:37 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:43799) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4AAy-0002mR-Ed for 45200@debbugs.gnu.org; Mon, 25 Jan 2021 17:17:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611613056; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=hny0JyIpFNLC2eroe7W3VBtqdV9rRLifZWdm3cvfOiA=; b=g0yvkslXqYu8LrXqCPB/LKwArFoqvmsb0lyofsAfuuFgnglXiSKpdCJeikahnHq8Qq5q2m 8vOHHNYQOcjUPXV9BSWKng9v+Z3ctz6YcuNdyZ8YhNtP6Pnp8Go95yG9lR+iQEfJ5LEd/F fPzLZ4jYH3TQISuuBedl1iPvoldS4PU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-550-K-0WOXQ5NP-BflxqTQvmoA-1; Mon, 25 Jan 2021 17:17:34 -0500 X-MC-Unique: K-0WOXQ5NP-BflxqTQvmoA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C34168030A0; Mon, 25 Jan 2021 22:17:32 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8112160C62; Mon, 25 Jan 2021 22:17:29 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 10PMHRsj017497; Mon, 25 Jan 2021 17:17:27 -0500 From: DJ Delorie To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <834kj64a36.fsf@gnu.org> (message from Eli Zaretskii on Sun, 24 Jan 2021 17:40:29 +0200) Date: Mon, 25 Jan 2021 17:17:27 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org, Hi-Angel@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Eli Zaretskii writes: > Thanks, but is it really a good idea to call malloc_trim each time we > free some chunk of memory? malloc_trim() is very expensive relative to free(), partly because of what it needs to do, and partly because it flushes the fastbins cache. If you call it every, say, 1000 iterations of free, that might suffice. Or perhaps after each GC run. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 25 17:28:11 2021 Received: (at 45200) by debbugs.gnu.org; 25 Jan 2021 22:28:11 +0000 Received: from localhost ([127.0.0.1]:40273 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4ALD-00032f-Cx for submit@debbugs.gnu.org; Mon, 25 Jan 2021 17:28:11 -0500 Received: from forward103o.mail.yandex.net ([37.140.190.177]:43691) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4ALB-00032Q-Nz for 45200@debbugs.gnu.org; Mon, 25 Jan 2021 17:28:10 -0500 Received: from myt5-89d0765c5c65.qloud-c.yandex.net (myt5-89d0765c5c65.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3e1f:0:640:89d0:765c]) by forward103o.mail.yandex.net (Yandex) with ESMTP id 1739C5F8143F; Tue, 26 Jan 2021 01:28:03 +0300 (MSK) Received: from myt6-016ca1315a73.qloud-c.yandex.net (myt6-016ca1315a73.qloud-c.yandex.net [2a02:6b8:c12:4e0e:0:640:16c:a131]) by myt5-89d0765c5c65.qloud-c.yandex.net (mxback/Yandex) with ESMTP id dfxNVCLwcf-S2GiwwO8; Tue, 26 Jan 2021 01:28:03 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611613683; bh=QloOjdXFwtmNfZvGv58YywFTSj55xekC3unwjLSVW7s=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=nCliLCXNSgH4pKDsLr9KuiCKLYRhoNXoQUFckh/mQTljxl9z5NPqKBavspJutAAIT E8e4qf6smuLaaWQWJLgEGoyQK0u4B37JmxYYmdAeVLMb1CDVAGoS22RHyoPpHpg9YN bBLFqgWg2zwH1ehi4rAvGwtmwzIo6flrM8RsW9qY= Authentication-Results: myt5-89d0765c5c65.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt6-016ca1315a73.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id uCYM5eBz4z-S1JGJPFD; Tue, 26 Jan 2021 01:28:01 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Eli Zaretskii Date: Tue, 26 Jan 2021 01:28:01 +0300 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, 2021-01-25 at 17:17 -0500, DJ Delorie wrote: > Eli Zaretskii writes: > > Thanks, but is it really a good idea to call malloc_trim each time we > > free some chunk of memory? > > malloc_trim() is very expensive relative to free(), partly because of > what it needs to do, and partly because it flushes the fastbins cache. > If you call it every, say, 1000 iterations of free, that might suffice. > Or perhaps after each GC run. (un-ccing Glibc maintainers as this question is Emacs-specific) After a GC run, would that be at the end of `garbage_collect` function as Stefan mentioned? From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 26 09:49:50 2021 Received: (at 45200) by debbugs.gnu.org; 26 Jan 2021 14:49:50 +0000 Received: from localhost ([127.0.0.1]:41168 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4PfC-0008Dg-EX for submit@debbugs.gnu.org; Tue, 26 Jan 2021 09:49:50 -0500 Received: from eggs.gnu.org ([209.51.188.92]:34172) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4PfB-0008DU-4Q for 45200@debbugs.gnu.org; Tue, 26 Jan 2021 09:49:49 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51979) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l4Pf5-00076O-I9; Tue, 26 Jan 2021 09:49:43 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1810 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l4Pf4-0002JC-AX; Tue, 26 Jan 2021 09:49:43 -0500 Date: Tue, 26 Jan 2021 16:49:53 +0200 Message-Id: <83lfcf21ny.fsf@gnu.org> From: Eli Zaretskii To: DJ Delorie In-Reply-To: (message from DJ Delorie on Mon, 25 Jan 2021 17:17:27 -0500) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org, Hi-Angel@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: DJ Delorie > Cc: Hi-Angel@yandex.ru, 45200@debbugs.gnu.org, monnier@iro.umontreal.ca, > carlos@redhat.com, fweimer@redhat.com > Date: Mon, 25 Jan 2021 17:17:27 -0500 > > Eli Zaretskii writes: > > Thanks, but is it really a good idea to call malloc_trim each time we > > free some chunk of memory? > > malloc_trim() is very expensive relative to free(), partly because of > what it needs to do, and partly because it flushes the fastbins cache. How expensive? Can you give some quantitative measure? > If you call it every, say, 1000 iterations of free, that might suffice. > Or perhaps after each GC run. There's no guarantee that a full GC will call 'free', even though that's quite likely. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 26 09:56:01 2021 Received: (at 45200) by debbugs.gnu.org; 26 Jan 2021 14:56:01 +0000 Received: from localhost ([127.0.0.1]:42394 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4PlA-0000Dz-UV for submit@debbugs.gnu.org; Tue, 26 Jan 2021 09:56:01 -0500 Received: from eggs.gnu.org ([209.51.188.92]:35324) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4Pl9-0000Dt-Rl for 45200@debbugs.gnu.org; Tue, 26 Jan 2021 09:56:00 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52055) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l4Pl4-0000fX-0X; Tue, 26 Jan 2021 09:55:54 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2186 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l4Pku-0002hT-GW; Tue, 26 Jan 2021 09:55:46 -0500 Date: Tue, 26 Jan 2021 16:55:55 +0200 Message-Id: <83k0rz21dw.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: (message from Konstantin Kharlamov on Tue, 26 Jan 2021 01:28:01 +0300) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Konstantin Kharlamov > Cc: 45200@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Tue, 26 Jan 2021 01:28:01 +0300 > > On Mon, 2021-01-25 at 17:17 -0500, DJ Delorie wrote: > > Eli Zaretskii writes: > > > Thanks, but is it really a good idea to call malloc_trim each time we > > > free some chunk of memory? > > > > malloc_trim() is very expensive relative to free(), partly because of > > what it needs to do, and partly because it flushes the fastbins cache. > > If you call it every, say, 1000 iterations of free, that might suffice. > > Or perhaps after each GC run. > > (un-ccing Glibc maintainers as this question is Emacs-specific) Please don't, I invited them to these discussions because they can help us make the right decisions. > After a GC run, would that be at the end of `garbage_collect` function as Stefan > mentioned? That'd be easy to implement, but I'm not yet sure it's the best alternative. In particular, I'd like to have some idea regarding how much time such a call could take. In some usage patterns Emacs calls GC very frequently, which slows down command execution and makes Emacs less responsive. So much so that some people raise the GC threshold too high trying to avoid this slowdown, and raising the threshold too much is generally not a good idea. I don't want us to slow GC down even more so that more people would shoot themselves in the foot by raising the threshold. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 26 10:06:52 2021 Received: (at 45200) by debbugs.gnu.org; 26 Jan 2021 15:06:52 +0000 Received: from localhost ([127.0.0.1]:42403 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4Pvg-0000To-1h for submit@debbugs.gnu.org; Tue, 26 Jan 2021 10:06:52 -0500 Received: from forward100p.mail.yandex.net ([77.88.28.100]:42108) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4Pve-0000Ta-9P for 45200@debbugs.gnu.org; Tue, 26 Jan 2021 10:06:50 -0500 Received: from forward100q.mail.yandex.net (forward100q.mail.yandex.net [IPv6:2a02:6b8:c0e:4b:0:640:4012:bb97]) by forward100p.mail.yandex.net (Yandex) with ESMTP id 1C6FA5982E30; Tue, 26 Jan 2021 18:02:07 +0300 (MSK) Received: from vla1-38e2a3439e30.qloud-c.yandex.net (vla1-38e2a3439e30.qloud-c.yandex.net [IPv6:2a02:6b8:c0d:1e04:0:640:38e2:a343]) by forward100q.mail.yandex.net (Yandex) with ESMTP id 170877080009; Tue, 26 Jan 2021 18:02:07 +0300 (MSK) Received: from vla5-047c0c0d12a6.qloud-c.yandex.net (vla5-047c0c0d12a6.qloud-c.yandex.net [2a02:6b8:c18:3484:0:640:47c:c0d]) by vla1-38e2a3439e30.qloud-c.yandex.net (mxback/Yandex) with ESMTP id QRsqf9GJTi-26HCep6n; Tue, 26 Jan 2021 18:02:07 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1611673327; bh=7T7ov+06wul3Se1WJ6VbYBPxKeEgJev2apwiGeBedVY=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=ctMbI8JRJanaxkAu7B0XVzDXSd+iOMx0jQJdj+MwnUVA/Hp0Lniq354rYYHEl0Gxa HekwGlM+aAnfxewFopgEHLI3hMGaNWq7fp1FYeJ33kKMW8mS2JWl5FXgaC7jGqaWAx wbBVyfltQot7Y4RCUE0y7cuGc7dMtZtNecuM+Jhw= Authentication-Results: vla1-38e2a3439e30.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by vla5-047c0c0d12a6.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id 5eSjcoaa5t-26oiQcAq; Tue, 26 Jan 2021 18:02:06 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <6e6b41b5e9e78d4360cb2f90118af2e43ddba548.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Eli Zaretskii Date: Tue, 26 Jan 2021 18:02:06 +0300 In-Reply-To: <83k0rz21dw.fsf@gnu.org> References: <83k0rz21dw.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Tue, 2021-01-26 at 16:55 +0200, Eli Zaretskii wrote: > > From: Konstantin Kharlamov > > Cc: 45200@debbugs.gnu.org, monnier@iro.umontreal.ca > > Date: Tue, 26 Jan 2021 01:28:01 +0300 > > > > On Mon, 2021-01-25 at 17:17 -0500, DJ Delorie wrote: > > > Eli Zaretskii writes: > > > > Thanks, but is it really a good idea to call malloc_trim each time we > > > > free some chunk of memory? > > > > > > malloc_trim() is very expensive relative to free(), partly because of > > > what it needs to do, and partly because it flushes the fastbins cache. > > > If you call it every, say, 1000 iterations of free, that might suffice. > > > Or perhaps after each GC run. > > > > (un-ccing Glibc maintainers as this question is Emacs-specific) > > Please don't, I invited them to these discussions because they can > help us make the right decisions. Sorry, okay. > > After a GC run, would that be at the end of `garbage_collect` function as > > Stefan > > mentioned? > > That'd be easy to implement, but I'm not yet sure it's the best > alternative.  In particular, I'd like to have some idea regarding how > much time such a call could take.  In some usage patterns Emacs calls > GC very frequently, which slows down command execution and makes Emacs > less responsive.  So much so that some people raise the GC threshold > too high trying to avoid this slowdown, and raising the threshold too > much is generally not a good idea.  I don't want us to slow GC down > even more so that more people would shoot themselves in the foot by > raising the threshold. Yeah, that's true, there's an existing advice to make GC only run on idle. Which is the reason btw this bugreport eventually came to life ☺ From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 26 10:30:18 2021 Received: (at 45200) by debbugs.gnu.org; 26 Jan 2021 15:30:18 +0000 Received: from localhost ([127.0.0.1]:42435 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4QIM-00014V-Iy for submit@debbugs.gnu.org; Tue, 26 Jan 2021 10:30:18 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:34654) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4QIK-00014F-Aj for 45200@debbugs.gnu.org; Tue, 26 Jan 2021 10:30:16 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id EE5FD44051C; Tue, 26 Jan 2021 10:30:10 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id B37FF4404F2; Tue, 26 Jan 2021 10:30:09 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1611675009; bh=RkCy4eFPMJcPjd1BsQHNknBf+28LknmqecdxtG3fqC8=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=fN4RJg4S10tI8V3e/iV0TDMaSml/TXJcZLwwYsHEcAbEl/Bh6kbsOu3t/mNYSChyI TNw58MSWp79wDO7q0Hu9SUNgaRo4Qo/OIWIl+5bohgZxvjwcQdEfDXBtIBQhlLz3FZ /u02o6LtxFVfskGzaIG+qg+BTAysxpt4xVahdGdlcIc6DSyPGaR9W8HwyQierEKHCC JyhASs6vzTF5abrAqKCStx0qDrIQ+SMWKCOiqGoQ5zux4GXFH4290AA6f1SVnezKSr Y8/w+57ToFWFKllsVrmt9dvyBl1yDRl7QNXgcOTlWPmSAyLKKklaiB/u1SdWJYN6Hu B7KpRpGWDsATQ== Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6ED7812005E; Tue, 26 Jan 2021 10:30:09 -0500 (EST) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> Date: Tue, 26 Jan 2021 10:30:07 -0500 In-Reply-To: <83k0rz21dw.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 26 Jan 2021 16:55:55 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.077 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org, Konstantin Kharlamov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > That'd be easy to implement, but I'm not yet sure it's the best > alternative. In particular, I'd like to have some idea regarding how > much time such a call could take. In some usage patterns Emacs calls > GC very frequently, which slows down command execution and makes Emacs > less responsive. The most important case where this happens is during phases of creation of large new structures (e.g. while loading a large package like Org), where a lot of memory is allocated without generating much garbage, so every time we run the GC that time is mostly wasted (since we don't recover any garbage) and calling malloc_trim would also be useless in those cases. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 26 11:14:06 2021 Received: (at 45200) by debbugs.gnu.org; 26 Jan 2021 16:14:06 +0000 Received: from localhost ([127.0.0.1]:42499 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4Qyk-0002ES-B0 for submit@debbugs.gnu.org; Tue, 26 Jan 2021 11:14:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56369) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l4Qyi-0002EL-MN for 45200@debbugs.gnu.org; Tue, 26 Jan 2021 11:14:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611677644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=18ylx5cVCoIZ5hHAxqTuL0XUbDGXLdXZSeQxL5Ze2Nc=; b=MBh5TTIUGe4zUxRDWfiZXUasoWMywa5GJvDiOxE58PtCjDEmmgfgqmlHD/nGYL9RwJ8/6u fUITTPlKVWsF6kWKU1A3h9oRfYJsVev3Ys5HHiNv0nS0Fd9YraLZb9dznop/XCxHqhkILg zAJ6BoPJ6v+eaxLeEO+RBRaJmq1NBKs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-512-fzf6eqwSPDmDufsFaOy4Mg-1; Tue, 26 Jan 2021 11:13:59 -0500 X-MC-Unique: fzf6eqwSPDmDufsFaOy4Mg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4FC8010054FF; Tue, 26 Jan 2021 16:13:58 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 12F475F9A6; Tue, 26 Jan 2021 16:13:53 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 10QGDqr6029994; Tue, 26 Jan 2021 11:13:52 -0500 From: DJ Delorie To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <83lfcf21ny.fsf@gnu.org> (message from Eli Zaretskii on Tue, 26 Jan 2021 16:49:53 +0200) Date: Tue, 26 Jan 2021 11:13:52 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org, Hi-Angel@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Eli Zaretskii writes: >> malloc_trim() is very expensive relative to free(), partly because of >> what it needs to do, and partly because it flushes the fastbins cache. > > How expensive? Can you give some quantitative measure? malloc_trim() is to free() as garbage collection is to just dropping a reference. >> If you call it every, say, 1000 iterations of free, that might suffice. >> Or perhaps after each GC run. > > There's no guarantee that a full GC will call 'free', even though > that's quite likely. malloc_trim() is our garbage collector, if you call it after GC that would be "appropriate" at least ;-) From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 02 16:17:15 2021 Received: (at 45200) by debbugs.gnu.org; 2 Feb 2021 21:17:15 +0000 Received: from localhost ([127.0.0.1]:36281 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l732x-0003tp-1L for submit@debbugs.gnu.org; Tue, 02 Feb 2021 16:17:15 -0500 Received: from forward105j.mail.yandex.net ([5.45.198.248]:53230) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l732s-0003tZ-IZ for 45200@debbugs.gnu.org; Tue, 02 Feb 2021 16:17:14 -0500 Received: from iva7-553ca43bc648.qloud-c.yandex.net (iva7-553ca43bc648.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:2e99:0:640:553c:a43b]) by forward105j.mail.yandex.net (Yandex) with ESMTP id E935EB22401; Wed, 3 Feb 2021 00:17:03 +0300 (MSK) Received: from iva5-057a0d1fbbd8.qloud-c.yandex.net (iva5-057a0d1fbbd8.qloud-c.yandex.net [2a02:6b8:c0c:7f1c:0:640:57a:d1f]) by iva7-553ca43bc648.qloud-c.yandex.net (mxback/Yandex) with ESMTP id zU606MBpXS-H2HiaLJ4; Wed, 03 Feb 2021 00:17:03 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612300623; bh=qBTGpSobwMuMwMUZctn8IQ3WGGe6TbgGXH1NwEMfpnY=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=Uj4hdj0B+sCcXN4e8cV1GoMKPxibCpDjUIPvgaX7VCG53Fj+R9hZZ/G3mmg6mJwIK CUZ319vE06sP35/GZvWZTg/g3KSSsQ2J4NjUntgVTbQThzUH+dSyakHRYv6RjlCIaw CTrBKi1erdTWIjeuRRiaNdu0fsYjASTbxULZIMrA= Authentication-Results: iva7-553ca43bc648.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva5-057a0d1fbbd8.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id dJjaVGW32F-H2nOTHMe; Wed, 03 Feb 2021 00:17:02 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier , Eli Zaretskii Date: Wed, 03 Feb 2021 00:17:02 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> Content-Type: multipart/mixed; boundary="=-cKi0qJ2/NeN4hWbvfzJw" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-cKi0qJ2/NeN4hWbvfzJw Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit On Tue, 2021-01-26 at 10:30 -0500, Stefan Monnier wrote: > > That'd be easy to implement, but I'm not yet sure it's the best > > alternative.  In particular, I'd like to have some idea regarding how > > much time such a call could take.  In some usage patterns Emacs calls > > GC very frequently, which slows down command execution and makes Emacs > > less responsive. > > The most important case where this happens is during phases of creation > of large new structures (e.g. while loading a large package like Org), > where a lot of memory is allocated without generating much garbage, so > every time we run the GC that time is mostly wasted (since we don't > recover any garbage) and calling malloc_trim would also be useless in > those cases. So, I just did some benchmarking; the patch I bencharked it with was modified per Stefan's comment (i.e. `malloc_trim(0)` is moved to the end of `garbage_collec()`). I'm attaching it here as well. Benchamarking was done as follows: I ran built emacs as `src/emacs -Q`; and used this code (benchmark-run (let ((i 1000000)) (while (> i 0) (print "hello") (setq i (- i 1))))) which I ran 3 times. First with the patch; then I git-checked out HEAD^ (i.e. to the code without my patch), built it, and ran 3 times again. This code you may recognize, it was the reproducer to show Emacs taking 200M of memory, except I cut out the line disabling GC, and the line that calls GC manually. IOW I used a known memory-heavy testcase, and ran it with vanilla Emacs configuration. Results are: with malloc_trim: (8.920371394 232 2.106283245) (9.038083601 231 2.060810826) (9.140798641 231 2.0594013240000004) without malloc_trim: (8.987097209 232 2.070143482) (8.700478084 231 1.7745506179999997) (8.781121056 231 1.7870093610000004) The difference is just 3-4% (8.7 / 9 ≈ 0.9666666667). It looks to me insignificant enough to not show up anywhere during interactive work with Emacs. --=-cKi0qJ2/NeN4hWbvfzJw Content-Disposition: attachment; filename="1.patch" Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="1.patch"; charset="UTF-8" RnJvbSBiYzA5Y2U0ZTI1ZmYyYzM2MWQxOWFjZTM1ZTU1NmI3Mzg2NTFjMGEyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBLb25zdGFudGluIEtoYXJsYW1vdiA8SGktQW5nZWxAeWFuZGV4 LnJ1PgpEYXRlOiBTdW4sIDI0IEphbiAyMDIxIDE3OjU1OjIzICswMzAwClN1YmplY3Q6IFtQQVRD SF0gRm9yY2UgR2xpYmMgdG8gZnJlZSB0aGUgbWVtb3J5IGZyZWVkCgpjb25maWd1cmUuYWM6IGNo ZWNrIHdoZXRoZXIgbWFsbG9jX3RyaW0gaXMgc3Vwb3J0ZWQKc3JjL2FsbG9jLmMgKGdhcmJhZ2Vf Y29sbGVjdCk6IGNhbGwgbWFsbG9jX3RyaW0oKSBpZiBwb3NzaWJsZQooYnVnIzQ1MjAwKQotLS0K IGNvbmZpZ3VyZS5hYyB8IDMgKysrCiBzcmMvYWxsb2MuYyAgfCAzICsrKwogMiBmaWxlcyBjaGFu Z2VkLCA2IGluc2VydGlvbnMoKykKCmRpZmYgLS1naXQgYS9jb25maWd1cmUuYWMgYi9jb25maWd1 cmUuYWMKaW5kZXggMDhmM2MwY2Q4NS4uNDE2Y2ZiZTM4OSAxMDA2NDQKLS0tIGEvY29uZmlndXJl LmFjCisrKyBiL2NvbmZpZ3VyZS5hYwpAQCAtNDU1Niw2ICs0NTU2LDkgQEAgQUNfREVGVU4KIGRu bCB0aGUgY3VycmVudCBDRkxBR1MgZXRjLgogQUNfQ0hFQ0tfRlVOQ1Moc25wcmludGYpCiAKKwor QUNfQ0hFQ0tfRlVOQ1MobWFsbG9jX3RyaW0pCisKIGRubCBDaGVjayBmb3IgZ2xpYi4gIFRoaXMg ZGlmZmVycyBmcm9tIG90aGVyIGxpYnJhcnkgY2hlY2tzIGluIHRoYXQKIGRubCBFbWFjcyBuZWVk IG5vdCBsaW5rIHRvIGdsaWIgdW5sZXNzIHNvbWUgb3RoZXIgbGlicmFyeSBpcyBhbHJlYWR5CiBk bmwgbGlua2luZyB0byBnbGliLiAgQWx0aG91Z2ggZ2xpYiBwcm92aWRlcyBubyBmYWNpbGl0aWVz IHRoYXQgRW1hY3MKZGlmZiAtLWdpdCBhL3NyYy9hbGxvYy5jIGIvc3JjL2FsbG9jLmMKaW5kZXgg Yjg2ZWQ0ZWQyNi4uMzEyNDM3OGZiOCAxMDA2NDQKLS0tIGEvc3JjL2FsbG9jLmMKKysrIGIvc3Jj L2FsbG9jLmMKQEAgLTYxNDYsNiArNjE0Niw5IEBAIGdhcmJhZ2VfY29sbGVjdCAodm9pZCkKICAg ICAgIGlmICh0b3RfYWZ0ZXIgPCB0b3RfYmVmb3JlKQogCW1hbGxvY19wcm9iZSAobWluICh0b3Rf YmVmb3JlIC0gdG90X2FmdGVyLCBTSVpFX01BWCkpOwogICAgIH0KKyNpZmRlZiBIQVZFX01BTExP Q19UUklNCisgIG1hbGxvY190cmltKDApOyAvKiB3b3JrIGFyb3VuZCBmb3IgaGlnaCBtZW1vcnkg Y29uc3VtcHRpb24sIHNlZSBidWcgNDUyMDAgKi8KKyNlbmRpZiAvKiBIQVZFX01BTExPQ19UUklN ICovCiB9CiAKIERFRlVOICgiZ2FyYmFnZS1jb2xsZWN0IiwgRmdhcmJhZ2VfY29sbGVjdCwgU2dh cmJhZ2VfY29sbGVjdCwgMCwgMCwgIiIsCi0tIAoyLjMwLjAKCg== --=-cKi0qJ2/NeN4hWbvfzJw-- From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 02 23:45:29 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 04:45:29 +0000 Received: from localhost ([127.0.0.1]:36533 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7A2i-00063p-5a for submit@debbugs.gnu.org; Tue, 02 Feb 2021 23:45:29 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:41890) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7A2g-00063c-Fc for 45200@debbugs.gnu.org; Tue, 02 Feb 2021 23:45:27 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id BBC74101134; Tue, 2 Feb 2021 23:45:20 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 4AD581006F2; Tue, 2 Feb 2021 23:45:19 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612327519; bh=XnEwho14LfC8HXZMA28MlezTFKOTasLMEbB4JpSy2Ow=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=liHIVTcqQ1rkeH5fIEjhKQoGybI8pb5dr83k+cJ5eKT8e19plIcE9mHVPREn6AqrM +F+eMEWVichIO/sdFgNJ5DiDld1DgzX51hSB2UWFQHq/tRqHO1nzMqS79ZWprpJwPG FmD/I46jXFIUZMtpQ9GU8vOgote0CO2rFenpINmlIP/bg+Itfv2ulFR77BGh+P8Ets zbqzBPnAPTLaFW5GaWb7mO50RG1//rCnjApBXXfWrKwd0+dNddIXlRXC8Rjnkdq87F jJ0XhPu3O7m4uITohczcxldQ9WdpZrkGRE62HVxIb1JK23zfdULEnQJw58GLLGMbRq DvjAo1+SfGibg== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id C01181201F1; Tue, 2 Feb 2021 23:45:18 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> Date: Tue, 02 Feb 2021 23:45:17 -0500 In-Reply-To: <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> (Konstantin Kharlamov's message of "Wed, 03 Feb 2021 00:17:02 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.026 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, Eli Zaretskii , dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Results are: > > with malloc_trim: > (8.920371394 232 2.106283245) > (9.038083601 231 2.060810826) > (9.140798641 231 2.0594013240000004) > > without malloc_trim: > (8.987097209 232 2.070143482) > (8.700478084 231 1.7745506179999997) > (8.781121056 231 1.7870093610000004) > > The difference is just 3-4% (8.7 / 9 =E2=89=88 0.9666666667). It looks to= me > insignificant enough to not show up anywhere during interactive work > with Emacs. It's indeed not too costly, but what about the upside? Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 02 23:50:55 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 04:50:55 +0000 Received: from localhost ([127.0.0.1]:36538 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7A7z-0006CD-Li for submit@debbugs.gnu.org; Tue, 02 Feb 2021 23:50:55 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:8956) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7A7x-0006By-Mw for 45200@debbugs.gnu.org; Tue, 02 Feb 2021 23:50:54 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 518E180C19; Tue, 2 Feb 2021 23:50:48 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id C444B807A5; Tue, 2 Feb 2021 23:50:42 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612327842; bh=OZHixiDZqfoUdoJcxjfq9OdDtiwqDQX6Ls3kWZVsMI0=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=WwIO2+ScaUQtTWS0qca67/ECOYui6097je5AfQDUZDsch7JAp4QNItqpsME451pOm X1JLa6hSksPodYG1XDdPYrdIgk0SQ1MxkU1UnAiwk9LdO0YQquQLf9WBeMyE4YBffC BuqW1hqIkSgxbVNF93u3rbNK2mGF68efRSW1cq/++8A4gygeQBegfkZHbJcLxYOcpZ iv/t96mGum37JErZL9Xo0zHpeog9DHUHCFE+Mz3nac4oMrmMKaAUJj+02nESQubdjQ UANrBExIIJgPTUBhX8zb+uktRk55N+PAU3BqNjTWcFAUz5L/Nd7tf8LSekyITYahNB DjBCyQWfh8s2w== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6CCC312005C; Tue, 2 Feb 2021 23:50:42 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> Date: Tue, 02 Feb 2021 23:50:41 -0500 In-Reply-To: (Stefan Monnier's message of "Tue, 02 Feb 2021 23:45:17 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.008 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, Eli Zaretskii , dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> with malloc_trim: >> (8.920371394 232 2.106283245) >> (9.038083601 231 2.060810826) >> (9.140798641 231 2.0594013240000004) >> >> without malloc_trim: >> (8.987097209 232 2.070143482) >> (8.700478084 231 1.7745506179999997) >> (8.781121056 231 1.7870093610000004) >> >> The difference is just 3-4% (8.7 / 9 =E2=89=88 0.9666666667). It looks t= o me >> insignificant enough to not show up anywhere during interactive work >> with Emacs. > > It's indeed not too costly, but what about the upside? BTW, maybe a better way forward than trying to convince us that it's a good default (which could be hard if the upside is a reduction of the memory use by a few percent: for many people it might be less relevant/noticeable than the corresponding few percents lost in speed) is to provide a patch that adds a new *ELisp* function that calls `malloc_trim`, which you can then add to `post-gc-hook` in your init file when your usage pattern makes it useful. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 01:04:34 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 06:04:34 +0000 Received: from localhost ([127.0.0.1]:36574 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7BHF-0007wk-Pf for submit@debbugs.gnu.org; Wed, 03 Feb 2021 01:04:34 -0500 Received: from forward103p.mail.yandex.net ([77.88.28.106]:49812) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7BHB-0007wT-8f for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 01:04:32 -0500 Received: from iva5-d37fe543de32.qloud-c.yandex.net (iva5-d37fe543de32.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:6a6:0:640:d37f:e543]) by forward103p.mail.yandex.net (Yandex) with ESMTP id 728BA18C0160; Wed, 3 Feb 2021 09:04:22 +0300 (MSK) Received: from iva8-174eb672ffa9.qloud-c.yandex.net (iva8-174eb672ffa9.qloud-c.yandex.net [2a02:6b8:c0c:b995:0:640:174e:b672]) by iva5-d37fe543de32.qloud-c.yandex.net (mxback/Yandex) with ESMTP id dEruPcMapr-4LIO8n5g; Wed, 03 Feb 2021 09:04:22 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612332262; bh=3uIUWppCNTatPRpTltII+niAeTdkH3p+YUqqd5GRy5s=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=Ts94VpL9wh9aFMBElmq5qnLtDo3/35WXSYGPG35vxgHepBSsanyGiugctoI0cqpKV qo1cpWVVPwuWXMI5meTboqUj0LfC6YAeiypeYciwbqKUByewNyaIZmrFAHNJhUZnKQ OPBHJIoPu3rBA7pkoAgRd02FZRcTORvjmgmycmXM= Authentication-Results: iva5-d37fe543de32.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva8-174eb672ffa9.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id 23k5Lf1a3K-4LoS21np; Wed, 03 Feb 2021 09:04:21 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier Date: Wed, 03 Feb 2021 09:04:21 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, Eli Zaretskii , dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Tue, 2021-02-02 at 23:50 -0500, Stefan Monnier wrote: > > >     with malloc_trim: > > >         (8.920371394 232 2.106283245) > > >         (9.038083601 231 2.060810826) > > >         (9.140798641 231 2.0594013240000004) > > > > > >     without malloc_trim: > > >         (8.987097209 232 2.070143482) > > >         (8.700478084 231 1.7745506179999997) > > >         (8.781121056 231 1.7870093610000004) > > > > > > The difference is just 3-4% (8.7 / 9 ≈ 0.9666666667). It looks to me > > > insignificant enough to not show up anywhere during interactive work > > > with Emacs. > > > > It's indeed not too costly, but what about the upside? > > BTW, maybe a better way forward than trying to convince us that it's > a good default (which could be hard if the upside is a reduction of the > memory use by a few percent: for many people it might be less > relevant/noticeable than the corresponding few percents lost in speed) > is to provide a patch that adds a new *ELisp* function that calls > `malloc_trim`, which you can then add to `post-gc-hook` in your init > file when your usage pattern makes it useful. Upside indeed is the memory reduction. Well, I didn't send the patches for my only benefit, but for benefit of others people. The new ELisp function is something that not too many people would benefit from, and I mean, including those who disable GC. That is because it would be an opt-in feature, which you need to know about to enable it, and not many will ever find out about it. For my-only benefit I could just continue building Emacs with my patch applied locally, as I do now. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 02:07:50 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 07:07:50 +0000 Received: from localhost ([127.0.0.1]:36608 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7CGS-0001DM-Td for submit@debbugs.gnu.org; Wed, 03 Feb 2021 02:07:49 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7CGO-0001D3-Ku for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 02:07:47 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37181) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l7CGI-0000aJ-3x; Wed, 03 Feb 2021 02:07:38 -0500 Received: from [2a02:14f:80:8d61::1] (port=45338) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1l7CGH-0002do-0h; Wed, 03 Feb 2021 02:07:37 -0500 Date: Wed, 03 Feb 2021 09:07:35 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier From: Eli Zaretskii Message-ID: <67DDB032-C3DF-4748-A3EF-EEC374EDB251@gnu.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On February 3, 2021 8:04:21 AM GMT+02:00, Konstantin Kharlamov wrote: > On Tue, 2021-02-02 at 23:50 -0500, Stefan Monnier wrote: > > > > =C2=A0=C2=A0=C2=A0 with malloc_trim: > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (8=2E920371394 232 2=2E= 106283245) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (9=2E038083601 231 2=2E= 060810826) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (9=2E140798641 231 2=2E= 0594013240000004) > > > >=20 > > > > =C2=A0=C2=A0=C2=A0 without malloc_trim: > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (8=2E987097209 232 2=2E= 070143482) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (8=2E700478084 231 1=2E= 7745506179999997) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (8=2E781121056 231 1=2E= 7870093610000004) > > > >=20 > > > > The difference is just 3-4% (8=2E7 / 9 =E2=89=88 0=2E9666666667)= =2E It looks > to me > > > > insignificant enough to not show up anywhere during interactive > work > > > > with Emacs=2E > > >=20 > > > It's indeed not too costly, but what about the upside? > >=20 > > BTW, maybe a better way forward than trying to convince us that it's > > a good default (which could be hard if the upside is a reduction of > the > > memory use by a few percent: for many people it might be less > > relevant/noticeable than the corresponding few percents lost in > speed) > > is to provide a patch that adds a new *ELisp* function that calls > > `malloc_trim`, which you can then add to `post-gc-hook` in your init > > file when your usage pattern makes it useful=2E >=20 > Upside indeed is the memory reduction=2E How much memory reduction? Can you show the numbers? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 02:15:30 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 07:15:31 +0000 Received: from localhost ([127.0.0.1]:36614 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7CNu-0001Rq-JM for submit@debbugs.gnu.org; Wed, 03 Feb 2021 02:15:30 -0500 Received: from forward100j.mail.yandex.net ([5.45.198.240]:57726) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7CNq-0001RX-Mj for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 02:15:28 -0500 Received: from sas1-46fc755e9ad1.qloud-c.yandex.net (sas1-46fc755e9ad1.qloud-c.yandex.net [IPv6:2a02:6b8:c08:168c:0:640:46fc:755e]) by forward100j.mail.yandex.net (Yandex) with ESMTP id 97AB950E3DB2; Wed, 3 Feb 2021 10:15:20 +0300 (MSK) Received: from sas1-37da021029ee.qloud-c.yandex.net (sas1-37da021029ee.qloud-c.yandex.net [2a02:6b8:c08:1612:0:640:37da:210]) by sas1-46fc755e9ad1.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 2UQ3eyOtp1-FJHaR8mv; Wed, 03 Feb 2021 10:15:20 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612336520; bh=xxpteFaYoJ1GJKNf8TzYXqW77FqeCJ1VsCEyYZ4Bxls=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=mj0JerLQSj6NJuS6WeNmgblyDjCDGWEr3nFLxTcdMiYcshSBXpVsfBS025wUXPCBn nYb8n1BGc56c3fzRY1CIMshQwm4zJ4a5Sq9l8rCZVQSzdVNBXcNl+dY3JbcytdXSGS aKBs8g0pSh1eKXKRaxOrpOmGphpsXzYwNIFTKeO0= Authentication-Results: sas1-46fc755e9ad1.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-37da021029ee.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id a6ThhMwydP-FJnafNgq; Wed, 03 Feb 2021 10:15:19 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Eli Zaretskii , Stefan Monnier Date: Wed, 03 Feb 2021 10:15:19 +0300 In-Reply-To: <67DDB032-C3DF-4748-A3EF-EEC374EDB251@gnu.org> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <67DDB032-C3DF-4748-A3EF-EEC374EDB251@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, 2021-02-03 at 09:07 +0200, Eli Zaretskii wrote: > On February 3, 2021 8:04:21 AM GMT+02:00, Konstantin Kharlamov > wrote: > > On Tue, 2021-02-02 at 23:50 -0500, Stefan Monnier wrote: > > > > >     with malloc_trim: > > > > >         (8.920371394 232 2.106283245) > > > > >         (9.038083601 231 2.060810826) > > > > >         (9.140798641 231 2.0594013240000004) > > > > > > > > > >     without malloc_trim: > > > > >         (8.987097209 232 2.070143482) > > > > >         (8.700478084 231 1.7745506179999997) > > > > >         (8.781121056 231 1.7870093610000004) > > > > > > > > > > The difference is just 3-4% (8.7 / 9 ≈ 0.9666666667). It looks > > to me > > > > > insignificant enough to not show up anywhere during interactive > > work > > > > > with Emacs. > > > > > > > > It's indeed not too costly, but what about the upside? > > > > > > BTW, maybe a better way forward than trying to convince us that it's > > > a good default (which could be hard if the upside is a reduction of > > the > > > memory use by a few percent: for many people it might be less > > > relevant/noticeable than the corresponding few percents lost in > > speed) > > > is to provide a patch that adds a new *ELisp* function that calls > > > `malloc_trim`, which you can then add to `post-gc-hook` in your init > > > file when your usage pattern makes it useful. > > > > Upside indeed is the memory reduction. > > How much memory reduction?  Can you show the numbers? Sure. When I run the testcase for the problem that I posted in 3rd email on this thread¹, vanilla Emacs takes up 260.3M of PSS and never goes below, not even upon re-running (garbage-collect) manually. The patched Emacs on the other hand has 64.3M of PSS right after the testcase finishes. 1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45200#11 From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 02:40:02 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 07:40:03 +0000 Received: from localhost ([127.0.0.1]:36633 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Cle-000244-M9 for submit@debbugs.gnu.org; Wed, 03 Feb 2021 02:40:02 -0500 Received: from mout.gmx.net ([212.227.17.22]:33421) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Clc-00023W-Qz for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 02:40:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612337990; bh=K5fejfNmO6xd3ohLKXviF/8V5lP8eMPnuR8jht7YxZw=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=FAvCx8biLwyAfba2ZHAIFJ7XzYn3I5IzOSXMgDcHUsbeyl5QfrxA4zBEbhHshJ8qJ 0hgYmqPAMJkMxvUVm1jPE7HfY0IvNKKyNYiPQqu/i5soN2RX+Ta10P897skN37SSlY C8vFdiNKQeNd9bUYuodfwRoz4HqPAumgxMuNPdCU= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.1.100] ([46.125.249.123]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MGz1f-1lBLNN1qct-00E7Fh; Wed, 03 Feb 2021 08:39:50 +0100 Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> From: martin rudalics Message-ID: <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> Date: Wed, 3 Feb 2021 08:39:48 +0100 MIME-Version: 1.0 In-Reply-To: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:jpYzOR6MmOWbH51cp+h1AgKIG5HcVLfPW30ikzHXB6epZOLLW+O ovrqBwzilRIRNxxwguWnSzvgtjIMoQ2wPd28Zr/TvFi+szui4N/LKvqsbszFsdCVkfCOlXQ wwEReFXv8qAWFfvl4A/8Dv4HhmIqXuoVc3fkEbc8XjZNUTxJo35+oWDkOSGdQ4nhm7SaGHy RrDZsqTvrOV3meTDsxNgQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:lR4ulWz6NWc=:5ITJoSbrKiJig8qkQQLiNs T4EueZdSBAJMySLrKIm6oZGbnSw4j7gLbCEJrV7zwBQKE5Thb1qZA4knZ8LVmR45/K60tH04K M6K7SQOwEkb6GKmVVJUHgEEzOl+tTIoMBfvWXijUpSN/j6vETeDS/fhumvaOHcIBU0ar01zVE frG8fNtbu4h3rsU0LfjAqKCZF+xu8AChq5OMh25/EqMAmk0fNZlwy8GGFyYh82SK940F3TqzZ 7XsBpD4wrOaDXxHNolQK1oiTRm7zYu6y46y+/35BxhTjpBaw9HHgsygx1S50WDENHPRo9YiZ/ RDXdciROL1CD4+jQyBrq6JXpGe1yHz94i0kR1tbh/hW7G9PvKPTcV5Hkt+HiaR//oxIYkGH4P EDtQYv60DXxpI8rZucmKy7hI6Wm+784yVStXXII3hFTI0RCmgGaf8C2WRUv21oNiKiWMDzg/w 9995P4lIRS+/YmS+JTAtoo67wF3Ktqh4v1L6klayGtOhgxv3cw9rXVFxSFHe1Q8SXNaBFApvt 5DkoRDy8GHaA7uzWmVRy32OcAUNyJ0qVYTs4HPUksBZlzduBSlZiaZN62BHEZaeMNCjRr4SlQ 8T+31QntBq3ilen2IbxJWOWHeZuMhJm56dgdkjU6PfU+gU6vmsh8LZvaUQUtZZ70yVfh2WBic KY0+16/LQi6tM90WDwYc5nOW31rOhZXtTInv+4oKtrVN778wET22IuzNWVS8nBUqwAy07K9jR gC7c8BqjktfkWPY8MQSAdXQpa9qxNdwKIbCOZl8M5x+nldWs4zlILn7zHBPxCP/4Ze+gaIXQj zqjUuo+Rh7DSf2WZUzDh77fV0nFCtnheC7sg0TSmIVIZ7ipDH3yqfgk051V5qPV3WxEzaOyOY E09ZHO8+UrZCEYeg4GDQ== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Well, I didn't send the patches for my only benefit, but for benefit > of others people. The new ELisp function is something that not too > many people would benefit from, and I mean, including those who > disable GC. Does anyone really disable GC and get away with it? > That is because it would be an opt-in feature, which you > need to know about to enable it, and not many will ever find out about > it. I wouldn't be afraid of that. In the past months we had a sufficient number of candidates caring about the memory consumption of their Emacs sessions. An opt-out feature OTOH would get us soon a couple of people who decide that your feature is responsible for some new slowness and recommend to throw it out. So making this an opt-in feature and at the same time giving some recommendations of what users can do when they detect that their sessions consumes more memory than expected would be a finely balanced solution IMO. martin From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 03:24:06 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 08:24:06 +0000 Received: from localhost ([127.0.0.1]:36689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7DSC-0003Ey-Cf for submit@debbugs.gnu.org; Wed, 03 Feb 2021 03:24:06 -0500 Received: from forward104o.mail.yandex.net ([37.140.190.179]:43303) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7DS8-0003Ei-Cd for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 03:23:59 -0500 Received: from iva1-c9520d34ceae.qloud-c.yandex.net (iva1-c9520d34ceae.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:7695:0:640:c952:d34]) by forward104o.mail.yandex.net (Yandex) with ESMTP id 045BB940395; Wed, 3 Feb 2021 11:23:49 +0300 (MSK) Received: from iva6-2d18925256a6.qloud-c.yandex.net (iva6-2d18925256a6.qloud-c.yandex.net [2a02:6b8:c0c:7594:0:640:2d18:9252]) by iva1-c9520d34ceae.qloud-c.yandex.net (mxback/Yandex) with ESMTP id pf9sBNAi9G-NmHOlUJi; Wed, 03 Feb 2021 11:23:48 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612340628; bh=ogWAbhvGPHRYWhSknX5xBVnKdlKTxHAEA7+IJSW8cPA=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=g0cbTTEdz8khg2oQi7HwxUirihf1Nx1FWT3PG8L+KZDH/GyA6PMnCFY31y+ThHK1d RhGJBfnZbM1tkMLEassdhLehloczdtZmNPjLqdu5a5eXQbRiWMB2x33TeHUYkYmeEL ksQqjcnLdNSfRU5jKMjO0JimemWKAWPBt71Xf/j4= Authentication-Results: iva1-c9520d34ceae.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva6-2d18925256a6.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id bbVdZmRHjM-Nlo4oFVg; Wed, 03 Feb 2021 11:23:47 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 11:23:43 +0300 In-Reply-To: <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 08:39 +0100, martin rudalics wrote: >  > Well, I didn't send the patches for my only benefit, but for benefit >  > of others people. The new ELisp function is something that not too >  > many people would benefit from, and I mean, including those who >  > disable GC. > > Does anyone really disable GC and get away with it? Sure. There's for example this post¹, which is probably where I got the idea to only run GC on idle from. The question is highly upvoted, it has 4k views, and the only positive answer basically says "it's okay as long as you got memory" (well, I got memory), and notes that there's also a point in only disabling GC during startup. Similar advice to only disable it during startup has this pretty upvoted reddit post² Here's another example³, except in this case the author only disables GC while they're inside minibuffer, and enables it back upon exiting. This then got propogated here as well⁴ On top of that, there are countless advices on just increasing the `gc-cons-threshold` (not inifitely increasing, just making some sane larger value), that would surely reinforce an idea of just making GC run only on idle, if one ever comes across such advice or just figures it out. >  > That is because it would be an opt-in feature, which you >  > need to know about to enable it, and not many will ever find out about >  > it. > > I wouldn't be afraid of that.  In the past months we had a sufficient > number of candidates caring about the memory consumption of their Emacs > sessions.  An opt-out feature OTOH would get us soon a couple of people > who decide that your feature is responsible for some new slowness and > recommend to throw it out.  So making this an opt-in feature and at the > same time giving some recommendations of what users can do when they > detect that their sessions consumes more memory than expected would be a > finely balanced solution IMO. Well, I have 2 things to say on that. First of, so far the impact seemed to be small. If one ever comes to blame the new feature, they surely should have actual measurements to support that hypothesis. Second, most importantly, what `malloc_trim(0)` does is it restores the correct behavior. I mean, what's the point of ever freeing memory if it's not freed, right? The problem we're dealing here with is an actual bug in glibc⁵. What this implies is that if the fix indeed hurts performance someplace, well, then it's that this place requires additional performance-related fixes. As opposed to just ignoring the bug because of performance got somewhere decreased. Things like, changing the slow algorithm, or modifying GC behavior for specific usecases… It is the general rule that correct behavior rarely correlates with good performance. You might for example omit locks in thread-heavy application, and you get better performance at cost of thread races. Things like that. --------- 1: https://emacs.stackexchange.com/questions/34342/is-there-any-downside-to-setting-gc-cons-threshold-very-high-and-collecting-ga 2: https://www.reddit.com/r/emacs/comments/4j828f/til_setq_gcconsthreshold_100000000/d34gbsp/?context=3 3: https://bling.github.io/blog/2016/01/18/why-are-you-changing-gc-cons-threshold/ 4: https://javaguirre.me/2016/10/19/fixing-my-performance-problems-on-emacs 5: https://sourceware.org/bugzilla/show_bug.cgi?id=27103 From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 04:36:06 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 09:36:06 +0000 Received: from localhost ([127.0.0.1]:36825 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7EZy-0005H9-EY for submit@debbugs.gnu.org; Wed, 03 Feb 2021 04:36:06 -0500 Received: from mout.gmx.net ([212.227.17.22]:50221) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7EZu-0005GW-OM for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 04:36:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612344950; bh=Hi++wHhZeWRRvBpXCMBZF86eqHyManfK4/IPeCJWUu8=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=AqbTVPyI5YIISXIxFHnaRjyWDkw9fFvfRypOQ+E1gdqnOkDHUu0MqqN453IKOlnyV b9JdaWbU5xQSVqHpL8TRVX3eJmP4is0wrgE7LL1Cy1zGVMbfvAEfwB5IbbG51UFSjH 65AwnFF6LScC2zMuAUZ41bHy6/aNIomRgMfa+rwc= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.1.100] ([46.125.249.123]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MKKUp-1lNNJh001H-00LpGy; Wed, 03 Feb 2021 10:35:50 +0100 Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> From: martin rudalics Message-ID: <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> Date: Wed, 3 Feb 2021 10:35:48 +0100 MIME-Version: 1.0 In-Reply-To: <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:0S+9JcfZRkETxrIH/o4siBzOp++Q549ANPfReUxUpzGrGeIxFY3 58AAa6fbY0u2lKQlBgnLPCh2N+gPq3Ll350Hbf0yNl8L5GtxGIePE3mFol92L2fOiGWdsBE r2Q72TIkBg50WlraNpLtHq9P+F1J6RXHLNMDNYbVVSaa5ryAfWhy9Ibxa+iwZUzWC4I8MwP 1/k0t4xVWUl3cHLgAEh1g== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:SokkLOwCKdE=:ZjOJRe/ymgQEHLc0w+3wt3 YOBYVmJU1jWh9V4u5jq83uDK4XWxe3Z9pac6KPCNI8VovhbsHR54ih6XR/u1Tw/c+UhCSn57Y 3V2nAJZLI58Bd8S6px1Ksz22ACLo0ClMtcKLHChReN4ocmS3upNqAlo+LzWjbCDw4S6IdVO3O X069wVhD1JtW/bhs/KEYVyOzRIw8fr2DJiUAnwOQUPijUbdZsO41BOXTnlF6Lk7fe8wbq8eBh HoxptD3uE36jgngB1mvZnflzP7gyfk1LZ+UienzG71vECcdy0pYSDZeeIU4O1vzReh0gQbTrw V+r3t8hvRHhVnYFFj46B7fL2on4fHf5k/NYL3GvnevPsxUcbDnPVmt8muJU/cahTvS85OU9Ew /aQB00JorvOTJXrctPLvTNOchEPxBfkNejZ0oEJ9IhaPc5yusZ+9tX4p4MIXuATCzgyhptE7Y geehNmf51wnPcMBFrSrPLQ3nuqmd/kDqknV3JXUX4nOWgWLMXJYgxP9QPW7ScHA5d3hq8S9oT 40X7c52VJMkH4rwwZzKC3in7IuA9UZZ7mNwMDi4Y5D1KIJQ6BSnpUTdqEpfPKt5+Y8BopJRSx TaH1Xx94uzzMJhPm2WXGEYbLNBQIeLEqIGoH1L+nVprK15yJYwnZe1+sfK5MPg94imQZL4a35 z+8waSyIeahdEBuURCZobuXy7iLS9FsqG9VUqY4v+yfFLZgn3VMxjSKClz9zR5WN9Sog75bID BWgy9LqOl8rqTVzhndKf6THVPOCB5vSjRRl8QhSMYQqqy9mFyV+6YFuG3QrQ9N4vrlR9ueRLU xXpQLgBd1CNKOUhAH3psD2/MAU0CsaUO/q6FTnSGZDeIDEiomytfbwov3jWEc5JFDo7nc7S2Q cyEjuus4nuEv0rY4YKAg== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> Does anyone really disable GC and get away with it? > > Sure. There's for example this post=C2=B9, which is probably where I g= ot the idea to only run GC on idle from. The question is highly upvoted, = it has 4k views, and the only positive answer basically says "it's okay a= s long as you got memory" (well, I got memory), and notes that there's al= so a point in only disabling GC during startup. Similar advice to only di= sable it during startup has this pretty upvoted reddit post=C2=B2 > > Here's another example=C2=B3, except in this case the author only disa= bles GC while they're inside minibuffer, and enables it back upon exiting= =2E This then got propogated here as well=E2=81=B4 > > On top of that, there are countless advices on just increasing the `gc= -cons-threshold` (not inifitely increasing, just making some sane larger = value), that would surely reinforce an idea of just making GC run only on= idle, if one ever comes across such advice or just figures it out. What I meant was if people really disabled GC for the rest of their session and got away with it. But that was only a rhetorical question to which the answer is clearly no. All the examples you cite have a culprit - an application that produces too much garbage. Identifying those applications would be much better than working around them by disabling GC, for example, while a user is in a minibuffer dialog. That latter case even must have an easy to identify responsible, just that nobody cares. And remember that every cons eventually collected must have been allocated first. We always pay twice here. > First of, so far the impact seemed to be small. If one ever comes to > blame the new feature, they surely should have actual measurements to > support that hypothesis. Hardly. Sometimes we're lucky as in the "The Emacs master is much slower than the emacs-27 branch" thread. More often we're not. > Second, most importantly, what `malloc_trim(0)` does is it restores > the correct behavior. I mean, what's the point of ever freeing memory > if it's not freed, right? The problem we're dealing here with is an > actual bug in glibc=E2=81=B5. What this implies is that if the fix ind= eed > hurts performance someplace, well, then it's that this place requires > additional performance-related fixes. As opposed to just ignoring the > bug because of performance got somewhere decreased. Things like, > changing the slow algorithm, or modifying GC behavior for specific > usecases=E2=80=A6 The important use case IMO is: - A user detects, maybe after many hours of use, that memory consumption increases. - The user switches on your option and if memory consumption now decreases, there's at least a first workaround. Personally, I never care about Emacs consuming RAM. On my machine, Firefox dwarfs everything else in this regard. martin From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 04:50:01 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 09:50:01 +0000 Received: from localhost ([127.0.0.1]:36844 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7EnQ-0005f3-IN for submit@debbugs.gnu.org; Wed, 03 Feb 2021 04:50:01 -0500 Received: from forward106p.mail.yandex.net ([77.88.28.109]:39737) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7EnN-0005el-LK for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 04:49:59 -0500 Received: from sas1-714ad325b0ec.qloud-c.yandex.net (sas1-714ad325b0ec.qloud-c.yandex.net [IPv6:2a02:6b8:c08:112c:0:640:714a:d325]) by forward106p.mail.yandex.net (Yandex) with ESMTP id 222CC1C80683; Wed, 3 Feb 2021 12:49:50 +0300 (MSK) Received: from sas1-e00c2743cdb8.qloud-c.yandex.net (sas1-e00c2743cdb8.qloud-c.yandex.net [2a02:6b8:c14:3a22:0:640:e00c:2743]) by sas1-714ad325b0ec.qloud-c.yandex.net (mxback/Yandex) with ESMTP id rwhwOzud1B-nmHGVtaR; Wed, 03 Feb 2021 12:49:49 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612345789; bh=MkuvrcRIpH+qDLQ8GqLsMJC8db38y2ETm738RKT3H5w=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=qD6Jvxp34IOcrsANx2LOxmo3KFh4v1xapP4zsmyV3tMSHXJUdnFYAAsWDcfuukJZj Cgbchd8K8j/UjfMgGpNfjpy4DKys4EpRV484a3PxTcW/TnNNjBPTBhbL8ZvB5cQ+RC X/dP/Lc6ly3Fw/fZMeAw4WrlTU7FTvp0L2BNVGRg= Authentication-Results: sas1-714ad325b0ec.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-e00c2743cdb8.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id GVfKEJa6IS-nmnOEAgf; Wed, 03 Feb 2021 12:49:48 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 12:49:47 +0300 In-Reply-To: <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 10:35 +0100, martin rudalics wrote: >  >> Does anyone really disable GC and get away with it? >  > >  > Sure. There's for example this post¹, which is probably where I got the > idea to only run GC on idle from. The question is highly upvoted, it has 4k > views, and the only positive answer basically says "it's okay as long as you > got memory" (well, I got memory), and notes that there's also a point in only > disabling GC during startup. Similar advice to only disable it during startup > has this pretty upvoted reddit post² >  > >  > Here's another example³, except in this case the author only disables GC > while they're inside minibuffer, and enables it back upon exiting. This then > got propogated here as well⁴ >  > >  > On top of that, there are countless advices on just increasing the `gc- > cons-threshold` (not inifitely increasing, just making some sane larger > value), that would surely reinforce an idea of just making GC run only on > idle, if one ever comes across such advice or just figures it out. > > What I meant was if people really disabled GC for the rest of their > session and got away with it.  But that was only a rhetorical question > to which the answer is clearly no.  I'm not sure where you got this answer from. The first link I referred to has a user which does exactly that, and it is a highly upvoted question with 4k views. To me that seem to imply the answer is "yes". > All the examples you cite have a > culprit - an application that produces too much garbage.  Identifying > those applications would be much better than working around them by > disabling GC, for example, while a user is in a minibuffer dialog.  That > latter case even must have an easy to identify responsible, just that > nobody cares.  Sure, I completely agree with you. But that is orthogonal to the problem of free()ed memory not actually being freed. > And remember that every cons eventually collected must > have been allocated first.  We always pay twice here. First of, glibc has per-thread cache (since 2.26 version, I think), specifically for the usecase of apps allcating and freeing memory too often. The cache is not affected by `malloc_trim()`, I was assured by people on #glibc IRC channel on that. Second, if Emacs indeed sees it's gonna allocate memory again right away, then it shouldn't have even freed the memory in the first place. No free — no malloc_trim(). From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 05:35:48 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 10:35:48 +0000 Received: from localhost ([127.0.0.1]:36906 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7FVj-0006wu-Qm for submit@debbugs.gnu.org; Wed, 03 Feb 2021 05:35:48 -0500 Received: from forward100j.mail.yandex.net ([5.45.198.240]:39382) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7FVg-0006wa-DT for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 05:35:47 -0500 Received: from forward102q.mail.yandex.net (forward102q.mail.yandex.net [IPv6:2a02:6b8:c0e:1ba:0:640:516:4e7d]) by forward100j.mail.yandex.net (Yandex) with ESMTP id 1A36250E05EC; Wed, 3 Feb 2021 13:35:37 +0300 (MSK) Received: from vla1-9b89fd466824.qloud-c.yandex.net (vla1-9b89fd466824.qloud-c.yandex.net [IPv6:2a02:6b8:c0d:1a03:0:640:9b89:fd46]) by forward102q.mail.yandex.net (Yandex) with ESMTP id 132B63A2000D; Wed, 3 Feb 2021 13:35:37 +0300 (MSK) Received: from vla5-047c0c0d12a6.qloud-c.yandex.net (vla5-047c0c0d12a6.qloud-c.yandex.net [2a02:6b8:c18:3484:0:640:47c:c0d]) by vla1-9b89fd466824.qloud-c.yandex.net (mxback/Yandex) with ESMTP id zZ7s4QBKr2-ZaHa9ACd; Wed, 03 Feb 2021 13:35:37 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612348537; bh=+6b3074yt5N2l/19TRrhwBE5Zn8h9Ez9mvgzF1u2uOI=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=CDiFsR9ggE5N+uuEvoQ3h6JWXvyaG8G9aUVwcMGXAVRWa7PwckjPl1HyI5q0l0vEg CFQZVlmzZTtVGey7FO4E5T/QcFfyo0wWjoRK/60VwLi8ZqkxH1DHuk7OOzRlPg2cq4 kvcP7c6ey0yzkmPDJYXS4i1FvNBlBJx/JuI6ADNw= Authentication-Results: vla1-9b89fd466824.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by vla5-047c0c0d12a6.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id gafDcr98Y0-ZZnuGfYj; Wed, 03 Feb 2021 13:35:35 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 13:35:35 +0300 In-Reply-To: <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, 2021-02-03 at 12:49 +0300, Konstantin Kharlamov wrote: > On Wed, 2021-02-03 at 10:35 +0100, martin rudalics wrote: > >  All the examples you cite have a > > culprit - an application that produces too much garbage.  Identifying > > those applications would be much better than working around them by > > disabling GC, for example, while a user is in a minibuffer dialog.  That > > latter case even must have an easy to identify responsible, just that > > nobody cares.  > > Sure, I completely agree with you. But that is orthogonal to the problem of > free()ed memory not actually being freed. Sorry, I figured, I might need to elaborate here. The usecase of GC being temporarily disabled is merely a reproducer for the problem with free(). There may be other usecases as well, which is why I consider the questions of "disabling GC" and the "free not freeing memory" kinda separate. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 06:06:33 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 11:06:33 +0000 Received: from localhost ([127.0.0.1]:36931 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7FzU-0007os-RW for submit@debbugs.gnu.org; Wed, 03 Feb 2021 06:06:33 -0500 Received: from mout.gmx.net ([212.227.15.15]:49189) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7FzP-0007oU-7l for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 06:06:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612350377; bh=vEt73JenGqBQGKP4Yr8bu4bvfxvpI1+P4vnpoNMjbtQ=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=JIW0kxCgTrd1LN9Vrv/ZKUtnwRL6+yjcaRQSdc7gp4X8LR2gJRrSkt1AsPmO+3T4y 1iSlPs2/HyCpwgqSgkDCyvzsV4j1vkz7M9TPMwGjZRRI3nnVB2sMvVAY5XPVB9zdE3 udRdCBn5ZdXhcZuCSoa+zxSP3BrVurQ4VV6ikUeE= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.1.100] ([46.125.249.123]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1Mzhj9-1ltsMr3PhR-00vbkS; Wed, 03 Feb 2021 12:06:16 +0100 Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> From: martin rudalics Message-ID: <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> Date: Wed, 3 Feb 2021 12:06:14 +0100 MIME-Version: 1.0 In-Reply-To: <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:BbCZ797bBSz0YU2QGfSo9niJs9S3P+Y7Nbi0Q/crFlx7VW7bgC6 kKIXVpYTDuiQO+aMtWeW70RArCb4W64ZXYGW1qCO0lJHXUU9Qz5svPaX2dR98eoEaaG9Ln3 9OZTfEbZO45DkJT+P6fkgcw2NiEQYMN2la3N0cNFISCLuL0VH/smumLDPUdo+cQAG205O20 Ng+myWgQSoICJYVBYmdZw== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:SAToO54cp80=:Tj46zvTZM3fAVNd8wUg7Bo rnLsbt6IS32jGsR++3mug8c7+0zR+bm6+qbPtP6Jtk/WJVTVqhFjGki+bES2oWbBe2qMZ0LiF fSHDwnwZg4Ug+IycEX60/Il7INbzY1Pp/Gfx0DYkkJuh1C2tSf8nuj+A1/stOpszMso1CaqVt 9wEagB0/pSUNkMORqjz2WuY/qhJjFRV8svQovCjPXd3oDdzezS/Fm54Xaz+ZflPe86zC3hOO5 bzX1ePQjK0sOK30j+l3uRGj2eXnEWC4QDRe366ZMUJSWl53ZfyHVF7Ov/Fos0elXQjpOh9Kau p3N5PgW0f3tV7ie8gh7vaf0OBpfYaYWEvtJ9qUE/dCl0ISSIWQf/CE+sowch90vS9fINEeHz2 GOWXezmybK9MKYfvtgUB8YCX1PFkLLWZBGxW0Pj2/SyWEXnSD1VpoQ3yjMGc2vFrrcaGyNy/U wDoTlr0ehXk9aw4yDcaaRgVIEtk8bQIAQdQh6KTvj2rLqh2OhObrl2Wq2S1SuOKg7NaUUh6py civsbcOh6Rhj46nVYRWjNPj1CwRmRtn1cPR3IfWtqmJ5HN55RRj0xjg/vphKpEF39EDZ+0/wZ WIuI09QLcSbek1eaJAwVEAZZYkJCyWjE+V18axWiO/zcNqSi2l6IdkdhQ6Mu1NSQnJ+k+XXY8 vHHQ6kiTO5jWTx16npH3sP8J/tOOCyI+hFCTfLbZo3ZPl0pNms9EOHFjff1T/HwbylLT2w6/S fFPXIEZWltD/+r86Bo1+vuqiYpV1+XQwH9e9N3e5RusueZUZffIpNGo0GqALMzD2ZTDU5iR5d 95buEnf7cfV4qMBCsiKOqDr7Wgv7lAmB+qmU/yr+gcjBeIGr1Zu+jFl9lypeRsRyXXjZFuKwj HD1pkAWwqkoBxQ3uo4fw== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) >> What I meant was if people really disabled GC for the rest of their >> session and got away with it. But that was only a rhetorical question >> to which the answer is clearly no. > > I'm not sure where you got this answer from. The first link I referred to has a > user which does exactly that, and it is a highly upvoted question with 4k views. If the first link refers to 1: https://emacs.stackexchange.com/questions/34342/is-there-any-downside-to-setting-gc-cons-threshold-very-high-and-collecting-ga then (setq gc-cons-threshold (eval-when-compile (* 1024 1024 1024))) (run-with-idle-timer 2 t (lambda () (garbage-collect))) obviously means that this author does GC as soon as Emacs is idle. > To me that seem to imply the answer is "yes". Not to me. martin From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 06:08:19 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 11:08:19 +0000 Received: from localhost ([127.0.0.1]:36935 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7G1D-0007sJ-7T for submit@debbugs.gnu.org; Wed, 03 Feb 2021 06:08:19 -0500 Received: from forward100j.mail.yandex.net ([5.45.198.240]:43181) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7G19-0007rp-7q for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 06:08:17 -0500 Received: from forward101q.mail.yandex.net (forward101q.mail.yandex.net [IPv6:2a02:6b8:c0e:4b:0:640:4012:bb98]) by forward100j.mail.yandex.net (Yandex) with ESMTP id 034EF50E09EF; Wed, 3 Feb 2021 14:08:08 +0300 (MSK) Received: from vla5-e173276f42e0.qloud-c.yandex.net (vla5-e173276f42e0.qloud-c.yandex.net [IPv6:2a02:6b8:c18:351e:0:640:e173:276f]) by forward101q.mail.yandex.net (Yandex) with ESMTP id EF4E9CF40002; Wed, 3 Feb 2021 14:08:07 +0300 (MSK) Received: from vla5-047c0c0d12a6.qloud-c.yandex.net (vla5-047c0c0d12a6.qloud-c.yandex.net [2a02:6b8:c18:3484:0:640:47c:c0d]) by vla5-e173276f42e0.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 2cZtgxqWy0-87ISFEED; Wed, 03 Feb 2021 14:08:07 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612350487; bh=865N9zsrtIyP50EMpn08AnEzROasxKtqjZfygNkyhp0=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=dKJToqRunnUQXPKq500QFjPFI0q4Y7rtAoTe6ImaQlEvNEZL9Lbl3xI0e5LsnGp8R kNiiGgrzaTn07grZ5oq5aXPlij08Y+uwfBKzeSDsC1AHDzMC1jHncyKOMFJthAvCXK vkmj61GowQBgkEXRw44DC2rpRWZ/yFB8B5dAksV4= Authentication-Results: vla5-e173276f42e0.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by vla5-047c0c0d12a6.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id wuav4Hy7Tv-86neP3Y9; Wed, 03 Feb 2021 14:08:06 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 14:08:06 +0300 In-Reply-To: <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, 2021-02-03 at 12:06 +0100, martin rudalics wrote: >  >> What I meant was if people really disabled GC for the rest of their >  >> session and got away with it.  But that was only a rhetorical question >  >> to which the answer is clearly no. >  > >  > I'm not sure where you got this answer from. The first link I referred to > has a >  > user which does exactly that, and it is a highly upvoted question with 4k > views. > > If the first link refers to > > 1: > https://emacs.stackexchange.com/questions/34342/is-there-any-downside-to-setting-gc-cons-threshold-very-high-and-collecting-ga > > then > > (setq gc-cons-threshold (eval-when-compile (* 1024 1024 1024))) > (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > obviously means that this author does GC as soon as Emacs is idle. Yes. I think you misread this thread, the case where the problem first came up is exactly running GC on idle. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 06:17:04 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 11:17:04 +0000 Received: from localhost ([127.0.0.1]:36960 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7G9g-00088A-6O for submit@debbugs.gnu.org; Wed, 03 Feb 2021 06:17:04 -0500 Received: from forward106o.mail.yandex.net ([37.140.190.187]:49190) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7G9e-00087W-3w for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 06:17:02 -0500 Received: from sas1-65bb95216cf6.qloud-c.yandex.net (sas1-65bb95216cf6.qloud-c.yandex.net [IPv6:2a02:6b8:c08:1f01:0:640:65bb:9521]) by forward106o.mail.yandex.net (Yandex) with ESMTP id 363BD506024A; Wed, 3 Feb 2021 14:16:55 +0300 (MSK) Received: from sas8-6bf5c5d991b2.qloud-c.yandex.net (sas8-6bf5c5d991b2.qloud-c.yandex.net [2a02:6b8:c1b:2a1f:0:640:6bf5:c5d9]) by sas1-65bb95216cf6.qloud-c.yandex.net (mxback/Yandex) with ESMTP id nBi8rpWmDS-GsIWiViO; Wed, 03 Feb 2021 14:16:55 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612351015; bh=BC1VTCQvrvHBy2BuFaLkgIcdq43itNgyGX+pN8byNuU=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=ZTN9CAB6JIpCxpCUw7jE/bfIQRf6zzCFGjfqeUabN0TfqaNgzk3GmU7EwPLpHvzKR uIT/VDTowwpeuk3GP5xVvGtRM4CA3TWlyUlW3SxO3AtEFvEwPi7fbOdXQPJOc0qxlC deSFJ7ixd0lyJB40oCn96h4AL1NMap/EHgn4AZc4= Authentication-Results: sas1-65bb95216cf6.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas8-6bf5c5d991b2.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id tzsRTl0ZxJ-GsnSl1tM; Wed, 03 Feb 2021 14:16:54 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 14:16:53 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 14:08 +0300, Konstantin Kharlamov wrote: > On Wed, 2021-02-03 at 12:06 +0100, martin rudalics wrote: > >  >> What I meant was if people really disabled GC for the rest of their > >  >> session and got away with it.  But that was only a rhetorical question > >  >> to which the answer is clearly no. > >  > > >  > I'm not sure where you got this answer from. The first link I referred to > > has a > >  > user which does exactly that, and it is a highly upvoted question with 4k > > views. > > > > If the first link refers to > > > > 1: > > https://emacs.stackexchange.com/questions/34342/is-there-any-downside-to-setting-gc-cons-threshold-very-high-and-collecting-ga > > > > then > > > > (setq gc-cons-threshold (eval-when-compile (* 1024 1024 1024))) > > (run-with-idle-timer 2 t (lambda () (garbage-collect))) > > > > obviously means that this author does GC as soon as Emacs is idle. > > Yes. I think you misread this thread, the case where the problem first came up > is exactly running GC on idle. But either way, I'm not sure why it matters. If you want, I can for you specifically craft up a reproducer that works with GC running on idle. The reason I haven't done this in the first place is just to simplify the reproducer, but if it's something you're interested in, just ping me up, I'll make one. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 07:56:20 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 12:56:21 +0000 Received: from localhost ([127.0.0.1]:37212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Hhk-0004vn-PV for submit@debbugs.gnu.org; Wed, 03 Feb 2021 07:56:20 -0500 Received: from mout.gmx.net ([212.227.15.19]:45215) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Hhi-0004vW-Ap for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 07:56:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612356966; bh=dSbgo9usI/ljjM5byVRsMaxIaeqFqkMzGeS0nU7tsFw=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=RLIpD9WxLLR4G7aR/W3UJhs0LxTCG11ScsBtWm61Jz9W2m0R0+Iko4YfzRqM0xeJx SsKoNsFrSyoc6nrz1T8b8HcZQ9Df/3JCcxpRFwxGKrFNc97QVnkAzZZN3Ia2aG9pFD 5xGH/NT7F0WD5862kaU9eWxVEvPD7XiNx9C5gb0Y= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.1.100] ([46.125.249.123]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1M8ygO-1l3Pou1w98-0065q7; Wed, 03 Feb 2021 13:56:06 +0100 Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> From: martin rudalics Message-ID: <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> Date: Wed, 3 Feb 2021 13:56:04 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:1eHMJRScAj7rdYEu7fQ2YUFnVFsRKjqLEP/StfAWUw7Vqz8UY3V vKaPO9S6y0oHs0KqdPAHjgFUQJwbrFfKJq+MKyyjtkaZdt8Oqfvchu28/dB8/+1RPEmHG1I vEkQgcA5vFSAe66edCJuB4Kb+B9M3fe4r1JVFR9LPUPw91wd6NUsIazLEgw2TcFBV7tnvei 0r8zf964mA+hp+nTDezHg== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:K1DU4Ayi3q4=:jReH/SD7ENKij01jD/Qx6u lzmbdOVICxEwhgNvUnC7vVfoXleOsVImToFZe2v+Wli1klF4qU9FBkpq7p5VLVK4qaFGsIgP7 I2GoEC0rWbQigd/s6rx0Yk4aYZ+kAANSakgn5e7dAldhoHTt14lpQWQ0p24SQxU20S5uvwaZB FHq8vx6D1MRa0td9Uz3RBUZ6PUjXN4NBAFpNBLbzVdMFPQDQcUl4mM/YQ717poRIaaF/J0WXr +VP9UKl6H0qgwZ9wgieojfj23fvLcV2bSVWESs7b2nN0ZRECm/UxGL5CHNvUXx2JeeCgTXlcR AXi83hhdWEQeSjr9B5evBiuY3rY1Okmyq2k15KFG2xh1QbvZXGgxy/ly2QXTLc8fiLRs3ewV7 bkpkmDlcmQdmthy1D0crU172+dbL3L2LboCqP3FONgWIXx4zrun2fT3WUalvPx1VAmKEatNZi Mar020GVRSDfp2OavH9kr/SJCLHaIhiKa0BqyZOSU1Wr2XootAlSh25pXuy1oLBIKzhGKfPIx bTfFrpiIrYkVef+JutxYneScFkFfv4tP8tqdVJno1gLyYZoH+gJCFssyd3Ry7T80NFD/Yj43j XJ8mWQ5vWpq0a3cIFRNXayxUZbyhEY2kHq3YyR4UsKb4GV2pWxp8V0Jleyf3xLwqVbT22hFR6 VC8LeNwTgH57PajtOXLfVfGGdRH1RdJ6ficggwJ+e4mE45BayJq+YZuYoMbYFZWxtb/ctsLu4 RaeSVngVF2cpvi8m8aoXl71KC62eLRsggYCxsIst/ucx93mwwu864yLCjb6T/+pNuSimjeybY qlEiqGuFFnaHuDR4lXMmyH8g7Y4llPQfNzEsJzhcIvGGlTX7ckTI3HXEzto4a2O3U1DH8SyKX Y5AkjKYtr4rOZ+RfrSZw== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Yes. I think you misread this thread, the case where the problem first > came up is exactly running GC on idle. In your "Force Glibc to free the memory freed" patch I nowhere see that you do that only when running GC on idle. What am I missing? martin From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 08:00:52 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 13:00:52 +0000 Received: from localhost ([127.0.0.1]:37227 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Hm8-00055H-FC for submit@debbugs.gnu.org; Wed, 03 Feb 2021 08:00:52 -0500 Received: from forward101j.mail.yandex.net ([5.45.198.241]:47983) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Hm6-00054z-NT for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 08:00:52 -0500 Received: from sas1-c7095e9a6207.qloud-c.yandex.net (sas1-c7095e9a6207.qloud-c.yandex.net [IPv6:2a02:6b8:c14:3d85:0:640:c709:5e9a]) by forward101j.mail.yandex.net (Yandex) with ESMTP id 875201BE1029; Wed, 3 Feb 2021 16:00:43 +0300 (MSK) Received: from sas8-b61c542d7279.qloud-c.yandex.net (sas8-b61c542d7279.qloud-c.yandex.net [2a02:6b8:c1b:2912:0:640:b61c:542d]) by sas1-c7095e9a6207.qloud-c.yandex.net (mxback/Yandex) with ESMTP id mWzt915DhA-0gIiEbC5; Wed, 03 Feb 2021 16:00:43 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612357243; bh=L7jxNh4gj/UFu/ZAI5A4NidewkWRtPJBK59wVleAeiE=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=jlQc4uzQqCgK8pbXK3eTp5PunXd5uSJkDpdMeZojXhhb83OzrIEmNsmaf2c6jY6kJ cs0B5lrRD230I84fu37zr8Ez4keuztVVxEia1WIBhePfqri34IvqbyeD0hq8/AI5P8 9VEDxxz/0J5JUMLz/wIkK3rsvtA20sagi8UA3mQs= Authentication-Results: sas1-c7095e9a6207.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas8-b61c542d7279.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id oJfaUpbwSG-0gn42acC; Wed, 03 Feb 2021 16:00:42 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: martin rudalics , Stefan Monnier Date: Wed, 03 Feb 2021 16:00:42 +0300 In-Reply-To: <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, 2021-02-03 at 13:56 +0100, martin rudalics wrote: >  > Yes. I think you misread this thread, the case where the problem first >  > came up is exactly running GC on idle. > > In your "Force Glibc to free the memory freed" patch I nowhere see that > you do that only when running GC on idle.  What am I missing? Haha, you seem to have jumped over explanations straight to discussing a discussion about an alternative implementation, even though the discussion has never happened :D Now I see what you mean, your idea is to run `malloc_trim(0)` only on idle. That sounds okay with me. Stefan, what do you think? Will it be okay if I implement a patch that runs `malloc_trim(0)` when Emacs is idle? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 09:44:28 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 14:44:28 +0000 Received: from localhost ([127.0.0.1]:37376 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JON-0001tx-Nn for submit@debbugs.gnu.org; Wed, 03 Feb 2021 09:44:28 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36882) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JOL-0001tb-Op for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 09:44:26 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43894) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l7JOF-0001Cq-TI; Wed, 03 Feb 2021 09:44:19 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4291 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l7JOF-00072h-E2; Wed, 03 Feb 2021 09:44:19 -0500 Date: Wed, 03 Feb 2021 16:44:17 +0200 Message-Id: <83v9b943em.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> (message from Konstantin Kharlamov on Wed, 03 Feb 2021 11:23:43 +0300) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Konstantin Kharlamov > Date: Wed, 03 Feb 2021 11:23:43 +0300 > Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, dj@redhat.com > > Second, most importantly, what `malloc_trim(0)` does is it restores the correct behavior. I mean, what's the point of ever freeing memory if it's not freed, right? The purpose of freeing memory is to make it available for further allocations by the program. You seem to interpret "freeing memory" to mean that memory should be returned to the OS, but that's not necessarily true, and isn't required to allow the freeing program be able to allocate more memory in the future. > The problem we're dealing here with is an actual bug in glibc⁵. What this implies is that if the fix indeed hurts performance someplace, well, then it's that this place requires additional performance-related fixes. As opposed to just ignoring the bug because of performance got somewhere decreased. Things like, changing the slow algorithm, or modifying GC behavior for specific usecases… The fact that the bug you reported didn't get any responses in more than a month, let alone wasn't fixed, could be a sign that not everyone agrees this is a bug... From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 09:51:23 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 14:51:23 +0000 Received: from localhost ([127.0.0.1]:37385 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JV5-0002A0-JI for submit@debbugs.gnu.org; Wed, 03 Feb 2021 09:51:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39702) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JV4-00029k-Oi for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 09:51:23 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44051) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l7JUy-0004NV-0o; Wed, 03 Feb 2021 09:51:16 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4733 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l7JUx-0007UD-9b; Wed, 03 Feb 2021 09:51:15 -0500 Date: Wed, 03 Feb 2021 16:51:14 +0200 Message-Id: <83tuqt4331.fsf@gnu.org> From: Eli Zaretskii To: Konstantin Kharlamov In-Reply-To: <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> (message from Konstantin Kharlamov on Wed, 03 Feb 2021 12:49:47 +0300) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Konstantin Kharlamov > Date: Wed, 03 Feb 2021 12:49:47 +0300 > Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, dj@redhat.com > > Second, if Emacs indeed sees it's gonna allocate memory again right away, then it shouldn't have even freed the memory in the first place. You are not serious here, are you? How could Emacs possibly know, at the level that does GC, whether the Lisp program that currently runs will need more memory in the future? The program itself may know that (and then it may not, if memory is allocated by some low-level primitive), but GC itself? If you think it can have powers for such prophecy, we should probably use that to write a program to advise us which shares to buy, and get rich -- fast. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:01:50 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:01:50 +0000 Received: from localhost ([127.0.0.1]:38574 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JfC-0002qg-F8 for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:01:50 -0500 Received: from forward105p.mail.yandex.net ([77.88.28.108]:41995) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JfB-0002qT-JZ for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:01:49 -0500 Received: from iva6-d2c82ec8bc26.qloud-c.yandex.net (iva6-d2c82ec8bc26.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:610c:0:640:d2c8:2ec8]) by forward105p.mail.yandex.net (Yandex) with ESMTP id 758BE4D41CF8; Wed, 3 Feb 2021 18:01:32 +0300 (MSK) Received: from iva5-057a0d1fbbd8.qloud-c.yandex.net (iva5-057a0d1fbbd8.qloud-c.yandex.net [2a02:6b8:c0c:7f1c:0:640:57a:d1f]) by iva6-d2c82ec8bc26.qloud-c.yandex.net (mxback/Yandex) with ESMTP id b3DqxIQIm8-1VF0SMU4; Wed, 03 Feb 2021 18:01:32 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612364492; bh=a8neNDqH4HPC/U0P9PAjGDuqs3bEbejcmqTl3PyLivI=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=Wn9Ci36dUrYCHlXGI3Rize5JLS8WOc08KJpr8m5106NGdqy9qs+AsLELlD4naCw0M uSQ0WL1Cixj8Oo71xFPjCOGCKX9Pq6xOEbLTMe0G/jIO50Nec4SXlBI/wNfuKpt4AE RyMlHu6Au0jIbiP32hqclCE2PdZ+MXeGueph+R4c= Authentication-Results: iva6-d2c82ec8bc26.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva5-057a0d1fbbd8.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id yGWKXVmDqI-1VlWk3Aj; Wed, 03 Feb 2021 18:01:31 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <49855294df24eec433160c90ebb5bf2054671c06.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Eli Zaretskii Date: Wed, 03 Feb 2021 18:01:30 +0300 In-Reply-To: <83tuqt4331.fsf@gnu.org> References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <83tuqt4331.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 16:51 +0200, Eli Zaretskii wrote: > > From: Konstantin Kharlamov > > Date: Wed, 03 Feb 2021 12:49:47 +0300 > > Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, > > dj@redhat.com > > > > Second, if Emacs indeed sees it's gonna allocate memory again right away, > > then it shouldn't have even freed the memory in the first place. > > You are not serious here, are you?  How could Emacs possibly know, at > the level that does GC, whether the Lisp program that currently runs > will need more memory in the future?  The program itself may know that > (and then it may not, if memory is allocated by some low-level > primitive), but GC itself?  If you think it can have powers for such > prophecy, we should probably use that to write a program to advise us > which shares to buy, and get rich -- fast. Yeah, you're right, never mind this text, I think I kinda misinterpreted two sentences I was answering to. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:12:28 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:12:28 +0000 Received: from localhost ([127.0.0.1]:38590 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JpU-0003Dk-Hc for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:12:28 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:35689) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JpS-0003Db-M1 for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:12:27 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4DW4t10K1Bz1qvqk; Wed, 3 Feb 2021 16:12:24 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4DW4t05CKtz1t5kj; Wed, 3 Feb 2021 16:12:24 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id TT49uWTmW9zr; Wed, 3 Feb 2021 16:12:23 +0100 (CET) X-Auth-Info: fbB67kY4y3ZEq7PClOgHU/AQIS8qmj47xiev0tVvLQeXuJrVKULeI9EuTDrHNav4 Received: from igel.home (ppp-46-244-188-241.dynamic.mnet-online.de [46.244.188.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Wed, 3 Feb 2021 16:12:23 +0100 (CET) Received: by igel.home (Postfix, from userid 1000) id 5F54C2C3753; Wed, 3 Feb 2021 16:12:23 +0100 (CET) From: Andreas Schwab To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <83v9b943em.fsf@gnu.org> X-Yow: .. here I am in 53 B.C. and all I want is a dill pickle!! Date: Wed, 03 Feb 2021 16:12:23 +0100 In-Reply-To: <83v9b943em.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 03 Feb 2021 16:44:17 +0200") Message-ID: <87h7mtxk14.fsf@igel.home> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.4 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, monnier@iro.umontreal.ca, Konstantin Kharlamov , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) On Feb 03 2021, Eli Zaretskii wrote: > The purpose of freeing memory is to make it available for further > allocations by the program. You seem to interpret "freeing memory" to > mean that memory should be returned to the OS, but that's not > necessarily true, and isn't required to allow the freeing program be > able to allocate more memory in the future. Especially as asking the OS to (de)allocate memory is a more expensive operation. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:15:12 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:15:12 +0000 Received: from localhost ([127.0.0.1]:38594 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Js8-0003KO-0O for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:12 -0500 Received: from mout.gmx.net ([212.227.15.18]:34691) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Js3-0003Jk-QT for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612365296; bh=JnFh8c6RbHH7QzYhvFG/3goND39i02X3B9bSDtm/lbQ=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=d4UBCKokfhtKKTtBGqv7o+yrwWHauXp1J2ndXWnEmw5KddGuuRPeg5BcDp69NOx7g frKlmB1KjuxPMEhzUnO7HtGfuJMbnjNBSCngqFXAYEGldUYG0sj4gZYHHkztSDv7je ATvu36doUBaa/fpdDw6z9krBYB3nPFJXDDBGeXIc= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.1.100] ([46.125.249.123]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MFKGZ-1l9jVB158z-00Fn3M; Wed, 03 Feb 2021 16:14:56 +0100 Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , Stefan Monnier References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> From: martin rudalics Message-ID: <63edbd63-29d8-0a32-55a4-28fd526b5d7a@gmx.at> Date: Wed, 3 Feb 2021 16:14:54 +0100 MIME-Version: 1.0 In-Reply-To: <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:NZNRmQ3PvstIfVEFo/Nz7Ezlwmt4nfhkRlUu7eZHOqvrF1f5Ytt 53H9yGL/IokZkCo1QMBBq0KfVtvyrQkDJjfqwmOjL7Zs07y0VQ/Zaoh7MUIe0So8V50sJSd eW0lvmfISzDiJrSzShnnyrxmQZTh+SYvH7E2RDahYAm1mnWOa8k9ILwuijCaQziqc8y5uk1 DKj9pvEGbUoVRk3iW1/Sw== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:MM2PDbmxD+E=:GeRZrwW21nR4UW43cKoshl uRy9oAeDf7SfuyHBw2qTeLs3ivod6XcYn4UwdsuHJc4kM17VWRWQ7ad+kApiPZii6ynkgln8y zsDVH2tHjSn8ocSEXBMnHWjXBomhCQsnGNDelsvSL7aEyWMDADwY5WRR94+Z2I7MwAPhBM4pX FMwUQV7g2Zm5HzzrGHdp/fUXg6HmWCZbe/VXLMs2hmisWrDMxHcO8AXLhzY1rH8Ca4Zq0Ayg8 wMKWny5WxinBj9QtaoZLtjiLHAqFphZ8CRAwTln/Tg3eE5DbONWNzt0RDHLkDMtSb4fRjs0IZ ggtsc3xPcOTJLl7Hw3w68z0zJ6AAzu7c/rZPe4xTnsI2bzRE3fVqE5srw4mXGxY9tNgGhK2O1 FP4NQ+vDelJDRyAL02kzEL9UA98ahWG3mpl+V00MPGZfhUhTU9u6EKFX4jsbYPtnpEUKGXusR s5/QCJFg46qeCe9jk8wHhIw+6kGE/c0zP3DIh7jvQuZ/p7+290ZMZvYC3Eek8wwPIomq2AS7P rwPo7jTS111kfwOmWvUiqcf0Y1IN8NrK9C7nFt5AIKQQT2f27zTkQHYze7UOzRwmtGfuaP6Rq 9NDNB4nEFxJDyvA1VQD+TVygot0tEhzMXYfFHWdbI59kbf4juqNKBpTPbLhyYlHp/T7yG9ZJF F+1Pdn8yJMh8Lf7cJGjkKJZKFMrV9b+T2S/FbTyN+e7jJvE1GW9h9KyiWIVnNAX9UgDSTby6r uqPvMe0L0Irtsz5DlVwEcbkXlgFt5KqGOQiABXqSbHGztN6m0k4qPn5vNkBOJqgpQfnGAzeER 9parVeix2mjUJ0zT5FlKDvpdHLm/YfRL9pmwbIbcT0ixh1DPoqqI7kmTabfWRhqorKDzCbrfr 27zUwRlMVVWiNS6cneTA== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Now I see what you mean, your idea is to run `malloc_trim(0)` only on > idle. That sounds okay with me. What I meant was a function say 'garbage-collect-and-trim' that would do the same as =E2=80=98garbage-collect=E2=80=99 but in addition run 'malloc= _trim(0)' so the person that earlier used (setq gc-cons-threshold (eval-when-compile (* 1024 1024 1024))) (run-with-idle-timer 2 t (lambda () (garbage-collect))) would now run (setq gc-cons-threshold (eval-when-compile (* 1024 1024 1024))) (run-with-idle-timer 2 t (lambda () (garbage-collect-and-trim))) instead. martin From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:15:21 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:15:21 +0000 Received: from localhost ([127.0.0.1]:38598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JsH-0003Kw-9V for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:21 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:56995) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JsG-0003Kb-2P for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:20 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 96126440989; Wed, 3 Feb 2021 10:15:14 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 375CA440977; Wed, 3 Feb 2021 10:15:13 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612365313; bh=hP4WjyZoIaXxles4sr3Aux/ECz0ZNoLANy+39BqV1aA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=BPn3ioXxIF67trY1/69GgiCwK/eBFOZenB24e0mNqwuGuaN2O5wvanReV3cuhE1j/ JLVSA9lvUsVN5SlrLnU8+lA3XaZ9qf4QZiug4p8rTwB0t2IfEe5lI48ji3mgaP5HxP DBOC2TIEk4xmFOEXvwD1JAIA0JreLwqAPauOcgBiBM7GKc6ESJYgSDrLnzqnwxLXJ6 KkBaYbCnAlwPDk0CNu+PcxDU91HxHSVlMcjBH2Jo58iF2Kd3+RTm5dHny1j8bQ7UJ+ icmkiUhLLRnJti/VAUq2zkd/ujgifolXcu54rmSZk9jEMVykkTsGTR4WB3BgceGq0o KmRqryXBwnJoA== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E56E3120298; Wed, 3 Feb 2021 10:15:12 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Date: Wed, 03 Feb 2021 10:15:11 -0500 In-Reply-To: <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> (Konstantin Kharlamov's message of "Wed, 03 Feb 2021 16:00:42 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.102 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Stefan, what do you think? Will it be okay if I implement a patch that runs > `malloc_trim(0)` when Emacs is idle? That's yet another reason why we should provide that function to ELisp, and then have ELisp decide when it's called (whether after GC, or on idle or what). Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:15:27 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:15:27 +0000 Received: from localhost ([127.0.0.1]:38602 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JsN-0003LK-HN for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:27 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:15532) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7JsL-0003Ku-Iy for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:15:25 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5B774440981; Wed, 3 Feb 2021 10:15:20 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D41024408F6; Wed, 3 Feb 2021 10:15:18 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612365318; bh=8fYMXYB+zpglYsh8FQyUSLeFQyVlkJuV7y8A93NygEI=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Edktm6pHBF+rFMbOCFNsXu/LHHlQCC2jk9bxdGFcT962fopdE4IALhBaj9NRzswsB 3QgrTSJfaYzkLKsNejHoP87oIQT9X+j+pW0B8wMnv8cdaNIlxOy4MB6JziyIHoQVj7 7ercPvz8irEannzchcZQNz0zjKnnrn+Yp/xM7Wtj/TNWf30hdRM19LcVKzWSw+gb1f 2zpb7gAin/PlRZ5QAP+iUYiJr5jm568AAN4rMMgwxxt/ox1KwCieaJGpvPIxQWDcAS ZDu/XjruCmcfM3PnLJijsWHo6ltVIv1TDpaTx4kEeSWUAXnEcxArFvYso24ufBqxlx QW78KC/2HewUg== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 96F7812047A; Wed, 3 Feb 2021 10:15:18 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> Date: Wed, 03 Feb 2021 10:15:17 -0500 In-Reply-To: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> (Konstantin Kharlamov's message of "Wed, 03 Feb 2021 09:04:21 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.101 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, fweimer@redhat.com, Eli Zaretskii , dj@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Upside indeed is the memory reduction. I expect so, but the measurement is the thing we need here. How much is the memory reduced (in the case where the GC vars are left to their default value)? > Well, I didn't send the patches for my only benefit, but for benefit of > others people. The new ELisp function is something that not too many people > would benefit from, and I mean, including those who disable GC. That is > because it would be an opt-in feature, which you need to know about to > enable it, and not many will ever find out about it. If it's very useful even with the default GC settings, then there's a good reason to say it should be enabled by default, but if not then I'd expect that those people who set their GC vars manually will learn to complete their GC settings by adding that function to their `post-gc-hook`. > For my-only benefit I could just continue building Emacs with my patch > applied locally, as I do now. Yes, I have my share of local patches as well ;-) But I think there's a good case to be made that it can benefit others, Stefan PS: BTW, we can arbitrarily reduce the percentage of CPU time spent in malloc_trim by only calling it every N times rather than after every GC. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 10:29:53 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 15:29:53 +0000 Received: from localhost ([127.0.0.1]:38626 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7K6K-0003q2-Sz for submit@debbugs.gnu.org; Wed, 03 Feb 2021 10:29:53 -0500 Received: from forward106p.mail.yandex.net ([77.88.28.109]:39024) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7K6H-0003pj-V7 for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 10:29:51 -0500 Received: from sas1-43b74f7725b7.qloud-c.yandex.net (sas1-43b74f7725b7.qloud-c.yandex.net [IPv6:2a02:6b8:c14:391a:0:640:43b7:4f77]) by forward106p.mail.yandex.net (Yandex) with ESMTP id 9802D1C80E0B; Wed, 3 Feb 2021 18:29:42 +0300 (MSK) Received: from sas1-f4dc5f2fc86f.qloud-c.yandex.net (sas1-f4dc5f2fc86f.qloud-c.yandex.net [2a02:6b8:c08:cb28:0:640:f4dc:5f2f]) by sas1-43b74f7725b7.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 2QJtszOois-TgH4EiU7; Wed, 03 Feb 2021 18:29:42 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612366182; bh=9yx27pYdf2pENoAnClA7ZBbTfFjOcv6qBjMa/9wuavA=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=KqjRvjHSVrnQL/cfH5fwJ9E4IbUGgFK3M/JwEqPXuFzFz6vdpOvK6nuPAhERBUyt0 HKyAidwMnR7wTubizxHEul30UudwbnTqx0UOM2IJYVpcm6+xZ4m1R62lcp+ggkKZWg OhgIbctNEqW2hCl31U8ZR7i+aaOGxaZl4rbzKPkw= Authentication-Results: sas1-43b74f7725b7.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-f4dc5f2fc86f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id lQhcgwPPir-TfoSHnpo; Wed, 03 Feb 2021 18:29:41 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier Date: Wed, 03 Feb 2021 18:29:41 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 10:15 -0500, Stefan Monnier wrote: > > Stefan, what do you think? Will it be okay if I implement a patch that runs > > `malloc_trim(0)` when Emacs is idle? > > That's yet another reason why we should provide that function to ELisp, > and then have ELisp decide when it's called (whether after GC, or on > idle or what). Okay, let's discuss this idea. It seems that running `malloc_trim(0)` on idle (let's say, after 10 seconds of idling, which implies a user might have switch to a browser, or something) should resolve all performance concerns. Especially given, measurements show it has pretty negligible impact even when located on return from GC. To answer you question in another email about memory benefits given default Emacs settings: well, today I tried creating a testcase that would reproduce problem with default settings, but haven't really succeeded at it. I have a testcase where Emacs without the patch has ≈20M more memory than the one with the patch, but that's pretty small difference, and offhand I didn't manage to get it increased any further. So, I'm thinking of wiring the functional of memory trimming to on-idle hook, without possibility to disable it. Given how small performance impact in this case would be, I see no reason to even implement an option to disable it. Thoughts? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 11:02:33 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 16:02:33 +0000 Received: from localhost ([127.0.0.1]:38661 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Kbx-0004sO-B7 for submit@debbugs.gnu.org; Wed, 03 Feb 2021 11:02:33 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:17320) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Kbv-0004s7-Ca for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 11:02:31 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id BCFC9440981; Wed, 3 Feb 2021 11:02:25 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4BD1844096F; Wed, 3 Feb 2021 11:02:24 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612368144; bh=QNI1cI1t0eioy4SY2k2V6zeWp0/h1/n1uX7BFS+/G4o=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=T+ywGqW0mRlNiJwf7YBYkiv6rqd6/1g0kwIEbukVAjpDcjpcRcG3C27rVmwxp6np1 BmRIW9oBe4TBA3SDRPatPCMojhUTcmDxihhIhWNn2n2r6ade4fzHhAvemMBI6bIAdb 6hA0+EITqoTukymdhkO/N3asfyH8GwiQ/wuG5gileK9iD8mdU73neiIGs4Q/8Cwjvo zYe42WKYhFqtychUPSS+8UHAVaVHMELpQrM/fVOaSc7vgB4pgmsho5KuqjHX1PAFTC PZIQfu2fWnUYi/9p3DIcaD6JvfcAVEIajk3sDs5Qif+bOFgLvXJ2+ZpXOmI4oIqxJo Pp3rNdgbqiLDw== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 020651202A7; Wed, 3 Feb 2021 11:02:23 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Date: Wed, 03 Feb 2021 11:02:22 -0500 In-Reply-To: (Konstantin Kharlamov's message of "Wed, 03 Feb 2021 18:29:41 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.099 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > To answer you question in another email about memory benefits given defau= lt > Emacs settings: well, today I tried creating a testcase that would reprod= uce > problem with default settings, but haven't really succeeded at it. I have > a testcase where Emacs without the patch has =E2=89=8820M more memory tha= n the one > with the patch, but that's pretty small difference, and offhand I didn't > manage to get it increased any further. Thanks. At least that seems to indicate that glibc does its job properly for the way we normally use it. > So, I'm thinking of wiring the functional of memory trimming to on-idle > hook, without possibility to disable it. That seems hard to do (luckily), since AFAICT idle hooks only exist via `run-with-idle-timer` and those can always be disabled with things like `cancel-timer`. > Given how small performance impact in this case would be, I see no > reason to even implement an option to disable it. > Thoughts? My main thought is that if `malloc_trim` indeed makes a significant difference, it's a sign that Emacs's own code did its job (it called `free` as it should) and that the problem is in how glibc decided not to return the memory to the OS. That's a behavior that can (and will) change over time outside of our control. So calling `malloc_trim` every time I stop typing for 10s, just on account of "maybe glibc didn't reclaim quite as much memory as we'd like this time" doesn't sound very appealing to me. It sounds like an ad-hoc quick hack. I love being able to use ad-hoc quick hacks, but I don't like enabling such things by default when the only use-cases where they're known to be useful are fairly specialized (and discouraged by Emacs maintainers ;-) So I think we need more info: do the glibc maintainers consider it normal for glibc to behave this way? Why does it behave this way? Would the equivalent of `malloc_trim` happen anyway "at some point in the future"? E.g. If you create a test case where you disable GC, let the memory use grow to 1GB, then reset the GC vars to their default and keep using Emacs modestly, will the memory ever be returned to the OS or is an explicit call to `malloc_trim` really indispensable? But until we get all the answers to these questions, we can already install the code that exposes `malloc_trim` to ELisp. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 11:35:20 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 16:35:20 +0000 Received: from localhost ([127.0.0.1]:38700 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7L7g-0005uR-DR for submit@debbugs.gnu.org; Wed, 03 Feb 2021 11:35:20 -0500 Received: from forward105o.mail.yandex.net ([37.140.190.183]:60404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7L7d-0005u3-7D for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 11:35:19 -0500 Received: from sas1-f45a06513e06.qloud-c.yandex.net (sas1-f45a06513e06.qloud-c.yandex.net [IPv6:2a02:6b8:c08:b315:0:640:f45a:651]) by forward105o.mail.yandex.net (Yandex) with ESMTP id A5CF942004CB; Wed, 3 Feb 2021 19:35:09 +0300 (MSK) Received: from sas2-e7f6fb703652.qloud-c.yandex.net (sas2-e7f6fb703652.qloud-c.yandex.net [2a02:6b8:c14:4fa6:0:640:e7f6:fb70]) by sas1-f45a06513e06.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 00bDFJtm6O-Z9I0IXc2; Wed, 03 Feb 2021 19:35:09 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612370109; bh=g1p/+di+AOlqV0fs18sOxT02JmGa6LIz8fsJkvSIldo=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=Rtz7SUv2eWRHEuFQkGG27TJfAF/PaRrfp1pESMhSnntq2Pt86mivAakWgqE1wDSfU /MOu+D9/zXGSb3j+GmtbywLKv6dcZBoky8otHp/1/HNUPo8iiQ5r0KDjEQJ1QBZCWY xLZh6Z6yCBfWiirp04iFxHKbUSgb0kyKxN5g8xR8= Authentication-Results: sas1-f45a06513e06.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas2-e7f6fb703652.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id JAqwHAvyMr-Z8nSY1KT; Wed, 03 Feb 2021 19:35:08 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier Date: Wed, 03 Feb 2021 19:35:08 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, 2021-02-03 at 11:02 -0500, Stefan Monnier wrote: > > To answer you question in another email about memory benefits given default > > Emacs settings: well, today I tried creating a testcase that would reproduce > > problem with default settings, but haven't really succeeded at it.  I have > > a testcase where Emacs without the patch has ≈20M more memory than the one > > with the patch, but that's pretty small difference, and offhand I didn't > > manage to get it increased any further. > > Thanks.  At least that seems to indicate that glibc does its job > properly for the way we normally use it. > > > So, I'm thinking of wiring the functional of memory trimming to on-idle > > hook, without possibility to disable it. > > That seems hard to do (luckily), since AFAICT idle hooks only exist via > `run-with-idle-timer` and those can always be disabled with things like > `cancel-timer`. Ah, okay then. I just didn't know how on-idle code should be implemented; if it would be just making a ELisp wrapper and adding it to the hook, which people can then remove if they want to, that's fine. I just thought there's some additional work required to make it "disableable", I'm not opposed to the idea of it being so per se. > > Given how small performance impact in this case would be, I see no > > reason to even implement an option to disable it. > > Thoughts? > > My main thought is that if `malloc_trim` indeed makes a significant > difference, it's a sign that Emacs's own code did its job (it called > `free` as it should) and that the problem is in how glibc decided not > to return the memory to the OS. > > That's a behavior that can (and will) change over time outside of > our control.  So calling `malloc_trim` every time I stop typing for 10s, > just on account of "maybe glibc didn't reclaim quite as much memory as > we'd like this time" doesn't sound very appealing to me. > > It sounds like an ad-hoc quick hack.  I love being able to use ad-hoc > quick hacks, but I don't like enabling such things by default when the > only use-cases where they're known to be useful are fairly specialized > (and discouraged by Emacs maintainers ;-) I totally agree with you, and I too consider it a hack. > So I think we need more info: do the glibc maintainers consider it > normal for glibc to behave this way?  Why does it behave this way? Very good question! I hope Glibc mainainers that are on CC list will be able to answer. Even though I created a report on Glibc just some months ago, the problem per se existed for a long time. And I've seen Carlos O'Donell leaving a small comment on a similar issue with Ruby 2 years ago, which implies they're aware of this situation. > Would the equivalent of `malloc_trim` happen anyway "at some point in > the future"?  E.g. If you create a test case where you disable GC, let > the memory use grow to 1GB, then reset the GC vars to their default and > keep using Emacs modestly, will the memory ever be returned to the OS or > is an explicit call to `malloc_trim` really indispensable? No, the memory will never be returned to OS. I can tell that right away, because the only difference would be `free()` getting called more often. I think it is worth mentioning here that Glibc usually does return memory to the OS without any need in malloc_trim(0). What happens in affected applications (such as here) is that an application stumbles upon a very special allocation pattern, which kinda breaks Glibc algorithms of returning memory. That's btw to the point of whether this GLibc behavior is a bug: well, sometimes it works, sometimes it doesn't — doesn't look like purposeful behavior to me :) > But until we get all the answers to these questions, we can already > install the code that exposes `malloc_trim` to ELisp. > > >         Stefan > From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 11:52:06 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 16:52:06 +0000 Received: from localhost ([127.0.0.1]:38740 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7LNu-0006RP-8y for submit@debbugs.gnu.org; Wed, 03 Feb 2021 11:52:06 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:48311) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7LNr-0006Qe-Vc for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 11:52:05 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5A267440977; Wed, 3 Feb 2021 11:51:58 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id A82F3440958; Wed, 3 Feb 2021 11:51:56 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612371116; bh=0Fl18BMF21QG7/Eb6X+sKYKdfD/7d9h6MD3fJtST7eI=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=U9melYBQM0gWy/Szv/Bai2kQim2mui0sF6R6JEepCmrqVI24iHkqEvJkBJvige+6S xUb4yqYD4QzOZWNVu6vlwHYP3uVSvbnHd4GJIckTnV7x0DnYjce7fZavmU7XtRimfA /+I7GNsOXJNMK8M0iRT+HXxMpILdki6EnENw7N+o2lE75/mILhBYx8gLdtsQF0DhuA pfEQqqJZA1dKYxNQd4UNi5AU9UicwiJVRHNeJCQfZMCLRrsc4EVQvv81qY56eeNMPM mk+cQqfWu4hWusL/bBzCr5y/r/gPjMu8VLMZsJSfayVlDCYbp32ZBrEGj7hgn1UG3G 41jpKJlUxftoQ== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 35F3212005C; Wed, 3 Feb 2021 11:51:56 -0500 (EST) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> <6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru> Date: Wed, 03 Feb 2021 11:51:55 -0500 In-Reply-To: <6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru> (Konstantin Kharlamov's message of "Wed, 03 Feb 2021 19:35:08 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.099 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> So I think we need more info: do the glibc maintainers consider it >> normal for glibc to behave this way?=A0 Why does it behave this way? > Very good question! I hope Glibc mainainers that are on CC list will be a= ble to > answer. Even though I created a report on Glibc just some months ago, th= e problem per > se existed for a long time. And I've seen Carlos O'Donell leaving a small= comment on a > similar issue with Ruby 2 years ago, which implies they're aware of this = situation. I mean, I understand why glibc doesn't always immediately release memory to the OS, but I'd like to better understand whether we can expect that there is a mechanism in glibc to avoid it keeping 500MB of free memory around eternally when the subsequent actual use remains below 100MB. >> Would the equivalent of `malloc_trim` happen anyway "at some point in >> the future"?=A0 E.g. If you create a test case where you disable GC, let >> the memory use grow to 1GB, then reset the GC vars to their default and >> keep using Emacs modestly, will the memory ever be returned to the OS or >> is an explicit call to `malloc_trim` really indispensable? > > No, the memory will never be returned to OS. I can tell that right > away, because the only difference would be `free()` getting called > more often. I'm not sure I follow: if I were to implement a malloc library, calls to `free` would be exactly the place where I'd decide that some memory should be returned to the OS, so even if a given call to `free` might not return the now-unused 400MB, I'd expect that this memory might be finally returned after some number of other/unrelated calls to `free`. > I think it is worth mentioning here that Glibc usually does return > memory to the OS without any need in malloc_trim(0). What happens in > affected applications (such as here) is that an application stumbles > upon a very special allocation pattern, which kinda breaks Glibc > algorithms of returning memory. Indeed. The question is whether it's a bug in glibc or whether it's something that the upper layer should deal with. Knowing a bit better why/when this happens would also help the upper layer decide when to call `malloc_trim` if indeed that's needed (another useful info would be a function that returns the mount of memory there is to trim, assuming that info can be obtained much more cheaply than a call to `malloc_trim`). Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 14:25:27 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 19:25:27 +0000 Received: from localhost ([127.0.0.1]:38887 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7NmI-00021i-Np for submit@debbugs.gnu.org; Wed, 03 Feb 2021 14:25:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:31268) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7NmG-00021Z-M6 for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 14:25:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612380324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to; bh=KFrdeOkd0NtgcrodlOyHjs2e2AEuyyYn7Lg12thKroM=; b=UPa2d5svjUmIS2xEpMWaFoIm+tAtWE+pOqAn0z+a1nS8trWqg4BndDJSGSZAeJfRA2M22d DSABf76H+Rmzi+xPTq2vARkomBwMqcAzrjZI+z2GZzGIj06v85YJ5tMkLygn8aeM4FurQo ZujQxIOWNStWO24M9J5WoHltmDuKCDQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-436-nRWJAYdoOaCTcb4wrzFH3Q-1; Wed, 03 Feb 2021 14:25:23 -0500 X-MC-Unique: nRWJAYdoOaCTcb4wrzFH3Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B72398710E3; Wed, 3 Feb 2021 19:25:21 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4CE891971D; Wed, 3 Feb 2021 19:25:18 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113JPG7L000768; Wed, 3 Feb 2021 14:25:16 -0500 From: DJ Delorie To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <83v9b943em.fsf@gnu.org> (message from Eli Zaretskii on Wed, 03 Feb 2021 16:44:17 +0200) Date: Wed, 03 Feb 2021 14:25:15 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, hi-angel@yandex.ru, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Eli Zaretskii writes: >> The problem we're dealing here with is an actual bug in glibc=E2=81=B5. = What >> this implies is that if the fix indeed hurts performance someplace, >> well, then it's that this place requires additional >> performance-related fixes. As opposed to just ignoring the bug >> because of performance got somewhere decreased. Things like, changing >> the slow algorithm, or modifying GC behavior for specific usecases=E2=80= =A6 > > The fact that the bug you reported didn't get any responses in more > than a month, let alone wasn't fixed, could be a sign that not > everyone agrees this is a bug... Right, glibc's malloc maintains a cache of re-usable chunks of memory, for performance reasons. That cache, obviously, costs memory. malloc_trim() flushes that cache and returns memory to the kernel. That has two effects: first, your memory footprint is smaller, but second, your performance will suffer until the cache is refilled. We (glibc devs) don't consider "poor cache performance for my application" to be a bug. A patch which improves cache performance *for most apps* would be considered, but a patch that improves one app's speed at the cost of most other apps, would be rejected. You (emacs devs) need to decide whether you care more about memory footprint or memory performance, and tune malloc accordingly. malloc_trim() is one knob to tune, there are others. But don't say it's a "bug" if our defaults don't match your optimal settings. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 14:31:08 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 19:31:08 +0000 Received: from localhost ([127.0.0.1]:38893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Nro-0002At-FM for submit@debbugs.gnu.org; Wed, 03 Feb 2021 14:31:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30716) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Nrl-0002Al-TT for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 14:31:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612380665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=q5VVjmHdKd+bkbCz/0woviTvgE4nx65r1wWa4fpwEsE=; b=hQ28ldIwc+3Ib4pvx3IHsNVfVgS7AzZZFZjW2/DzvnMJtfmaEtkTUCsuHyNTTEQL9M65M/ mH0elOGaxNJrJN+Z/erS0ml17y5a5+FPOiZ3OWghDhKApXyjcx2a7+BNeaEYJz8BS1jejf mXdbmW3mWKc46UZLAEpfctooDDYy304= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-45-aSbwW-leOTu3tv4O_nZ12A-1; Wed, 03 Feb 2021 14:31:03 -0500 X-MC-Unique: aSbwW-leOTu3tv4O_nZ12A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 793EBBBF0F; Wed, 3 Feb 2021 19:31:02 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2BC11100AE4B; Wed, 3 Feb 2021 19:30:58 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113JUuTM000841; Wed, 3 Feb 2021 14:30:57 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 11:51:55 -0500) Date: Wed, 03 Feb 2021 14:30:56 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, rudalics@gmx.at, fweimer@redhat.com, 45200@debbugs.gnu.org, hi-angel@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: > I'm not sure I follow: if I were to implement a malloc library, calls to > `free` would be exactly the place where I'd decide that some memory > should be returned to the OS, Heh. malloc() and free() are on the hot path for many time-critical things; if you wasted time in free() when you didn't have to, people would complain. *This* is the tradeoff we (glibc devs) have to make - performance vs footprint. Gaining even a few percent in a malloc/free benchmark is a *huge* win for some folks. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 14:36:19 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 19:36:19 +0000 Received: from localhost ([127.0.0.1]:38897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Nwp-0002I2-2o for submit@debbugs.gnu.org; Wed, 03 Feb 2021 14:36:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36410) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Nwk-0002Hp-9d for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 14:36:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612380974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to; bh=V1zFb2kVxc6ebEi1zVRb4i69G4O040fLEA2V3VgTib8=; b=COjq/yLxRG7lt0/fJs3QSuprFeKjGI8vrEXLFItxt18uAN0vr/e142YctXcpNIKKqO72bu ISaQbP/85hGb4uiIISA2ZaUzmeRfLAE0sYHYJgyWM8GiwLX/D+Da6eEyaCSzFoy0KJxEUD BWrN1vlRDTJI3y3biFZMwJh86UJPJDU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-p8_KA5PgNKCyZhIcZ7D8dg-1; Wed, 03 Feb 2021 14:36:12 -0500 X-MC-Unique: p8_KA5PgNKCyZhIcZ7D8dg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 378A8107ACC7; Wed, 3 Feb 2021 19:36:11 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E6FE05C238; Wed, 3 Feb 2021 19:36:07 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113Ja6Lv000907; Wed, 3 Feb 2021 14:36:06 -0500 From: DJ Delorie To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru> (message from Konstantin Kharlamov on Wed, 03 Feb 2021 19:35:08 +0300) Date: Wed, 03 Feb 2021 14:36:05 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, rudalics@gmx.at, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Konstantin Kharlamov writes: >> So I think we need more info: do the glibc maintainers consider it >> normal for glibc to behave this way?=C2=A0 Why does it behave this way? > > Very good question! I hope Glibc mainainers that are on CC list will be a= ble to > answer. The answers are "yes" and "performance". Sorry. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 14:50:06 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 19:50:06 +0000 Received: from localhost ([127.0.0.1]:38914 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7OAA-0002cW-0R for submit@debbugs.gnu.org; Wed, 03 Feb 2021 14:50:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:32946) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7OA7-0002bv-JF for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 14:50:04 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50699) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l7OA1-0004og-HT; Wed, 03 Feb 2021 14:49:57 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3505 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l7OA0-00008N-PA; Wed, 03 Feb 2021 14:49:57 -0500 Date: Wed, 03 Feb 2021 21:49:54 +0200 Message-Id: <838s853p99.fsf@gnu.org> From: Eli Zaretskii To: DJ Delorie In-Reply-To: (message from DJ Delorie on Wed, 03 Feb 2021 14:25:15 -0500) Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed References: X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, hi-angel@yandex.ru, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: DJ Delorie > Cc: hi-angel@yandex.ru, rudalics@gmx.at, monnier@iro.umontreal.ca, > carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org > Date: Wed, 03 Feb 2021 14:25:15 -0500 > > You (emacs devs) need to decide whether you care more about memory > footprint or memory performance, and tune malloc accordingly. > malloc_trim() is one knob to tune, there are others. Which other knobs are there that aren't deprecated and could disappear without a warning, though? We used to use some knobs in the past, then one day we were told we should stop because those knobs will no longer be available. We still use some knobs (see alloc.c), but could some of you please review that and see if the optimizations we attempt through those knobs are reasonable? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 15:24:33 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 20:24:33 +0000 Received: from localhost ([127.0.0.1]:38952 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7OhU-0003Qj-TY for submit@debbugs.gnu.org; Wed, 03 Feb 2021 15:24:33 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:47010) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7OhT-0003QX-Cx for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 15:24:31 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1810544098C; Wed, 3 Feb 2021 15:24:26 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 8A004440974; Wed, 3 Feb 2021 15:24:24 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612383864; bh=WOpYATkMILoG7l2Y5EFF721MOq3ssNrf2gUqe0bjLjw=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=OqKl1R/K7q0piTC4p5L32/+JxuVceg97fZquRbdJ0wDS8tKg+BMwTEi03GQ0kXbCh /TcXN+R8EppDZwN4ryLjid64d5YvuF/CmWk0klisiZ5syQ3VUg2sagv4CsttXiV4Ov cJoR+rUsHr2U630jORn4xDao3JC8om0eRdzYQfAo77EYKbpYbCvcS1lTIfc3xjCuDs dTJ4sCW8nFEOXBEfRo75hGhdeeokZBVVfuIb8Goq6LvH72fgTN8gHxsREDc2rSAAix c5PHGPB54KZu5VzMOeWWbRpYnVP2agMwOFtJy15Xn2j/HzfUyR8hAA1yL/lj0bfrub iF2CR0CxE+SUg== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3B077120218; Wed, 3 Feb 2021 15:24:24 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 15:24:23 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 14:25:15 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.096 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) DJ Delorie [2021-02-03 14:25:15] wrote: > Right, glibc's malloc maintains a cache of re-usable chunks of memory, > for performance reasons. Right, so far as good. > We (glibc devs) don't consider "poor cache performance for my > application" to be a bug. A patch which improves cache performance *for > most apps* would be considered, but a patch that improves one app's > speed at the cost of most other apps, would be rejected. I understand that as well. But I'm wondering why glibc is willing to keep *indefinitely* an unused 200MB of memory, which is more than double the mount of memory in use for the rest of the application's life. I mean I understand that you can't predict the future, but I expected that "at some point" glibc should decide that those 200MB have been left unused for long enough that they deserve to be returned to the OS. The doc of `malloc_trim` suggests it's occasionally called by `free` and `mallopt` suggests via `M_TRIM_THRESHOLD` that there's a limit to how much extra spare memory glibc keeps around, so this suggests that indeed memory is trimmed "every once in a while". So what causes Emacs's experience to be different? If I read `mallopt`s man page carefully I see it say: M_TRIM_THRESHOLD When the amount of contiguous free memory at the top of the heap grows sufficiently large, free(3) employs sbrk(2) to release this memory back to the system. (This can be useful in programs so my guess is that in Emacs's case those 200MB of extra memory are *not* contiguous? Would that be it? If so, it does seem like a weakness of glibc's heuristic. Is there a way for Emacs to ask malloc how much memory (or some approximation of it) would be recovered by `malloc_trim`? > You (emacs devs) need to decide whether you care more about memory > footprint or memory performance, and tune malloc accordingly. > malloc_trim() is one knob to tune, there are others. But don't say it's > a "bug" if our defaults don't match your optimal settings. In our normal use, glibc's tuning works fine. We're just seeing some corner case behaviors which seem clearly undesirable, so I think it would make sense to try and arrange for glibc to handle them better. I suspect the issue is how to make it so the malloc library can keep track of the size of the "cache" without slowing down the most common execution path. Once that is done, it should be easy to check the size of the "cache" every once in a while (e.g. when requesting more memory from the OS) and call malloc_trim when that cache is "way too big". Tho of course, "way too big" might just mean we're just about to re-allocate it, so we'd probably want to include some notion of time so we only call `malloc_trim` if it's been "way too big" for a while. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 15:29:05 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 20:29:05 +0000 Received: from localhost ([127.0.0.1]:38957 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Olt-0003X3-KG for submit@debbugs.gnu.org; Wed, 03 Feb 2021 15:29:05 -0500 Received: from forward101j.mail.yandex.net ([5.45.198.241]:58018) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Olq-0003WY-Fh for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 15:29:03 -0500 Received: from myt5-5c2d79a89c75.qloud-c.yandex.net (myt5-5c2d79a89c75.qloud-c.yandex.net [IPv6:2a02:6b8:c12:1f22:0:640:5c2d:79a8]) by forward101j.mail.yandex.net (Yandex) with ESMTP id 27F3B1BE1294; Wed, 3 Feb 2021 23:28:55 +0300 (MSK) Received: from myt5-aad1beefab42.qloud-c.yandex.net (myt5-aad1beefab42.qloud-c.yandex.net [2a02:6b8:c12:128:0:640:aad1:beef]) by myt5-5c2d79a89c75.qloud-c.yandex.net (mxback/Yandex) with ESMTP id L6n3n3QCmb-StIeUCpd; Wed, 03 Feb 2021 23:28:55 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612384135; bh=QBZKKmPzFXtXKcNNv5UtjiW0OKQ+78/cgxqsmIYoEMk=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=irTFt88ORRN7ct5HCPqmQT0CfElA+sdk4OBnbKt3Y2TJ2JQkIyGToZp7a9ayFBHTH sTydc1s1tCQONL6sg2sCwAfUKCI068HDZxJdE8gQrXxQ3ys0yshHAg4/qy8+1PSUXc qhwlvIuSgFbxqpbrAo9G/TElqaOVMEH6s/fRxCno= Authentication-Results: myt5-5c2d79a89c75.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt5-aad1beefab42.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id 0KI4iIblaP-SsnCMNNi; Wed, 03 Feb 2021 23:28:54 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <542372cc86c88931a80574dd61c5342023424cca.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: DJ Delorie Date: Wed, 03 Feb 2021 23:28:54 +0300 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, rudalics@gmx.at, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, 2021-02-03 at 14:36 -0500, DJ Delorie wrote: > Konstantin Kharlamov writes: > > > So I think we need more info: do the glibc maintainers consider it > > > normal for glibc to behave this way?  Why does it behave this way? > > > > Very good question! I hope Glibc mainainers that are on CC list will be able > > to > > answer. > > The answers are "yes" and "performance".  Sorry. > Thanks for chiming in. Let me rephrase a question of mine: right now, Glibc behaves in an odd way: α) sometimes, call to `free()` frees memory completely β) sometimes, call to `free()` leaves memory lingering forever, till malloc_trim is called additionally. The answer I'd like to know is why do you not do always either α or always β? (I'm leaving out here the question of which way is correct and why, I'm rather curious just why the difference in behavior). From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 15:42:24 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 20:42:24 +0000 Received: from localhost ([127.0.0.1]:38980 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Oyl-0003rW-PX for submit@debbugs.gnu.org; Wed, 03 Feb 2021 15:42:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Oyi-0003rN-Uv for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 15:42:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612384940; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=SchxRDK8NZSNl+opd9cfPeLz0psV4KLmDWg5JsEEbtk=; b=I63cvQYJN+7tu0RfXYGRMmDA7a8IVP/IXQZ5+1cvLbQ5r9gQU7f+tdWH3EL1tCzOXuANzx JVWu7REnRUx8iww4ALD6kW3CRhJoG1+UQnnWrw6PtoZcHBBqLqGPiyz2SDdoMOlVtdXm43 eCDAC+soyMoBKRONsLlvDyZixPlRxaM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-55-baaWDf1oPR6Q_PrEVon4Nw-1; Wed, 03 Feb 2021 15:42:18 -0500 X-MC-Unique: baaWDf1oPR6Q_PrEVon4Nw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F171B801967; Wed, 3 Feb 2021 20:42:16 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DDC358899; Wed, 3 Feb 2021 20:42:13 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113KgB2q001712; Wed, 3 Feb 2021 15:42:11 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 15:24:23 -0500) Date: Wed, 03 Feb 2021 15:42:11 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: > I understand that as well. But I'm wondering why glibc is willing to > keep *indefinitely* an unused 200MB of memory, which is more than double > the mount of memory in use for the rest of the application's life. To be blunt, 200Mb is peanuts compared to some applications, and it's *nothing* compared to an enterprise application. Keeping 200M around to quickly satisfy memory requests of various sizes (not all cached chunks are the same size) is IMHO reasonable. > I mean I understand that you can't predict the future, but I expected > that "at some point" glibc should decide that those 200MB have been left > unused for long enough that they deserve to be returned to the OS. Where will we store that lifetime information? Yet another word of memory used, yet another syscall to check the time? Or completely reorganize the malloc internals into an LRU system so we can peel off the old end, at the expense of performance? I agree that we could do better at detecting long-unused chunks, but it's expensive (in terms of both development and runtime) to do so, and typically at the expense of some other desired metric. I would ask the Emacs devs why they wait until gc to free() memory instead of keeping track of uses more accurately and free()ing it right away. It's a similar type of compromise. > The doc of `malloc_trim` suggests it's occasionally called by `free` and > `mallopt` suggests via `M_TRIM_THRESHOLD` that there's a limit to how > much extra spare memory glibc keeps around, so this suggests that indeed > memory is trimmed "every once in a while". Only when the available memory is "at the top of the heap". Most cached memory is not; one unfree'd chunk at the top of the heap can keep the whole heap in memory. We used to have code that munmap()'d large "holes" in the cache, but the resulting performance was horrible. > so my guess is that in Emacs's case those 200MB of extra memory are > *not* contiguous? Would that be it? Right, or just not at the top of the heap. > Is there a way for Emacs to ask malloc how much memory (or some > approximation of it) would be recovered by `malloc_trim`? Not easily. You can call mallinfo2() before and after a malloc_trim(), but then you'd have *done* the trim ;-) > In our normal use, glibc's tuning works fine. We're just seeing some > corner case behaviors which seem clearly undesirable, so I think it > would make sense to try and arrange for glibc to handle them better. IIRC the original problem was not an unfreed 100M but an unfreed *many Gb*. Those are two different problems. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 15:52:05 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 20:52:05 +0000 Received: from localhost ([127.0.0.1]:38988 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7P89-00045r-3U for submit@debbugs.gnu.org; Wed, 03 Feb 2021 15:52:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:45066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7P86-00045M-59 for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 15:52:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612385521; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to; bh=iQmEUX6iyTFyLxtE39ggwxhxca4iQpyLHe+ss4nTJ5A=; b=GHdYc16nGgQQon0u4oQ4bOpHGvi0l/bf1MxYEcXp10srMg2RMFJOqnLXUc0ew5CD39A75B d+gjXNPy6RUNkdWeBN+ETuvw6Nun/PNWaqMYSw//mg/9qABp2/LS2TwOul0PP3jinJxR3G ZptydP30pQMrf2DkQB8SQxtmdvk7uvw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-534-atzrEJ5jPpufJCqsiL9LEg-1; Wed, 03 Feb 2021 15:51:58 -0500 X-MC-Unique: atzrEJ5jPpufJCqsiL9LEg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CBFD9100CCC2; Wed, 3 Feb 2021 20:51:56 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9EA915D9F6; Wed, 3 Feb 2021 20:51:53 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113KppHg001856; Wed, 3 Feb 2021 15:51:52 -0500 From: DJ Delorie To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <542372cc86c88931a80574dd61c5342023424cca.camel@yandex.ru> (message from Konstantin Kharlamov on Wed, 03 Feb 2021 23:28:54 +0300) Date: Wed, 03 Feb 2021 15:51:51 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, rudalics@gmx.at, fweimer@redhat.com, monnier@iro.umontreal.ca, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Konstantin Kharlamov writes: > =CE=B1) sometimes, call to `free()` frees memory completely > =CE=B2) sometimes, call to `free()` leaves memory lingering forever, till= malloc_trim is called additionally. > > The answer I'd like to know is why do you not do always either =CE=B1 or > always =CE=B2? (I'm leaving out here the question of which way is correct > and why, I'm rather curious just why the difference in behavior). If the chunk being free'd happens to be the highest-addressed chunk in the heap, it "exposes" that hole in the heap below it, if that exceeds MALLOC_TRIM_THRESHOLD it causes a trim. If the chunk is *not* at the top of the heap, nothing happens. We only trim the top of the heap. Of course, this assumes that the caching logic doesn't decide to hold on to the chunk *anyway* (like, if it's put in tcache), in which case it's not freed despite being at the top of the heap. Note that any patch affecting tcache performance will need to be vigorously defended ;-) IIRC the fastbin cache also holds onto the top chunk. Since trimming cleans the fastbin, this often frees the top chunk. Disabling fastbins might be an interesting thing to try. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 16:01:13 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 21:01:13 +0000 Received: from localhost ([127.0.0.1]:38992 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7PGz-0004Jp-1O for submit@debbugs.gnu.org; Wed, 03 Feb 2021 16:01:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58538) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7PGv-0004Jf-CQ for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 16:01:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612386068; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=DnJ8/V7WFhL6/99DkI2l5CyQv1XZuy9/GwykyGVbPn8=; b=bCHSpeabz8h3lgYge5MWqLlEIEzFJw2xjnHMryXdlbxbI2GBsyJJTZGZwxHbtz0rd4a8Z1 dD1JNG5ZjHRDkVrKCpeYsv/DkqEn5UbPtK2rGYvwmRey8Vk2l0RyZM2qj0V+8hsxQaGFvE B+RqitpujZHzuGv6nUq2zd4frygqm9Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-15-z5n2GEt6MFm7RkoZRJG4bw-1; Wed, 03 Feb 2021 16:01:06 -0500 X-MC-Unique: z5n2GEt6MFm7RkoZRJG4bw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A77D6869ED9; Wed, 3 Feb 2021 21:00:40 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2E79660BFA; Wed, 3 Feb 2021 21:00:40 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113L0cEK002011; Wed, 3 Feb 2021 16:00:38 -0500 From: DJ Delorie To: Eli Zaretskii Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: <838s853p99.fsf@gnu.org> (message from Eli Zaretskii on Wed, 03 Feb 2021 21:49:54 +0200) Date: Wed, 03 Feb 2021 16:00:38 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, monnier@iro.umontreal.ca, hi-angel@yandex.ru, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Eli Zaretskii writes: > Which other knobs are there that aren't deprecated and could disappear > without a warning, though? We used to use some knobs in the past, > then one day we were told we should stop because those knobs will no > longer be available. There's a whole section in the glibc manual on "Memory Allocation Tunables", which is the "new" way to do this. "man mallopt" is the other way. Using these you could, for example, disable the tcache and fastbins completely and see how that works for you. Or limit the number of arenas. Or let smaller allocations go directly to mmap(). There are some environment variables which affect malloc, but we're trying to get rid of those. Maybe in ten or twenty years we'll come up with some better way, but that's the future's problem ;-) From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 17:07:15 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 22:07:15 +0000 Received: from localhost ([127.0.0.1]:39065 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7QIs-0005ru-RP for submit@debbugs.gnu.org; Wed, 03 Feb 2021 17:07:15 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:37696) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7QIq-0005rh-NH for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 17:07:13 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7443380D8E; Wed, 3 Feb 2021 17:07:07 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B27598050D; Wed, 3 Feb 2021 17:07:05 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612390025; bh=5vsSPTJktnPs0QpmeT1J7GjOhcDat+1BLPM2j4T7RJ0=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Gl4o/sg8i6E096/NobNVQopVr+bTC6ubepBO2780VZ70JxJxw0MxMqgf37oA67zEO 0l1vtBVaXAfnHWhE7/jvhhcA0Gao/pAaXJbQ3MrbWDvhogz2w9PQYNW7tWkDWbajj/ 9CGaElXcWCNoyLNQkWGHLo5O5Z5HOlEjY9Z5N5dRgAD/5Zp2uqls1BX51zo5oJOftG Kt3jrGkSCySElU983Kh+oogeGEc4eWugiwDL7xGVQnNnP7stvgCOfVDQmt7F0ZqDe7 iirW0E3yoY0fXjbv9R7s1ewHsWlmOsYGKZAr/IShcT5OJ0tFVmyJIG7NoukgK1LWeB iHE15dT9G0Apw== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 65354120218; Wed, 3 Feb 2021 17:07:05 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 17:07:04 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 15:42:11 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.008 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> I understand that as well. But I'm wondering why glibc is willing to >> keep *indefinitely* an unused 200MB of memory, which is more than double >> the mount of memory in use for the rest of the application's life. > To be blunt, 200Mb is peanuts compared to some applications, and it's I'm not worried about the absolute value, but about the proportion. I think in memory management code, having an overall overhead of 50% is generally considered acceptable (i.e. actual memory allocated is twice the memory used by the application), whether that comes from internal&external fragmentation or a stop© GC, ... But in our specific use case, there seems to be no limit to the overhead: if the application uses a heap of size N at some point in time it will never grown back down, so the overhead can end up being arbitrarily large. > *nothing* compared to an enterprise application. Keeping 200M around to > quickly satisfy memory requests of various sizes (not all cached chunks > are the same size) is IMHO reasonable. If the average allocation/deallocation rate justifies it, I fully agree. But if the variation of allocated space stays well below that for a long time, then those 200MB are truly wasted. >> I mean I understand that you can't predict the future, but I expected >> that "at some point" glibc should decide that those 200MB have been left >> unused for long enough that they deserve to be returned to the OS. > Where will we store that lifetime information? I haven't thought very much about it, so I'm sure it's easy to shoot holes through it, but I imagined something like: - one `static unsigned long hoard_size` keeps the approximate amount of space that is free but not returned to the OS. Not sure where/when to keep it up to date cheaply, admittedly. - one `static unsigned long smallest_recent_hoard_size`. This is updated whenever we allocate memory from the OS. - one `static unsigned long age_of_smallest_recent_hoard_size`. This is incremented every time we allocate memory from the OS (and reset whenever the value of smallest_recent_hoard_size is modified). Then you'd call `malloc_trim` based on a magic formula combining `age_of_smallest_recent_hoard_size` and the ratio of `smallest_recent_hoard_size / total_heap_size` (and you'd trim only what's necessary to release O(`smallest_recent_hoard_size`) memory). > Yet another word of memory used, Since 200MB is peanuts, I figure that extra 24B should be acceptable ;-) > yet another syscall to check the time? I didn't mean time as in an OS-notion of clock, no. > I agree that we could do better at detecting long-unused chunks, but > it's expensive (in terms of both development and runtime) to do so, and > typically at the expense of some other desired metric. No doubt. > I would ask the Emacs devs why they wait until gc to free() memory > instead of keeping track of uses more accurately and free()ing it > right away. It's a similar type of compromise. Delaying for some time is one thing. Delaying forever is another. >> The doc of `malloc_trim` suggests it's occasionally called by `free` and >> `mallopt` suggests via `M_TRIM_THRESHOLD` that there's a limit to how >> much extra spare memory glibc keeps around, so this suggests that indeed >> memory is trimmed "every once in a while". > Only when the available memory is "at the top of the heap". Ah, I see, that makes sense. I do remember such behavior in other/older libc libraries. > We used to have code that munmap()'d large "holes" in the cache, That's what I seem to remember, indeed. And our memory management code does play with `mallopt` in the hopes to encourage it to allocate using `mmap` in the hopes that it then deallocates via `munmap`. > but the resulting performance was horrible. Hmm... so that explains why we're seeing those problems again. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 17:21:55 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 22:21:56 +0000 Received: from localhost ([127.0.0.1]:39079 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7QX5-0006Cm-M5 for submit@debbugs.gnu.org; Wed, 03 Feb 2021 17:21:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41909) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7QX3-0006Cc-EG for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 17:21:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612390913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=t2lAnq69iLdNeQGImBCvsYXHpWVNaR2xKNW7u6/GxEI=; b=UdSMbeXNlGJMc42c9tgI+GrYJ/cVdYT6QfVcHuxAtYrH7muERdeJC8G8hlD3NNYJt4Rewb A8Nig0ZzayhOD7rFQ3MWw+bGY7XPjcWA64CngwgvqB2sLjiyzgzzyi78SDzWyUmpYl/2ZV N2BZke3alp7vGXkFOCZM/sA+ETfflsw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-517-HREB3BLJOsKngMUBVq__Xw-1; Wed, 03 Feb 2021 17:21:51 -0500 X-MC-Unique: HREB3BLJOsKngMUBVq__Xw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DE295106BC6A; Wed, 3 Feb 2021 22:21:49 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 846A96091B; Wed, 3 Feb 2021 22:21:46 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 113MLifK003418; Wed, 3 Feb 2021 17:21:44 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 17:07:04 -0500) Date: Wed, 03 Feb 2021 17:21:44 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: > I haven't thought very much about it, so I'm sure it's easy to shoot > holes through it, but I imagined something like: A bit of devil's advocacy just for giggles... > - one `static unsigned long hoard_size` keeps the approximate amount of > space that is free but not returned to the OS. > Not sure where/when to keep it up to date cheaply, admittedly. With full atomic access because malloc is multi-threaded; atomics are expensive compared to thread-local or nonexisting data. Updating this has to be in the critical path, too. > - one `static unsigned long smallest_recent_hoard_size`. > This is updated whenever we allocate memory from the OS. We map huge chunks at a time, but they stay unbacked until they're actually used. > Then you'd call `malloc_trim` based on a magic formula That's cheating ;-) >> Yet another word of memory used, > > Since 200MB is peanuts, I figure that extra 24B should be acceptable ;-) *per chunk*. You allocate 1 byte and you get 32 at the moment, because of these lower limits. We're already getting complaints about this particular overhead. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 18:32:59 2021 Received: (at 45200) by debbugs.gnu.org; 3 Feb 2021 23:33:00 +0000 Received: from localhost ([127.0.0.1]:39175 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Rdr-0001f3-Ka for submit@debbugs.gnu.org; Wed, 03 Feb 2021 18:32:59 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21591) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Rdq-0001eo-Cf for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 18:32:58 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id C781F80976; Wed, 3 Feb 2021 18:32:52 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 240F78063A; Wed, 3 Feb 2021 18:32:38 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612395158; bh=+Lk1WLoj32NkCn77sEIZxJ7opV7/J9YxMoSzj79fmow=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=CqwlJdS4bwr82nEfk2OvbbhtXJaaAoj/JbZPWiHhMPt12jF1GqqWKgSw/57KIyH6+ a/mLCzWJXfRtbhCAvaSUwugaE3W/4bXM1d/bdNMVSkVcRT+skTe8c3B/6o4LOerwVW ffwlESGlOaqw7RJUsopwElwkl/9MJsfoJCvn3ue6xU+KeLRJM4TZYVlS7YREEP4GkC fGRcuJV6QpaohM9I/caieNRta7iu1ltywF9aRWF5QRBtZOC4bG+KLD8d9Q6wk/gxSz TXZE5NTZRcH/LeHl8tsArynJSJLm9CD/mWOKciqOGp/vEBkLlnVJ2bD/a4fLRYw2M/ 1+hnJV3QRhc1g== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B97A012020B; Wed, 3 Feb 2021 18:32:37 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 18:32:36 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 17:21:44 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.008 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> - one `static unsigned long hoard_size` keeps the approximate amount of >> space that is free but not returned to the OS. >> Not sure where/when to keep it up to date cheaply, admittedly. > With full atomic access because malloc is multi-threaded; atomics are > expensive compared to thread-local or nonexisting data. Updating this > has to be in the critical path, too. Indeed, multithreading makes it all much more fun. I guess you'd want to make those info thread-local, which would force a rethink of its definition (what about data allocated in one thread bu freed in another, yadda, yadda). The key would be to make sure this is not updated for every call to `free` but only for some of them (that's why it says "approximate"). Not having the source code of `free` at hand, I'm free to assume that this is possible. E.g. we could say that we don't count objects smaller than a certain size or for objects smaller than 2^N (for some arbitrary 2^N), you only count it if it's placed at the beginning of the 2^N chunk (and you count it as having size 2^N)? >> - one `static unsigned long smallest_recent_hoard_size`. >> This is updated whenever we allocate memory from the OS. > We map huge chunks at a time, but they stay unbacked until they're > actually used. By "whenever we allocate memory from the OS" I really meant "in a code path that's not speed critical but which can be expected to be executed eventually". Maybe there isn't any such path except for those that may never be executed? >> Then you'd call `malloc_trim` based on a magic formula > That's cheating ;-) Like the rest of my hand waving wasn't? C'mon, give me some credit! [ Just to clarify: by "magic formula" I meant that this would be the part amenable to tuning to decide how often we're willing to pay for `malloc_trim`. ] >>> Yet another word of memory used, >> Since 200MB is peanuts, I figure that extra 24B should be acceptable ;-) > *per chunk*. No, those counters are meant to be global. Anyway. I see that we should presume that our glibc will not release our memory back to the OS unless we explicitly ask it to with `malloc_trim`. Thanks. One thing that remains a bit unclear for me is in the doc of `malloc_trim`; it says: The pad argument specifies the amount of free space to leave untrimmed at the top of the heap. If this argument is 0, only the minimum amount of memory is maintained at the top of the heap (i.e., one page or less). A nonzero argument can be used to maintain some trailing space at the top of the heap in order to allow future allocations to be made without having to extend the heap with sbrk(2). But this only talks about the free space at the "top". Since in our case most of the free space is not at the top, I wonder: say we have 64KB free at the top and 64MB free elsewhere and we call `malloc_trim` with a `pad` of 16MB, will it release ~48MB of the non-top free memory or will it free all 64MB of the non-top free memory or ... ? Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 19:32:07 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 00:32:07 +0000 Received: from localhost ([127.0.0.1]:39234 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7SZ5-0003K3-6u for submit@debbugs.gnu.org; Wed, 03 Feb 2021 19:32:07 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:41751) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7SZ3-0003Jv-Bm for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 19:32:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612398724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=yhHAm/K8uHsI7k/9aSpwqXZzhx1pEc7lLUoKf7Kfb6U=; b=OzF1gDxXM4/BVEkHhEh04zHsd/9AcYB7PVQDIItHqB5J3Mgh6J7HzykWYVzCuvl1qj73LP zZWYzqpo6kv0Hp5S53DmRIuJ+6f02f2vsXeTp+N+8Qwwuz0L+crnele+TOM+m05ya4Jzif dbIhCTg5l3f0+u8AVa4x3VwtGME3dJo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-436-QE3HWAlhPCCp6fEsGJXuow-1; Wed, 03 Feb 2021 19:32:03 -0500 X-MC-Unique: QE3HWAlhPCCp6fEsGJXuow-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E1C1B801963; Thu, 4 Feb 2021 00:32:01 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 965FD1971D; Thu, 4 Feb 2021 00:31:58 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 1140VuJA005176; Wed, 3 Feb 2021 19:31:56 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 18:32:36 -0500) Date: Wed, 03 Feb 2021 19:31:56 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: >>>> Yet another word of memory used, >>> Since 200MB is peanuts, I figure that extra 24B should be acceptable ;-) >> *per chunk*. > > No, those counters are meant to be global. To clarify, that extra word was the "chunk age" information, not the aggregate metrics. > One thing that remains a bit unclear for me is in the doc of > `malloc_trim`; it says: > > The pad argument specifies the amount of free space to leave untrimmed > at the top of the heap. If this argument is 0, only the minimum amount > of memory is maintained at the top of the heap (i.e., one page or > less). A nonzero argument can be used to maintain some trailing space > at the top of the heap in order to allow future allocations to be made > without having to extend the heap with sbrk(2). > > But this only talks about the free space at the "top". Since in our > case most of the free space is not at the top, I wonder: > say we have 64KB free at the top and 64MB free elsewhere and we call > `malloc_trim` with a `pad` of 16MB, will it release ~48MB of the > non-top free memory or will it free all 64MB of the non-top free memory > or ... ? No. Even the tiniest allocation still in use at the top of the heap locks the entire rest of the heap into memory. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 22:26:40 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 03:26:40 +0000 Received: from localhost ([127.0.0.1]:39367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VHz-0007ei-UB for submit@debbugs.gnu.org; Wed, 03 Feb 2021 22:26:40 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:8700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VHx-0007eR-VA for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 22:26:38 -0500 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 775E9101F42; Wed, 3 Feb 2021 22:26:32 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E70F3100DB3; Wed, 3 Feb 2021 22:26:30 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612409190; bh=Erd9LRnCA71m3WRdClw0r71DFNYgGztwoC9Ig5C1YFA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=MECiFCO+CnzgVFl0fuJZ//Kt8amp+p/Bfuw0WsCfrsAUi4csfqPxIg1lJVyua5sI8 sCYvbud2iaL6UnW8uK23cN0DfKom1JRk2U5A99XtlXbNY6ReVlkkmcSFpiDAWAQ40a nNlZjsNHMrHeZHjpdYqrDs4IPUdxEz7nl2e2SBUBMQ+foRLsbgc9hJPzyZBw+sPHvk ++dwo7g+ijubIQ3ohirM7MWlUPg23d7CHk3PyNMq7gf6q8+CWlaWm0e0xVPWR87PyC Lsk2Y4MQx4Fpa4xZBAHsYw2mR0B24NN04Qh6MykgPkb3T+f6dF3B9VxvMVFzhiY4/p MKaPQ+Hf7x+qw== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9732512021C; Wed, 3 Feb 2021 22:26:30 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 22:26:29 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 19:31:56 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.025 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> But this only talks about the free space at the "top". Since in our >> case most of the free space is not at the top, I wonder: >> say we have 64KB free at the top and 64MB free elsewhere and we call >> `malloc_trim` with a `pad` of 16MB, will it release ~48MB of the >> non-top free memory or will it free all 64MB of the non-top free memory >> or ... ? > > No. Even the tiniest allocation still in use at the top of the heap > locks the entire rest of the heap into memory. Hmm... then I don't understand: the user who reports the problem with Emacs claims that calling `malloc_trim` reduces the PSS size of Emacs tremendously (from 260MB down to 60MB). If `malloc_trim` can't release memory other than at the top, then how come glibc didn't recover those 200MB on its own (e.g. it seems 200MB is well beyond the default value of M_TRIM_THRESHOLD)? Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 22:38:57 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 03:38:57 +0000 Received: from localhost ([127.0.0.1]:39382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VTt-00084n-3X for submit@debbugs.gnu.org; Wed, 03 Feb 2021 22:38:57 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29383) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VTq-00084Y-IE for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 22:38:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612409933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=c40yvS0b5LKCGZXos/gmGw4miP9/1IvgbQ7sTHUaA68=; b=MFnkFcj/h6s0K9rYyruP7+1YmvP/jN9+dbPChgm546/3g5OOl8GsdVTAkwcvf2RNHq0PSg uORO/qpSfV7m+6djzo97CndYSsAtqVK8qydiQ1pImmhbSAB5+2x3TDVkFDW75MYtcoT4sf vVTFdieXeMKpj5EgJu7kCBpT3k0Tqe4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-emu34IGNMSWLr8kBJTCHtA-1; Wed, 03 Feb 2021 22:38:11 -0500 X-MC-Unique: emu34IGNMSWLr8kBJTCHtA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4A6F015720; Thu, 4 Feb 2021 03:38:10 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F127D722D1; Thu, 4 Feb 2021 03:38:06 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 1143c4E3000388; Wed, 3 Feb 2021 22:38:05 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 22:26:29 -0500) Date: Wed, 03 Feb 2021 22:38:04 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: >> No. Even the tiniest allocation still in use at the top of the heap >> locks the entire rest of the heap into memory. > > Hmm... then I don't understand: the user who reports the problem with > Emacs claims that calling `malloc_trim` reduces the PSS size of > Emacs tremendously (from 260MB down to 60MB). > > If `malloc_trim` can't release memory other than at the top, then how > come glibc didn't recover those 200MB on its own (e.g. it seems 200MB > is well beyond the default value of M_TRIM_THRESHOLD)? Well, let me make up a case that "works" and you tell me if this is common in emacs. Consider you've spent the day doing normal things and your heap is at 100 Mb. Now you do something memory-intensive for a few minutes, and stop, and gc runs. That "something" allocates a lot of memory, but it's all going to be at the top of the heap - aside from whatever fits in the first 100 Mb. It's all released, and GC free()'s it all. malloc_trim() at this point would coalesce it and return it to the system. As long as gc runs between each type of "task", a high-memory task will be trimmable because GC would have freed all that memory and it's all together at the top of the heap. What would cause a problem is, say, if in the middle of running a high memory task you *also* load a font or something and *that* locks the top of the heap. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 22:55:35 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 03:55:35 +0000 Received: from localhost ([127.0.0.1]:39387 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Vjz-00008g-Iy for submit@debbugs.gnu.org; Wed, 03 Feb 2021 22:55:35 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:4047) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7Vjy-00008S-7w for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 22:55:34 -0500 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id A2C2480D8E; Wed, 3 Feb 2021 22:55:28 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 126D4805B2; Wed, 3 Feb 2021 22:55:27 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612410927; bh=1A6zsPD3NZCcvjlcjE6Mte4HW7EaGsbNTC1Xfq9ogDI=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=kj5C5U4KBsMtqm42GcnD0r75Oad+j9G/67RlEq9M6PTDMVRyzTh2bcP9lw1u91DEc 1Iq2811vz5t+hBLTl9cScqXxLc/l/9100vJep+NAer54PktPm6zCaV4T5bjPYXBmPc zBy18lftBY5/Ri7duk/MM9sqMJqehNgv7MhI5bIg7K0csshcdw80+0v+IhIeanZMqj CGXUVdWwgx5sWFHLRgVr8LfqJmV65fh3+i6v+ZXIWiNwJm0L6aWjOxb3zx8MG82nOw JArfBzBDI4TK5x74n/hTdG40r/XZNC7ivhDkiVISgQy80EzqlJzKVcmbfvSb5QzvJT mGA3TUWtDBkHw== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A568412021C; Wed, 3 Feb 2021 22:55:26 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 22:55:25 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 22:38:04 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.008 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >>> No. Even the tiniest allocation still in use at the top of the heap >>> locks the entire rest of the heap into memory. >> >> Hmm... then I don't understand: the user who reports the problem with >> Emacs claims that calling `malloc_trim` reduces the PSS size of >> Emacs tremendously (from 260MB down to 60MB). >> >> If `malloc_trim` can't release memory other than at the top, then how >> come glibc didn't recover those 200MB on its own (e.g. it seems 200MB >> is well beyond the default value of M_TRIM_THRESHOLD)? > > Well, let me make up a case that "works" and you tell me if this is > common in emacs. > > Consider you've spent the day doing normal things and your heap is at > 100 Mb. Now you do something memory-intensive for a few minutes, and > stop, and gc runs. That "something" allocates a lot of memory, but it's > all going to be at the top of the heap - aside from whatever fits in the > first 100 Mb. It's all released, and GC free()'s it all. malloc_trim() > at this point would coalesce it and return it to the system. But wouldn't glibc release that memory even without calling `malloc_trim` simply because it's at the top and is larger than `M_TRIM_THRESHOLD`? Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 23:02:17 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 04:02:17 +0000 Received: from localhost ([127.0.0.1]:39391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VqT-0000Nl-BX for submit@debbugs.gnu.org; Wed, 03 Feb 2021 23:02:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21961) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VqR-0000NR-Eq for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 23:02:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612411335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=Zcng9GYw8HRmbv+q7fUqR6Z46XYLA6mL5hxLJk5JSFU=; b=UxWyZ8fN3BNCcwtigj6uzRBJWSt/s1Nk+LhSQ6oT6dIuGd2V9fOXD825mN91tBTiaxybol 5X2cOx2ltmCJT9Rz8avG814sTkzshdSt5zAz0AJpooFE5/aQi0+Av3qksoEhACQF9jLyaj /tGyBYMUri2ijncxT+CmHBzgEkZFp08= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-303-zWMPn8LCMaGNKTV4UbCUTg-1; Wed, 03 Feb 2021 23:02:11 -0500 X-MC-Unique: zWMPn8LCMaGNKTV4UbCUTg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 16CF7801961; Thu, 4 Feb 2021 04:02:10 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5FF145B695; Thu, 4 Feb 2021 04:02:06 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 1144255k000585; Wed, 3 Feb 2021 23:02:05 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 22:55:25 -0500) Date: Wed, 03 Feb 2021 23:02:05 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: > But wouldn't glibc release that memory even without calling > `malloc_trim` simply because it's at the top and is larger than > `M_TRIM_THRESHOLD`? Probably, if you disabled fastbins. Like I said earler, I think fastbins bypasses the top-of-heap consolidation, so the chunks might be free'd but in the fastbin cache. malloc_trim() flushes the fastbin cache. There are other ways to flush the fastbins but they're harder to programmatically invoke (i.e. "magic rules"). From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 23:04:26 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 04:04:26 +0000 Received: from localhost ([127.0.0.1]:39395 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VsX-0000S7-Ne for submit@debbugs.gnu.org; Wed, 03 Feb 2021 23:04:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51991) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7VsW-0000Rt-1i for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 23:04:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612411464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=wgnqOul7xoSElbgLydWg9wt+TGW08yqubvnxXQrctv4=; b=VbTa7hffSEehQmq+3s2CGuWwjtOQscjvIY04GNMLsSbGpoxDAOb8ze00P4ueBLYb8yYPKw MgPmDr8CnvvxFYgUFe595vqMBG+xTx+ktcWeN6k1apJHZt5rQNJ4Zv3aygGUq3RA7HvSoR dsVl3W+rKKbutwtq2aISSx4Hjkrx42I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-419-EEinBddfNNOQ0i3NF-Vu7Q-1; Wed, 03 Feb 2021 23:04:20 -0500 X-MC-Unique: EEinBddfNNOQ0i3NF-Vu7Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 224C7801960; Thu, 4 Feb 2021 04:04:19 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C70FB101E581; Thu, 4 Feb 2021 04:04:15 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 11444ENe000606; Wed, 3 Feb 2021 23:04:14 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 22:55:25 -0500) Date: Wed, 03 Feb 2021 23:04:14 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Also, fastbins bypass the chunk consolidation, so even if all the memory is free()'d it might still be in small-chunk form in the fastbins, ready for fast re-use. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 23:19:50 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 04:19:50 +0000 Received: from localhost ([127.0.0.1]:39411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7W7R-00010S-PH for submit@debbugs.gnu.org; Wed, 03 Feb 2021 23:19:50 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:56960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7W7Q-00010B-Kj for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 23:19:48 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1AAE84409AE; Wed, 3 Feb 2021 23:19:43 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id B1F56440996; Wed, 3 Feb 2021 23:19:41 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1612412381; bh=iuMj4I8c9DPAjiv8u/wJVLgb5fTDByuF7DrGwU1WSvI=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=OFFoFmj/CQB/zPP4S+RD/FdMJxqbbUKuTTtrsKuzFkFAnRQqHwZZzv0utiop+6N2I EAKFU2XQv8/5RCm5+jMB3tXmg9K6RkUH0DJQCJZXcUhVySDZ4KWO21jcb2tTO+ynuC QVr9GiZs7FiHErGFIjWR35kktiI07EZ3azkXuMgHeBnFMcn3eIa/K/C9X2u81EFPG2 +LoYKE7NUXEWVQZuEmJGnTiEwFG/DNjfLXNOOZ18RAMO0z6FjMpqTkATE1/M5G+zxL ErOrqwS7HD9jxTSYrj3V2GGv7zU2zpGxIYhwYvkLah9EX+FsKnRH2zd/vf0AJsWUYx bI3OuCRfcpWhw== Received: from alfajor (76-10-182-85.dsl.teksavvy.com [76.10.182.85]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 59E191204AE; Wed, 3 Feb 2021 23:19:41 -0500 (EST) From: Stefan Monnier To: DJ Delorie Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: Date: Wed, 03 Feb 2021 23:19:40 -0500 In-Reply-To: (DJ Delorie's message of "Wed, 03 Feb 2021 23:02:05 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.096 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> But wouldn't glibc release that memory even without calling >> `malloc_trim` simply because it's at the top and is larger than >> `M_TRIM_THRESHOLD`? > Probably, if you disabled fastbins. Like I said earler, I think > fastbins bypasses the top-of-heap consolidation, so the chunks might be > free'd but in the fastbin cache. malloc_trim() flushes the fastbin > cache. > There are other ways to flush the fastbins but they're harder to > programmatically invoke (i.e. "magic rules"). > Also, fastbins bypass the chunk consolidation, so even if all the memory > is free()'d it might still be in small-chunk form in the fastbins, ready > for fast re-use. [ I have no idea what "fastbins" are, but I'll nod along approvingly. ] OK, so `malloc_trim` can do a more thorough job and discover that there's indeed 200MB free at the top which malloc/free wouldn't have noticed? I guess that makes sense. Since our GC doesn't move objects, there will often be some left over live object near the top that will prevent most of those mythical 200MB from being released. Anyway, thanks for bearing with us and hand-holding me until I start to feel like I understand the situation. I have a much better understanding of what Emacs might be able to do. Stefan PS: One last thing: I must say I'm surprised to see the amount of effort that seems to be spent on trying to release that "top" free memory. >From where I stand, it seems that it will only very rarely allow you to release any non-trivial amount of memory. Why bother? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 03 23:26:58 2021 Received: (at 45200) by debbugs.gnu.org; 4 Feb 2021 04:26:59 +0000 Received: from localhost ([127.0.0.1]:39415 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7WEM-0001I2-Jw for submit@debbugs.gnu.org; Wed, 03 Feb 2021 23:26:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:36405) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l7WEJ-0001Hm-JQ for 45200@debbugs.gnu.org; Wed, 03 Feb 2021 23:26:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612412814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=btVU+mPuaY2HZZ2utA406Vd1cORyaIgD6kRWEVkY8I8=; b=H3kJib1PHlwplAwR9Acm2e6WV+MplgX48AeksuW4qwJUDOM89JY5ZfWjirl2qEmg4N991v MrgYe0WM7IK9qgX8r0a15jjm8J5suJ8eCClp0Ph2NzwtLy6fcRu5qSeOQILg7rVTTQqE4P tW1D06qb5gCiT9L6qZkcaXC/ercZZH4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-92-ZTNyMHMTPaelbhIXB-w1Hw-1; Wed, 03 Feb 2021 23:26:52 -0500 X-MC-Unique: ZTNyMHMTPaelbhIXB-w1Hw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C4FFD107ACED; Thu, 4 Feb 2021 04:26:50 +0000 (UTC) Received: from greed.delorie.com (ovpn-114-77.rdu2.redhat.com [10.10.114.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7502C71C8B; Thu, 4 Feb 2021 04:26:47 +0000 (UTC) Received: from greed.delorie.com.redhat.com (localhost [127.0.0.1]) by greed.delorie.com (8.14.7/8.14.7) with ESMTP id 1144Qj8g000797; Wed, 3 Feb 2021 23:26:46 -0500 From: DJ Delorie To: Stefan Monnier Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed In-Reply-To: (message from Stefan Monnier on Wed, 03 Feb 2021 23:19:40 -0500) Date: Wed, 03 Feb 2021 23:26:45 -0500 Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dj@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, carlos@redhat.com, rudalics@gmx.at, hi-angel@yandex.ru, eliz@gnu.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Stefan Monnier writes: > [ I have no idea what "fastbins" are, but I'll nod along approvingly. ] https://sourceware.org/glibc/wiki/MallocInternals Fastbins are one of the caches layered on the core malloc code. > OK, so `malloc_trim` can do a more thorough job and discover that > there's indeed 200MB free at the top which malloc/free wouldn't > have noticed? I guess that makes sense. Yes. > PS: One last thing: I must say I'm surprised to see the amount of > effort that seems to be spent on trying to release that "top" free > memory. From where I stand, it seems that it will only very rarely > allow you to release any non-trivial amount of memory. Why bother? We have to keep track of the top of heap anyway, in order to grow it. Adding a check to shrink it is pretty trivial, and it's our *only* way to return memory to the system. From debbugs-submit-bounces@debbugs.gnu.org Tue May 18 16:13:04 2021 Received: (at 45200) by debbugs.gnu.org; 18 May 2021 20:13:04 +0000 Received: from localhost ([127.0.0.1]:57384 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lj65Q-0008RY-95 for submit@debbugs.gnu.org; Tue, 18 May 2021 16:13:04 -0400 Received: from forward106o.mail.yandex.net ([37.140.190.187]:59656) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lj65N-0008Qa-6U for 45200@debbugs.gnu.org; Tue, 18 May 2021 16:13:03 -0400 Received: from sas1-de9e0dd89469.qloud-c.yandex.net (sas1-de9e0dd89469.qloud-c.yandex.net [IPv6:2a02:6b8:c14:3997:0:640:de9e:dd8]) by forward106o.mail.yandex.net (Yandex) with ESMTP id 2F5945060765; Tue, 18 May 2021 23:12:54 +0300 (MSK) Received: from sas1-27140bb19246.qloud-c.yandex.net (sas1-27140bb19246.qloud-c.yandex.net [2a02:6b8:c08:1803:0:640:2714:bb1]) by sas1-de9e0dd89469.qloud-c.yandex.net (mxback/Yandex) with ESMTP id me84m2Lgkc-CrKiLuZP; Tue, 18 May 2021 23:12:54 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1621368774; bh=u5r/UPXFKIwXeCWWsxWLy+cP3Gl9q8vZ4QltKGsp5xs=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=p1xNqUNKJLdkzxCk5KLnHawRvwI3/BS0Oy/w84hKI6kC1bOWa0ZzSZ3yLLgnYstEi jTQH14QEU56eesgRCgmOICCUUW3eS+Dbv0Ot82CEx7T3BVwlSAw9yPhIFv5VGzpffk a8Jd+GDAtwj0m0sgzcp80cJSKOqBJJmbrLY5Sd2w= Authentication-Results: sas1-de9e0dd89469.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-27140bb19246.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id Q7m2S8yAr8-CrLqMBGm; Tue, 18 May 2021 23:12:53 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier Date: Tue, 18 May 2021 23:12:53 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Okay, apparently I'm not the only one affected by the memory problem¹, so I'll try to make the necessary change for it to go over the finish line. I have a question though. So, wrapping `malloc_trim` in elisp and adding it to idle with `(run-with-idle-timer)` sounds simple. But, where in code should I do the `(run-with-idle-timer)` call, i.e. so it's called as is emacs is getting initialized? 1: https://www.reddit.com/r/emacs/comments/n13v5l/what_is_the_next_big_feature_after_native_comp/gwc5g77/?context=3 On Wed, 2021-02-03 at 11:02 -0500, Stefan Monnier wrote: > > To answer you question in another email about memory benefits given default > > Emacs settings: well, today I tried creating a testcase that would reproduce > > problem with default settings, but haven't really succeeded at it.  I have > > a testcase where Emacs without the patch has ≈20M more memory than the one > > with the patch, but that's pretty small difference, and offhand I didn't > > manage to get it increased any further. > > Thanks.  At least that seems to indicate that glibc does its job > properly for the way we normally use it. > > > So, I'm thinking of wiring the functional of memory trimming to on-idle > > hook, without possibility to disable it. > > That seems hard to do (luckily), since AFAICT idle hooks only exist via > `run-with-idle-timer` and those can always be disabled with things like > `cancel-timer`. > > > Given how small performance impact in this case would be, I see no > > reason to even implement an option to disable it. > > Thoughts? > > My main thought is that if `malloc_trim` indeed makes a significant > difference, it's a sign that Emacs's own code did its job (it called > `free` as it should) and that the problem is in how glibc decided not > to return the memory to the OS. > > That's a behavior that can (and will) change over time outside of > our control.  So calling `malloc_trim` every time I stop typing for 10s, > just on account of "maybe glibc didn't reclaim quite as much memory as > we'd like this time" doesn't sound very appealing to me. > > It sounds like an ad-hoc quick hack.  I love being able to use ad-hoc > quick hacks, but I don't like enabling such things by default when the > only use-cases where they're known to be useful are fairly specialized > (and discouraged by Emacs maintainers ;-) > > So I think we need more info: do the glibc maintainers consider it > normal for glibc to behave this way?  Why does it behave this way? > Would the equivalent of `malloc_trim` happen anyway "at some point in > the future"?  E.g. If you create a test case where you disable GC, let > the memory use grow to 1GB, then reset the GC vars to their default and > keep using Emacs modestly, will the memory ever be returned to the OS or > is an explicit call to `malloc_trim` really indispensable? > > But until we get all the answers to these questions, we can already > install the code that exposes `malloc_trim` to ELisp. > > >         Stefan > From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 00:11:48 2021 Received: (at 45200) by debbugs.gnu.org; 19 May 2021 04:11:48 +0000 Received: from localhost ([127.0.0.1]:57728 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljDYh-0007F7-RV for submit@debbugs.gnu.org; Wed, 19 May 2021 00:11:48 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:12125) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljDYf-0007Eu-Pe for 45200@debbugs.gnu.org; Wed, 19 May 2021 00:11:47 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 84455807A1; Wed, 19 May 2021 00:11:40 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id AD18680792; Wed, 19 May 2021 00:11:38 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1621397498; bh=3DKKlUFcRxJ6kEgQVaNvE1l2P7Vc3sJ2HfiSiNxQQOw=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=AzYi9RMBEtePMhCq3sI7ST2TkPDAnbJh8MWZsI7AL/vAE75fPe7YAY2iFELpwMsIm cH6U/2XR4LCWpb076S4/1X6L8eMe6qlCkHWg/Dlkk6JWVp8/ewrd9daYHc4rpHrRO1 q8UbbIWBiaHvcVyx1pPXS9Ewc55WTAqGV8QxZo4SfhN8QzpbB8TJoV+ut/B22LUCkg jyrlXZ2+pCrCqZ46bnUmPjVYvjFbN8B0Mpv7jI2UDEtV0RnYgYgDaq/8Ic3hGmGE7n pqRzHtDzB1fUcrmyZhSsyJVtsyGmL+w5Nep1eGBBMg3Boa+lx5ZbhyWw24wcCNSIB8 9DKi77VLCWVSQ== Received: from alfajor (76-10-140-76.dsl.teksavvy.com [76.10.140.76]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 5F31F120429; Wed, 19 May 2021 00:11:38 -0400 (EDT) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Date: Wed, 19 May 2021 00:11:32 -0400 In-Reply-To: (Konstantin Kharlamov's message of "Tue, 18 May 2021 23:12:53 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.036 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Okay, apparently I'm not the only one affected by the memory problem=C2= =B9, so I'll > try to make the necessary change for it to go over the finish line. Sadly, malloc_trim does not satisfy the request "a GC that actually releases the memory back to the system": we do release it back to glibc but glibc only does it when we're lucky enough that the memory we released is all at the very end of the heap (and that constraint apparently also applies to `mallov_trim`). A concurrent GC would be great, yes. XEmacs had that, pretty much (not concurrent, only incremental, but it should not be terribly hard to push it over the edge and make it concurrent). > I have a question though. So, wrapping `malloc_trim` in elisp and adding = it to > idle with `(run-with-idle-timer)` sounds simple. But, where in code shoul= d I do > the `(run-with-idle-timer)` call, i.e. so it's called as is emacs is gett= ing > initialized? I'd oppose calling `malloc-trim` in the default config. So the answer would be "in your ~/.emacs" (presumably next to the code that changes the GC config vars ;-) Stefan > > 1: > https://www.reddit.com/r/emacs/comments/n13v5l/what_is_the_next_big_featu= re_after_native_comp/gwc5g77/?context=3D3 > > On Wed, 2021-02-03 at 11:02 -0500, Stefan Monnier wrote: >> > To answer you question in another email about memory benefits given de= fault >> > Emacs settings: well, today I tried creating a testcase that would rep= roduce >> > problem with default settings, but haven't really succeeded at it.=C2= =A0 I have >> > a testcase where Emacs without the patch has =E2=89=8820M more memory = than the one >> > with the patch, but that's pretty small difference, and offhand I didn= 't >> > manage to get it increased any further. >>=20 >> Thanks.=C2=A0 At least that seems to indicate that glibc does its job >> properly for the way we normally use it. >>=20 >> > So, I'm thinking of wiring the functional of memory trimming to on-idle >> > hook, without possibility to disable it. >>=20 >> That seems hard to do (luckily), since AFAICT idle hooks only exist via >> `run-with-idle-timer` and those can always be disabled with things like >> `cancel-timer`. >>=20 >> > Given how small performance impact in this case would be, I see no >> > reason to even implement an option to disable it. >> > Thoughts? >>=20 >> My main thought is that if `malloc_trim` indeed makes a significant >> difference, it's a sign that Emacs's own code did its job (it called >> `free` as it should) and that the problem is in how glibc decided not >> to return the memory to the OS. >>=20 >> That's a behavior that can (and will) change over time outside of >> our control.=C2=A0 So calling `malloc_trim` every time I stop typing for= 10s, >> just on account of "maybe glibc didn't reclaim quite as much memory as >> we'd like this time" doesn't sound very appealing to me. >>=20 >> It sounds like an ad-hoc quick hack.=C2=A0 I love being able to use ad-h= oc >> quick hacks, but I don't like enabling such things by default when the >> only use-cases where they're known to be useful are fairly specialized >> (and discouraged by Emacs maintainers ;-) >>=20 >> So I think we need more info: do the glibc maintainers consider it >> normal for glibc to behave this way?=C2=A0 Why does it behave this way? >> Would the equivalent of `malloc_trim` happen anyway "at some point in >> the future"?=C2=A0 E.g. If you create a test case where you disable GC, = let >> the memory use grow to 1GB, then reset the GC vars to their default and >> keep using Emacs modestly, will the memory ever be returned to the OS or >> is an explicit call to `malloc_trim` really indispensable? >>=20 >> But until we get all the answers to these questions, we can already >> install the code that exposes `malloc_trim` to ELisp. >>=20 >>=20 >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Stefan >>=20 From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 00:26:16 2021 Received: (at 45200) by debbugs.gnu.org; 19 May 2021 04:26:16 +0000 Received: from localhost ([127.0.0.1]:57735 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljDmi-0007aD-A1 for submit@debbugs.gnu.org; Wed, 19 May 2021 00:26:16 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:46963) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljDmg-0007a0-Jt for 45200@debbugs.gnu.org; Wed, 19 May 2021 00:26:14 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 4D2E480625; Wed, 19 May 2021 00:26:09 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 109F680169; Wed, 19 May 2021 00:26:06 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1621398366; bh=SRpqRBiY2qiCIavtrHnNb1ld7d4wGD/kyNXGSfVJbDA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=MfsuVSGKncXG8YaszBn0YBik42A/3xV+DJf4wl9fujUdDbPP2oLVLbyXOKGGGseFt RxwpliTL/vah2mv7/ae6ffeX+BfFQrJ2LwcPi+SdbA4jcRJ+6ly7xCt6xN9/bKqdKu lr9TfYHQ5kx2cR1CC5obI7lFu3JNf1YG2X+qIY/NrD0NFCczait8U/feH8phMtakjb cuFYzAO1MWU9W6W2LlOPc9ZW7pbhtxMNsP++OeXkxcBBbQN5ex9yKCZ+DNN6ePzMrm xuXHM5Xc0c0aIMPFO9F3kiMNyzKccIW7bd5kExXXS37p70ouUduhHEY69Wn8vX70T+ VGMja0oGQYI3Q== Received: from alfajor (76-10-140-76.dsl.teksavvy.com [76.10.140.76]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9D642120262; Wed, 19 May 2021 00:26:05 -0400 (EDT) From: Stefan Monnier To: Konstantin Kharlamov Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Date: Wed, 19 May 2021 00:26:04 -0400 In-Reply-To: (Stefan Monnier's message of "Wed, 19 May 2021 00:11:32 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.036 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I'd oppose calling `malloc-trim` in the default config. BTW, to clarify, I'd oppose it based on the current data we have. If we can collect data that shows it to be sufficiently useful for the default config, then of course, I'd have no objection, Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 02:46:41 2021 Received: (at 45200) by debbugs.gnu.org; 19 May 2021 06:46:41 +0000 Received: from localhost ([127.0.0.1]:57813 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljFyT-0002hP-N6 for submit@debbugs.gnu.org; Wed, 19 May 2021 02:46:41 -0400 Received: from forward102p.mail.yandex.net ([77.88.28.102]:40287) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljFyP-0002h9-Vv for 45200@debbugs.gnu.org; Wed, 19 May 2021 02:46:31 -0400 Received: from sas1-121c281eae19.qloud-c.yandex.net (sas1-121c281eae19.qloud-c.yandex.net [IPv6:2a02:6b8:c08:cf03:0:640:121c:281e]) by forward102p.mail.yandex.net (Yandex) with ESMTP id 9E44654C204E; Wed, 19 May 2021 09:46:22 +0300 (MSK) Received: from sas1-e00c2743cdb8.qloud-c.yandex.net (sas1-e00c2743cdb8.qloud-c.yandex.net [2a02:6b8:c14:3a22:0:640:e00c:2743]) by sas1-121c281eae19.qloud-c.yandex.net (mxback/Yandex) with ESMTP id Re7Gcf3BEr-kMJeVahx; Wed, 19 May 2021 09:46:22 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1621406782; bh=XQAyBTr3wygmf5SjrAfChzxbas09GhBFnpVA2PaJVvI=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=dZGr5DhFcF55De7e0bAG7gt+tPDkJO9fpX/4kuzgCo5YjYu2Ngf54f5/OYHkFaytt v/xmDXALOnfXM1geVCt1/JjNMuSVQkYDO4ilH+sZhxi74T8btMvoiUN0SzGv58/Rc8 rJMtIIaXSD3P/XaMLzRUCZFkvvMBI7qPsk9DdqA8= Authentication-Results: sas1-121c281eae19.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-e00c2743cdb8.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id RNslD1Hz4k-kLLCcbQC; Wed, 19 May 2021 09:46:21 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <054dec43be450abe6f7f359f7580992ae22bc8a4.camel@yandex.ru> Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Stefan Monnier Date: Wed, 19 May 2021 09:46:21 +0300 In-Reply-To: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45200 Cc: carlos@redhat.com, martin rudalics , dj@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Sorry, a bit offtopic: I noticed, the debbugs page for this report does not show email anymore https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45200 the note at the bottom says This bug report was last modified 102 days ago. Apparently the bugtracker broke. Any ideas, where should I report it? From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 05:47:18 2021 Received: (at submit) by debbugs.gnu.org; 19 May 2021 09:47:18 +0000 Received: from localhost ([127.0.0.1]:58034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljInO-0007bV-1w for submit@debbugs.gnu.org; Wed, 19 May 2021 05:47:18 -0400 Received: from lists.gnu.org ([209.51.188.17]:43068) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljInL-0007bH-Ho for submit@debbugs.gnu.org; Wed, 19 May 2021 05:47:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44540) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljInK-0004Km-MP for bug-gnu-emacs@gnu.org; Wed, 19 May 2021 05:47:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43066) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljInH-0007AB-Uj; Wed, 19 May 2021 05:47:13 -0400 Received: from [2a02:14f:2:821f::1] (port=55250) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ljInH-0008Ex-Dd; Wed, 19 May 2021 05:47:11 -0400 Date: Wed, 19 May 2021 12:47:09 +0300 User-Agent: K-9 Mail for Android In-Reply-To: <054dec43be450abe6f7f359f7580992ae22bc8a4.camel@yandex.ru> References: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> <054dec43be450abe6f7f359f7580992ae22bc8a4.camel@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Autocrypt: addr=eliz@gnu.org; keydata= mQENBF+pf4UBCAC6vjkWLSAsQpe8YIGKLQzNOJx/IjGtCdFF8uzmO5jmME+SD8ROuJN+t5KXVw58 uzu75EFD0vHTY9e+udJ2gkpuy0NnzkFcbumdLLo2ERKCoSctZZRhzKXI5z5cHxCqW0B2ygHRrRLt oNlGID7bAgcgSViT1ptGqTXO7zGVu4Airok7dNzcPtHgns8GlR5YAFX0TvE6oGd0l2VPghNeVJKJ OjrbfhoDxl3ucFpqbqMH8z9HTLDOFpz8UaYYUdJMi3xX6vwTZxI2sM2RRVLUpZyllAkSMI4lln1O OgazM/62DJUs/rKIHKBnF6h3/qsJUjUYXaAHbrXY26mWllAd536lABEBAAG0I0VsaSBaYXJldHNr aWkgKGVsaXopIDxlbGl6QGdudS5vcmc+iQE4BBMBAgAiBQJfqX+FAhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAAKCRCRwSYvAeuNOYUQB/4/iIKKOG45ijNaRoTvmJJZMvj1S07WQxEm7c5SHEeE QbLOAxB9vESOV7sLueuN3oqEndtzyYt4x1WTSBmHFF7h5fcCMjBs41siOIp5Sj/xD0Bvaa0IKGCR SZ7PAo8Mq3wgajXpTpn9vxE2PmtzA8KdEE0K1+f9pVAfOpUIcCl44rIxLUW352XG0y7iz6c/O6LB 1deOKMiKFctKO7pBti1dJEm1ImewLH3H8uTbwspLOs3EB8xhsESxmTidnze68HX2jt+2EeMgCdki NU+LWbexQZPfIS7+ZmE06ll0v6+Jy7ZdTkCCRypKWTnW7pIFsq/p4kybV8O/kHSV6B4vvQBfuQEN BF+pf4UBCACvFrdx/m22lgObypSmSS4TNlNvQnMUorrMmp0U32hv5adt6CKXeMjk05F+GcIfVMrp xqMBn4sEUIXWhhogQJa9ZbWEP/HbS8XjMMbz0Q0Siaty9+DSspK/9u2GWKsz3uQzLCexIJtzmXvj AVmvoMCAU/F2t038ggygjYLRgyLRNLgbbartu2dMkvrfxRjheip60S4S3utOcwUf/qdoa1grNann CFluHr/ftXCeeuGB4H8iO0BXWNby6NZPizxJttx9gdcH8/OmDOJkXyRMTT/3sSem76CSOjfXcz7s aJlg680NQhG5TmuYERjJD4+U02K5RuqTsEnOuWeFy4p+/mslABEBAAGJAR8EGAECAAkFAl+pf4UC GwwACgkQkcEmLwHrjTno7Af/a1XoLHxAUkS43nmF8iazn3ZnuwWKWLEAsNrxk56yUxhUPRzNs0/f sABDQR1o0DyTqbScKOcOMSG2YMCctLiDd7FdfMWwkUsV9GUpPBiRtD60Ewmn9sbNJKrEoZ5L6sqO UEslJRVABu5taOzVIRfeUPPaMRjvCcr0d+epKjW81J9Aqj8SskuNkHwvHchTYFYVT22aemjjZ1MG OUm7QiybWQgYL6aSPV2gR+NQQ7pEhOBoEi6GLEiBkoYOIXvmxsqQLBrUPbsJq8lItYEaw4HGt8Ba PxtK2yZ9mSqC2xhWYr1j1YAIHffzubC0jxc5znXERsRANoJOwNUXmiddD7UM9A== Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: bug-gnu-emacs@gnu.org, Konstantin Kharlamov , Stefan Monnier From: Eli Zaretskii Message-ID: <86516D34-F7E3-4513-B201-4ECCB2A91321@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, dj@redhat.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On May 19, 2021 9:46:21 AM GMT+03:00, Konstantin Kharlamov wrote: > Sorry, a bit offtopic: I noticed, the debbugs page for this report > does not show > email anymore https://debbugs=2Egnu=2Eorg/cgi/bugreport=2Ecgi?bug=3D4520= 0 the > note at > the bottom says=20 >=20 > This bug report was last modified 102 days ago=2E >=20 > Apparently the bugtracker broke=2E Any ideas, where should I report it? There's nothing wrong with debbugs, it's your browser showing you a stale = version of that page=2E Trh reloading it while holding the Shift key or so= mething=2E From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 05:56:00 2021 Received: (at submit) by debbugs.gnu.org; 19 May 2021 09:56:00 +0000 Received: from localhost ([127.0.0.1]:58068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljIvo-0007rg-4J for submit@debbugs.gnu.org; Wed, 19 May 2021 05:56:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:53636) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljIvm-0007rY-IG for submit@debbugs.gnu.org; Wed, 19 May 2021 05:55:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46618) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljIvm-0003On-9H for bug-gnu-emacs@gnu.org; Wed, 19 May 2021 05:55:58 -0400 Received: from forward104p.mail.yandex.net ([77.88.28.107]:47124) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljIve-0004G0-Hy; Wed, 19 May 2021 05:55:57 -0400 Received: from forward103q.mail.yandex.net (forward103q.mail.yandex.net [IPv6:2a02:6b8:c0e:50:0:640:b21c:d009]) by forward104p.mail.yandex.net (Yandex) with ESMTP id AC3C14B015E0; Wed, 19 May 2021 12:55:44 +0300 (MSK) Received: from vla3-a1a926cfe7da.qloud-c.yandex.net (vla3-a1a926cfe7da.qloud-c.yandex.net [IPv6:2a02:6b8:c15:2591:0:640:a1a9:26cf]) by forward103q.mail.yandex.net (Yandex) with ESMTP id A748B61E0005; Wed, 19 May 2021 12:55:44 +0300 (MSK) Received: from vla1-1bc5b51c612f.qloud-c.yandex.net (vla1-1bc5b51c612f.qloud-c.yandex.net [2a02:6b8:c0d:89c:0:640:1bc5:b51c]) by vla3-a1a926cfe7da.qloud-c.yandex.net (mxback/Yandex) with ESMTP id PqwyOi32Fi-thKu5ZbC; Wed, 19 May 2021 12:55:44 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1621418144; bh=wReiCL7zCMF2cUyO+5EpR+NvjrIROxxwKcf0atdD6uI=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=LwMPPPzdVGvuUTvOjoychRZz6NwkjXXN/tklIYVY4g7/v67ZfSJqnY7E5Utrbt8PU KAYIou634ip0FeW1UKeUNWJAL7c/13KJri9KhdCZ/yze5UmCAsplo7XNsGaeB5nJ28 FnEAgUSLVyTc7GRnKT0t3exO9jqOvgvS5vSdFmCg= Authentication-Results: vla3-a1a926cfe7da.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by vla1-1bc5b51c612f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id B8K2sUTIon-thLCcei7; Wed, 19 May 2021 12:55:43 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed From: Konstantin Kharlamov To: Eli Zaretskii , bug-gnu-emacs@gnu.org, Stefan Monnier Date: Wed, 19 May 2021 12:55:43 +0300 In-Reply-To: <86516D34-F7E3-4513-B201-4ECCB2A91321@gnu.org> References: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> <054dec43be450abe6f7f359f7580992ae22bc8a4.camel@yandex.ru> <86516D34-F7E3-4513-B201-4ECCB2A91321@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=77.88.28.107; envelope-from=hi-angel@yandex.ru; helo=forward104p.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, dj@redhat.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) On Wed, 2021-05-19 at 12:47 +0300, Eli Zaretskii wrote: > On May 19, 2021 9:46:21 AM GMT+03:00, Konstantin Kharlamov > wrote: > > Sorry, a bit offtopic: I noticed, the debbugs page for this report > > does not show > > email anymore https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45200 the > > note at > > the bottom says > > > >         This bug report was last modified 102 days ago. > > > > Apparently the bugtracker broke. Any ideas, where should I report it? > > There's nothing wrong with debbugs, it's your browser showing you a stale > version of that page.  Trh reloading it while holding the Shift key or > something. Oh, now it works. For the record, I don't keep the tab opened (there's no reason for me), I always open it anew. So in a sense, the page is always "fresh". So I'm not sure what was the problem, but I confirm that now it works. From debbugs-submit-bounces@debbugs.gnu.org Wed May 19 06:09:42 2021 Received: (at submit) by debbugs.gnu.org; 19 May 2021 10:09:42 +0000 Received: from localhost ([127.0.0.1]:58085 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljJ94-0008Fi-3P for submit@debbugs.gnu.org; Wed, 19 May 2021 06:09:42 -0400 Received: from lists.gnu.org ([209.51.188.17]:37966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ljJ92-0008FU-Q1 for submit@debbugs.gnu.org; Wed, 19 May 2021 06:09:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljJ91-00042j-ON for bug-gnu-emacs@gnu.org; Wed, 19 May 2021 06:09:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43420) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljJ8z-0003ub-T4; Wed, 19 May 2021 06:09:38 -0400 Received: from [2a02:14f:2:821f::1] (port=55282) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ljJ8z-0004SW-8K; Wed, 19 May 2021 06:09:37 -0400 Date: Wed, 19 May 2021 13:09:34 +0300 User-Agent: K-9 Mail for Android In-Reply-To: References: <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> <054dec43be450abe6f7f359f7580992ae22bc8a4.camel@yandex.ru> <86516D34-F7E3-4513-B201-4ECCB2A91321@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Autocrypt: addr=eliz@gnu.org; keydata= mQENBF+pf4UBCAC6vjkWLSAsQpe8YIGKLQzNOJx/IjGtCdFF8uzmO5jmME+SD8ROuJN+t5KXVw58 uzu75EFD0vHTY9e+udJ2gkpuy0NnzkFcbumdLLo2ERKCoSctZZRhzKXI5z5cHxCqW0B2ygHRrRLt oNlGID7bAgcgSViT1ptGqTXO7zGVu4Airok7dNzcPtHgns8GlR5YAFX0TvE6oGd0l2VPghNeVJKJ OjrbfhoDxl3ucFpqbqMH8z9HTLDOFpz8UaYYUdJMi3xX6vwTZxI2sM2RRVLUpZyllAkSMI4lln1O OgazM/62DJUs/rKIHKBnF6h3/qsJUjUYXaAHbrXY26mWllAd536lABEBAAG0I0VsaSBaYXJldHNr aWkgKGVsaXopIDxlbGl6QGdudS5vcmc+iQE4BBMBAgAiBQJfqX+FAhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAAKCRCRwSYvAeuNOYUQB/4/iIKKOG45ijNaRoTvmJJZMvj1S07WQxEm7c5SHEeE QbLOAxB9vESOV7sLueuN3oqEndtzyYt4x1WTSBmHFF7h5fcCMjBs41siOIp5Sj/xD0Bvaa0IKGCR SZ7PAo8Mq3wgajXpTpn9vxE2PmtzA8KdEE0K1+f9pVAfOpUIcCl44rIxLUW352XG0y7iz6c/O6LB 1deOKMiKFctKO7pBti1dJEm1ImewLH3H8uTbwspLOs3EB8xhsESxmTidnze68HX2jt+2EeMgCdki NU+LWbexQZPfIS7+ZmE06ll0v6+Jy7ZdTkCCRypKWTnW7pIFsq/p4kybV8O/kHSV6B4vvQBfuQEN BF+pf4UBCACvFrdx/m22lgObypSmSS4TNlNvQnMUorrMmp0U32hv5adt6CKXeMjk05F+GcIfVMrp xqMBn4sEUIXWhhogQJa9ZbWEP/HbS8XjMMbz0Q0Siaty9+DSspK/9u2GWKsz3uQzLCexIJtzmXvj AVmvoMCAU/F2t038ggygjYLRgyLRNLgbbartu2dMkvrfxRjheip60S4S3utOcwUf/qdoa1grNann CFluHr/ftXCeeuGB4H8iO0BXWNby6NZPizxJttx9gdcH8/OmDOJkXyRMTT/3sSem76CSOjfXcz7s aJlg680NQhG5TmuYERjJD4+U02K5RuqTsEnOuWeFy4p+/mslABEBAAGJAR8EGAECAAkFAl+pf4UC GwwACgkQkcEmLwHrjTno7Af/a1XoLHxAUkS43nmF8iazn3ZnuwWKWLEAsNrxk56yUxhUPRzNs0/f sABDQR1o0DyTqbScKOcOMSG2YMCctLiDd7FdfMWwkUsV9GUpPBiRtD60Ewmn9sbNJKrEoZ5L6sqO UEslJRVABu5taOzVIRfeUPPaMRjvCcr0d+epKjW81J9Aqj8SskuNkHwvHchTYFYVT22aemjjZ1MG OUm7QiybWQgYL6aSPV2gR+NQQ7pEhOBoEi6GLEiBkoYOIXvmxsqQLBrUPbsJq8lItYEaw4HGt8Ba PxtK2yZ9mSqC2xhWYr1j1YAIHffzubC0jxc5znXERsRANoJOwNUXmiddD7UM9A== Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed To: Konstantin Kharlamov , bug-gnu-emacs@gnu.org, Stefan Monnier From: Eli Zaretskii Message-ID: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: carlos@redhat.com, fweimer@redhat.com, 45200@debbugs.gnu.org, dj@redhat.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On May 19, 2021 12:55:43 PM GMT+03:00, Konstantin Kharlamov wrote: > On Wed, 2021-05-19 at 12:47 +0300, Eli Zaretskii wrote: > > On May 19, 2021 9:46:21 AM GMT+03:00, Konstantin Kharlamov > > wrote: > > > Sorry, a bit offtopic: I noticed, the debbugs page for this report > > > does not show > > > email anymore > https://debbugs=2Egnu=2Eorg/cgi/bugreport=2Ecgi?bug=3D45200=C2=A0the > > > note at > > > the bottom says=20 > > >=20 > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0This bug report was = last modified 102 days ago=2E > > >=20 > > > Apparently the bugtracker broke=2E Any ideas, where should I report > it? > >=20 > > There's nothing wrong with debbugs, it's your browser showing you a > stale > > version of that page=2E=C2=A0 Trh reloading it while holding the Shift= key > or > > something=2E >=20 > Oh, now it works=2E=20 >=20 > For the record, I don't keep the tab opened (there's no reason for > me), I always open it anew=2E So in a sense, the page is always "fresh"= =2E > So I'm not sure what was the problem, but I confirm that now it works=2E Browsers ard sneaky, they remember where you have been looking even if you= close the tab=2E From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 04 18:20:25 2021 Received: (at 45200) by debbugs.gnu.org; 4 Dec 2021 23:20:25 +0000 Received: from localhost ([127.0.0.1]:55681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mteKP-0004gP-Cf for submit@debbugs.gnu.org; Sat, 04 Dec 2021 18:20:25 -0500 Received: from quimby.gnus.org ([95.216.78.240]:42942) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mteKN-0004g3-C9 for 45200@debbugs.gnu.org; Sat, 04 Dec 2021 18:20:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=yL5Ot2NsHKU0Z5NmjjCFQyRXbi8ugciEE3e+o/vlXoA=; b=q1U15agZO8NlY0Qv6g6w5igH6A 7q9s9gezCObQ0oX4P6MEu5amBWBd5EMYo9nQZmNI2WUAstywlGeGwdOHbjBpnp2h1/y897wxhRy2r REAvYBrqMoF5mH33L/EYXrd4WxWTAmC5gvUPxOblstqwfpTIFvZBzWmF6kr2tbpDN3aE=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mteKD-0001TB-TD; Sun, 05 Dec 2021 00:20:16 +0100 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> X-Now-Playing: Joni Mitchell's _Hejira_: "Amelia" Date: Sun, 05 Dec 2021 00:20:11 +0100 In-Reply-To: <834kj64a36.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 24 Jan 2021 17:40:29 +0200") Message-ID: <87tufovtxw.fsf_-_@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Eli Zaretskii writes: > Thanks, but is it really a good idea to call malloc_trim each time we > free some chunk of memory? Skimming this very long thread, I think the conclusion was that we shouldn't do anything in Emacs on this point. Does that sound correct? Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, monnier@iro.umontreal.ca, Konstantin Kharlamov , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: > Thanks, but is it really a good idea to call malloc_trim each time we > free some chunk of memory? Skimming this very long thread, I think the conclusion was that we shouldn't do anything in Emacs on this point. Does that sound correct? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 04 18:21:06 2021 Received: (at control) by debbugs.gnu.org; 4 Dec 2021 23:21:07 +0000 Received: from localhost ([127.0.0.1]:55690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mteL4-0004jE-Pk for submit@debbugs.gnu.org; Sat, 04 Dec 2021 18:21:06 -0500 Received: from quimby.gnus.org ([95.216.78.240]:42982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mteL3-0004hl-HR for control@debbugs.gnu.org; Sat, 04 Dec 2021 18:21:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=L0CbsavQ1ps8v2jc2BskPM+NraV8LTrQs2DzlhzJy50=; b=Ei43CdN1uD4cj1YMDoX9XLJhfk kbfLf4XjYCLvcn8vuhn1AMzFROFp8Zn6xBXgPiT00IP2kPDHht5m2OKYxYYRtNq79FahXAgaBFT4A mMEVJJmJrKIv/FWQGjolsFsMJN8+z4ahZ4i481kSHtPYkIz+Uf0SofFk3KXnVC2g7gK4=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mteKv-0001TY-O2 for control@debbugs.gnu.org; Sun, 05 Dec 2021 00:21:00 +0100 Date: Sun, 05 Dec 2021 00:20:57 +0100 Message-Id: <87r1asvtwm.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #45200 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: tags 45200 + moreinfo quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 45200 + moreinfo quit From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 01:24:01 2021 Received: (at 45200) by debbugs.gnu.org; 5 Dec 2021 06:24:01 +0000 Received: from localhost ([127.0.0.1]:56121 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtkwL-0006RJ-Fx for submit@debbugs.gnu.org; Sun, 05 Dec 2021 01:24:01 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20791) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtkwJ-0006R1-8v for 45200@debbugs.gnu.org; Sun, 05 Dec 2021 01:23:59 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 70ACD441B1E; Sun, 5 Dec 2021 01:23:53 -0500 (EST) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 07352441B18; Sun, 5 Dec 2021 01:23:52 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1638685432; bh=rIXXN1iE8tA1fXu8e1nkFU/WHndWOPjEksb2e4sdc3g=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=XgnIk1BYJGjipWq3feREihgKS+S8mkv+ApmOB5NGDdGP+Cu2d4U8+ajCKK8fyRRK+ gD2/ZndYLEMpWLksRCqh/EFrvQN7RDS/9fhv1mgdxIY0oSc6ImL7h2C7PKn7zz72u2 DjqvNC6Fd6W/1TeIfc8DCSVEhHXhA0xC4cYPrHOv/tluAwmxVfJkwmaKjaXU+uG1ZE 80mqwotISIiKnDrGiflGrqPMzPCEO7GrI/D7aapg3Uq31Rnd6wdWaot1ZWiz7v/J4R CudY/dwq2y4Hf87YX4RxWv26J+jbDHP9ixxZ7Wamff4ubtEX/kxTs1695RM92591EZ f7eBqpxyEFwDA== Received: from pastel (unknown [216.154.30.173]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A10441202CE; Sun, 5 Dec 2021 01:23:51 -0500 (EST) From: Stefan Monnier To: Lars Ingebrigtsen Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Message-ID: References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> <87tufovtxw.fsf_-_@gnus.org> Date: Sun, 05 Dec 2021 01:23:50 -0500 In-Reply-To: <87tufovtxw.fsf_-_@gnus.org> (Lars Ingebrigtsen's message of "Sun, 05 Dec 2021 00:20:11 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.037 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, Konstantin Kharlamov , Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Thanks, but is it really a good idea to call malloc_trim each time we >> free some chunk of memory? > Skimming this very long thread, I think the conclusion was that we > shouldn't do anything in Emacs on this point. Does that sound correct? For me the conclusion was that I'd be happy to see someone add a `malloc-trim` primitive to ELisp, so that users can experiment with it, e.g. by adding it to `post-gc-hook`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 02:07:43 2021 Received: (at 45200) by debbugs.gnu.org; 5 Dec 2021 07:07:43 +0000 Received: from localhost ([127.0.0.1]:56186 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtlcc-0007qb-RU for submit@debbugs.gnu.org; Sun, 05 Dec 2021 02:07:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48716) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtlcb-0007qP-DA for 45200@debbugs.gnu.org; Sun, 05 Dec 2021 02:07:41 -0500 Received: from [2001:470:142:3::e] (port=60666 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtlcV-0000j2-P2; Sun, 05 Dec 2021 02:07:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=hrcRIOFsPW98pfcdlhDQZiYnleYfrExBoBRgzN7L+Dg=; b=M3J3ZtfV5XUS 1ihuJpVh5JgpZMKja5/5ZxuXL798PVnmv+IuuuEgcuwyBpzrqnutwEeQtkkaKKQuG1sRzQdi5zjnM hZaI3eGexkhRIaIpsVxosQyLbdtAFfSFX0IP98AGWlfJrHtR4pnIXIh4UB90AUVHke2J95QAJCK72 X+uyWxo1L+d/VXCwqb1y2aRIFgtrsNMrq7lydZ+ov/aBNqIncBvoA7G/2yb4nIdmHlu22NCtHWO9v TWk+CAIBEB9f1kSg7NOFNdsoz11hvoSO66zYJhoRbXqQO2DWYKvFmQTHds/oG98jdBSq7egsTin7H HWmR9YBCJcCdnD2GJbEfCA==; Received: from [87.69.77.57] (port=2531 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtlcV-0004Ir-GW; Sun, 05 Dec 2021 02:07:35 -0500 Date: Sun, 05 Dec 2021 09:07:29 +0200 Message-Id: <83y24z4jim.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-Reply-To: <87tufovtxw.fsf_-_@gnus.org> (message from Lars Ingebrigtsen on Sun, 05 Dec 2021 00:20:11 +0100) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> <87tufovtxw.fsf_-_@gnus.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, monnier@iro.umontreal.ca, Hi-Angel@yandex.ru, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Lars Ingebrigtsen > Cc: Konstantin Kharlamov , carlos@redhat.com, > fweimer@redhat.com, dj@redhat.com, monnier@iro.umontreal.ca, > 45200@debbugs.gnu.org > Date: Sun, 05 Dec 2021 00:20:11 +0100 > > Eli Zaretskii writes: > > > Thanks, but is it really a good idea to call malloc_trim each time we > > free some chunk of memory? > > Skimming this very long thread, I think the conclusion was that we > shouldn't do anything in Emacs on this point. Does that sound correct? I don't really know whether there were any conclusions, and what they were. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 03:26:56 2021 Received: (at 45200) by debbugs.gnu.org; 5 Dec 2021 08:26:56 +0000 Received: from localhost ([127.0.0.1]:56243 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtmrI-0001Rs-LZ for submit@debbugs.gnu.org; Sun, 05 Dec 2021 03:26:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtmrH-0001Re-1v for 45200@debbugs.gnu.org; Sun, 05 Dec 2021 03:26:55 -0500 Received: from [2001:470:142:3::e] (port=33494 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtmrB-0004mx-CW; Sun, 05 Dec 2021 03:26:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=2xCovdL+GDkG3KPKiLpYGOBxQ2cvwJ5qNd1VhoNL+6M=; b=Or7JrHzPtRs2 ikj9LbO768/4ILi2HRJGnAUnFnMMcKLe0cjieDAN4xYz+dutG4h+YXR5Z0cOhMa8/0IjXn9UcHN5e Xh43vkMschHOeTrwHzcn0tM9wtOeZ3likSgmek66HqxHy3qbzcPsdGwFYPnu7yxnfQjM0MVNcZtRJ qFEv25WwTSAXzQbjovA6r5BjQhhioyPmfDIJXN9Z9RsgfX4TCjpqtgAZho45T+ZuCMwEdFH33Np/N E7/ckFKHmvC82Y3zqtHksDScP60Wbm0nGUrid+E7jpTufE8eb4g0dGxeKjVnrMJrasAK5dRDNBnN+ H7a9ti2MAD8ErJDhix182Q==; Received: from [87.69.77.57] (port=3415 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtmr2-0003Gh-7E; Sun, 05 Dec 2021 03:26:48 -0500 Date: Sun, 05 Dec 2021 10:26:34 +0200 Message-Id: <83lf0z4fut.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sun, 05 Dec 2021 01:23:50 -0500) Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> <87tufovtxw.fsf_-_@gnus.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, Hi-Angel@yandex.ru, larsi@gnus.org, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: Eli Zaretskii , Konstantin Kharlamov > , carlos@redhat.com, fweimer@redhat.com, > dj@redhat.com, 45200@debbugs.gnu.org > Date: Sun, 05 Dec 2021 01:23:50 -0500 > > >> Thanks, but is it really a good idea to call malloc_trim each time we > >> free some chunk of memory? > > Skimming this very long thread, I think the conclusion was that we > > shouldn't do anything in Emacs on this point. Does that sound correct? > > For me the conclusion was that I'd be happy to see someone add a > `malloc-trim` primitive to ELisp, so that users can experiment with it, > e.g. by adding it to `post-gc-hook`. That might be a good feature regardless, IMO. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 14:59:39 2021 Received: (at 45200) by debbugs.gnu.org; 5 Dec 2021 19:59:39 +0000 Received: from localhost ([127.0.0.1]:59566 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxfe-0003VQ-GO for submit@debbugs.gnu.org; Sun, 05 Dec 2021 14:59:39 -0500 Received: from quimby.gnus.org ([95.216.78.240]:55392) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxfc-0003V9-T1 for 45200@debbugs.gnu.org; Sun, 05 Dec 2021 14:59:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6YMC+JbyHYzcqiVxIUbAQEdwO1T/FUQX5+Ut0aQ4fWQ=; b=hJ70IR4T6PduUlgyUpqQIkY3nK 2bvDHc+Wzx0FCFAPOJoxVmep36o8mQOVk01Czp4hvNYHIbRc/rEvb6xJn8pwwGfm9RGscbD0TFWbm ZC74P9HX2MKMeB1v8yziuSCJon3tV8Ct08TUyKha+IPPTXzEWt2+giZOha/AF7WOmiEQ=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mtxfQ-0002Rb-9r; Sun, 05 Dec 2021 20:59:27 +0100 From: Lars Ingebrigtsen To: Stefan Monnier Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> <87tufovtxw.fsf_-_@gnus.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAADFBMVEUmIiSMfHPIwr// ///XfD68AAAAAWJLR0QDEQxM8gAAAAd0SU1FB+UMBRMvJqYCnRsAAAGPSURBVCjPRdFBa6NAFAfw l1KH6MmCKepJignqp3BDG8KeTFGpPZlFhySfws0hbHMyIUrrKQYDOp9ynTG2c5of/5nHezwAmKtW rRZRAvTMeasWSJTeMPkUiBG2VxuGCq5/IONVC+2GPzXamD0u9cxSpG/4E0Vm1Rjw72GPBuE9/53c rfcvHYwW9X72g8/dlkE2LtHg33HbJxF8XLZ2h6MOX8sDg2QetQHR06RDrklYP9+w0yVs9MjNNX4L HQrRzK0m/tAZNL3YuKsvo0f8Vy7YqKCFTQVyrnRYBCkMXwUG0ePPwD/NJJYEoxOgyW7eYVECsrbP FOPGPwBSpvcUavayhJHnMSjXaZk8ZSZHwV1f63B5eJ9S3JF0gJ6VIqdAxAX/hEhD0S7KsU6P5I2h iVMhfSQVQ3A9ocQnZwreKzMhCQhbKsrLbPbL6yD4ZbEYeBv2R30PC4dbWyxRg2UWcat2DtpbIHoV RDHD2JViByrMno1BenAWFWYFrnD/YCN3lFFkYEsguCKbxwUYiYpjc/Afhu+BQQa9QocAAAAldEVY dGRhdGU6Y3JlYXRlADIwMjEtMTItMDVUMTk6NDc6MzgrMDA6MDCFwShyAAAAJXRFWHRkYXRlOm1v ZGlmeQAyMDIxLTEyLTA1VDE5OjQ3OjM4KzAwOjAw9JyQzgAAAABJRU5ErkJggg== X-Now-Playing: Peter Gabriel's _Peter Gabriel 2_: "White Shadow" Date: Sun, 05 Dec 2021 20:59:19 +0100 In-Reply-To: (Stefan Monnier's message of "Sun, 05 Dec 2021 01:23:50 -0500") Message-ID: <871r2qrffs.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Stefan Monnier writes: > For me the conclusion was that I'd be happy to see someone add a > `malloc-trim` primitive to ELisp, so that users can experiment with it, > e.g. by adding it to `post-gc-hook`. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, Konstantin Kharlamov , Eli Zaretskii , 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Stefan Monnier writes: > For me the conclusion was that I'd be happy to see someone add a > `malloc-trim` primitive to ELisp, so that users can experiment with it, > e.g. by adding it to `post-gc-hook`. Sure, sounds good to me. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 14:59:45 2021 Received: (at control) by debbugs.gnu.org; 5 Dec 2021 19:59:45 +0000 Received: from localhost ([127.0.0.1]:59569 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxfl-0003Vk-3r for submit@debbugs.gnu.org; Sun, 05 Dec 2021 14:59:45 -0500 Received: from quimby.gnus.org ([95.216.78.240]:55412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxfj-0003VP-HR for control@debbugs.gnu.org; Sun, 05 Dec 2021 14:59:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6YnRH0yT0t611802xea2vVpJfKjCXhYxYMUpib22vBk=; b=PNruZkX0LmmCNDB3J046AWoF/2 jnySoz/7+7u+CM28BsW29jpBBKVehCGasezAiaBj2QXUadpU4x9aAo9qMnU4q9UELd3/4rvhrzbbj a2l39aDf4ZLo8GXvO0l7c9aZF9ZmXAAvQqhWVUvO5eNQxH0R1lo/Tb8Ue/lBDZPohnhY=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mtxfb-0002Ro-HX for control@debbugs.gnu.org; Sun, 05 Dec 2021 20:59:38 +0100 Date: Sun, 05 Dec 2021 20:59:35 +0100 Message-Id: <87zgpeq0uw.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #45200 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: severity 45200 wishlist quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) severity 45200 wishlist quit From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 05 15:00:16 2021 Received: (at control) by debbugs.gnu.org; 5 Dec 2021 20:00:16 +0000 Received: from localhost ([127.0.0.1]:59575 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxgG-0003nl-AB for submit@debbugs.gnu.org; Sun, 05 Dec 2021 15:00:16 -0500 Received: from quimby.gnus.org ([95.216.78.240]:55436) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtxgE-0003ex-Gr for control@debbugs.gnu.org; Sun, 05 Dec 2021 15:00:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=7gCfA1hIaA+Hss/NVN7w/qxmg3GuuKMlD+5v9pKMSLc=; b=Shm3Jnsrvld+I0pkm0tyTy3Ob3 GLtp1ZWWunnSKuB3FQ2yjCns54+xc9Vl/6XDSZO/Aq+Z9qiUGsnrJTlPmPeWa40dP2XmDL/XYFp39 mii40YbFeR9qzqTu5/l7EE9qtTYnINS0i8lHFXWuN87YTpmu8Z2T4tpWolzSz1CIdse4=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mtxg5-0002SA-A5 for control@debbugs.gnu.org; Sun, 05 Dec 2021 21:00:08 +0100 Date: Sun, 05 Dec 2021 21:00:04 +0100 Message-Id: <87y24yq0u3.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #45200 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: retitle 45200 Wishlist: There should be a `malloc-trim' function quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) retitle 45200 Wishlist: There should be a `malloc-trim' function quit From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 21 08:48:29 2022 Received: (at control) by debbugs.gnu.org; 21 Jan 2022 13:48:29 +0000 Received: from localhost ([127.0.0.1]:60738 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAuHF-0001vd-8K for submit@debbugs.gnu.org; Fri, 21 Jan 2022 08:48:29 -0500 Received: from quimby.gnus.org ([95.216.78.240]:35478) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAuHD-0001vQ-8T for control@debbugs.gnu.org; Fri, 21 Jan 2022 08:48:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=4/39f2TTv8kS+3s0TGPJnyrHU+TTO8mArXZP/JlRMcA=; b=blSsfgIYQ60lsoq5icMKWCjSgV 8mYWqButCH0PpNVtsvgHsx69JUTBkX3W2c4PM99s9X7sMS6O09T61Eqn1Nmc03qboCYJbtSxZU4TX 4yT2tDAZXXVU0E+2NOvH205Z9J+j0ElHMxpCddY1C7HIJGKzlYC5KRjQr/EXbH3ECXwU=; Received: from [84.212.220.105] (helo=giant) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nAuH5-0004px-JM for control@debbugs.gnu.org; Fri, 21 Jan 2022 14:48:21 +0100 Date: Fri, 21 Jan 2022 14:48:19 +0100 Message-Id: <87v8ydmc7w.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #45200 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: tags 45200 - moreinfo quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 45200 - moreinfo quit From debbugs-submit-bounces@debbugs.gnu.org Sun May 01 05:43:54 2022 Received: (at 45200) by debbugs.gnu.org; 1 May 2022 09:43:54 +0000 Received: from localhost ([127.0.0.1]:59557 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nl67O-0008Dz-1j for submit@debbugs.gnu.org; Sun, 01 May 2022 05:43:54 -0400 Received: from quimby.gnus.org ([95.216.78.240]:57360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nl67M-0008Dl-2p for 45200@debbugs.gnu.org; Sun, 01 May 2022 05:43:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Mnz4632UL0VspzyBj0EsCeaRRGciq6c4H0ngHu62XSw=; b=D9vQeqI46BHTQmfRM8BPIRV4iy 6DPTMKAFgN1y1j2cKiL+eVPLtkhsfSZyq/kKbOb56A8MWmhbXCd5CBV+0ykEWxhpb/6kjOI+weV6V 5gB+7WaLS1vK3X/UIHZJ5E4t/ynyQIeuTuNIDOWS3t3Sn1J9XasjT4LOl3CtyJbQTLiQ=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nl67B-0002cn-Do; Sun, 01 May 2022 11:43:44 +0200 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#45200: Wishlist: There should be a `malloc-trim' function References: <20210124152402.40270-1-Hi-Angel@yandex.ru> <834kj64a36.fsf@gnu.org> <87tufovtxw.fsf_-_@gnus.org> <83lf0z4fut.fsf@gnu.org> X-Now-Playing: David Byrne's _The Catherine Wheel (expanded)_: "Dense Beasts" Date: Sun, 01 May 2022 11:43:40 +0200 In-Reply-To: <83lf0z4fut.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 05 Dec 2021 10:26:34 +0200") Message-ID: <875ympr3jn.fsf_-_@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Eli Zaretskii writes: >> For me the conclusion was that I'd be happy to see someone add a >> `malloc-trim` primitive to ELisp, so that users can experiment with it, >> e.g. by adding it to `post-gc-hook`. > > That might be [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45200 Cc: fweimer@redhat.com, dj@redhat.com, carlos@redhat.com, Stefan Monnier , Hi-Angel@yandex.ru, 45200@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> For me the conclusion was that I'd be happy to see someone add a >> `malloc-trim` primitive to ELisp, so that users can experiment with it, >> e.g. by adding it to `post-gc-hook`. > > That might be a good feature regardless, IMO. I've now added malloc-trim to Emacs 29. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Sun May 01 05:43:56 2022 Received: (at control) by debbugs.gnu.org; 1 May 2022 09:43:56 +0000 Received: from localhost ([127.0.0.1]:59560 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nl67Q-0008ED-8E for submit@debbugs.gnu.org; Sun, 01 May 2022 05:43:56 -0400 Received: from quimby.gnus.org ([95.216.78.240]:57376) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nl67O-0008Do-Df for control@debbugs.gnu.org; Sun, 01 May 2022 05:43:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=nFWE/aIBBe4xzOBuaeI5KU9htqMNdcsMM9cVhhScREU=; b=XSHJ7ge98ILF22ayNZNhHtr1dQ yOBr434Xrxxt5gN8bKSa3i4Dr4phndvztfBaRjkSdZ1TX3PKdMbugFD6JabwKSurgbfe1oO+5AAKA 4O4f0JXuDNm/A+rXHHzLYa/z2eNYsRfcy9J/T6KIOZ6zWgDTHDDqjNQRMj8GRTTZ65ek=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nl67G-0002cs-K8 for control@debbugs.gnu.org; Sun, 01 May 2022 11:43:48 +0200 Date: Sun, 01 May 2022 11:43:46 +0200 Message-Id: <874k29r3jh.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #45200 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: close 45200 29.1 quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 45200 29.1 quit From unknown Fri Aug 15 16:17:53 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 29 May 2022 11:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator