From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 14:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 33275@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.15414268855530 (code B ref -1); Mon, 05 Nov 2018 14:09:02 +0000 Received: (at submit) by debbugs.gnu.org; 5 Nov 2018 14:08:05 +0000 Received: from localhost ([127.0.0.1]:34702 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJfXx-0001R7-1R for submit@debbugs.gnu.org; Mon, 05 Nov 2018 09:08:05 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48689) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJfXv-0001Qe-JI for submit@debbugs.gnu.org; Mon, 05 Nov 2018 09:08:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJfXp-0002BQ-7V for submit@debbugs.gnu.org; Mon, 05 Nov 2018 09:07:58 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:43099) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gJfXp-0002BD-2U for submit@debbugs.gnu.org; Mon, 05 Nov 2018 09:07:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58286) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJfXo-0002Nk-2Z for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 09:07:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJfXk-00025e-Nf for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 09:07:56 -0500 Received: from lamora.getmail.no ([84.210.184.7]:50622) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gJfXk-0001lD-G9 for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 09:07:52 -0500 Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id F1D6860840 for ; Mon, 5 Nov 2018 15:07:30 +0100 (CET) Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id r3Ad1Lq2lR-M for ; Mon, 5 Nov 2018 15:07:30 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 781E960842 for ; Mon, 5 Nov 2018 15:07:30 +0100 (CET) X-Virus-Scanned: amavisd-new at lamora.get.c.bitbit.net Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id z3IaSEm1gCTC for ; Mon, 5 Nov 2018 15:07:30 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by lamora.getmail.no (Postfix) with ESMTPSA id 4C5C960840 for ; Mon, 5 Nov 2018 15:07:30 +0100 (CET) From: Lars Ingebrigtsen Date: Mon, 05 Nov 2018 15:07:30 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) I was writing a function to do some image processing, and after it had run a few minutes, the Emacs window disappeared. After poking around a bit, it turned out that Emacs was killed by the out-of-memory task. I though that Emacs must have a serious memory leak somewhere, but `garbage-collect' didn't really seem to have anything mysterious in it. So after trying various things, it turned out that you can kill Emacs by doing this: (dolist (file (directory-files "/directory/with/many/images" t "png$")) (image-size (create-image file) t)) and this is because `image-size' puts the image into the image cache. Calling `clear-image-cache' releases the memory. I think putting it into the cache is the right thing to do, because if you're calling `image-size' on a file, then nine out of ten times you want to display it, and my use case here is atypical (batch-processing of a large number of huge images), but it's still not optimal that you can crash Emacs in this way. So perhaps the cache should be pruned more aggressively according to some smart algorithm or other? Or, at the very least, an innocuous-looking function like `image-size' should note in the doc string should note what's going on and recommend calling `image-flush' afterwards if we don't want to cache the image? In GNU Emacs 27.0.50 (build 50, x86_64-pc-linux-gnu, GTK+ Version 3.22.11) of 2018-08-26 built on stories Repository revision: 5c642b2dc1b666ae488225b76251750a8cf331be Windowing system distributor 'The X.Org Foundation', version 11.0.11902000 System Description: Debian GNU/Linux 9 (stretch) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 16:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143394917000 (code B ref 33275); Mon, 05 Nov 2018 16:06:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 16:05:49 +0000 Received: from localhost ([127.0.0.1]:35468 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhNp-0004Q4-Nn for submit@debbugs.gnu.org; Mon, 05 Nov 2018 11:05:49 -0500 Received: from eggs.gnu.org ([208.118.235.92]:55629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhNk-0004Po-Nx for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:05:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJhNG-0008Ob-ET for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:05:22 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57387) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJhNC-0008Is-Kb; Mon, 05 Nov 2018 11:05:08 -0500 Received: from [176.228.60.248] (port=1457 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJhNA-00058R-TM; Mon, 05 Nov 2018 11:05:06 -0500 Date: Mon, 05 Nov 2018 18:05:01 +0200 Message-Id: <83va5bcxxu.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 15:07:30 +0100) References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Date: Mon, 05 Nov 2018 15:07:30 +0100 > > So after trying various things, it turned out that you can kill Emacs by > doing this: > > (dolist (file (directory-files "/directory/with/many/images" t "png$")) > (image-size (create-image file) t)) > > and this is because `image-size' puts the image into the image cache. > Calling `clear-image-cache' releases the memory. > > I think putting it into the cache is the right thing to do, because if > you're calling `image-size' on a file, then nine out of ten times you > want to display it, and my use case here is atypical (batch-processing > of a large number of huge images), but it's still not optimal that you > can crash Emacs in this way. Doesn't this mean that your system's VM is misconfigured? How is it Emacs's fault that you've run a program that gobs so much memory as to trigger the OS's OOM killer? > So perhaps the cache should be pruned more aggressively according to > some smart algorithm or other? Or, at the very least, an > innocuous-looking function like `image-size' should note in the doc > string should note what's going on and recommend calling `image-flush' > afterwards if we don't want to cache the image? I'm okay with saying something about this in the docs, but frankly, you could do the same in any number of ways, like by visiting a file whose size is larger than the VM you allow your system to use, right? So there's a potential for causing such problems in many different ways, and warning about that only in the doc string of image-size seems insufficient, to say the least... From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 16:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143474118249 (code B ref 33275); Mon, 05 Nov 2018 16:19:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 16:19:01 +0000 Received: from localhost ([127.0.0.1]:35476 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhae-0004kH-SH for submit@debbugs.gnu.org; Mon, 05 Nov 2018 11:19:01 -0500 Received: from ryder.getmail.no ([84.210.184.14]:53694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhac-0004k0-5q for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:18:59 -0500 Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id 7996362C8A; Mon, 5 Nov 2018 17:18:48 +0100 (CET) Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id oGvfj7Lv1MK3; Mon, 5 Nov 2018 17:18:48 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id E386562C89; Mon, 5 Nov 2018 17:18:47 +0100 (CET) X-Virus-Scanned: amavisd-new at ryder.getmail.no Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id BinrjQqDg4EU; Mon, 5 Nov 2018 17:18:47 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by ryder.getmail.no (Postfix) with ESMTPSA id B931062C8A; Mon, 5 Nov 2018 17:18:47 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEVYQhQsJQN4TimCbkLl 66uAWDOunm1MQAQZAAACX0lEQVQ4jW2TTXObMBCGVyL2WaSTnAkMOWNv0bmdaHR2pCpnB+H9/z+h rwTJtE2XGZt5n/0WIre6/xp5kYq8OHGrX/HvBD8EbXX/CSvA+46aLwCZ5HwxdPcvQFIhyl9CECFL 9+jcj79kX8Gk0MnPP1T7LBWcCF2+f+qykC015OX0za/u+um/ZrsV96cjXl42b+8t81qB87VVDxXv 4uPQyAdAUy5HHhg20YGSXR295Vk18NRxbmbIHVEoPnRp+AnD+cPMfI5TRyN1zDHAgxWKvzfTpWfu wtTrmpJIxzL4iYapF01gTdFHIsSiq4n6Xsuko5VqTMrQULo9d30jUyO7ZTKGfpXRFuoauX4CIUOH Cm6BjpJ3kAFaGlMFke5kMVW3IKT0mN4A5tgdZbnv2X6k0iml7+5ljop4MsE64Vr8vgdIawZo29Yc cW6HLPmG9gtIMcUhBaJjpgERFmCT5xR1WYQ2LdHB4hTPfQCYUxpMPxOyUbHbleYQxgLCyAOqGFJ4 Wuy2TyFFLvkkK+iQSamSL5VU0FkWAAOdVKsVHUoNRo0o15KqZGtNDxBCyRPGKJcySTFVKun+UgcZ rVxqbQCNIERUEHpeSq94Si4UekhdBZGX6m1a1Ddo6yE9jdtSsrCq87UKD+Z43QALz+hXGV2YIZ3i DuypLqTUMTjw5QPES+kUo0MuK1jSZnN8qjOYtmtLfbpt7vwJJnnFF2Ipl10BML9uYz9K7kwjJDuY 4w7ucx6SLWBLldJpj7C+S9bvIHHiHWjnzsy4UTecd1m638Ezbhe+NMInuF/X9w3gAma27jc2rfja fW4ALQAAAABJRU5ErkJggg== Date: Mon, 05 Nov 2018 17:18:47 +0100 In-Reply-To: <83va5bcxxu.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 18:05:01 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: > I'm okay with saying something about this in the docs, but frankly, > you could do the same in any number of ways, like by visiting a file > whose size is larger than the VM you allow your system to use, right? > So there's a potential for causing such problems in many different > ways, and warning about that only in the doc string of image-size > seems insufficient, to say the least... If you visit a, say, 16GB file, then, indeed, it's expected that Emacs grows to 16GB (or more). But if you're just calling `image-size' on a bunch of files (each of which is a lot smaller than that), it's rather unexpected that that's going to blow up Emacs. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 16:25:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143504518779 (code B ref 33275); Mon, 05 Nov 2018 16:25:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 16:24:05 +0000 Received: from localhost ([127.0.0.1]:35488 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhfU-0004sT-6H for submit@debbugs.gnu.org; Mon, 05 Nov 2018 11:24:05 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhfP-0004sE-2H for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:23:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJhfF-0003M2-Sv for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:23:49 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57918) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJhfD-0003H3-HS; Mon, 05 Nov 2018 11:23:44 -0500 Received: from [176.228.60.248] (port=2762 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJhfD-0006wH-7u; Mon, 05 Nov 2018 11:23:43 -0500 Date: Mon, 05 Nov 2018 18:23:40 +0200 Message-Id: <83sh0fcx2r.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 17:18:47 +0100) References: <83va5bcxxu.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 17:18:47 +0100 > > If you visit a, say, 16GB file, then, indeed, it's expected that Emacs > grows to 16GB (or more). But if you're just calling `image-size' on a > bunch of files (each of which is a lot smaller than that), it's rather > unexpected that that's going to blow up Emacs. What was the sum of sizes of all the files you used to reproduce the problem? From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 16:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143577019922 (code B ref 33275); Mon, 05 Nov 2018 16:37:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 16:36:10 +0000 Received: from localhost ([127.0.0.1]:35506 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhrC-0005BD-GD for submit@debbugs.gnu.org; Mon, 05 Nov 2018 11:36:10 -0500 Received: from bouvier.getmail.no ([84.210.184.8]:57510) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJhr8-0005Af-1U for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 11:36:05 -0500 Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 3D7F24002D; Mon, 5 Nov 2018 17:35:52 +0100 (CET) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id V3dYAjJcGtdJ; Mon, 5 Nov 2018 17:35:51 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id BF5A74002E; Mon, 5 Nov 2018 17:35:51 +0100 (CET) X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Ftv9EFfbjJik; Mon, 5 Nov 2018 17:35:51 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by bouvier.getmail.no (Postfix) with ESMTPSA id 8BC2D4002D; Mon, 5 Nov 2018 17:35:51 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEVYQhQsJQN4TimCbkLl 66uAWDOunm1MQAQZAAACX0lEQVQ4jW2TTXObMBCGVyL2WaSTnAkMOWNv0bmdaHR2pCpnB+H9/z+h rwTJtE2XGZt5n/0WIre6/xp5kYq8OHGrX/HvBD8EbXX/CSvA+46aLwCZ5HwxdPcvQFIhyl9CECFL 9+jcj79kX8Gk0MnPP1T7LBWcCF2+f+qykC015OX0za/u+um/ZrsV96cjXl42b+8t81qB87VVDxXv 4uPQyAdAUy5HHhg20YGSXR295Vk18NRxbmbIHVEoPnRp+AnD+cPMfI5TRyN1zDHAgxWKvzfTpWfu wtTrmpJIxzL4iYapF01gTdFHIsSiq4n6Xsuko5VqTMrQULo9d30jUyO7ZTKGfpXRFuoauX4CIUOH Cm6BjpJ3kAFaGlMFke5kMVW3IKT0mN4A5tgdZbnv2X6k0iml7+5ljop4MsE64Vr8vgdIawZo29Yc cW6HLPmG9gtIMcUhBaJjpgERFmCT5xR1WYQ2LdHB4hTPfQCYUxpMPxOyUbHbleYQxgLCyAOqGFJ4 Wuy2TyFFLvkkK+iQSamSL5VU0FkWAAOdVKsVHUoNRo0o15KqZGtNDxBCyRPGKJcySTFVKun+UgcZ rVxqbQCNIERUEHpeSq94Si4UekhdBZGX6m1a1Ddo6yE9jdtSsrCq87UKD+Z43QALz+hXGV2YIZ3i DuypLqTUMTjw5QPES+kUo0MuK1jSZnN8qjOYtmtLfbpt7vwJJnnFF2Ipl10BML9uYz9K7kwjJDuY 4w7ucx6SLWBLldJpj7C+S9bvIHHiHWjnzsy4UTecd1m638Ezbhe+NMInuF/X9w3gAma27jc2rfja fW4ALQAAAABJRU5ErkJggg== Date: Mon, 05 Nov 2018 17:35:51 +0100 In-Reply-To: <83sh0fcx2r.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 18:23:40 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> From: Lars Ingebrigtsen >> Cc: 33275@debbugs.gnu.org >> Date: Mon, 05 Nov 2018 17:18:47 +0100 >> >> If you visit a, say, 16GB file, then, indeed, it's expected that Emacs >> grows to 16GB (or more). But if you're just calling `image-size' on a >> bunch of files (each of which is a lot smaller than that), it's rather >> unexpected that that's going to blow up Emacs. > > What was the sum of sizes of all the files you used to reproduce the > problem? I think it's sufficient to run Emacs over 2GB worth of .png files to make Emacs grow to 16GB (which will kill it on this laptop). But it'll vary on how well-compressed the .png files are, I guess, because I think Emacs caches the, er, pixmaps and not the files? I don't remember. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 17:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143888124313 (code B ref 33275); Mon, 05 Nov 2018 17:29:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 17:28:01 +0000 Received: from localhost ([127.0.0.1]:35520 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJifM-0006Ju-0K for submit@debbugs.gnu.org; Mon, 05 Nov 2018 12:28:01 -0500 Received: from eggs.gnu.org ([208.118.235.92]:60105) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJifH-0006Je-Bu for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 12:27:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJif4-0003wA-Bt for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 12:27:43 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59541) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJif1-0003nj-1T; Mon, 05 Nov 2018 12:27:36 -0500 Received: from [176.228.60.248] (port=2737 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJiey-0000qH-Rn; Mon, 05 Nov 2018 12:27:34 -0500 Date: Mon, 05 Nov 2018 19:27:28 +0200 Message-Id: <83o9b3cu4f.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 17:35:51 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 17:35:51 +0100 > > >> If you visit a, say, 16GB file, then, indeed, it's expected that Emacs > >> grows to 16GB (or more). But if you're just calling `image-size' on a > >> bunch of files (each of which is a lot smaller than that), it's rather > >> unexpected that that's going to blow up Emacs. > > > > What was the sum of sizes of all the files you used to reproduce the > > problem? > > I think it's sufficient to run Emacs over 2GB worth of .png files to > make Emacs grow to 16GB (which will kill it on this laptop). > > But it'll vary on how well-compressed the .png files are, I guess, > because I think Emacs caches the, er, pixmaps and not the files? I > don't remember. I think it's reasonable to expect 2GB worth of image files to take several times that in memory. (Btw, visiting a 16GB file usually takes twice that much during insert-file-contents.) I think we should try to add some code that would call display_malloc_warning and/or memory_full, before the system runs out of memory. That should be enough to prevent OOM in such cases. Can you spot why we never called memory_full in this case? From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 17:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143940825159 (code B ref 33275); Mon, 05 Nov 2018 17:37:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 17:36:48 +0000 Received: from localhost ([127.0.0.1]:35530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJinw-0006Xj-Fp for submit@debbugs.gnu.org; Mon, 05 Nov 2018 12:36:48 -0500 Received: from lamora.getmail.no ([84.210.184.7]:60571) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJinu-0006XW-NH for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 12:36:47 -0500 Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 18CD56083C; Mon, 5 Nov 2018 18:36:40 +0100 (CET) Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id OcUNj5dXngkp; Mon, 5 Nov 2018 18:36:39 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 9507C6083E; Mon, 5 Nov 2018 18:36:39 +0100 (CET) X-Virus-Scanned: amavisd-new at lamora.get.c.bitbit.net Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4mkZH3iNi6IM; Mon, 5 Nov 2018 18:36:39 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by lamora.getmail.no (Postfix) with ESMTPSA id 649906083C; Mon, 5 Nov 2018 18:36:39 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEW6gTlVGIFdHox+SDZD EmhXGoJRFnz8xUXqvZ6yAAACO0lEQVQ4jW2UzY+DIBDFaa3bM0a8S2rPJG7iVbvUO0Q8S4NzbtKD //4OoK2bLP1I01/ePN4wQsBo/t8iBdX/IjIyzTUH6Yquu1wenF+rWwDK6IpzJ9fFr73rV0WGlVzv 4NYNQbGWUoYpzuUPea/vewBGzYbry5Scl+d5we8ljR6mmtF9mGqRvAR5EdGc+hVYzpVr67p+hXdz vFUeaMN0VSBAxQZ4BHNmrAf4r2fNocNgHlhuQEbwRwFQMcgnEsyfQmzAWKqYzdvlvb56ju4EwDJr 5Q6gwnuABcOyy/QtMdtCPIjmwGBmWd4KQYSoSf1qTjJ4OABgmUOQkAZB0qS9YbEU2McwiXYDXz3N gjnM5q1IEJzkCizzyT8g7SieKilgBkvzCYGIAEtRHRRWKa+oxVZqxFreA8ZZtqL2wJsfu5JtwObT spxD7ldzkAVl5q1I6giei/fIqhAQGJ7HD4kdWdJebR4Acbt4JqG7JTUbQPN2A6mk2BNiLX0rsI8e YMBQCoHNA8DsCLAlJgQsEMjoQWIpFZOz0Vo3fcCxi8DSwlLnzUUEviU67spirwg5n9czxxzrdmcz tFtLfKmg8Mkt24/PSZZMmwBmjaVEnSTE9yS9lVT7Xo3A2OBz15hkNz6Yg9KhfYYOPkOpyqxgVvtJ TPtKr9t9mGH6gEPPowIKzfPdw/kVJ9EDRsfHHS7j7X65Ph7XfgNWfy4AvyKwOKJMDX3nXH7v3ICf YlMA0yX+rP5eMgEYpXc3UIWv8hctdBMsfua7RgAAAABJRU5ErkJggg== Date: Mon, 05 Nov 2018 18:36:39 +0100 In-Reply-To: <83o9b3cu4f.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 19:27:28 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: > I think it's reasonable to expect 2GB worth of image files to take > several times that in memory. (Btw, visiting a 16GB file usually > takes twice that much during insert-file-contents.) It would be reasonable if the code had asked for those images to be kept in memory, but it hasn't. Emacs decides that on its own without saying that it's doing that. It'd be similarly surprising if (dolist (file (directory-files "/directory/with/many/images" t "png$")) (with-temp-buffer (insert-file-contents-literally file))) were to lead to Emacs growing uncontrollably. At present, the image cache is purely based on timestamps, if I recall correctly? If we wish to fix this unexpected behaviour, we could also look at the size of the image cache and perhaps try to limit it, or if we wanted to be really fancy, we could somehow check whether there's any references to the images anywhere (beyond the image cache itself). I think the latter would mean adding some... fun... to the gc system? Or does Emacs have weak hashes? In which case the image cache could be a weak hash and the problem would resolve itself? > I think we should try to add some code that would call > display_malloc_warning and/or memory_full, before the system runs out > of memory. That should be enough to prevent OOM in such cases. Can > you spot why we never called memory_full in this case? Sorry, I'm not familiar with the memory_full stuff... -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 17:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154144013326264 (code B ref 33275); Mon, 05 Nov 2018 17:49:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 17:48:53 +0000 Received: from localhost ([127.0.0.1]:35555 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJizZ-0006pU-DN for submit@debbugs.gnu.org; Mon, 05 Nov 2018 12:48:53 -0500 Received: from ryder.getmail.no ([84.210.184.14]:35254) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJizU-0006oy-Dc for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 12:48:48 -0500 Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id 49F5262C86; Mon, 5 Nov 2018 18:48:34 +0100 (CET) Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 4LTxwgA4gwGJ; Mon, 5 Nov 2018 18:48:33 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id CDA8F62C87; Mon, 5 Nov 2018 18:48:33 +0100 (CET) X-Virus-Scanned: amavisd-new at ryder.getmail.no Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id YDlDPCmlSzp3; Mon, 5 Nov 2018 18:48:33 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by ryder.getmail.no (Postfix) with ESMTPSA id A2BF862C86; Mon, 5 Nov 2018 18:48:33 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEW6gTlVGIFdHox+SDZD EmhXGoJRFnz8xUXqvZ6yAAACO0lEQVQ4jW2UzY+DIBDFaa3bM0a8S2rPJG7iVbvUO0Q8S4NzbtKD //4OoK2bLP1I01/ePN4wQsBo/t8iBdX/IjIyzTUH6Yquu1wenF+rWwDK6IpzJ9fFr73rV0WGlVzv 4NYNQbGWUoYpzuUPea/vewBGzYbry5Scl+d5we8ljR6mmtF9mGqRvAR5EdGc+hVYzpVr67p+hXdz vFUeaMN0VSBAxQZ4BHNmrAf4r2fNocNgHlhuQEbwRwFQMcgnEsyfQmzAWKqYzdvlvb56ju4EwDJr 5Q6gwnuABcOyy/QtMdtCPIjmwGBmWd4KQYSoSf1qTjJ4OABgmUOQkAZB0qS9YbEU2McwiXYDXz3N gjnM5q1IEJzkCizzyT8g7SieKilgBkvzCYGIAEtRHRRWKa+oxVZqxFreA8ZZtqL2wJsfu5JtwObT spxD7ldzkAVl5q1I6giei/fIqhAQGJ7HD4kdWdJebR4Acbt4JqG7JTUbQPN2A6mk2BNiLX0rsI8e YMBQCoHNA8DsCLAlJgQsEMjoQWIpFZOz0Vo3fcCxi8DSwlLnzUUEviU67spirwg5n9czxxzrdmcz tFtLfKmg8Mkt24/PSZZMmwBmjaVEnSTE9yS9lVT7Xo3A2OBz15hkNz6Yg9KhfYYOPkOpyqxgVvtJ TPtKr9t9mGH6gEPPowIKzfPdw/kVJ9EDRsfHHS7j7X65Ph7XfgNWfy4AvyKwOKJMDX3nXH7v3ICf YlMA0yX+rP5eMgEYpXc3UIWv8hctdBMsfua7RgAAAABJRU5ErkJggg== Date: Mon, 05 Nov 2018 18:48:33 +0100 In-Reply-To: (Lars Ingebrigtsen's message of "Mon, 05 Nov 2018 18:36:39 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Lars Ingebrigtsen writes: > Or does Emacs have weak hashes? In which case the image cache could be > a weak hash and the problem would resolve itself? Emacs does have weak hashes... Hm... Anyway, there's code that tries to keep the cache small: /* If the number of cached images has grown unusually large, decrease the cache eviction delay (Bug#6230). */ delay = XFIXNUM (Vimage_cache_eviction_delay); if (nimages > 40) delay = 1600 * delay / nimages / nimages; But it works solely on the number of images in the cache, and not the size of the images. So it's a heuristic that could be tweaked pretty easily, I think? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 18:15:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154144169728547 (code B ref 33275); Mon, 05 Nov 2018 18:15:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 18:14:57 +0000 Received: from localhost ([127.0.0.1]:35572 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjOm-0007QI-Li for submit@debbugs.gnu.org; Mon, 05 Nov 2018 13:14:55 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50176) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjOh-0007Q1-AN for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:14:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJjOX-0000nS-OO for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:14:42 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60674) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJjOX-0000n8-J6; Mon, 05 Nov 2018 13:14:37 -0500 Received: from [176.228.60.248] (port=1767 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJjOW-0007DH-Ki; Mon, 05 Nov 2018 13:14:37 -0500 Date: Mon, 05 Nov 2018 20:14:27 +0200 Message-Id: <83muqncry4.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 18:48:33 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 18:48:33 +0100 > > Anyway, there's code that tries to keep the cache small: > > /* If the number of cached images has grown unusually large, > decrease the cache eviction delay (Bug#6230). */ > delay = XFIXNUM (Vimage_cache_eviction_delay); > if (nimages > 40) > delay = 1600 * delay / nimages / nimages; > > But it works solely on the number of images in the cache, and not the > size of the images. So it's a heuristic that could be tweaked pretty > easily, I think? I don't see why we should tweak our cache management for the benefit of exceptional use case. Last time we did it (with fonts) we are still leaking the wounds. I think it should be enough to catch excess memory usage and signal an error. I hope this is possible. From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 18:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154144193528920 (code B ref 33275); Mon, 05 Nov 2018 18:19:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 18:18:55 +0000 Received: from localhost ([127.0.0.1]:35577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjSe-0007WL-6U for submit@debbugs.gnu.org; Mon, 05 Nov 2018 13:18:55 -0500 Received: from eggs.gnu.org ([208.118.235.92]:51677) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjSY-0007W7-TL for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:18:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJjSN-0005ZX-TX for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:18:41 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60729) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJjSK-0005X0-Ia; Mon, 05 Nov 2018 13:18:33 -0500 Received: from [176.228.60.248] (port=2004 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJjSA-0007X6-OJ; Mon, 05 Nov 2018 13:18:24 -0500 Date: Mon, 05 Nov 2018 20:18:17 +0200 Message-Id: <83lg67crrq.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 18:36:39 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 18:36:39 +0100 > > It would be reasonable if the code had asked for those images to be kept > in memory, but it hasn't. Emacs decides that on its own without saying > that it's doing that. You cannot assume anything about memory management while a Lisp program runs. > It'd be similarly surprising if > > (dolist (file (directory-files "/directory/with/many/images" t "png$")) > (with-temp-buffer > (insert-file-contents-literally file))) > > were to lead to Emacs growing uncontrollably. That it doesn't is just sheer luck: the way we manage buffer memory is special. With any other Lisp object, it could well grow uncontrollably. Our way of keeping that in check is to signal memory-full error when we are about to run out of memory. Not sure yet why this doesn't happen in your original recipe. > > I think we should try to add some code that would call > > display_malloc_warning and/or memory_full, before the system runs out > > of memory. That should be enough to prevent OOM in such cases. Can > > you spot why we never called memory_full in this case? > > Sorry, I'm not familiar with the memory_full stuff... OK, I will try to look into this soon. From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 18:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: larsi@gnus.org Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154144295430468 (code B ref 33275); Mon, 05 Nov 2018 18:36:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 18:35:54 +0000 Received: from localhost ([127.0.0.1]:35588 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjj7-0007vL-V8 for submit@debbugs.gnu.org; Mon, 05 Nov 2018 13:35:54 -0500 Received: from eggs.gnu.org ([208.118.235.92]:57795) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJjj5-0007v6-LC for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:35:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJjiu-00046C-M7 for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 13:35:42 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:32857) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJjiq-0003yn-Pk; Mon, 05 Nov 2018 13:35:38 -0500 Received: from [176.228.60.248] (port=3075 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJjij-0006OB-8t; Mon, 05 Nov 2018 13:35:33 -0500 Date: Mon, 05 Nov 2018 20:35:21 +0200 Message-Id: <83k1lrcqza.fsf@gnu.org> From: Eli Zaretskii In-reply-to: <83muqncry4.fsf@gnu.org> (message from Eli Zaretskii on Mon, 05 Nov 2018 20:14:27 +0200) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83muqncry4.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > Date: Mon, 05 Nov 2018 20:14:27 +0200 > From: Eli Zaretskii > Cc: 33275@debbugs.gnu.org > > I don't see why we should tweak our cache management for the benefit > of exceptional use case. Last time we did it (with fonts) we are > still leaking the wounds. ^^^^^^^ I meant "licking", of course ;-) > I think it should be enough to catch excess memory usage and signal an > error. I hope this is possible. I guess we lost the ability to do that automatically when we stopped using vm-limit.c on modern platforms? We still could use the memory limit calculated by get_lim_data in strategic places, like cache_image, to catch situations where the cache size exceeds some fraction of what ret_lim_data returns. Would that work? From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 19:07:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15414448151508 (code B ref 33275); Mon, 05 Nov 2018 19:07:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 19:06:55 +0000 Received: from localhost ([127.0.0.1]:35616 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkD9-0000OG-Db for submit@debbugs.gnu.org; Mon, 05 Nov 2018 14:06:55 -0500 Received: from ryder.getmail.no ([84.210.184.14]:42278) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkD5-0000O1-TY for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:06:54 -0500 Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id 02FB862C86; Mon, 5 Nov 2018 20:06:43 +0100 (CET) Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id gdRvsPZ94HYt; Mon, 5 Nov 2018 20:06:41 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id 9EF9062C87; Mon, 5 Nov 2018 20:06:41 +0100 (CET) X-Virus-Scanned: amavisd-new at ryder.getmail.no Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ub5QvtFJto0G; Mon, 5 Nov 2018 20:06:41 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by ryder.getmail.no (Postfix) with ESMTPSA id 6C3A762C83; Mon, 5 Nov 2018 20:06:41 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83lg67crrq.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAIVBMVEX+/thGAw9eUVv+/93+ //fq3LH58cKsqZr+/dP9+s7+/99hXvwTAAACQklEQVQ4jW3TMY/aMBQHcEup2pQtLJ1tiYvUCSlt VWV2qyoTQnLCdUI6Ac1NlVVjcVOWqmGrSgtnpui4tJw/ZZ8dc+QQb0rej/d37ATE0ulJMVsobd+n x2vkLieSBEGXyIYmDtZqygNTXdJzv7OQKzXDFgJChAszoNS6GWjJBNmBDT5Al4TNCMqhr4ZBcByZ O1DtJDuilAHoqz/4BNYONi2ArEszYqD88RQW0DNQnoO1nTiJWiyhiWDgCLiBEgTdlo9RXUKwXXx5 gO/YPSjjBkbQUqhsQS9vg4/KJXbnlw1N3heAJUCBLEBHMolfkfC29C0gVLgJIjnu8QvoAxQIijfQ DTme86+lXcPW0PZlKGZkNLSwNG0P/bIPK8R1HQ6yn27C63voBYalRaIpFUWGfFjERHme95aEnA+S e0oZypEPZaK8OI64yOTnj78pzaFxiDIgxUxe1hVN2hBF8W4mplJkCw0AJ2H3AUk61n9ZmonxTfJp FSFY3E7EsYaC7zlPt3q/6ts+ACQZmLCMkHDwfAxH0WzQ8yzUjHfJXNfjl+MDRHEDmUy3u3rcP4EK /iu53r3L+4V/OCsbpQGuk11HuX08rgFZ+TZ56BTwGgoXdYB0FOW6qOx78M3EzsJGMjFd6Y6unlVm j8glVRIqvdKR3tGkDTMDbKv7+g2lVFelOSoownGIxfD1v4uHkJp6jyJzVnv4OOFEhuIu/MAtJE3U Psu45DK70d+otEBRBI9rLmrGxoN7+lgAcev2CZztU8SuzsN/F61wm8Zh404AAAAASUVORK5CYII= Date: Mon, 05 Nov 2018 20:06:41 +0100 In-Reply-To: <83lg67crrq.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 20:18:17 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> It would be reasonable if the code had asked for those images to be kept >> in memory, but it hasn't. Emacs decides that on its own without saying >> that it's doing that. > > You cannot assume anything about memory management while a Lisp > program runs. In general, if you don't have a reference to an object, you can be pretty sure that Emacs is going to garbage-collect it. If you can't assume that, then programming in Emacs Lisp becomes impossible. Fortunately that's not the case, and programming in Emacs Lisp is nice and easy. >> It'd be similarly surprising if >> >> (dolist (file (directory-files "/directory/with/many/images" t "png$")) >> (with-temp-buffer >> (insert-file-contents-literally file))) >> >> were to lead to Emacs growing uncontrollably. > > That it doesn't is just sheer luck: the way we manage buffer memory is > special. With any other Lisp object, it could well grow > uncontrollably. What other non-referenced Lisp object can realistically make Emacs grow in this way? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 19:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15414452182087 (code B ref 33275); Mon, 05 Nov 2018 19:14:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 19:13:38 +0000 Received: from localhost ([127.0.0.1]:35621 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkJe-0000Xb-7A for submit@debbugs.gnu.org; Mon, 05 Nov 2018 14:13:38 -0500 Received: from ryder.getmail.no ([84.210.184.14]:42825) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkJc-0000XN-Jp for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:13:37 -0500 Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id E2E9562C86; Mon, 5 Nov 2018 20:13:28 +0100 (CET) Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id bhxOkkO24Vop; Mon, 5 Nov 2018 20:13:28 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by ryder.getmail.no (Postfix) with ESMTP id 77E6262C87; Mon, 5 Nov 2018 20:13:28 +0100 (CET) X-Virus-Scanned: amavisd-new at ryder.getmail.no Received: from ryder.getmail.no ([127.0.0.1]) by localhost (ryder.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id II4H5AMbQaO7; Mon, 5 Nov 2018 20:13:28 +0100 (CET) Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by ryder.getmail.no (Postfix) with ESMTPSA id 3065862C86; Mon, 5 Nov 2018 20:13:28 +0100 (CET) From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83muqncry4.fsf@gnu.org> <83k1lrcqza.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAIVBMVEX+/thGAw9eUVv+/93+ //fq3LH58cKsqZr+/dP9+s7+/99hXvwTAAACQklEQVQ4jW3TMY/aMBQHcEup2pQtLJ1tiYvUCSlt VWV2qyoTQnLCdUI6Ac1NlVVjcVOWqmGrSgtnpui4tJw/ZZ8dc+QQb0rej/d37ATE0ulJMVsobd+n x2vkLieSBEGXyIYmDtZqygNTXdJzv7OQKzXDFgJChAszoNS6GWjJBNmBDT5Al4TNCMqhr4ZBcByZ O1DtJDuilAHoqz/4BNYONi2ArEszYqD88RQW0DNQnoO1nTiJWiyhiWDgCLiBEgTdlo9RXUKwXXx5 gO/YPSjjBkbQUqhsQS9vg4/KJXbnlw1N3heAJUCBLEBHMolfkfC29C0gVLgJIjnu8QvoAxQIijfQ DTme86+lXcPW0PZlKGZkNLSwNG0P/bIPK8R1HQ6yn27C63voBYalRaIpFUWGfFjERHme95aEnA+S e0oZypEPZaK8OI64yOTnj78pzaFxiDIgxUxe1hVN2hBF8W4mplJkCw0AJ2H3AUk61n9ZmonxTfJp FSFY3E7EsYaC7zlPt3q/6ts+ACQZmLCMkHDwfAxH0WzQ8yzUjHfJXNfjl+MDRHEDmUy3u3rcP4EK /iu53r3L+4V/OCsbpQGuk11HuX08rgFZ+TZ56BTwGgoXdYB0FOW6qOx78M3EzsJGMjFd6Y6unlVm j8glVRIqvdKR3tGkDTMDbKv7+g2lVFelOSoownGIxfD1v4uHkJp6jyJzVnv4OOFEhuIu/MAtJE3U Psu45DK70d+otEBRBI9rLmrGxoN7+lgAcev2CZztU8SuzsN/F61wm8Zh404AAAAASUVORK5CYII= Date: Mon, 05 Nov 2018 20:13:27 +0100 In-Reply-To: <83k1lrcqza.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 20:35:21 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: > We still could use the memory limit calculated by get_lim_data in > strategic places, like cache_image, to catch situations where the > cache size exceeds some fraction of what ret_lim_data returns. Would > that work? If I understand you correctly, you seem to favour introducing code that will break code that's working today (if you insert an image into a buffer today in a memory-pressure situation, that won't signal an error today) instead of tweaking a non-documented ad-hoc caching strategy (by taking the image cache size into consideration), or fixing the caching (by using weak hash tables), and I'm not quite sure why. `image-size' didn't use to cache images, but was introduced as an optimisation. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 19:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15414459083301 (code B ref 33275); Mon, 05 Nov 2018 19:26:01 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 19:25:08 +0000 Received: from localhost ([127.0.0.1]:35635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkUj-0000r5-0d for submit@debbugs.gnu.org; Mon, 05 Nov 2018 14:25:08 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkUe-0000qS-2L for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:25:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJkUQ-0002bZ-U2 for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:24:53 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34047) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJkUQ-0002bU-Pl; Mon, 05 Nov 2018 14:24:46 -0500 Received: from [176.228.60.248] (port=2142 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJkUQ-0008Ru-CO; Mon, 05 Nov 2018 14:24:46 -0500 Date: Mon, 05 Nov 2018 21:24:42 +0200 Message-Id: <83in1bcop1.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 20:06:41 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83lg67crrq.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 20:06:41 +0100 > > In general, if you don't have a reference to an object, you can be > pretty sure that Emacs is going to garbage-collect it. Of course. But you cannot control when that GC will happen. > > That it doesn't is just sheer luck: the way we manage buffer memory is > > special. With any other Lisp object, it could well grow > > uncontrollably. > > What other non-referenced Lisp object can realistically make Emacs grow > in this way? Theoretically, anything. In practice, I've seen such situations with fonts. From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 19:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15414461343604 (code B ref 33275); Mon, 05 Nov 2018 19:29:02 +0000 Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 19:28:54 +0000 Received: from localhost ([127.0.0.1]:35639 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkYM-0000w1-V5 for submit@debbugs.gnu.org; Mon, 05 Nov 2018 14:28:54 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49725) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJkYI-0000vl-05 for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:28:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJkY7-0006Og-Mt for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 14:28:40 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34108) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJkY7-0006Ob-JF; Mon, 05 Nov 2018 14:28:35 -0500 Received: from [176.228.60.248] (port=2388 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJkY7-0004Dn-6x; Mon, 05 Nov 2018 14:28:35 -0500 Date: Mon, 05 Nov 2018 21:28:32 +0200 Message-Id: <83h8gvcoin.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 20:13:27 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83muqncry4.fsf@gnu.org> <83k1lrcqza.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 20:13:27 +0100 > > If I understand you correctly, you seem to favour introducing code that > will break code that's working today (if you insert an image into a > buffer today in a memory-pressure situation, that won't signal an error > today) instead of tweaking a non-documented ad-hoc caching strategy (by > taking the image cache size into consideration), or fixing the caching > (by using weak hash tables), and I'm not quite sure why. Because your use case is not important enough to change image caching strategy that works well for many years. > `image-size' didn't use to cache images, but was introduced as an > optimisation. And that optimization had a reason, right? Btw, why doesn't xmalloc return NULL in your case before invoking the OOM killer? Is that expected behavior on GNU/Linux systems? From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 06 Nov 2018 15:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154151885713545 (code B ref 33275); Tue, 06 Nov 2018 15:41:02 +0000 Received: (at 33275) by debbugs.gnu.org; 6 Nov 2018 15:40:57 +0000 Received: from localhost ([127.0.0.1]:37478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gK3TJ-0003WL-V3 for submit@debbugs.gnu.org; Tue, 06 Nov 2018 10:40:57 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34173) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gK3TE-0003Vt-QU for 33275@debbugs.gnu.org; Tue, 06 Nov 2018 10:40:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gK3T4-0002L5-Vn for 33275@debbugs.gnu.org; Tue, 06 Nov 2018 10:40:43 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:55586) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gK3T4-0002KU-R1; Tue, 06 Nov 2018 10:40:38 -0500 Received: from [176.228.60.248] (port=1731 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gK3T4-0006dl-A5; Tue, 06 Nov 2018 10:40:38 -0500 Date: Tue, 06 Nov 2018 17:40:36 +0200 Message-Id: <83bm72ciyz.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Mon, 05 Nov 2018 18:48:33 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Mon, 05 Nov 2018 18:48:33 +0100 > > At present, the image cache is purely based on timestamps, if I recall > correctly? If we wish to fix this unexpected behaviour, we could also > look at the size of the image cache and perhaps try to limit it, or if > we wanted to be really fancy, we could somehow check whether there's any > references to the images anywhere (beyond the image cache itself). > > I think the latter would mean adding some... fun... to the gc system? > > Or does Emacs have weak hashes? In which case the image cache could be > a weak hash and the problem would resolve itself? > [...] > Emacs does have weak hashes... Hm... > > Anyway, there's code that tries to keep the cache small: > > /* If the number of cached images has grown unusually large, > decrease the cache eviction delay (Bug#6230). */ > delay = XFIXNUM (Vimage_cache_eviction_delay); > if (nimages > 40) > delay = 1600 * delay / nimages / nimages; > > But it works solely on the number of images in the cache, and not the > size of the images. So it's a heuristic that could be tweaked pretty > easily, I think? After thinking about this and looking at the code, I don't think I understand your proposal(s). Here are the issues that I saw with what I think you proposed: . The hash table used by the image cache is not a Lisp hash-table, so weakness doesn't apply. We cache pointers to C structs, so using a Lisp hash-tables would need "some work". . The code you show above is called from redisplay, so it will not be executed as long as a Lisp program (such as the one you show at the beginning of this discussion) runs, and you will still have the OOM killer get you before that code get a chance to clear the cache. . Running such code from GC could be tricky, because freeing images needs to remove references of those images from the glyph matrices, and that cannot be safely done from arbitrary places. . I don't think you can ensure GC will run during your program anyway, because most of the memory allocated by your program is not used to cons Lisp objects, so maybe_gc called by the Lisp interpreter will most probably decide there's no need for GC and do nothing. Therefore, I don't think that avoiding to cache the images would prevent the OOM situation in your case. I think if we want to prevent the OOM killer from killing Emacs due to many cached images, we should inspect inside cache_image the current VM size of the process (like system_process_attributes does), compare it to the memory limits, and display a warning and/or signal an error when we get dangerously close to the limit. If we want to avoid similar situations with other objects, we could do these tests inside xmalloc and mmap_alloc/mmap_realloc. (We used to do something similar in the past, but that code needed a libc hook for memory allocation from the OS, something I understand we cannot rely on now.) IOW, I don't see how we could prevent being OOM-killed, except if we monitor the committed memory at the point where it is allocated, as otherwise the opportunistic memory allocation strategy can always trip us. From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 09 Nov 2018 12:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15417661364030 (code B ref 33275); Fri, 09 Nov 2018 12:23:01 +0000 Received: (at 33275) by debbugs.gnu.org; 9 Nov 2018 12:22:16 +0000 Received: from localhost ([127.0.0.1]:42284 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gL5nj-00012w-Ow for submit@debbugs.gnu.org; Fri, 09 Nov 2018 07:22:15 -0500 Received: from hermes.netfonds.no ([80.91.224.195]:49350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gL5nh-00012k-9K for 33275@debbugs.gnu.org; Fri, 09 Nov 2018 07:22:14 -0500 Received: from cm-84.212.221.165.getinternet.no ([84.212.221.165] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1gL5nc-0001fe-Nv; Fri, 09 Nov 2018 13:22:10 +0100 From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83muqncry4.fsf@gnu.org> <83k1lrcqza.fsf@gnu.org> <83h8gvcoin.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAElBMVEWHhWtoTzJ8ZTuAdVa8 tIZjKBaOPNBXAAACdklEQVQ4jW2Uy5LjIAxFZcPsI3D2PJw9D3tPbGXfdjf//ysjSDpTUxVV2RAd pCuhJFDos8HmXFI+KceGp3UHokUfJwbWOW19tAoVokPleEEGObkPNsPoP/kZbD61hfeZH2JJ1z4Y GJGXNDvny+aMIPZxcjCwPXl7FPs4wmpeWcM19c2PPYPgvWkdNODVU+wlW8ZXVa6XNQ+coBVubr1X 8z5IozN904vcWsTb/G+rBlgjcbmoXh6rHP42OGRlw+AsXx2fQHt1jdkJxmA1FKWjispaRqc9lNUT 7GEAKAPkIFBblSNiRgwL7MB+IbLIJcZ4WTKfiBkakLAnBjMIldYgvJCiAQEzDzJkKCX7KIvwKYtQ GEgGe8hZFsieZy0SQDYBSoswmcFQohlkCSMIIVgjLESpSd4hx5XA70GMHWRHhqtd7jOAJOBMIjWQ QY7aWjGAjF4SVwi5A9a/YooaAqQgnPf8gp5Kys3mqEcnioBtjNmkAB1weB6mPUKrfct+3CLnKUBU xqRZl2QgWgzsG/QIItTp4IoK8CEyQXbwuND+pSwDGKumhSLKPV9meBgaqnYXgOEquTFaVRgVTPD4 dr5+Xz0DvUg10+pmd/E/8KjnZa3WMviRNMx0dy7OWwMHz/7WAN8mzrQhVtNBdZp/OgxgKBuDSxHy TwP2dvqhifM13xnwUGhlYNuXav16prpTB7KBg0GqLUI+gW2RXTy6W32lGjkNTvyam/jpkEG+ToUU +276DRCPWitqCSe5beVLUr3cAxV7ufzae6oLbbWD88BrxXpt4HbeTrk8nqA28M9wmNYXwPqf4XJ9 gQ82AfFsIcP730gW2Td/AYait0ePzXUgAAAAAElFTkSuQmCC Date: Fri, 09 Nov 2018 13:22:08 +0100 In-Reply-To: <83h8gvcoin.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 21:28:32 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) (Sorry for the late response; things got in the way as usual...) Eli Zaretskii writes: >> `image-size' didn't use to cache images, but was introduced as an >> optimisation. > > And that optimization had a reason, right? Yeah, the normal use case when you're doing images is that you're going to end up displaying it. And `image-size' has to render the image to see how big it is, so during normal usage, having `image-size' cache the image is a big win. Hm... Perhaps `image-size' should just have a DONT-CACHE optional parameter? > Btw, why doesn't xmalloc return NULL in your case before invoking the > OOM killer? Is that expected behavior on GNU/Linux systems? No, you've got overcommit on Linux, so you often don't get a NULL when you need it... -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 09 Nov 2018 12:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.15417664144494 (code B ref 33275); Fri, 09 Nov 2018 12:27:01 +0000 Received: (at 33275) by debbugs.gnu.org; 9 Nov 2018 12:26:54 +0000 Received: from localhost ([127.0.0.1]:42289 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gL5sE-0001AP-BJ for submit@debbugs.gnu.org; Fri, 09 Nov 2018 07:26:54 -0500 Received: from hermes.netfonds.no ([80.91.224.195]:49478) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gL5sC-0001AI-VZ for 33275@debbugs.gnu.org; Fri, 09 Nov 2018 07:26:53 -0500 Received: from cm-84.212.221.165.getinternet.no ([84.212.221.165] helo=stories) by hermes.netfonds.no with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1gL5sA-0002IL-EV; Fri, 09 Nov 2018 13:26:52 +0100 From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83bm72ciyz.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAElBMVEWHhWtoTzJ8ZTuAdVa8 tIZjKBaOPNBXAAACdklEQVQ4jW2Uy5LjIAxFZcPsI3D2PJw9D3tPbGXfdjf//ysjSDpTUxVV2RAd pCuhJFDos8HmXFI+KceGp3UHokUfJwbWOW19tAoVokPleEEGObkPNsPoP/kZbD61hfeZH2JJ1z4Y GJGXNDvny+aMIPZxcjCwPXl7FPs4wmpeWcM19c2PPYPgvWkdNODVU+wlW8ZXVa6XNQ+coBVubr1X 8z5IozN904vcWsTb/G+rBlgjcbmoXh6rHP42OGRlw+AsXx2fQHt1jdkJxmA1FKWjispaRqc9lNUT 7GEAKAPkIFBblSNiRgwL7MB+IbLIJcZ4WTKfiBkakLAnBjMIldYgvJCiAQEzDzJkKCX7KIvwKYtQ GEgGe8hZFsieZy0SQDYBSoswmcFQohlkCSMIIVgjLESpSd4hx5XA70GMHWRHhqtd7jOAJOBMIjWQ QY7aWjGAjF4SVwi5A9a/YooaAqQgnPf8gp5Kys3mqEcnioBtjNmkAB1weB6mPUKrfct+3CLnKUBU xqRZl2QgWgzsG/QIItTp4IoK8CEyQXbwuND+pSwDGKumhSLKPV9meBgaqnYXgOEquTFaVRgVTPD4 dr5+Xz0DvUg10+pmd/E/8KjnZa3WMviRNMx0dy7OWwMHz/7WAN8mzrQhVtNBdZp/OgxgKBuDSxHy TwP2dvqhifM13xnwUGhlYNuXav16prpTB7KBg0GqLUI+gW2RXTy6W32lGjkNTvyam/jpkEG+ToUU +276DRCPWitqCSe5beVLUr3cAxV7ufzae6oLbbWD88BrxXpt4HbeTrk8nqA28M9wmNYXwPqf4XJ9 gQ82AfFsIcP730gW2Td/AYait0ePzXUgAAAAAElFTkSuQmCC Date: Fri, 09 Nov 2018 13:26:50 +0100 In-Reply-To: <83bm72ciyz.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 06 Nov 2018 17:40:36 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: > After thinking about this and looking at the code, I don't think I > understand your proposal(s). Here are the issues that I saw with what > I think you proposed: > > . The hash table used by the image cache is not a Lisp hash-table, so > weakness doesn't apply. We cache pointers to C structs, so using a > Lisp hash-tables would need "some work". Yeah, it'd have to be rewritten substantially to use Lisp hash tables. > . The code you show above is called from redisplay, so it will not be > executed as long as a Lisp program (such as the one you show at the > beginning of this discussion) runs, and you will still have the OOM > killer get you before that code get a chance to clear the cache. That's true, but calling the cache pruning function from the function that enters images into the cache would be trivial, I think. > . Running such code from GC could be tricky, because freeing images > needs to remove references of those images from the glyph matrices, > and that cannot be safely done from arbitrary places. Ah, I see. Yes, that's a major complication... > I think if we want to prevent the OOM killer from killing Emacs due to > many cached images, we should inspect inside cache_image the current > VM size of the process (like system_process_attributes does), compare > it to the memory limits, and display a warning and/or signal an error > when we get dangerously close to the limit. I think that sounds like a good idea, yes. Having such a check is good for Emacs sanity, but we should also have a way for programmers to avoid ending up in that pathological situation to begin with. So either a DONT-CACHE parameter to `image-size' or just a docstring thing. :-) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 10 Nov 2018 09:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Lars Ingebrigtsen Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154184332323833 (code B ref 33275); Sat, 10 Nov 2018 09:49:01 +0000 Received: (at 33275) by debbugs.gnu.org; 10 Nov 2018 09:48:43 +0000 Received: from localhost ([127.0.0.1]:44176 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gLPsd-0006CH-CJ for submit@debbugs.gnu.org; Sat, 10 Nov 2018 04:48:43 -0500 Received: from eggs.gnu.org ([208.118.235.92]:43222) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gLPsW-0006Bx-39 for 33275@debbugs.gnu.org; Sat, 10 Nov 2018 04:48:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gLPsL-0003k4-M4 for 33275@debbugs.gnu.org; Sat, 10 Nov 2018 04:48:26 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56037) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gLPsJ-0003h7-46; Sat, 10 Nov 2018 04:48:19 -0500 Received: from [176.228.60.248] (port=2205 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gLPsI-0002P5-Nx; Sat, 10 Nov 2018 04:48:19 -0500 Date: Sat, 10 Nov 2018 11:48:06 +0200 Message-Id: <834lcpb6w9.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Lars Ingebrigtsen on Fri, 09 Nov 2018 13:26:50 +0100) References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83bm72ciyz.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Lars Ingebrigtsen > Cc: 33275@debbugs.gnu.org > Date: Fri, 09 Nov 2018 13:26:50 +0100 > > So either a DONT-CACHE parameter to `image-size' or just a docstring > thing. :-) How about both? From unknown Fri Jun 13 06:16:12 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33275: 27.0.50; Image cache pruning Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Sep 2019 08:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33275@debbugs.gnu.org Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.156905390328977 (code B ref 33275); Sat, 21 Sep 2019 08:19:02 +0000 Received: (at 33275) by debbugs.gnu.org; 21 Sep 2019 08:18:23 +0000 Received: from localhost ([127.0.0.1]:59027 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iBab1-0007XJ-Co for submit@debbugs.gnu.org; Sat, 21 Sep 2019 04:18:23 -0400 Received: from quimby.gnus.org ([80.91.231.51]:48810) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iBaax-0007X7-ET for 33275@debbugs.gnu.org; Sat, 21 Sep 2019 04:18:19 -0400 Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=marnie) by quimby.gnus.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iBaat-00081M-IA; Sat, 21 Sep 2019 10:18:18 +0200 From: Lars Ingebrigtsen References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> <83bm72ciyz.fsf@gnu.org> <834lcpb6w9.fsf@gnu.org> Date: Sat, 21 Sep 2019 10:18:14 +0200 In-Reply-To: <834lcpb6w9.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 10 Nov 2018 11:48:06 +0200") Message-ID: <87sgoq12rd.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Eli Zaretskii writes: >> So either a DONT-CACHE parameter to `image-size' or just a docstring >> thing. :-) > > How about both? DONT-CACHE seems unnecessary since `image-flush' exists, so I've just added a reference to that function in the `image-size' function. It should point people who need this towards doing something like Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> So either a DONT-CACHE parameter to `image-size' or just a docstring >> thing. :-) > > How about both? DONT-CACHE seems unnecessary since `image-flush' exists, so I've just added a reference to that function in the `image-size' function. It should point people who need this towards doing something like (dolist (file (directory-files-recursively "~/pics" "jpg$")) (let ((spec (create-image file))) (image-size spec t) (image-flush spec))) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 21 04:18:26 2019 Received: (at control) by debbugs.gnu.org; 21 Sep 2019 08:18:27 +0000 Received: from localhost ([127.0.0.1]:59030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iBab4-0007Xb-LK for submit@debbugs.gnu.org; Sat, 21 Sep 2019 04:18:26 -0400 Received: from quimby.gnus.org ([80.91.231.51]:48824) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iBab2-0007XP-1u for control@debbugs.gnu.org; Sat, 21 Sep 2019 04:18:24 -0400 Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=marnie) by quimby.gnus.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iBaaz-00081X-Cf for control@debbugs.gnu.org; Sat, 21 Sep 2019 10:18:23 +0200 Date: Sat, 21 Sep 2019 10:18:20 +0200 Message-Id: <87r24a12r7.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #33275 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: tags 33275 fixed close 33275 27.1 quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 33275 fixed close 33275 27.1 quit