Subject: Re: bug#75672: 31.0.50; scratch/igc memory usage/collection issues

From: Pip Cet <[email-masked]>
Date: Mon, 20 Jan 2025 21:07:55 +0000

"Eli Zaretskii" <[email-masked]> writes:

>> Date: Mon, 20 Jan 2025 14:19:29 +0000
>> From: Pip Cet <[email-masked]>
>> Cc: Alexis Purslane <[email-masked]>, [email-masked]
>>
>> "Eli Zaretskii" <[email-masked]> writes:
>>
>> >> Cc: [email-masked]
>> >> Date: Mon, 20 Jan 2025 02:33:15 +0000
>> >> From:  Alexis Purslane via "Bug reports for GNU Emacs,
>> >>  the Swiss army knife of text editors" <[email-masked]>
>> >>
>> >> committed                                    1       400138240  400138240             0
>> >> pause-time                                 nil             0.1        nil           nil
>> >> reserved                                     1       536875008  536875008             0
>> >> spare                                      nil            0.75        nil           nil
>> >> spare-committed                              1       217436160  217436160             0
>> >
>> > AFAIU, these 3 values mean that MPS uses about 1.15GB of memory
>> > (unless "reserved" includes "committed").  If "reserved" means memory
>>
>> MPS "reserved" means "mmapped, not necessarily faulted in or used".
>> MPS "committed" means "faulted in and possibly in use by MPS or the
>> client program".
>> MPS "spare-committed" means "in use by MPS, not the client program".
>>
>> So committed - spare-commited is how much usable memory MPS thinks has
>> been allocated, about 183 MB.
>
> I agree, but did I say something different?

Not at all!  I was merely summarizing the (unusual) definitions MPS
uses, and explain why they amount to 183MB of MPS memory in use by
Emacs, not 1.15GB.

> The question now becomes: how was the 1.5GB figure Alex reported
> measured? did it include the reserved memory, or didn't it?

Even if it did, MPS accounts for at most 512MB: reserved is an upper
limit on how much memory is in use.  So we'll have to look for other
culprits, or for very strange MPS/igc bugs.

>> It's virtual memory: MPS assumes virtual memory is very cheap.  That's
>> not really true for processes that fork() a lot, so this assumption may
>> have to be fine-tuned.
>
> AFAIK, fork'ed process uses copy-on-write, so we should be okay.

It does use copy-on-write, but on my machine, at least, Linux appears to
copy the entire page table on every fork, which makes fork quite a bit
more expensive in terms of CPU time than I thought it would be.  This is
noticeable for large processes, as I've previously described.

Pip