On Tue, Mar 04 2025, Pip Cet wrote:

> Gerd, Helmut, Eli:
>
> See https://github.com/Ravenbrook/mps/issues/304 for the description of
> a Linux-specific MPS issue that currently causes hard crashes when we
> create "too many" mappings for the Linux kernel (this is purely a kernel
> issue, so no GNU/ prefix here), which causes 'mprotect' to fail with
> ENOMEM.

For me, this code 
;; -*- lexical-binding: t -*-

(defun test ()
  (let* ((npages 120000)
	 (pages (make-vector npages nil)))
    (dotimes (i npages)
      (aset pages i (make-vector (/ 4096 2) 0)))
    (message "nmaps: %s" 
	     (shell-command-to-string
	      (format "wc -l /proc/%d/maps" (emacs-pid))))
    (dotimes (i npages)
      (when (zerop (% i 2))
	(let ((page (aref pages i)))
	  (aset page 0 1))))))

crashes with:
protix.c:117: Emacs fatal error: assertion failed: unreachable code

Though this seems to require >4GB.

> We have to find a workaround for this.

We could emit a warning (similar to [1]) and telling users to increase
max_map_count.  E.g. if COMMIT_LIMIT / PAGE_SIZE > max_map_count.

Increasing max_map_size seems pretty harmless[2] and the reason[3] why
it is so low, seems to be some no-longer relevant limit for core
dumps[3].

We could also set COMMIT_LIMIT to max_map_count / PAGE_SIZE, so that MPS
tries harder to reuse memory.  Beside issue #288[4], we would likely
see situations where MPS gets slower and slower because it can free some
memory but not enough to make much progress on non-GC tasks.

[1] https://alchemistsimulator.github.io/howtos/workarounds/max-map-count/index.html#symptoms
[2] https://www.suse.com/support/kb/doc/?id=000016692
[3] https://github.com/torvalds/linux/blob/v5.18/include/linux/mm.h#L178
[4] https://github.com/Ravenbrook/mps/issues/288

> Increasing AREA_GRAIN_SIZE may delay running into the problem, or there
> might be a way to handle a failed mprotect call and at least recover the
> Emacs session, but ultimately the maximum safe size of an Emacs session
> appears to be ARENA_GRAIN_SIZE * /proc/sys/vm/max_map_count, or about
> 512 MB of MPS memory (minus whatever mappings libraries, malloc, and
> mmap require).  That's obviously not sufficient.

My guess is that AREA_GRAIN_SIZE is used for mmap/mmunmap but the memory
barriers always work at page granularity.  I could be wrong of course.

> One open question is whether Linux is actually capable of merging
> adjacent mappings when they're 'mprotect'ed to have the same
> permissions.

That's easy to verify with the attached C code.  Without the second call
to change_protection, the number mappings increases until the process
aborts.

Helmut