Conversation

the correct solution is to have overcommit disabled and then monitor VM pressure so you can add or remove swap pages as needed. every other OS besides Linux and BSD get this right.

Quote Tweet

Rich Felker

@RichFelker

Apr 17, 2021

Replying to @pcwalton and @slava_pestov

Overcommit is just an operating system bug. Write software assuming it doesn't exist and users who don't like shit crashing can turn it off.

Replying to

While it sounds correct, you end up with DBs like Scylla that allocate a whole TB of "virtual" memory "because performance" and then things break without overcommit. I think even KDE's file indexer DB is similar? Apps are used to it and it's probably too late to change. :(

Replying to

and

Memory mapping massive amounts of file data is perfectly compatible with overcommit. It gets paged as needed and doesn't count as accountable memory. Similarly, fresh PROT_NONE / PROT_READ anonymous mappings don't count as accountable memory. Reserving address space is fine.

12:45 PM · Apr 18, 2021Twitter Web App

Replying to

and

VIRT doesn't show the amount the amount of accountable memory. Not using overcommit means you can't aggressively reserve writable, anonymous memory. You can still use a ton of non-accountable VM. It's not a measurement of potential memory usage and RLIMIT_AS is an anti-pattern.