The os will drop program text if it encounters memory pressure even if swap is disabled. Anything that isn’t anonymous memory is fair game and it does it for even well behaved programs on the system.
I mentioned that, yes. It's also problematic, particularly on HDD systems. (SSDs are fast enough that it's mostly okay.) This is why I said "Linux's low memory handling is deeply flawed" rather than just "turning off swap fixes everything".
Without swap, even with SSD's, the system gets into a very bad state when a cgroup is close to running out of memory.
The behavior ends up being that the kernel ends up constantly swapping out everything but anonymous memory until it can't anymore and then it finally terminates the process when it can't page anything else out. That can unfortunately impact other process on the system during that time.
It seems like there are enough failure modes that everyone has a different story about bad behavior, and unfortunately I don't think there's one solution a sysadmin can adopt (e.g., swap or not, sysctl settings, cgroup settings) that reliably prevents them all. I've been guilty in the past of telling people turning off swap dramatically improves things, but in your scenario it sounds like the opposite is true. /shruggie
The PSI-based oom daemon seems like an improvement, but I imagine there are still gaps there if it doesn't react quickly enough. Adding more parts can often paper over problems but rarely completely solve them...I think what's needed is a rethink: don't page these vital things out. Instead kill a process (chosen by userspace if possible, anything if not). Reserve enough for the killing rather than doing more reclamation to make it happen.
