The idea of having programs that don't crash under memory pressure is one of the things that I really hope developers will start striving for more in the near future.
A database that doesn't OOM crash is great, and the same is also true for more consumer-grade software.
I have 32gb of RAM on my PC and whenever I'm encoding a video with DaVinci Resolve, I always see Firefox and Discord crash almost instantly.
What would happen in your utopia is that DaVinci Resolve would fail to start because Firefox or Discord would reserve too much memory. Or maybe Firefox would demand you close open tabs before allowing you to open a new one. We have a solution to the problem you describe, it's called paging to disk and hoping that we won't need to read or write to that memory very often (which works great for something like Firefox where a bunch of pages are sitting idly in the background).
Too bad we can't design some protocol for applications to "work together" to ensure each has the memory they need. Like some sort of "application collective memory management" where each application has certain memory it needs/wants and depending on the priority of that need and the current priorities of the other applications, it could know to reduce memory usage during those times when some other application has a higher priority. For one, it would help in your case, because background video processing might be a lower priority, or perhaps you want it to finish as fast as possible, so that Firefox and etc. just "freeze" themselves for a while and drop as much memory as is practicable (ie: deleting currently rendered pages from memory and re-rendering them later, or paging those out to disk for a while).
The E/Agoric people have been trying to steer the whole thing in the cryptocurrency direction for about 35 years now, some of them for more than 50 years. If you are enthusiastic about their previous work, consider that possibly they might have been right about this, however unfashionable cryptocurrency might be on HN this year.
Eh. They’ve certainly always foresaw people using capability systems they were building to sell computing services to each other, but I don’t see anything pointing to the particular stylized flavour of computing of cryptographic smart contracts before those developed independently(?) over the last decade.
While I’ll admit that I find the corporate gloss of the Agoric website thoroughly repulsive, I would actually like to see cryptocurrencies succeed as a boring payment instrument. It’s just that my interests have always lain more in userspace resource allocation (etc.) among applications on a single machine, and in that context both E and the original agoric papers are completely separable from their money aspects, while any developments in the cryptographic consensus direction seem unlikely to be. Conversely, it’s not like the original object-capability work seems universally attractive to me—Shapiro advocated some system mechanisms that are, as far as I can tell, DRM-complete. But those are, again, separable from the rest of the ideas.
It's definitely true that E smart contracts were conceived to run on a secured server (or in tamperproof hardware), which was known to be feasible, rather than via a decentralized consensus mechanism, which was not. But they've been building smart contracts since last millennium.
Moreover, they didn't just foresee people selling computing services to each other; some of them built what we would call cloud computing platforms where the platform owner sold computation and the platform users could sell services to each other on it, starting in 01968, which was successful for decades: https://en.wikipedia.org/wiki/Tymshare
That's the experience that gave rise to the object-capability approach in the first place.
And, as far as I know, the Digital Silk Road paper from Agorics was the first workable proposal for a secure decentralized digital money.
It doesn't work very well and can't be made to work better. free()ing memory doesn't return memory to the system fast enough and it's easy to accidentally create more memory pressure while trying to respond to this (like by touching a page that's compressed/swapped out).
The only safe thing you can do is free entire pages at once, but that's not necessarily effective.
Of course you can only return pages at a time. That is the minimum size virtual memory works in.
You don't want malloc to instantly release memory to the system on free(), and malloc() is then constrained by the same page multiple at a time restriction as anything else so it can't release fragmented memory.
What you should be doing is trying to save state so that you can return to original state, not fighting to keep going.
Well, no, because the tradeoff with static allocation is that you need to allocate all the memory you want upfront. If you're requesting memory via dynamic allocation, you can be less wasteful by only requesting what you need and returning what you don't need to the OS.
In the case of e.g. Firefox, do you really want it to always be doing bookkeeping for 100 tabs when you only need 1? Or having to restart Firefox every time if you've hit the upper limit of tabs? Or just let Firefox dynamically allocate memory as you open more tabs?
There are many compelling reasons for dynamic memory allocation, and "the OS essentially concealing crappy software from the user via a billion layers of abstraction and hacks" is a fundamental misunderstanding of its purpose.
Somehow DaVinci Resolve is capable of using all the free memory it can find and not more than that, otherwise it would never be able to complete any rendering job. That's one "utopic" program that's pretty real.
> We have a solution to the problem you describe, it's called paging to disk and hoping
No. The solution is to be smart about memory usage, and to handle failures when trying to allocate memory. DaVinci would become unusable if it started swapping when rendering.
Maybe for some programs swapping is a fine solution, but the argument that software engineers shouldn't concern themselves with designing for OOM conditions is absolutely ridiculous.
That's not a 'utopia', that's what we should be concretely be striving for.
It’s difficult to see how that would be possible without a hard break from legacy systems when fork/exec practically forces overcommit and very dubious resource accounting on us. (One consumer system that handled OOM conditions with remarkable grace was Symbian, but of course it forced an extremely peculiar programming style.)
That's a fair point, but it's also true that we're never going to break away from legacy systems until we start writing software that has a chance of functioning correctly when running out of resources.
The good news is that we do have ecosystems where you don't have overcommit, like webassembly and embedded devices, and if you're writing libraries meant to also work on those platforms, then you will have some good building blocks that can be used to iterate towards more robust software.
That's how you're expected to approach library writing in Zig.
The purpose of abstractions is so you don't have to think about what's under them.
malloc() can fail if you try to allocate many GBs at once, but if you let it fail for 32 byte allocations then what are you even going to do about that? The only solution is to crash your process, but that will lose user data if you haven't saved it yet, and purity isn't worth that.
As far as I'm aware, witing to a file doesn't require any memory allocation on any of the major OSs, so you can most definitely save the user's work when running out of memory.
> The only solution is to crash your process
I disagree. I think this is a bad mindset that we allowed ourselves to slid into. There's a lot more than can be done when an allocation fails, starting from what I mentioned above.
That said, I'll agree that at least you need a programming language that can help you design and maintain allocation-free codepaths in your program, and that 's hard if your language has builtins that allocate implicitly, as you will have to avoid them all.
Zig has the philosophy of keeping all memory allocations explicit precisely for this reason. In the beginning people used to say that it was too extreme, but now Rust is also embracing this approach in order to be used in the Linux Kernel (one place where the "only solution is to crash" learned helplessness will not be well received), so hopefully there will be even more options in the future.
> As far as I'm aware, witing to a file doesn't require any memory allocation on any of the major OSs, so you can most definitely save the user's work when running out of memory.
Course it does. You can’t run any code in a typical app environment without allocating heap objects as you go, and even if you didn’t, anything that accesses a swapped out page is “allocating physical memory”.
And once it gets into the kernel, file systems have to allocate their own objects to track new writes. Though if the kernel is failing to allocate memory you have worse problems.
It would be nice if you could specify e.g. a Postgres table be backed by a (really big) "ring buffer" rather than the usual storage.
Maybe that breaks some relational features on the table, but being able to painlessly query the most recent X records for some interesting transient log data would be nice.
I ended up using a ring buffer in Redis recently to store the last N values of some database lookups that are slow. It works great! Didn't take that much logic, either, since Redis has easy transactions. I just have the "current buffer index 'i'" as a key-value, the "buffer length 'N'" as a key-value, then 'N' numbered entries as key-values. I prefix these with the data identifier that it caches, so you can have several ring buffers active. you can do this with a Postgres table just as easily, I assume!
I also am building a ring buffer data structure for my hobby database that I use for personal stuff that uses FoundationDB [though I have been unhappy with the state of the FDB Go bindings, which are somewhat out of date versus the C client library for FDB and aren't very Go-like in design (the C libraries are the "gold standard" and they use FFI in all the other bindings to connect to those)].
Added later: it was a bit harder to deal with resizing the buffer, so when that happens I do some data shifing/dropping depending on which way we're resizing, effectively initializing the buffer anew because that was easier (again, a Redis transaction saves me a lot of concurrency work!), but lucky for me we would only resize each buffer at most 1 time per minute due to how our configs work.
OOM is usually a good reason to crash. Sure, a DB might be able to do something creative with its cache, but if you can't allocate, there's not much the average program can do anymore. I guess browsers can kill their child processes, but that's not a long-term solution. If you're out of memory, your computer will be lagging from paging/swapping memory in-and-out, so killing yourself helps relieve some pressure.
Also, are you sure it's the programs failing and not some OOM killer coming around? Could your pagefile/swap be full?
The article says it rejects incoming connections. that seems like a much better behaviour than OOM - clients can back-off and retry and keep the overall system stable.
This was in fact one of our motivations for static allocation—thinking about how best to handle overload from the network, while remaining stable. The Google SRE book has a great chapter on this called "Handling Overload" and this had an impact on us. We were thinking, well, how do we get this right for a database?
We also wanted to make explicit what is often implicit, so that the operator has a clear sense of how to provision their API layer around TigerBeetle.
I can easily imagine that kind of design getting into a state where it's "up" but not accepting any connections indefinitely (if it's using just enough memory to run itself, but doesn't have enough to accept any connections). Crashing early is often a good way to reduce the state space and ensure consistent behaviour, since you will always have to handle crashing anyway.
> using just enough memory to run itself, but doesn't have enough to accept any connections
The situation you described is exactly what is prevented with static allocation. It's a very predictable design - it will successfully handle X requests and will fail any requests above that.
An important thing implicit in this design - or at least that I'm assuming - is that there is no queue of requests. Queuing is a major source of problematic behaviour during overload. It's /queuing/ that causes RAM to increase with load (in a system with fixed max of requests being processed). It's /queuing/ that is why latency increase as you near load limits (vs errors).
I'm with you that early errors are better, and the scheme used here achieves that on a request level.
Crashing and losing all existing connections often merely leads to an avalanche of subsequent requests, accelerating overload of your systems. I've seen this countless times.
Being crash tolerant is one thing. But "crash early, crash often" is absolutely horrible advice.
> Crashing and losing all existing connections often merely leads to an avalanche of subsequent requests, accelerating overload of your systems. I've seen this countless times.
Sure, but again, that's a scenario that you need to handle anyway.
> Being crash tolerant is one thing. But "crash early, crash often" is absolutely horrible advice.
I've found it to be good advice. It's a big part of why Erlang systems have been so reliable for decades.
In Erlang a "process" is more akin to a thread or fiber in most other languages, and "crashing" more like an exception that is caught just across a logical task boundary. Importantly, in Erlang a process "crashing" doesn't kill all other connections and tasks within the kernel-level Erlang process.
And that's why Erlang is so resilient, precisely because the semantics of the language make it easier to isolate tasks and subtasks, minimizing blast radius and the risk of reaching an inconsistent or unrecoverable state. I often use Lua (as a high-level glue language) to accomplish something similar: I can run a task on a coroutine, and if a constraint or assertion throws an error (exceptions in Lua are idiomatically used sparingly, similar to how they're used in Go), it's much easier to recover. This is true even for malloc failures, which can be caught at the same boundaries (pcall, coroutine.resume, etc) as any other error.[1] It's also common to use multiple separate Lua VM states in the same kernel process, communicating using sockets, data-copying channels, etc, achieving isolation behaviors even closer to what Erlang provides, along with the potential performance costs.
[1] While more tricky in a language like Lua, as long as your steady-state--i.e. event loop, etc--is free of dynamic allocations, then malloc failure is trivially recoverable. The Lua authors are careful to document which interfaces and operations might allocate, and to keep a core set of operations allocation free. Of course, you still need to design your own components likewise, and ideally such that allocations for connection request, subtasks, etc can be front-loaded (RAII style), or if not then isolated behind convenient recovery points. Erlang makes much of this discipline perfunctory.
Isolation is indeed a cornerstone of what makes this style work, but "let it crash" is another one, and just as important IMO. "In this language crashing would take down other connections" does not make it safe to continue without crashing and doesn't remove the need to crash and recover, or something equivalent to that - rather it's an argument for finding a way to separate connection tracking from more complicated logic that might get into unexpected states.
When SQL Server goes OOM, it's often because the system can't free memory quickly enough, not that there is literally not enough memory to continue operating. Sure, it could be more aggressive with freeing memory, but at some point it just makes more sense to restart the entire system so that overall performance doesn't nosedive.
Software for critical infrastructure often handles no/low memory situations in a graceful manner. I've worked on network switch software where we were expected to work indefinitely with zero remaining memory. A single packet drop is unacceptable, at 10-40G line rate. Any new malloc needs to be justified with a design document.
So yeah, average user software can do a lot, it's just that we've given up on reliability and robustness as an industry. That's why sometimes your phone fails to make a 911 call, and sometimes you need to reset your car.
This presumes that one is indeed using a pagefile or swap partition. I know of quite a few folks who skip that on SSDs out of fear of SSD wear.
Personally, in this day and age I don't really give a damn about SSD wear (that's what backups are for), so I'll happily create a swap partition, but not everyone is as comfortable with that.
A database that doesn't OOM crash is great, and the same is also true for more consumer-grade software.
I have 32gb of RAM on my PC and whenever I'm encoding a video with DaVinci Resolve, I always see Firefox and Discord crash almost instantly.