> 90% of the time, the gains from streaming aren't worth the added effort.
I gotta disagree with that estimate. Virtually any time I have a backend service operating on (mostly) arbitrarily-sized user input, I use streaming so that I can make better guarantees about how much memory my service needs. This, in turn, lets you give your customers much higher service limits (unless you want to scale your fleet's memory just to handle 100th-percentile style use cases).
The number of times I've seen backend services fall over, with a heap graph that looks like a repeated sawtooth pattern to OOM, because a customer's objects were unusually sized (but within limits..)..
Yeah this is an important accidental DOS vector, and streaming APIs are a classic way to fix them.
But you do have to be careful that you're not just overloading some other system (like consuming disk space with files that don't need to be retained). Keep good stats on all of your exhaustible resources, kids.
I gotta disagree with that estimate. Virtually any time I have a backend service operating on (mostly) arbitrarily-sized user input, I use streaming so that I can make better guarantees about how much memory my service needs. This, in turn, lets you give your customers much higher service limits (unless you want to scale your fleet's memory just to handle 100th-percentile style use cases).
The number of times I've seen backend services fall over, with a heap graph that looks like a repeated sawtooth pattern to OOM, because a customer's objects were unusually sized (but within limits..)..