This may come as a great big surprise to you, but memory capacity is legitimately a problem, most especially at scale, and even where you can just "spin up a new instance".
Taking your "$15" stick of memory comment, that's a cost of $1.5m dollars when you start talking about a hundred thousand servers. You could pay the salary of a number of SDEs for the lifetime of that generation of servers for that much money.
Working in a memory constrained approach can lead to lower latency and faster applications, and make the difference between a service being profitable or not.
Without going in to details that would put my NDA at risk, when I worked on an AWS team, there was a change went through that improved the efficiency of the service by about a third. The original code was well written, extremely logical, and exactly what almost anyone would write when presented with the problem. You could have shown it to developers anywhere and they wouldn't have found fault with it.
Stopping and thinking very carefully about largely fictitious constraints revealed a potential other approach, one that was somewhat left-field, but both worked and came without any risks.
When your service is operating on many tens or hundreds of thousands of servers worldwide, or more, being able to retire up to a third of your servers is far from negligible in annual costs.
Taking your "$15" stick of memory comment, that's a cost of $1.5m dollars when you start talking about a hundred thousand servers. You could pay the salary of a number of SDEs for the lifetime of that generation of servers for that much money.
Working in a memory constrained approach can lead to lower latency and faster applications, and make the difference between a service being profitable or not.
Without going in to details that would put my NDA at risk, when I worked on an AWS team, there was a change went through that improved the efficiency of the service by about a third. The original code was well written, extremely logical, and exactly what almost anyone would write when presented with the problem. You could have shown it to developers anywhere and they wouldn't have found fault with it.
Stopping and thinking very carefully about largely fictitious constraints revealed a potential other approach, one that was somewhat left-field, but both worked and came without any risks.
When your service is operating on many tens or hundreds of thousands of servers worldwide, or more, being able to retire up to a third of your servers is far from negligible in annual costs.