I always figured it was the other way around. When you're small it's pretty easy to get by with a stupidly simple solution but as you grow you end up needing to spend much more to build something scalable and at that point, using the cloud makes sense. The biggest success that cloud providers have had is convincing users that they need to spend $100k and that a much simpler $5k solution that's built using off the shelf components just won't cut it.
I see the cloud mostly for startup-ish companies hoping to grow rapidly but which want to avoid large upfront expenses to be ready for said growth.
A stable company where growth as a percentage isn't likely to be significant can run things cheaper on their own in most cases. At least if you consider the cost of the inevitable departure from the cloud provider either to switch another or to go on-prem. And if you aren't willing to make that exit, you can guarantee your cloud provider won't stop cranking up the fees until the threat of you leaving surfaces.
I think this is a pretty key point. If a business is going through any kind of rapid change, cloud providers offer a lot of off-the-shelf help for that, be it ability to scale, hosted infrastructure, or PoPs in new geographies. If the company is relatively static with easily predictable future requirements, you can get a lot more bang-for-your-buck by handling things on your own and developing your own in-house expertise.
There is also a third approach that is the best if you have a predictible base load with surges sometimes imo: hybrid cloud
You basically run the base load in your own data center and the surges go to the cloud. My university is evaluating this because sometimes you have multiple labs that need a lot of compute resources at the same time and local compute cluster has finite capacity.
It's not though. With your own stuff you have at least one DC sitting idle, with all that private gear doing nothing. Doesn't matter if you don't use a single byte of bandwidth. With AWS at least some of that is not there.
If you're set up for HA you're paying for the idle hardware either way, and if you save on electricity that might benefit the DC option but not the cloud option. Overall not much difference there.
Bandwidth is the one thing where the cloud clearly wins wrt idle servers... except that DC bandwidth is a hundred times cheaper than AWS bandwidth, so you should prefer buying 133% or 150% or 200% DC bandwidth by a mile.
Whether you are paying for HA depends on your Recovery Time Objective (RTO). You can have a bunch of suspended EC2 instances and non EC2 resources where you only pay per use in another region.
You can redirect traffic to another region and have autoscaling spin up EC2 instances, etc.
Sure, if you can wait for it to load from unallocated resources (and risk failure) then it's a very different scenario.
But, very notably, you can have a suspended cloud backup even if your main servers aren't cloud. And the added complexity for datacenter-to-cloud HA doesn't have to be significantly higher than the cloud-to-cloud version.
Entirely depends on use case. If you "just" need a lot - a lot of storage, bandwidth, CPU power - going on prem is way cheaper when you get up to "few racks of servers".
If you complicated your architecture enough - and the cloud makes it oh so easy to make rube goldberg architecture - keeping many different services running or even developing in-house can take a lot.
And it's not like cloud costs you zero in ops work either, just need different set of skills.
But it is not like on-prem stagnated - there is plenty automation in that space too. Our team of 3 manages 7 racks of servers and few dozen projects on them (anything from "very legacy" to 30+ node k8s cluster and ceph storage with it) and the hardware management still isn't majority of our work
