This makes too much sense to me to think people use Kubernetes for anything else. Why must one use Kubernetes for EVERYTHING unless there is some higher-order reason?
I use Kubernetes to host something like 6 of our own services, and it excels at that and is fairly simple. Databases and other things use different services.
We did the reverse of go. We started with purely stateless k8s and got really comfortable with the platform before moving to support stateful loads. For databases, many vendors have dedicated operators which introduces best practice deployments with less operational fuss provisioning the instances. Tooling like Strimzi for Kafka as an example here.
Do you really think they have ALL the operations coded properly for all operational conditions?
Stateless is so much easier for operations. It either is running, or not, A/B upgrades, yada yada.
Stateful has backups, restores, outages, patching, migrations, corruptions, fixes. For distributed systems it gets even hairier. Do they support your distribution? What if you have a multicloud or blended internal/cloud? Does the operator provide a turnkey restore from backup, and are you testing it?
Operators shouldn't be viewed as replacements for stateful operations knowledge, but it's probably what they'll be used for.
If you're using a large scale distributed stateful/database system, you need an operations team to support it, or pay for that.
You have a lot less problems in a single VM: There‘s (hopefully) a clear path on how to install software etc. and it is easy to understand. If your distributed PostgreSQL operator written by someone fails you‘re out of luck and it will be really really hard restoring your system. With non-kubernetes non-automated solutions not written by someone you may have some docs saying „Update the replication address and restart“.
The way to restore to a running system is much easier in old traditional vm approaches.
In K8 paths are clearer: Every bit of component that goes into a deployment is versioned. They deploy the same in every build. Its 100% certain.
The problem with stateful app scaling and HA comes from two things: the need to make files and database multi read-write and high available. Other than that, app scaling is pretty easy in K8 even for stateful apps.
I use Kubernetes to host something like 6 of our own services, and it excels at that and is fairly simple. Databases and other things use different services.