Haha, it sounds like you have it covered. Even more so if you were to run on GKE (which I use and adore).
When not to use it is a tough question. If I was ever in charge of a company, kubernetes would be the only way of running my product that I would consider. I am a fan of kubernetes as I use it every day but I have also been on the other side of the fence. I have run production systems on bare metal, VMs, EC2 instances, etc. The operational burden of anything non-kube is too much and takes time away from solving big problems such as stability, scaling, deploy, monitoring and more. The solutions to the problems become standard, boring and consistent.
I say the above as someone that spent over a year migrating an entire platform/product from ECS to GKE. It is not perfect but so many silly day to day interruptions have been eliminated. Retired and broken instances are a thing of a past. Scaling is easy. Stability is easier.
Side effects of the move are that our Ops team is 1/2 the size it was a year ago (attrition/covid), we are running 3 times the number of product stacks for 1/3rd the cost. I should really blog about that one!
Kubernetes support is a top Tailscale request and the community's starting to do it on their own (a bit, in less than ideal ways sometimes) so soon enough here we'll have to end our little Kubernetes vacation and get back into it and make it Tailscale support Kubernetes really well.
When not to use it is a tough question. If I was ever in charge of a company, kubernetes would be the only way of running my product that I would consider. I am a fan of kubernetes as I use it every day but I have also been on the other side of the fence. I have run production systems on bare metal, VMs, EC2 instances, etc. The operational burden of anything non-kube is too much and takes time away from solving big problems such as stability, scaling, deploy, monitoring and more. The solutions to the problems become standard, boring and consistent.
I say the above as someone that spent over a year migrating an entire platform/product from ECS to GKE. It is not perfect but so many silly day to day interruptions have been eliminated. Retired and broken instances are a thing of a past. Scaling is easy. Stability is easier.
Side effects of the move are that our Ops team is 1/2 the size it was a year ago (attrition/covid), we are running 3 times the number of product stacks for 1/3rd the cost. I should really blog about that one!