Ehhh. This is an unpopular viewpoint, but to be frank: I've rolled up k8s at scale, and to be honest? I think you might as well just use ECS if you're going to already be into AWS. And it plugs into pretty much everything AWS does without having to either reinvent every wheel or having to build translation layers (or use ones you don't fully understand). Failing that, use EC2 and run singleton Docker containers on each instance (easy enough to do with a short cloud-init script). You already have resource segmentation. You shouldn't have enough headroom in your cluster as-is to not be using all of it (otherwise it's just deadweight loss).
The set of shops that actually benefit from k8s or the like are a rounding error and so many person-hours are burned building stuff like this that it hurts. Simplicity is better than complexity until your needs require complexity.
If you'd like to chat further, feel free to reach out - email's in my profile.
The set of shops that actually benefit from k8s or the like are a rounding error and so many person-hours are burned building stuff like this that it hurts. Simplicity is better than complexity until your needs require complexity.
If you'd like to chat further, feel free to reach out - email's in my profile.