That seems to be people main justification for using Kubernetes. We really need to find an solution that will allow us to deploy software programmatically, without dragging along the complexity of Kubernetes.
I know, Kubernetes is “easy” on AWS, Azure or GCP. Well great, but many of us can’t use managed Kubernetes. Now we’re stuck managing on-prem cluster, just so the developers can deploy three containers and an ingress controller while complaining that they need to update their YAML because some API changed again.
Kubernetes is great, for some project, just not most. Sadly we’re don’t have any good alternative for deploying to VMs.
I was just looking into Harbormaster. Does it play well with Traefik? Seems like a good combo in theory at least. I'm already running Traefik, but haven't found a nice way to automatically upgrade and make sure docker-compose is running for all projects I'm hosting.
Hmm, I've never used Traefik with it, I use Caddy in host mode. Does Traefik autodetect the containers from the Docker socket? I might look into it because that would make automatic ingress painless.
Harbormaster is basically a thin layer/manager over Compose. If Traefik works with a bunch of running containers, it'll work with Harbormaster.
> Does Traefik autodetect the containers from the Docker socket?
Yes, it can do that. you can put annotations(? tags? I forget what the docker terminology is) on containers that Traefik reads so it knows "expose port xyz on this container as https://example.com".
That would be really cool. I might mess around with it around Christmas. If I run into any rough edges around labels (Traefik uses labels like `traefik.enable=true` https://doc.traefik.io/traefik/reference/dynamic-configurati...) I'll submit an issue or PR.
> Does Traefik autodetect the containers from the Docker socket?
Given the overall expertise you've demonstrated here on HN time and again, I'm sure you are aware of this but a general warning to everyone else reading this might be in order: Careful with exposing the Docker socket to Traefik, see https://github.com/traefik/traefik/issues/4174
Ah, yes, I'm unclear on whether Docker intends to be secure these days (though there are things like Sysbox or running rootless that help), but exposing the socket inside a root container is removing a few layers of defense, thank you for pointing this out.
Unfortunately I don't think there's a way for Harbormaster to do anything about it, but I'll play around with Traefik and see if there's a more secure way to deploy it, thank you.
Not really what I expected. I was thinking more in terms of a kubectl replacement, but this is actually pretty clever for the projects that want to run a git-ops setup.
Oh hmm, how do you use kubectl for programmatic deployments? I've used it before but it required an intermediate machine with cluster access to run kubectl on, which is less convenient than git-ops.
It depends on the customer and the project. Most want git-ops, but often they also want kubectl. The overhead of having to build, instrument and manage the infrastructure to run software on a Kubernetes cluster is often a pretty big surprise for many of our customers. So in order to debug, developers still resort to kubectl to get a shell in a pod or viewing logs.
This is also why I'm against Kubernetes for most projects. It require a high level of maturity in the development team to do right. When you see some one who knows what they're doing, Kubernetes is amazing. For most people however, picking something more "classic" is going to make their life much easier. This is especially in the light of what most want is just some easy way to saying: "Run this container image on those two hosts".
So far the best solution I've been able to come up with is Ansible and AWX. Developers can then bump a version number in an group_vars file and click "launch" in AWX.
I find Nomad (coupled with Terraform, Consul and Vault) hits the sweet spot for me.
It provides all the orchestration, coordination and resource management in source-controlled templates but doesn’t add at all the layers of abstraction and indirection that k8s does.
There are a few solutions that try to abstract away the innerworkings of k8s, like Garden and Tilt. They enable you to use a Compose-like syntax for deployment, which covers the 80% use case of "I just wanna deploy an app, bro"
The issue is that escaping the complexity of Kubernetes is challenging because of that 20%. You want to use Vault for pulling secrets in dynamically and customize how your service mesh routes traffic to your app? Unless you're using something like OpenShift that gives you one way to do it, you're gonna mess with some YAML.
Can you describe what kubevirt is? I'd like to understand, but having spent 15 mins on their github, blog and docs site, there's no one-liner or even a summary of features offered.
> the technology provides a unified development platform where developers can build, modify, and deploy applications residing in both Application Containers as well as Virtual Machines in a common, shared environment
That seems to be people main justification for using Kubernetes. We really need to find an solution that will allow us to deploy software programmatically, without dragging along the complexity of Kubernetes.
I know, Kubernetes is “easy” on AWS, Azure or GCP. Well great, but many of us can’t use managed Kubernetes. Now we’re stuck managing on-prem cluster, just so the developers can deploy three containers and an ingress controller while complaining that they need to update their YAML because some API changed again.
Kubernetes is great, for some project, just not most. Sadly we’re don’t have any good alternative for deploying to VMs.