I really wish Rancher didn't abandon Rancher 1.6 and moved to k8s.
This was a perfect solution for a small business and bare metal.
I am trying to move on k3s but it is just too complex to run anything and there is still not solved problem of exposing services to internet.
What I want is to declare I want this service to be under this domain and this IP - so for that you still need to configure your load balancer (bare metal) manually, setup certificates etc. I am writing a tool to automate this, but it's been a pain.
> What I want is to declare I want this service to be under this domain and this IP - so for that you still need to configure your load balancer (bare metal) manually, setup certificates etc. I am writing a tool to automate this, but it's been a pain.
So this is not going to work for several reasons.
One being that on bare metal you don't have a cloud provider, so there is no load balancer it can talk to.
Second - it will setup a hostname and a certificate on the ingress, but there is no way to contact it from outside world. The domain still needs A record pointing at the server and in the cluster that may be a local IP or a set of IPs.
What I have in mind is an external server that is not being a part of the cluster that bears the role of load balancer. It will contact the cluster and look for services and then setup up a reverse proxy based on their declared hostname, then setup certificates and update DNS records at DNS provider.
As far as I know something like this does not exist.
Maybe Traefik has such a capability, but their documentation is so complex I have no idea.
Actually I'm using it on bare metal and it works. Initial setup wasn't very hard but I think it could be more intuitive. Overall I think documentation for self-hosting kubernetes sometimes a bit incomplete.
Yes, I need to add A records with IPs for each domain, but that's one time setup. I did it manually, but you can automate it [1] (depends on what you use for DNS provider but you can extend it to support your provider or maybe there is another existing solution).
I'm not sure that one server in front of the cluster is more reliable than using all cluster nodes for load balancing. I guess that in automated solutions like [1] cluster's node could be automatically deleted from DNS if it went down.
My setup is not so big so I don't have real need for load balancing, but it seems possible with existing solutions.
Sure it does, I ran kube-vip[1](but there are many others, e.g. metallb) as my cloud controller, all it needs are valid static IPs/range/dhcp and it will assign these to LoadBalancer services(which you usually only need one of for your ingress) and it will either ARP or use BGP to route external traffic.
As for DNS records, external-dns[2] works perfectly as long as your DNS as some way to doing automatic updates.
The problem with kube-vip is that it has poor documentation. I have read it many times and still don't know how I could use it.
Last time I was running something assigning IP addresses to the dedicated server interface I got it null routed and provider threatened to terminate the service because it was interfering with other clients network.
So if I see things like ARP, BGP, DHCP it is not clear what exactly it does on the network and how that would work in the real world.
I am missing an example where I have a server with a static IP from which I want to access the exposed services that are on a private network.
All I really want is an automatically configured reverse proxy that will direct traffic to appropriate services and take care of certificates and DNS.
Before the Kubernetes I used Rancher 1.6 and that was super simple. For instance I would start a wordpress container and then all I needed to do was to add a reverse proxy entry with its hostname as a backend and point where the certificates are (that was before lets encrypt).
Closest I could get was exposing a NodePort and having nginx to reverse proxy to the nodes at given port, but that seems more complex / fragile, as I need o keep track which service uses which port and it is still manual, so I might as well just use containers without Kubernetes.
I am trying to move on k3s but it is just too complex to run anything and there is still not solved problem of exposing services to internet.
What I want is to declare I want this service to be under this domain and this IP - so for that you still need to configure your load balancer (bare metal) manually, setup certificates etc. I am writing a tool to automate this, but it's been a pain.