Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.
If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.
Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.
The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).
Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.
as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.
Maybe. This is a bit of a cottage industry that shouldn't need to exist. It's so hard to get data and visibility into these things in AWS right now. Billing is hard, it's impossible to easily tell what is costing me what in every region, etc. AWS should do all this natively as part of their service, but everything is so poorly designed from a user standpoint that it's very time consuming to do, so third-party services like Hakuna pop up to take advantage of the inefficiency by saving some money. I'm halfway convinced AWS does this on purpose so you have to pay them for support or training for people new to cloud service providers, or AWS specifically, increasing their revenue.
Of course they do this on purpose, it does not require a lot to give you metrics on usage and tell you if you are over provisioned, when mint.com can suggest a lower interest credit card or a higher yielding account based on my financial history, AWS cab easily do the same.
But, why would they. They rather laugh all the way to the bank!
I think you’re underestimating how hard of a problem it is to provide continuous billing information. I’ve seen SAAS companies with an engineering team dedicated just to the monthly reports, handling things like pipeline delays and data cleanliness and conversion to both a clean UI and fiddly excel spreadsheets.
Tag every resource you provision (probably on 3-5) dimensions and you’ll never wonder where your money is going. It’s a great system, but most people (including me for the first 3 years) skip the tags screen by default.
I just had a conversation about this product idea a couple days ago. How long before Amazon acquires it and cripples the functionality? Could be a good exit strategy, at least.
I don't think they'd cripple it. Amazon takes a pretty long term view of things. They might well introduce something like this into their load balancer or api gateway though. It'd be nice to have in front of fargate because of the faster cold start times.
True, I think lambda would be a better choice for new projects. However, for old monolith projects, this sounds like a great way to cut costs on dev/staging servers.
Related: if you're looking for a service that starts/stops instances on a schedule (we find this really good for QA and development instances), check out https://www.parkmycloud.com/ . You can also set an instance to "always parked" and unpark it for a certain number of hours or until a certain date/time.
Interesting model, although I suggest looking at the GCP Cloud Run approach using Knative to start and server containers on demand.
That's the next generation of Lambda that all clouds and vendors are moving towards, and increases developer agility with much faster cold-start times. If we could have Cloud Run today across multiple clouds and locations with geo-loadbalancing stitched together automatically, that would be valuable.
> Hakuna Cloud is a software-as-a-service HTTPS proxy. You don't need to change existing software or infrastructure, and you don't need to install additional tools on your servers.
> Each cloud server must have an FQDN/DNS name configured as a CNAME to our load balancers.
> When your server stops receiving requests, it will be stopped. As soon as a new request arrives, Hakuna will start it.
Interesting idea. It's like a proxy that kind of makes an instance/vm-based service act like a serverless service, without moving to containers or rewriting.
Seems kind of niche but I can see the use: there's a lot of services that have a time-based usage pattern (during working hours, or used interactively for a few minutes/hours sparsely through the day).
What are the cold start times like with this (at least for a typical, simple app - say on asp.net on Windows or something hosted via nginx on
Linux)? What happens if an instance is being stopped and a new request comes in - does the request have to wait for shutdown plus startup?
At Hakuna we use our own service to start and stop our Jira and NPM servers; the instances are automatically stopped at night or whenever nobody needs to do `npm install`.
In our use case, the EC2 instance starts in about 50 seconds, with another minute needed to start the Jira service.
We have a demo, deployable directly from our CLI, that starts a Nginx server on an Oracle Cloud instance in less than 40 seconds.
If the instance is being stopped and a new request arrives, it will have to wait shutdown + startup, yes.
Is there UI feedback to the user during the wait, or does the browser just show "waiting for response" for the whole time? If a user refreshes the browser a bunch of times during the wait, will the Hakuna proxy give up on those requests or still pass all of them through to the target server?
We cannot provide UI feedback during the wait: being a HTTPS proxy, we would need the SSL certificate to reply to the request, otherwise it would be some kind of MitM.
If the client closes the socket before the start of the server, the proxy gives up on its requests.
You should probably check your website on mobile. For me on Firefox Android, the unmoving header takes up 60-70% of the screen and the actual scrolling content is crammed into the bottom fraction.
Interesting. Why proxying though rather than monitoring DNS queries on the CNAME? And updating to point to the right ip when the server is live (This could maybe help with the 10gb base limit + $0.08/gb?)
Not trying to be an armchair coach but rather understand the architecture decisions and trade-offs that I must have missed
I think they can point to some kind of a "Loading..." page that they can host? (SSL can be managed with Let's encrypt, since the CNAME is effectively controlled by them)
It seems like it does exactly (or a subset) of what Google Cloud Run already does. Just shove an application into a container and scale up/down depending on use. Other cloud providers probably have this too. So is the value add that this is less expensive or what?
> The HTTPS trigger is intercepting all my traffic?
> No, your data are safe if your server support HTTPS protocol. All the data exchanged between your server and your clients is encrypted and not accessible by us.
Unless there's an IP allocated to each user, I don't think this is accurate. With SSL, the HTTP headers are encrypted, so there would be no way to know where know where to route the request without first decrypting the data, and thus having access to the data.
SNI was added on top of TLS to allow one server to serve multiple certificates. SNI is different than the hostname header, and in fact no headers are available until the traffic is decrypted post-handshake, which Hakuna can't see if it's not terminating TLS. But SNI lets the hostnames be available outside the encrypted TLS stream on hello.
From a comment above, it seems like Hakuna requires a FQDN of each AWS server it's serving traffic to, so if you're not MITM'ing traffic, this FQDN I'm guessing sits in with SNI and is used for routing rather than serving certificates. I don't think I've personally dealt with this use case on SNI, but it makes sense.
The whole header is encrypted, including the Host header and request line. Certificate selection and routing relies on SNI[1]. The server name in the TLS client hello message is almost always, but not necessarily, a copy of the value of the Host header field.
I had long wondered if it would be possible to have a custom autoscaler that just stopped/started instances rather than terminating and re-creating, in order to respond to load increases more quickly than amazon's autoscaling groups. You still pay for the EBS even when it's stopped and deploys involve briefly starting all of the stopped instances, but EBS is a fraction of the overall spend and maybe in some cases the complexity is worth it?
Hi, I'm a developer at Hakuna. Thank you for the feedback.
With DO, we destroy the instance so that nothing is charged for it, and save a snapshot to be used to restart the instance; you pay only for the snapshot storage - $0.05/GB per month.
What is the cold start time of a DO instance image to working VM? Isn't it like 2 minutes?
If a DO droplet for my marketing site cost me $5 per month to run.. I could use your service and save a buck or two, at the risk of a $5000 client timing out on my page as it tries to spin up?
Scaling based on demand is important, but it seems you are better using k8s with some metric based scaling, over trying to hack it on DO?
I think this is not something you'd use for customer facing services. It's fine for internal stuff of the first employee of the day has to wait two minutes, and it gets shut down at night after it hasn't been used for an hour.
So is this the bring your own server version of “inactive app hibernation” that you see in the free tiers of PaaS providers like Heroku? If so, that’s neat!
Hi, I work at Hakuna Cloud as a developer. Thanks for your feedback.
If you don't like the idea of updating your DNS records, Hakuna provides a hostname creation feature for trying the product (*.demo.hakuna.cloud CNAME records that point to our load balancers).
You can install the Hakuna CLI on your laptop, no need to have it on the cloud instances. Also, most of the features you can find on the CLI are also available on the web dashboard.
- Using the Kubernetes Vertical Pod Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/vertica...) for CPU and memory scaling, and switching to metrics like requests per second and connection count for horizontal scaling
- Collecting metrics on container CPU and memory allocation vs utilization
- Writing scripts to auto-submit PRs to SREs with better recommended sizing based on actual usage
- Tuning our VM instance sizes and autoscaler configs
A few engineers were able to save the company several times their salary with a few months of work, and plan to 10x the savings over the next year