Hi HN, we're Lucas and Lucas, the authors of Layerform (https://github.com/ergomake/layerform). Layerform is an open-source tool for setting up development environments using plain .tf files. We allow each engineer to create their own "staging" environment and reuse infrastructure.
Whenever engineers run layerform spawn, we use plain .tf files to give them their own "staging" environment that looks just like production.
Many teams have a single (or too few) staging environments, which developers have to queue to use. This is particularly a problem when a system is large, because then engineers can't run it on their machines and cannot easily test their changes in a production-like environment. Often they end up with a cluttered Slack channel in which engineers wait for their turn to use staging. Sometimes, they don't even have that clunky channel and end up merging broken code or shipping bugs to production. Lucas and I decided to solve this because we previously suffered with shared staging environments.
Layerform gives each developer their own production-like environment.This eliminates the bottleneck, increasing the number of deploys engineers make. Additionally, it reduces the amount of bugs and rework because developers have a production-like environment to develop and test against. They can just run "layerform spawn" and get their own staging.
We wrap the MPL-licensed Terraform and allow engineers to encapsulate each part of their infrastructure into layers. They can then create multiple instances of a particular layer to create a development environment.The benefit of using layers instead of raw Terraform modules is that they're much easier to write and reuse, meaning multiple development environments can run on top of the same infrastructure.
Layerform's environments are quick and cheap to spin up because they share core pieces of infrastructure. Additionally, Layerform can automatically tag components in each layer, making it easier for FinOps teams to manage costs and do chargebacks.
For example: with Layerform, a product developer can spin up their own lambdas and pods for staging while still using a shared Kubernetes cluster and Kafka instance. That way, development environments are quicker to spin up and cheaper to maintain. Each developer's layer also gets a tag, meaning FinOps teams know how much each team's environments cost.
For the sake of transparency, the way we intend to make money is by providing a managed service with governance, management, and cost-control features, including turning off environments automatically on inactivity or after business hours. The Layerform CLI itself will remain free and open (GPL).
You can download the Layerform CLI right now and use it for free. Currently, all the state, permissions, and layer definitions stay in your cloud, under your control.
After the whole license change thing, I think it's also worth mentioning we'll be building on top of the community's fork and will consider adding support for Pulumi too.
We'd love your feedback on our solution to eliminate “the staging bottleneck". What do you think?
Agree that "preview apps are hard" is a massive bottleneck - especially when specs are evolving and a feature may spend some time being QA'd by technical and non-technical team members before being ready to land. Not to mention that processes for securely populating seed data from recent production data are highly domain-specific!
We've built our own system for preview apps on k8s, with a complex Github action that tries to encapsulate everything we can into a set of helm charts that are installed into a dedicated namespace for all the databases and services for that specific PR.
Of course, this can only go so far, and doesn't easily allow for tracking assets outside of k8s. But given that we can keep many of those shared, we've in part stayed away from Terraform due to adding yet another learning curve - but also for lack of tooling for this shared-some-things-but-not-all per-environment customizability - the exact problem you're solving!
I'm excited to see where this system goes - and am excited to see what value-add layers you build on the UI and governance/cost-control side!