Good stuff. We are doing the exact same thing at Grofers (more or less). Our philosophy of our service registry is exactly the same - using services to connect kubernetes objects to AWS resources to monitoring dashboards to github repos.
The place where we differ is:
* YAML instead of JSON
* A lot of python based tooling to do the plumbing
* We don't always rely on terraform for provisioning everything so we have monitoring of new resources in AWS and policies on how those untagged resources will be treated
Good to know that there are more people thinking in a similar way. Validates our approach. :)
Would love to chat in person to learn more about the setup and exchange ideas if you are up for it.
Hey! I'm always up for chatting about this type of thing. Great to trade ideas and experiences, I find you always come away with something new to think you.
If you want to email me at me@lawrencejones.dev, we can sort a virtual coffee.
I really like what you're doing here, but oh my goodness that Jsonnet cries out for a more readable format which lets one focus on the semantics more than the mechanics of the mixins.
This looks way more legible to me:
(service make-it-rain (gocardless/make-it-rain)
;; Example service called make-it-rain, powering a dashboard of falling
;; gold coins whenever anyone takes a payment via GoCardless.
;;
;; Banking teams love money, which is why they created this dashboard.
;; It's officially owned by banking-integrations, but core-banking
;; sometimes optimise the React code.
;;
;; It consumes data about new payments from Google Pub/Sub, and has a
;; separate Google Cloud Platform project for each of its environments,
;; of which there are two: staging and production.
(team banking-integrations)
(alerts-channel make-it-rain-alerts)
(google-services pubsub.googleapis.com)
(environments
;; By default, every environment should have banking-integrations as
;; admins, and core-banking as operators (they provide on-call cover
;; for the falling gold coins).
(* (admins banking-integrations)
(operators core-banking))
(staging
(google-project gc-prd-make-it-stag-833e)
(argocd.new (cluster compute-staging-brava)
(namespace make-it-rain)))
(production
;; Unlike most services, the production environment should permit
;; a non-engineering team to open consoles. Sometimes we take a
;; manual payment outside of GoCardless, and banking-operations
;; open a make-it-rain console and run a script, so we don't miss
;; any gold coins.
(operators banking-operations)
(google-project gc-prd-make-it-prod-1eb1)
(argocd.new (cluster compute-banking)
(namespace make-it-rain)))))
It would be really easy to write, too. Basically a service takes a mandatory name & repo, then all other elements are mixins. You could take a couple of different approaches with the way the environment mixin works, too. You could still compile it to JSON, if that's what you need downstream, too.
I think there's a huge win in having a more man-readable, machine-computable format that JSON or Jsonnet: as the system and the tooling grow, I think eventually the Jsonnet will become an unreadable mess of spaghetti (some might argue the example already is!), while a well-thought-out S-expression alternative would not, I think, have the same problem.
So firstly, I really like this. I find it nicely readable, I like that the structure is almost all you have to write, and I'm quite a fan of lisp-y syntax anyway, so no surprises there.
The primary reason we choose Jsonnet for the registry library was because it's already become our data munger of choice at GoCardless, and we're keen to limit the number of tools we use to make our work more accessible to people outside our team, along with our new joiners.
So while your example works really quite beautifully for this use case, we'd naturally prefer to choose a tool that can work for all our use cases. And Jsonnet has a bit win over the language you've sketched here, in its inheritance and mixin semantics, which allow us to write configuration like complex Kubernetes manifests that are composeable and allow flexible overriding of fields from the caller.
Someone else has mentioned dhall-lang in the comments. We looked at this and several alternatives before we began a push for Jsonnet about a year ago- we ended up choosing Jsonnet over these because:
- Jsonnet has first-class support in Golang, our primary language for infrastructure code
- Everyone who knows JSON already knows how to write valid Jsonnet. This was important, as we expect application engineers to contribute to the Jsonnet config, and asking them to learn something totally novel is a bit much
- There were several tools like Ksonnet and Tanka from which we could take inspiration, especially around managing of Kubernetes resources, which was the big motivation behind a total revamp
If I were working on a team solo, I think dhall-lang would look really appealing to me. I like strongly typed languages and have a bit of experience in Haskell, so it looks very normal to me. That said, cuelang (https://github.com/cuelang/cue) looks super interesting, though it has a very different take to Jsonnet.
This is a roundabout way of saying I really like your example, and I see our choice of Jsonnet as another version of our choice of Golang: the network effects were too great to ignore, and it's good enough to work for almost everything, if not the best in any one area.
Author of the post here, happy to answer any questions about our experience with building the registry, what things it tracks, how we might handle infrastructure deletions, etc.
The place where we differ is: * YAML instead of JSON * A lot of python based tooling to do the plumbing * We don't always rely on terraform for provisioning everything so we have monitoring of new resources in AWS and policies on how those untagged resources will be treated
Good to know that there are more people thinking in a similar way. Validates our approach. :)
Would love to chat in person to learn more about the setup and exchange ideas if you are up for it.