yes! As always, it's mostly a tooling issue. Surely, some aspects of the language like lazy evaluation, make it even harder, but fundamentally there is no reason we cannot significantly improve on the overall experience.
And this is not just about debugging when things go wrong. It's also hard to navigate the codebase and understand where are the templates that you care about. Things can happen at multiple levels (that's the feature!) and so it can be quite hard to figure out which files you need to touch if you want to change something in the output. I'm working on a tool to answer that question: "if I wanted to change this field in the generated out put, what are the places in the input files that contribute to produce this value"
The tool lives in https://github.com/mkmik/ursonnet . I got the basics working but it doesn't work on my larger codebases due to some bug I didn't have yet time hunting down. Having some interest/feedback/help from the community would help making this a reality.
Its mind-bogging amount of work to write correct algebraic types that make sense for most JSON and YAML config files. Pretty much every property is optional, but not really as some combinations are allowed while others not. If you are lucky there might be a JSON Schema with `oneof`s you can automatically map to unions, if not there will just be documentation (e.g. Concourse YAML). Sometimes there will even be two algebraic datatypes encoded in one object, so you would need 2 -> 1 value mapping (or 4 union tags for the possible combinations, whatever suits your fancy)
Since jsonnet is side-effect free, types are somewhat less needed. You can instead execute your code, which will generate values and which can report arbitrary runtime errors (all the way to the language server, even)
(Types would still be awesome for writing funcitons, though)
I can't seem to wrap my head around CUE. Most of my Jsonnet is creating a nice abstraction layer using functions. It seems CUE specifically does not like that approach. Maybe I'm just looking at it the wrong way.
Yeah, code functions aren’t the right mindset for CUE. The essence of it is narrowing constraints and defaults. You have a bunch of constraints that make a template, like Server. Then something extends that, like StageServer. Then it has to get so specific (MyAppStageServer) that all the values are concrete or can be filled with defaults. The constraint narrowing for a field is something like int, 1-8, 1-4, 2. This makes multiple template use (like multiple inheritance) not a nightmare.
Configuration as code, but in a quirky language - how about no? I'm sure someone uses this, someone created this, and so it might seem like it's useful, but we have enough clownishness out there with yaml, json combined with js, json with eval and so on.
Create a framework in a normal language, but one which is very limited by design with regard to what is able to access - Skylark/Starlark is a good example[0], "limited Python" and use that. The rest works the same way, take an input in this language/framework, generate outputs.
Tried it[0], worked reasonably well. Be prepared for strong opposition from traditional “devops” folks (really just rebranded sysadmins) and boneheaded management “who don’t mind yaml” and will drag everyone down.
My employer (who I don’t speak for) used this for a long time. Im not normally particularly opinionated about tech choice, but I can very strongly say do not use jsonnet unless you enjoy adding an esoteric unsupported language to your job configs for no reason.
Not OP, but I can highly recommend just using the programming language you already use to make key-value map to and then converting that to JSON. That way you only have 1 syntax to worry about. In my case that’s Python (using dictionaries and json.dumps)
You could also try for a templating framework (like jinja2) but then you have 3 syntaxes colliding: the programming language you call the templating engine with (e.g. Python for jinja2) the templating language itself (e.g. jinja2) and then JSON as well.
This works for some languages but not others (some typed languages are particularly ill-suited to the problem of writing CLI tools to dump heterogenous dictionaries and/or objects with many optional fields)
I can definitely sympathize here - in every context, just straight JSON/YAML configuration seems never expressive enough, but the tooling created in response always seems to come with sharp edges.
Here are some of the things I appreciate about Jsonnet:
- It evals to JSON, so even though the semantics of the language are confusing, it is reasonably easy to eval and iterate on some Jsonnet until it emits what one is expecting - and after that, it's easy to create some validation tests so that regressions don't occur.
- It takes advantage of the fact that JSON is a lowest-common-denominator for many data serialization formats. YAML is technically a superset of JSON, so valid JSON is also valid YAML. Proto3 messages have a canonical JSON representation, so JSON can also adhere to protobuf schemas. This covers most "serialized data structure" use-cases I typically encounter (TOML and HCL are outliers, but many tools that accept those also accept equivalent JSON). This means that with a little bit of build-tool duct-taping, Jsonnet can be used to generate configurations for a wide variety of tooling.
- Jsonnet is itself a superset of JSON - so those more willing to write verbose JSON than learn Jsonnet can still write JSON that someone else can import/use elsewhere. Using Jsonnet does not preclude falling back to JSON.
- The tooling works well - installing the Jsonnet VSCode plugin brings in a code formatter that does an excellent job, and rules_jsonnet[0] provides good bazel integration, if that's your thing.
I'm excited about Jsonnet because now as long as other tool authors decide to consume JSON, I can more easily abstract away their verbosity without writing a purpose-built tool (looking at you, Kubernetes) without resorting to text templating (ahem Helm). Jsonnet might just be my "one JSON-generation language to rule them all"!
---
Though if Starlark is your thing, do checkout out skycfg[1]
Why do you think jsonnet is quirky? It has pretty normal components overall: local bindings with lexical scope, imports, functions. Certainly more pedestrian / closer to most general purpose languages. The only strange bit may be thunks / lazy evaluation, but its definitely possible to opt out of relying on their behavior.
It always bothered me that the "tutorial" (https://jsonnet.org/learning/tutorial.html) is using mixing cocktails as examples, rather than actual networks/infrastructure which is a much more likely use cases for using jsonnet (or, I'm much less into mixing than my fellow developers/operators).
The excellent Jsonnet training course [1] by u/Duologic somewhat fills this niche by having you play around with k8s resources and using external packages.
Cue is a little different I think. It doesn't attempt to make writing configs less tedious through the use of functions.
It is more about schemas and reliably merging documents as far as I can tell. But weirdly given its focus on schemas I couldn't find any way for a document to link to a schema in-band!
FWIW, "YAML Typing = None" is for sure wrong, that's what the `!!map` annotations (https://yaml.org/spec/1.1/#tag/syntax) are for, and a very common RCE vector for dynamic languages since they can cause execution when the parser instantiates the types
We use Jsonnet with Tanka (by Grafana) for managing our Kubernetes manifests. It does take a bit to pick it up properly. The language itself is straight-forward (coming from Nix it was easy), however, learning how to use ksonnet is the challenging part.
I'm still tinkering with how to organize things. I don't think I've fully grasped the purpose of mixins yet.
The Prometheus Operator and Thanos projects only officially provide jsonnet libraries and example implementations, rather than Helm charts (which are provided by the community, but I detest Helm charts anyway) so I've picked up Jsonnet in the last few weeks to build our new monitoring stack and I'm really impressed.
It's a massive learning curve but it has become really powerful and flexible.
I started trying to use Tanka but found the mental model a bit strange and struggled passing parameters into environments, so I tried Kapitan instead and that has been much more productive.
Original creator of both of those here. I recommend just generating things out straight with the `jsonnet -m` command, and then doing `kubectl diff` on a PR and apply on merge. Scaled huge amount of infra this way and it’s easily understandable and extensible.
I have to admit I'm a little starstruck to see your username replying to me. Thanks for all the effort and expertise you bring to bear on observability!
We're using exactly that diff-on-PR/apply-on-merge approach, checking the generated manifests in for easier reviews, but I found just using jsonnet -m made it difficult to DRY things out (e.g. set some URL values differently per environment) so I'd be curious to hear how you approach that?
Would you wrap the customised kube-thanos/prometheus-operator "calls" in another library?
Nothing I dread more than needing to edit the tangled web of jsonnet files that constitute our grafana dashboards. A jumble of spaghetti with hard to find documentation.
I'm not the person you're replying to, but in our company we edit them in the UI but check the results into source control (so that we can re-deploy the dashboards consistently). I imagine for more complicated dashboards you could get stuck editing them manually, and it's probably the kind of thing where once you do it one time, you're stuck doing it for the rest of time.
The nix language, albeit for a package manager, is really similar. You get json-like "attrsets" and can have variables and functions. Then of course you can export it to different formats as well.
We use Jsonnet extensively in Aperture project for generating control policies.
- Policy spec is expressed in protobuf format
- GRPC gateway plugin is used to generate Swagger (OpenAPI v2) spec from proto files
- Jsonnet bindings are generated from Swagger spec.
- Blueprints are implemented using Jsonnet bindings. The users generate policies from blueprints by providing configuration in yaml (using aperturectl CLI) or via jsonnet mixin.
The issue with normal programming languages is that they are rarely properly sandboxed and deterministic. Jsonnet can't wipe your hard disk or steal your passwords.
Something like Starlark is probably a good option.
self should refer to the immediately containing object, the outermost object should be called top (or whatever). Or perhaps found relatively with self.out.out.out...
Imagine having to take a nested configuration fragment that refers to its own pieces via the absolute self, and trying to plant it in some other configuration, at a different nesting level.
yep, mixing lexical and dynamic scope was the hell indeed.
A convention I use with jsonnet is to use locals to "anchor" some important objects in the lexical scope.
My rule is to name the locals with exactly the same name as the field whose object they refer to. Example:
Adding these locals gives the right amount of friction to avoid using this all the time, while at the same time having a predictable naming for the locals. The jsonnet LSP tool can easily resolve that local since it's only a lexical thing
Some of our customers use Jsonnet to generate JSON-based configurations, including Terraform and even Postman collections.
They choose to use it because:
- it allows them to keep things DRY when replicating a configuration artifact across environments
- it is accessible for folks who are not as comfortable with program control flow and other programming constructs. They can start by copying and pasting JSON.
- it provides more advanced control flow and logic features for those who need them
- it provides features for template parameter validation
- it is not that hard to learn
Some things could be improved for sure. Better IDE support, error messages which are not the best. Maybe also a more confident community.
Overall, Jsonnet does seem to hit a pragmatic sweet spot for a bunch of use cases, regardless of the merits of other approaches mentioned in this great discussion!
I evaluated Jsonnet and Starlark for configuration generation at a previous job, and found Starlark more practical -- the similarity to Python made it much simpler for everyone to pick up and use.
Honest question, why not just use… JavaScript? I get that this is scoped to only templating json, but the syntax is already diverging pretty heavily from actual JavaScript, while also trying to pretend to be similar? You’re using operators and a self/this keyword and things, but I can only assume they obey different rules.
Could just be config.js. (Or just generate JSON in your app programmatically, it’s a pretty simple well-defined syntax and there’s already a hash-to-json library for whatever platform you’re using)
Scoping to templating json lets the library enforce a really strict contract that you don't get with javascript. For example,
* Accessing a missing field is totally fine in JavaScript and returns undefined; this is an error in Jsonnet.
* JSonnet guarantees no side-effects, not true of JS.
* When you run a jsonnet file, you have a consistent CLI that handles things like external vars, whereas you'd have to write your own wrapper to do this in JS.
* JSonnet lets you export to multiple different config formats, whereas JS only support JSON really. Again, you'd have to write a wrapper to do this.
* JavaScript would let you return any valid JS object, your wrapper would need to validate.
Like yes, you could write your own JS library to mimic a lot of these behaviors, but the constraints are what make the tool useful. I do wish it had better typing though, so you could more easily inspect the structure of the resulting object.
If the constraints are what makes it special, why is it just not very constrained? If you really wanted something totally focused on having multiple configs sharing some base config template, then you wouldn’t be able to do anything other than overwrite specific keys entirely. (json deep merge) Jsonnet seems to let you concat strings, do some math using it’s standard lib (?!), reference other parts of the object etc.
I also don’t think “object matches type description” is a niche that JS is lacking, there’s tons of libraries out there to assert that.
You’re just trading footguns for different footguns IMO.
Yep, it's all about them tradeoffs. JS gives you absolute freedom, JSonnet doesn't. JS is easier to debug, JSonnet isn't because it's very declarative.
Yes that's a common problem with config languages. They're torn between competing priorities:
1. Fast to parse with a small engine, good error messages, safe to evaluate.
2. Powerful, can express config with arbitrary logic.
In Conveyor we try an alternative approach. The config is HOCON, which is a superset of JSON syntax designed for human readability/writability/convenience first and foremost, so it's got a very nice and clean feel to it. You can see an example here:
It can be parsed with a normal-sized config library and the errors you get are reasonable.
But then what if you hit the limits of what it can express? We added support for "hashbang includes":
include "#!script.js"
You can embed arbitrary commands in the config which are executed when found unless the app is running in untrusted mode. The script is expected to produce more config on stdout, which is then included. This lets you encode only the minimal needed logic using a full programming language, whilst the rest stays declarative.
In case you want a fresh opinion, every time I stumble upon a config like this, I wish it was just a .json, .js, .py, heck even .pl or .php file rather than that.
Why. It’s unclear what’s the syntactic structure of it at a glance. Yes, in stricter languages there’s more of /[()[\]{}"'`,;]/, but also I know for sure what’s a literal, what’s an identifier, what’s a key, what’s a number or a date. The whole structure and ordering is obvious.
Same issue I have with nginx, terraform and other ad-hoc half-languages half-formats half-templates. The worst part is that you have to hack them once a year, but it never “sinks in” for good even if you read the docs.
You can also convert HOCON to JSON and back. I find it a lot easier to work with configs when you can get rid of superfluous syntax, can use comments, substitutions, include files etc. But you don't have to use those.
I honestly... don't know. I have been using Jsonnet to programatically generate Grafana dashboards, because AFAIK the only official library to generate them is written in Jsonnet [1].
In my experience, it works, but it doesn't really give me any distinct advantage and instead it gives me some headaches. Maybe for things that look almost like JSON it'd be helpful, but the moment you start dealing with more complex generations you start finding lack of typing, lack of IDE support, lack of easy debugging, etc, fairly problematic. For example, something I really dislike is that due to how the expressions are evaluated, the only way to add debug/trace statements is to use them to "transform" a value you're going to use, if you don't use the result of the trace in the final output, the trace does not appear.
Also, I really dislike the error messages. Again, due to the lazy evaluation design, when you mess up something the error message might appear deep in some unrelated call stack and with a jarring lack of context. Debugging it is a real pain.
yes! As always, it's mostly a tooling issue. Surely, some aspects of the language like lazy evaluation, make it even harder, but fundamentally there is no reason we cannot significantly improve on the overall experience.
And this is not just about debugging when things go wrong. It's also hard to navigate the codebase and understand where are the templates that you care about. Things can happen at multiple levels (that's the feature!) and so it can be quite hard to figure out which files you need to touch if you want to change something in the output. I'm working on a tool to answer that question: "if I wanted to change this field in the generated out put, what are the places in the input files that contribute to produce this value"
The tool lives in https://github.com/mkmik/ursonnet . I got the basics working but it doesn't work on my larger codebases due to some bug I didn't have yet time hunting down. Having some interest/feedback/help from the community would help making this a reality.
Grafana have been working on a replacement for grafonnet but unfortunately haven’t shared much detail. It sounds like they completely rewrote the dashboard spec using CUE, but there will be converters available at release.
In some of the discussions where the community is trying to figure out how to future-proof current dashboard definitions, Grafana Labs has also recommended this Python tool (unofficially I suppose) by Weaveworks, called grafanalib:
https://github.com/weaveworks/grafanalib
Thema is mentioned there, as “the presumed successor to grafonnet,” but that hasn’t officially been confirmed and it sounds like bigger changes might be underway. Discussions are also happening elsewhere, like their Slack, but that link is the most complete overview I’ve found.
Thema is still under heavy development for sure. I chat with Sam about it on the CUE slack from time to time. I have a related effort I am working on for data model evolution, tracking, and transformation generations as part of https://github.com/hofstadter-io/hof
Agreed. Tomorrow they’ll realize that templating often requires string capitalization and other operations and will start reinventing Math, String.prototype and exception handling, if not yet.
It’s just a new language with a new runtime, for some reason marketed as a json templater. Literally any language with “json” module is at least equivalent to it, but more familiar to developers.
> Honest question, why not just use… JavaScript? I get that this is scoped to only templating json, but the syntax is already diverging pretty heavily from actual JavaScript, while also trying to pretend to be similar?
Jsonnet's stated aim is to be a superset of JSON rather than something similar to JavaScript.
Having used Jsonnet for a while, it's nice in that it makes it relatively easy to take existing JSON and incrementally turn it into templates. You wind up with the ability to create some nice abstractions for elements of JSON. The close coupling between the syntax of the emitted output and the Jsonnet script itself makes it far easier to write a correct template than when using something like gotmpl to create Yaml in a Helm chart.
Whether or not this is worth another language is a judgement call, but it's not that hard to learn or work with, so I've tended to find Jsonnet a nice tool to have around.
Used it previously to template some Terraform module configurations. The way the flow would work would be YAML config -> Jsonnet + defaults -> Terraform + Spinnaker
We wanted to avoid having to inject a more complex piece to the puzzle. It worked okay at first, but I knew from the get-go that wasn't going to hold for very long and eventually there was a Python app that took the YAML config and the Jsonnet+defaults and mashed them together before sending the info to TF + Spinnaker.
jsonnet has lazy evaluation. This allows you to create mutual references between different parts of your configuration without having to manually topologically sort the evaluation order yourself.
kubecfg does add some features, like https:// imports, oci:// imports (oci bundles in OCI registries, transitively bundling all imported files with jsonnet-deps).
But yes, I strive to keep the "one file, one target, import whatever you need but explicitly" as much as possible.
I'm pouring some more time into the project and trying to implement some ideas I had for a long time but never managed to get them out. For example "Flags From Files" (https://github.com/kubecfg/kubecfg/blob/flagspec/docs/rfcs/r...) or "Caching + optional vendoring of immutable external deps".
Yes the go version takes way longer materialize our configs compared at my work compared to sJsonnet (from Databricks). The first time you switch it adds (or removes? I forget) a bunch of newlines at the end of files which is annoying to handle in git but otherwise it’s faster and the same.
The discoverability of golang templates is terrible, IMHO, since it's missing a "dir(locals())" equivalent and every execution environment gets to make its own rules about what pipelines/functions are exposed
Separately, I dooooo nooooooot understand why every joker has to invent their own new thing when we have like 50 or so templating languages already. Golang may be an outlier in that competition due to the Google Promotion Packet Effect(tm) but how they came up with `{{ range }}{{ end }}` as sane syntax is some true facepalm, to say nothing of the same landmine that ansible stepped on by not switching jinja2's default characters: `{{` is not _yaml safe_
Helm does a poor job of documenting their functions, and when you use helm, you're not really using go templates, you're using helm's wrapper around go templates.
If you use go templates directly- it's very obvious what functions are available, there's the default ones from the Go template package, and there's whatever you pass explicitly. The go template package is very small and well documented.
What I love about Go templates, is that I can pass any function I want, using good old Go code (meaning I can use SDKs and whatever else). This is really powerful- we're able to have functions that go out to APIs and return data and it's very easy to manage
> helm, you're not really using go templates, you're using helm's wrapper around go templates.
Well, that's a tautology when having any contact with golang templates outside of quite literally just importing "text/template" and good luck, as your subsequent comment somehow says and yet misses the point. I draw one's attention to the sibling comment of yet another golang templating gizmo that injects its own cutesy functions with completely random naming into the evaluation namespace
But if I was doing it again now, I would just use https://cuelang.org/ or https://dagger.io Jsonnet is really hard to debug.