I'd say all of these problems have answers in CUE.
> I want to be able to write a "generic environment" module for each application and then parameterize it accordingly for each environment.
This is pretty solidly in the target use case range, i'd say - managing variations of the same "object type" over some dimension is a lot of what's targeted by the way that CUE treats directory hierarchies when loading files: https://cuelang.org/docs/concepts/packages/#instances
The main thing you have to consider in designing a layout is that you have to take a compositional approach to how you define individual config instances. That is, you can't start from prod's config, then override a value or two for staging.
If i were to do it - i have not, this is not how i currently use CUE - my first approach would probably be by defining defaults at the "policy" level (per the above link), which effectively allows you to get exactly one "override"-ish behavior.
Lots of possible approaches to this, though.
> but you can't quite emulate functions as far as I can tell
How would you solve this with directory structure and "function structs" respectively? I'm having trouble wrapping my head around the former and ran into shadowing problems with the latter.
I think parent means a General Purpose language, i.e. capable of computations.
Personally, on one hand I know allowing computations into configuration immediatly destroys any hope of having a tidy, rational schema in real word projects.
On the other hand though, i do believe configuration and code should be build with related tools, possibly the same tool- or at least tools using the same syntax!
(a bit like the json syntax is the same as a Python dict syntax, except this is the terrible example that is so poorly thought out that does more harm than good)
This unlocks a much greater degree of freedom and power than all the gluing together technologies that we have to do...
If this is the sense in which the comment is intended - CUE is capable of some kinds of computation. It's just not Turing complete.
My 2c - thus far, i've found the language features enabled by this constraint much more useful than the expressiveness lost - at least, for the purposes i've chosen.
Very excited to see someone doing this! Right now, Grafana is [planned to] relying on an anemic CUE->Typescript translator for getting its schema to the frontend - https://github.com/sdboyer/cuetsy. (Somebody also pointed me to Project Cambria recently, which could be an interesting compilation target for what we have https://www.inkandswitch.com/cambria.html)
Being able to work with CUE natively in TS, though, would be a huge gamechanger for what we can do with CUE in Grafana
Our implementation is in TS. However that's just an API. Do you have some ideas on how you'd like the TS type system to work with the Cue type system. It's an open question for us.
CUE is currently centered around its CLI, but AIUI, that's not the long-term goal. i read in some CUE issue somewhere that the goal is shifting towards enabling frameworks rather than driving people to the CLI, though i don't have a link.
Our use of CUE in Grafana is an example of framework-style usage. It is a hard requirement that users never have to install the CUE CLI to perform any of our planned CUE-related tasks; rather, the needed tooling is baked into Go packages we export, and things like grafana-cli. (Avoiding a dependency on the CUE CLI also gives us a defense mechanism against breaking changes)
re: Grafana (i'm the author of the linked issue) - i'm quite excited, i do think there's a world of possibilities here.
Two-way sync with a git repo is one possible path, and we've talked a lot internally at GL about how to best support it. My sense is that we can do it with relatively little friction and likely will - but if you're just syncing with a git repo, there's still a lot of arbitrary, opaque repo layout decisions that still have to be made (how do you map a filesystem position for a dashboard to a position in Grafana? In a way that places the dashboards next to the systems they're intended to observe? With many teams? With many Grafana instances?) which induce new kinds of friction at scale.
Fortunately - and not mutually exclusively with the above - by building the system for schema in CUE, we've made a composable thing that we can make into larger systems. That's what we're starting to do with Polly: https://github.com/pollypkg/polly
This seems really exciting. I haven't had the chance to use Grafana yet; from the linked issue, am I understanding correctly that you'll be able to serialize dashboards to Cue schema, and hence get all the niceties of a structured representation - versioning, non-visual editing, and reproducibility?
I recall seeing another project HN which created dashboards out of a yaml description. This seems like a fantastic idea, given that a lot of business panels and dashboard apps can be implemented with a limited set of UI interactions.
Yep, this was the one! It looks great though I hadn't got around to trying it.
Do you think something like Cue makes for a better representation for lowdefy apps than YAML, since it seems to offer better abstraction ability and hence easier to compose?
It's an interesting thought. I'll definitely spend some time to consider how this could work.
Although, writing apps in yaml or json works really well currently. We mostly express logic through Lowdefy operators. We like supporting yaml / json since it is easy to write code to create or update such apps in any language.
That said, it's not like we're planning on replacing all the "export JSON" buttons in the Grafana UI with "export CUE." One of the interesting properties of defining schemas in CUE is how it allows us to remove schema-defined default values from a dashboard's JSON. The JSON representation can actually look a lot more like a concise CUE representation.
> versioning
Versioning of the Grafana schema is the essential design goal of the "scuemata" system that is under discussion in that epic issue
Like, editing something other than raw code in an editor? Yes, this is also something directly enabled by the schema (again, see the Polly doc, the "Produce" heading). For data-intensive tasks, such editing experiences are the only way to see your logic in the context of data, and therefore IMO prerequisite for confidence
> reproducibility
Yup. This already isn't "hard" to do today, but reproducibility gets more complicated at scale - those questions about how to map what's on disk to what's in your Grafana (or whatever app) in my parent comment become more complicated, leading to friction, leading to staleness.
When dep removed the vendor folders from the other libraries, How does dep handle multiple libraries that depend on the same library but different versions of it?
sadly, gb is one of the extant systems i haven't had a ton of time to explore. but...
i guess the analogue to what you're describing would be
1. <add an import path>
2. `dep ensure`
3. `git commit -am "all of your vendor'd source`
a bit more detail, starting at a high level: gb is a replacement toolchain. dep is focused strictly on dependency management. there is no notion of a `dep build`.
with dep, there's no explicit command to actually fetch a dep; the import graph is still queen, as is customary in go. so there's no direct analogue to `gb vendor fetch github.com/some/pkg`. if you want to add a dep, import it in your source code, and run `dep ensure`. (there's some flux in exactly how that works right now, but what i'm describing is the state we're moving towards)
`dep ensure` is really the workhorse command, and pretty much the only thing you'll ever need to run. (in fact, the only three subcommands we currently plan on having are `init`, `ensure`, and `status`).
> I want to be able to write a "generic environment" module for each application and then parameterize it accordingly for each environment.
This is pretty solidly in the target use case range, i'd say - managing variations of the same "object type" over some dimension is a lot of what's targeted by the way that CUE treats directory hierarchies when loading files: https://cuelang.org/docs/concepts/packages/#instances
The main thing you have to consider in designing a layout is that you have to take a compositional approach to how you define individual config instances. That is, you can't start from prod's config, then override a value or two for staging.
If i were to do it - i have not, this is not how i currently use CUE - my first approach would probably be by defining defaults at the "policy" level (per the above link), which effectively allows you to get exactly one "override"-ish behavior.
Lots of possible approaches to this, though.
> but you can't quite emulate functions as far as I can tell
Function-like capability is present, just in a form that's less familiar. I think of them as "function structs." This post has a bunch of examples https://github.com/cuelang/cue/issues/139#issuecomment-55677.... It seems there's a plan to add a more comfortable notation (https://github.com/cuelang/cue/issues/943), but it's fundamentally possible now.