>If you write your config in a "full-blown" programming language, then your configs are full-blown programs in that programming language.
Yes, which means we have well-documented functionality and tooling of that language to deal with various use cases. Which is not going to be the case with your ad-hoc format based on YAML or JSON.
>Code-as-configuration demands tremendous discipline from the team.
No more discipline than any other form of programming.
>Whereas if you abuse YAML (or JSON or XML or whatever) to create a limited and hard/impossible-to-extend DSL, you still have much more control
And here is the crux of the issue. Tools that are designed so that someone can keep "more control" rather than for tool users to solve real problems. The industry is sliding back towards bad old days of batch processing because of conceit and lack of lateral thinking.
Organizations don't have access to an infinite pool of highly disciplined software engineers, the less discipline or skill required to get something done quickly and safely, the more things they can get done with more people and more kinds of people divided into teams with different responsibilities and different kinds of code.
This is an important point. There was even a discussion here a few months back (I remembered it being more recent, but it was 4 months ago) on an article titled "Discipline Doesn't Scale" [0]. Discipline works up to a point, but the more your system relies on discipline, the more fragile it becomes as you scale (in people, in size of the system). At some point you'll hit a wall where your system is too big or you have too many people and discipline falters as a consequence, or you get slowed down maintaining discipline beyond what's reasonable for your field and customers.
I consider myself a highly disciplined software engineer, and I still want as many guard rails for myself as possible. I am a human, I make mistakes; my schema validator does not make mistakes.
The pool of people who understand any popular scripting language is incomparably larger than the pool of people who understand your clever dialect of YAML, JSON or XML.
With a schema, you have a fully-documented and soundly, statically typed DSL. If it were a Ruby library, you'd have to read the docs anyway, and you also lose static parsing and validation.
The best of both worlds is to use a de facto standardized non-executable format like INI or JSON that nearly every language supports.
Then if you need to, you can create complex or overly long configuration files in Python by inserting keys into a dictionary and dumping to ConfigParser (or however your favourite language does things). For example, its useful when writing a test for many permutations of something similar.
Meanwhile the parsing side is simple enough to be re-implemented in an hour when the time comes to rewrite your whole stack in C+Verilog for real ultimate performance.
The 2 main things are:
1) Using your own bespoke config format or some pet format that's not widely supported adds needless friction to writing little duct tape scripts, testing harnesses, and misc tools. It also adds unnecessary difficulty when porting parts of your program to new languages.
2) Using a Turing complete config format even if it's not bespoke makes all the drawbacks in (1) even more apparent.
Yes, which means we have well-documented functionality and tooling of that language to deal with various use cases. Which is not going to be the case with your ad-hoc format based on YAML or JSON.
Really? Unless it's written in Haskell or something else with a very strong type system, you won't do better than JSONSchema for validating the config file.
And here is the crux of the issue. Tools that are designed so that someone can keep "more control" rather than for tool users to solve real problems. The industry is sliding back towards bad old days of batch processing because of conceit and lack of lateral thinking.
Too much freedom is a bad thing. The industry is not "sliding" anywhere. We tried code-as-configuration, it required too much discipline, so the pendulum is swinging back. As pointed out elsewhere, hopefully Dhall will save us from all this by being the happy balance between expressive and chaos-limiting.
Yes, which means we have well-documented functionality and tooling of that language to deal with various use cases. Which is not going to be the case with your ad-hoc format based on YAML or JSON.
>Code-as-configuration demands tremendous discipline from the team.
No more discipline than any other form of programming.
>Whereas if you abuse YAML (or JSON or XML or whatever) to create a limited and hard/impossible-to-extend DSL, you still have much more control
And here is the crux of the issue. Tools that are designed so that someone can keep "more control" rather than for tool users to solve real problems. The industry is sliding back towards bad old days of batch processing because of conceit and lack of lateral thinking.