> Once you start writing complex scripts you have a choice -- you either do it imperatively, or declaratively.
I don't think declarative programming exists. I think it's just a regular old program with a poorly defined interface. Moreover, I think the claims of idempotence are overblown to the point of near falsehood.
Declarative Infrastructure is really just Configuration Management applied to cloud infrastructure rather than operating system software. Neither have really solved anything, other than turning the management of complexity into a Sisyphean task. Forever pushing drifting state back up the hill.
Compare this to Immutable Infrastructure, where state never drifts. One never "fixes" a container once deployed, or a package once built and installed. One merely rolls back or upgrades. Any uncertainty is resolved in the build and test process, and in providing both all the dependencies and the execution environment.
I think eventually people will wise up to the fact that Terraform is just puppet for infrastructure. I think the real fix is to make the infrastructure less like an operating system and more like versioned packages. Install everything all at once. If anything changes, reinstall everything. Never allow state change.
SQL is due for replacement. The combination of schema and data in one constantly mutating hodge podge with no atomic immutable versioning or rollback is absolutely ancient. Migrations are an okay hack but definitely not good enough.
ZFS and LVM prove filesystems can do snapshots and restores on a version of filesystem history without a lot of pain, so clearly we just need more work here to make it an everyday thing. Versioning should be the default and probably also an infinite transaction log, seeing as capacity and performance is ridiculous now.
And couldn't we lock writes, revert a perpetual write journal/transaction log to some previous version, and then continue a new write history tree? If you run out of space, overwrite the old history. If you don't run out of space, allow reverting back.
And allow bulk atomic updates by specifying a method to write files that aren't 'live' until you perform some ioctl, and then atomically expose them and receive the new history version. Then you could do immutable version-controlled storage on a filesystem, right?
Blob/object stores should be much simpler to do the same with. Just an API rather than ioctl.
In this way, replacing a data store immutably will just be replacing a reference to a storage version, the same as using snapshots, but built into the filesystem/API.
Hm, isn’t there still nondeterminism from a path dependency, because reinstalling a datastore that has arbitrary history isn’t exactly equivalent to creating a datastore with none?
> I don't think declarative programming exists. I think it's just a regular old program with a poorly defined interface. Moreover, I think the claims of idempotence are overblown to the point of near falsehood.
I think this really depends on how you define the term "declarative programming" -- pinning down a singular meaning and a singular interpretation is really hard. If we think about it like a spectrum, there's a clear difference between ansible and terraform like there is with python and prolog. That's "declarative" enough for me.
Idempotence is also really tricky and hard -- I'm not surprised most large codebases can't handle it, but getting close is definitely worth something.
> Declarative Infrastructure is really just Configuration Management applied to cloud infrastructure rather than operating system software. Neither have really solved anything, other than turning the management of complexity into a Sisyphean task. Forever pushing drifting state back up the hill.
While I agree on declarative infrastructure being configuration management applied to cloud infra (especially in the literal sense), I would argue that they have solved things. In the 90% case they're just what the doctor ordered when compared to writing every ansible script yourself (or letting someone on ansible universe give it to you) -- and ansible actually supports provisioning! The thing with this declarative infrastructure push is that it's encouraged the companies themselves to maintain providers (with or without the help of zealous open source committers), so now someone else is writing your ansible script and it has a much better chance of staying up to date.
> Compare this to Immutable Infrastructure, where state never drifts. One never "fixes" a container once deployed, or a package once built and installed. One merely rolls back or upgrades. Any uncertainty is resolved in the build and test process, and in providing both all the dependencies and the execution environment.
People are often using these two concepts in tandem -- the benefits of immutable infrastructure are well known, and I'd argue that declarative infrastructure tools make this easier to pull off not harder (again, because you don't have to write/maintain the script that puts your deb/rpm/vm image/whatever on the right cloud-thing).
> I think eventually people will wise up to the fact that Terraform is just puppet for infrastructure. I think the real fix is to make the infrastructure less like an operating system and more like versioned packages. Install everything all at once. If anything changes, reinstall everything. Never allow state change.
Agreed, but I'm not sure this is very practical, and there's a lot of value in going part of the way. There is a lot of complexity hidden in "reinstall everything" and "never allow state change", and getting that going without downtime -- it requires the cooperation of the systems involved most of the time, and you'll never get away from the fact that there is efficiency lost.
But again, we were talking about the scripts you'll have to write -- in a world that is not yet ready for fully immutable infrastructure, it's just a question of how you write the scripts, not whether an option exists that will prevent you from writing them all together (because there isn't, and most things are not fully immutable-ready yet).
> there's a clear difference between ansible and terraform
The only difference I can see is that Terraform attempts more of an estimation of what might happen when you apply. Otherwise they're the same.
Terraform has multiple layers of unnecessary complexity which were added with good intention (the belief that you could "plan" changes before applying them) but don't actually work in practice. Your state file never reflects the actual state, so it's pretty much meaningless. The plan step is (in theory) supposed to tell you what will happen before you hit apply. But actually knowing it beforehand is impossible.
Part of that is the fault of the providers that don't do the same validation as the actual APIs you're calling do. But the other part is the fact that the system is mutable; it's always changing, so you can never know what will happen until you pull the trigger. The only way to say "only apply these changes if they will actually work" is to move the logic into the system, turning them into transactions (ala SQL).
Honestly, the only reason I use Terraform at all is because writing a bunch of scripts is not scalable. With large teams, you have to use some kind of uniform library/tooling to manage changes. Terraform is currently the best "free" option for that, but I don't find Ansible any more or less reliable, it's just more annoying to use. I definitely don't use them for any "declarative" approach they may have. And in fact, for regular app deployments, I actively do not use Terraform/Ansible at all, and instead write deployment scripts that can manage my particular deployment model requirements. I intentionally abandon the "declarative" model because it's so uncertain (and unwieldy).
> The thing with this declarative infrastructure push is that it's encouraged the companies themselves to maintain providers (with or without the help of zealous open source committers), so now someone else is writing your ansible script and it has a much better chance of staying up to date.
I agree with you here, it's very good that companies can invest in supporting a provider so people can benefit from common solutions. I'm not sure that is specific to declarative infrastructure as much as just being more proactive about supporting their users using their services, though. For example, NewRelic didn't have a Terraform provider until one of their customers wrote one, and eventually they took it over. It's still not great (I have to supplement a lot of missing features with custom scripts calling their APIs directly), but it's better than nothing.
I don't think declarative programming exists. I think it's just a regular old program with a poorly defined interface. Moreover, I think the claims of idempotence are overblown to the point of near falsehood.
Declarative Infrastructure is really just Configuration Management applied to cloud infrastructure rather than operating system software. Neither have really solved anything, other than turning the management of complexity into a Sisyphean task. Forever pushing drifting state back up the hill.
Compare this to Immutable Infrastructure, where state never drifts. One never "fixes" a container once deployed, or a package once built and installed. One merely rolls back or upgrades. Any uncertainty is resolved in the build and test process, and in providing both all the dependencies and the execution environment.
I think eventually people will wise up to the fact that Terraform is just puppet for infrastructure. I think the real fix is to make the infrastructure less like an operating system and more like versioned packages. Install everything all at once. If anything changes, reinstall everything. Never allow state change.