We run our Terraform pipeline via Azure DevOps. We separate the plan/apply steps, stash both the binary and json version of the plan, stash the logs of the plan, and require increasing levels of approval as it moves through each environment on it's way from dev to prod. To go to prod it needs 4 people to have reviewed the plan - the person whose PR triggered the release, a director, a team lead, and another engineer. This is enforced using Azure DevOps' baked in pipelines and approval functionalities. To keep from nuking things we completely disable destructive operations from our AWS/Azure Terraform user and in fact, limit what the user can do as much as possible, only giving it the permissions needed to create what we use, and adding as necessary. For deletes we add a very targeted permission that we revoke after deployment.
It can seem onerous at times, but ultimately it's a far smoother process than legacy manual or scripted deployments and is very repeatable and visible. It's also a good thing to have lots of eyes on critical infrastructure changes anyway.
It can seem onerous at times, but ultimately it's a far smoother process than legacy manual or scripted deployments and is very repeatable and visible. It's also a good thing to have lots of eyes on critical infrastructure changes anyway.