No questions, just a suggestion: implement the part where the terraform plan is added as a comment in the PR. We set this up at my current employer and it makes the review process much quicker (also, commenting on lines in the ~plan~ terraform code changes is the bee's knees).
Don't have the apply be automatic after a review is approved; terraform apply's occasionally go sideways and need human intervention (remember: rollbacks are not automatic). A human should always kick off the apply and monitor state change activity.
Woah can you comment on lines in a comment? /me dashes to investigate github
And thanks for the suggestion. So far it's been on a someday maybe list, but if it really does help that much, maybe we'll bump it to someday maybe soon.
Great post. I think the area where things start to go off the rails with Terraform is variables/modules though. There are edge cases and gotchas with modules and not a lot of best practice guides.
Shameless plug, if anybody is looking for a guide on getting started with Terraform and Google Cloud Platform, I wrote a short blog post:
Shameless plug, if anybody wants to generate a Terraform module with a testing harness preconfigured then I hacked up a module scaffolder last night/this morning: https://github.com/howdoicomputer/terrashovel
Great post! I too had the issue with AWS volume attachments and the need to separate them out. Luckily it happened early during some of my first provisioning and we realized it before standing up more databases. I now have a policy where each server has a separate EBS volume, using a volume attachment Terraform resource, for any data. Inline EBS volumes are just used for the OS.
I've run into another similar type issue now and I think that I'm going to have to do state surgery like you. I want to refactor some of my terraform resources into modules, but this changes the resource names. I can't see any way to do it without either standing up new infrastructure or modifying the state directly.
We ran into similar issues. From there I went one step further and wrote a tool to do the attachments for me (instead of using Terraform's aws_volume_attachment):
We're using Terraform for our AWS env and while it worked great for getting stuff out there, we're now scrambling to figure out how to get Terraform to do non-destructive updates to the environment. For instance, I need to update our base OS for the application servers we're running. How do I do this without incurring a downtime? Or I need to now interject a new reverse proxy between our ELB's and app servers, but want to do it without causing an interruption. We're doing our research and this blog post is very helpful, but if you have any pointers...
You pretty much have to take the same approach as you would outside of terraform: create the new thing (launch configuration, standalone instance) and attach it to the ELB before spinning down the old one. Terraform does have some rudimentary aid to that end in the form of the create_before_destroy[0] flag, though it doesn't work out with uniquely named things.
You should use 2 ASG ( blue / green ) and 1 ELB for that, then when you need to update the OS of one of the ASG you just update the launch configuration. It's easy to do in Terraform.
1) Update the launch configuration of your un-used ASG ( with the new AMI ID )
2) Apply terraform to deploy the new ASG
3) Make sure it's working ( your local app on your new instances )
4) Connect the ELB to your new ASG
5) Set the old ASG to 0 instances to drain the connections
This is a pretty common question. Check out this blog post for how to get Terraform to bring up a new ASG with a new AMI before removing the old one from an ELB:
Is it possible to query information directly from a state file? Some interpolation would help. I ask this because I often refer to resource information from another project, and I don't see that modules are helpful.
I posted my work-around here [1] some days ago . The real code was not there because I haven't had permission for my company, but it's very short and easy to write by anyone.
Terraform v0.9 only handles remote state and locking properly. Handling multiple applies of state is something scheduled for 0.10
Basically terragrunt handles some stuff today in a less robust, less portable, way that your team is likely to need today -- whilst waiting for core terraform support.
(disclaimer: I follow terragrunt and terraform very closely but do not actually use terragrunt in producution anymore).
