On: Create Terraform files using Python scripts

[busterarm]

You can just write information out to files in Terraform with no stress.

In terraform resources this is what that looks like:

    resource aws_iam_user email {
      name = "email"
    }

    resource aws_iam_access_key email {
      user = aws_iam_user.email.name
    }

    resource local_file smtp_password {
      content = aws_iam_access_key.email.ses_smtp_password_v4
      filename = "SES_PASSWORD.secret"
    }

So what's the plumbing that you would have to do? Under the hood, Pulumi is using the Terraform providers...

(I left out username, because I don't see where you're setting emailUsername)

In my pipelines I don't bother writing out to files things that are in terraform state. I just create an output for that state (potentially set to sensitive) and then use that output in my CI/CD. Remote state stays encrypted and without wide access and I don't have to worry about secrets being in files anywhere.

That's where the bash scripts do things with outputs. It could by python or whatever, it doesn't matter really. But with bash I can easily just set variables to `terraform output -json | jq <select output &/|| do stuff>`.

Mainly all I do is write terraform outputs to vault (i have simple bash automation to do all of this) and then I can use the Vault secrets in other CI/CD pipelines.

[hardwaresofton → busterarm]

> You can just write information out to files in Terraform with no stress.

This wasn't the point -- it was that I wanted to do something that I know how to do in a fully featured programming language, and I can "just do it". Writing local files is a very simple example -- unless you're arguing that terraform's capabilities amount to an entire language's ecosystem, this is just the tip of the iceberg.

> So what's the plumbing that you would have to do? Under the hood, Pulumi is using the Terraform providers...

I think I didn't explain it well enough. Pulumi and Terraform are almost the same tool, but the difference is how the pieces are plumbed together. I prefer plumbing with a programming language more than shell scripts, utilities and/or some other programming languages.

Also, Pulumi's system of custom resources which are just pieces of code sitting in your codebase is fantastic and novel (terraform has custom providers but this feels significantly more heavy weight).

> In my pipelines I don't bother writing out to files things that are in terraform state. I just create an output for that state (potentially set to sensitive) and then use that output in my CI/CD. Remote state stays encrypted and without wide access and I don't have to worry about secrets being in files anywhere.

If you're really adjusted to Terraform, and it works great for you, then awesome -- I'm not out to change how you do things. It sounds like you've fully bought in to the terraform way of doing things, and it's working for you, and that's great.

> That's where the bash scripts do things with outputs. It could by python or whatever, it doesn't matter really. But with bash I can easily just set variables to `terraform output -json | jq <select output &/|| do stuff>`.

Here it is again... My point is that the plumbing matters, and a fully baked programming language offers the possibility of better plumbing. I didn't touch on it much, but just having access to custom resources with Pulumi might be able to cut down the external plumbing to zero, and enable creating more reusable pieces.

[busterarm → hardwaresofton]

> Also, Pulumi's system of custom resources which are just pieces of code sitting in your codebase is fantastic and novel (terraform has custom providers but this feels significantly more heavy weight).

Pulumi literally uses Terraform's custom providers as its dependencies under the hood to make this work.

Moreover, you're entrusting your entire production stack to an ultra-aggressive, hypergrowth, early stage startup...

[jen20 → busterarm]

Your statement about providers as dependencies is inaccurate in a number of respects:

- Pulumi has the ability to use the CRUD logic from Terraform providers to reify resources, but that is one of a number of different approaches it can use.

The Kubernetes new Azure Resource Manager providers are instead built via the API specs and additional annotations, and involve no aspects of Terraform.

- (Less importantly, but still:) Terraform does not have the notion of “custom” providers as a technical construct - first- and third-party providers implement the same protocol and have the same capabilities. There are just “providers”.

Disclaimer: I have contributed to Pulumi and now use it day to day at a large company. I also was a core maintainer of Terraform at HashiCorp for several years.

[hardwaresofton → busterarm]

> Pulumi literally uses Terraform's custom providers as its dependencies under the hood to make this work.

Right but you can see that the interface is easier as a custom resource though right? The Pulumi docs on making a custom resource are just extending a class. Making a good one I'm sure is fraught with peril but it's so much easier than trying to extend terraform as far as I can see, I'm glad that Pulumi has done this for me.

> Moreover, you're entrusting your entire production stack to an ultra-aggressive, hypergrowth, early stage startup...

Source code is Apache 2.0 and available[0]... I'm not against them trying to profit, but they've created a useful thing that is licensed very permissively (they could have gone with BSL or whatever else)...

[0]: https://github.com/pulumi/pulumi