Run Terraform In An OSS Environment: No Terraform Enterprise

Picture this: You work in a company that is pro-OSS (open source software) tools; they’ll only accept a paid SaaS tool only and only if it is the last option. This company is on a cloud migration journey (AWS) and they want to adopt IaC from the onset. Terraform is the preferred tool and you’ve been added to the cloud migration team, your first Epic is to do a POC (Proof of Concept) on Terraform — without using Hashicorp’s SaaS offering; Terraform Cloud/Enterprise. What will you do?

Well, first thing you’ll do is start knowledge gathering. Terraform, being a quite opinionated tool, has a lot of ways to achieve this. But then you’ll likely not see the complete implementation of an end to end pipeline that is similar to Terraform Enterprise, sitting on a GitHub account somewhere. But then here’s how I did the POC for a Dev, Staging and Prod environments (three AWS Environment). Like I’ve stated, it is an opinionated tool, my approach is not a global standard.

Pre-requisite:

  • Knowledge of Terraform
  • Experience storing secrets in AWS Secret Manager.
  1. AWS Accounts: Get two other separate AWS Accounts. One will be dedicated for administration duties and the other one will be dedicated to testing modules (alias AWSTerraform and AWSModuleTests). The former will house the buckets that will be destinations for remote state files, it will also store the secrets (AWS Credentials) for other accounts, using AWS Secret Manager. I personally prefer Vault for secret storage but I wasn’t ready to manage vault instances. The other three accounts are going to be called AWSDev, AWSStage, AWSProd.

2. Decide if you’ll use public Terraform modules or write your own modules. It was the latter for me. So you’ll write modules. These modules are tested on AWSModuleTests, which also serves as the environment for testing new provisioned resources on continuous integration pipeline.

This is a simple VPC module with private subnet(s).

main.tf

variable.tf

3. Use the modules: In a multi-tenant environment, Terraform OSS offers workspace features that enables the management of different environments. I didn’t use Workspaces, instead I adopted the folder approach. This approach was such that each environment had its own folder arranged as shown below. Pretty messed up approach, considering that it does not use a VCS multi-branch approach but scratch that, with the right script, it works.

Let’s take a look at what’s inside the vpc folder in dev.

terraform.tf

main.tf

backend.tf

4. Write your deployment script… Depending on the CI/CD tools you use, you can decide to add a test phase to your pipeline or use policy agent tools. Chef Inspec is a good tool, though the learning curve is not as smoothas it looks at first glance. For the POC, test phase was not part of it so I implemented a script using makefile.