Templating GCP Infrastructure With Terraform Modules

Now, watch out. Here comes Genesis. We’ll do it for you in six minutes!

As I mentioned in my Cloud Resume Challenge post about IaC, one of the great things about Terraform code is its reusability. There are countless ways to architect and configure your cloud environment but chances are that you’ll spend a lot of time reusing the same essential building blocks. By separating your Terraform files out into parameterised and semantically organised modules, you can build a repository of these blocks, that need minimal configuration in order to be repurposed and reused.

Here, I’m going to build a modular Terraform config to provision everything necessary for a single Compute Engine VM behind an HTTP(S) External Load Balancer. I’m also going to create a new network & subnet, along with the necessary firewall rules. I’ll set up the config to be as generic as possible, and configurable via a single .tfvars file. In doing this, I should end up with a Terraform set up for an extremely common use case, as well as individual modules that I can use elsewhere.

Terraform will read any .tf file from the directory in which it’s called but will ignore any subdirectories. For this reason, it’s possible to split resources (or groups of related resources) out into separate folders, and only call the ones you need:

As you can see above, I’ve created a directory structure in which each group of resources has its own folder, each of which has a main.tf (to declare the resources themselves), a variables.tf file to define any variables which need to be set and passed to the modules, and an outputs.tf file to define any outputs from each module that will need to be passed to other modules (which I’ll cover later.) In addition, I have equivalent files in the root of my directory (which will be used to call the other modules) and a .tfvars file in which the specific configurations for each environment can be set.

For example, here is the content of ./tf-modules/gcp-wordpress-vm/main.tf:

Here, I’m following the usual process to provision the resources but I’m making extensive use of variables (of the format var.machine_type etc) to parameterise things. Obviously it would be possible to parameterise everything here (OS image, boot disk size etc) but I’ve left some things as they would be for common use cases. For each module folder, it’s necessary to declare these input variables, which I’ve done in the separate ./tf-modules/gcp-wordpress-vm/variables.tf file:

It’s only necessary to declare the variables here (just ‘variable "machine_type" {}‘ would be fine) but I’m trying to get into the habit of being verbose with these, to make them easier to reuse!

Finally, in each module folder, it’s necessary to output any values that need to be referred to in other modules. Within a single file in Terraform, you can simply refer to resources by name when creating other resources (e.g. google_compute_instance_group.webserver_group.id.) However, in the modular structure, modules can’t “see” other modules, so it’s necessary to output such values, so that they can be accessed later (as I’ll demonstrate shortly), like so:

As I said, Terraform only reads .tf files from the directory in which it’s called, so it’s necessary to refer to each of these distinct modules in a single root module, in order to plan and provision your resources. Here’s the main.tf file at the root of my directory structure:

You’ll see that each module is separately defined, and its location in the file structure referred to with the ‘source’ variable. Here, we’re also pointing to the variables that need to provided to each module, which are defined in ./variables.tf but actually set in another file called (in this case) ./example.tfvars. In this way, this configuration of resources can be applied in multiple environments, and only the ./example.tfvars file needs to be modified. Here’s an example configuration:

The final thing to note is that some of the variables in main.tf (the backend_group for the load balancer module, for example) are set to refer to the outputs of other modules. In cases such as the creation of a backend service, it’s necessary to supply the name of the instance group:

By setting this with var.backend, then setting that variable to module.gcp-https-server-vm.instance_group_id in the root (./main.tf) module, the value output from the module.gcp-https-server-vm can be easily passed to the gcp-https-load-balancer module.

In a new repo, it’s then only necessary to run:

$ terraform init

…to initialise all the modules. you can then run:

$ terraform apply -var-file=example.tfvars

…to provision everything in GCP.

If the above sounds useful, please feel free to checkout the full config on GitHub, and please give me a shout if you have any questions or suggestions for improvements.