Handling infrastructure on multi-environment cloud platforms can be a difficult task. In this article I will show how to use Gitlab CI/CD pipelines and Terraform Cloud to provision infrastructure to multiple AWS accounts. 

CI/CD stands for continuous integration and continuous deployment/delivery. Gitlab pipelines allows us to create CI/CD pipelines for task such as code deployment and provisioning infrastructure. Our use case for this article is that we have two different Terraform Cloud workspaces, one development environment and one production environment. We want to deploy these Terraform Cloud workspaces whenever we update our source code in Gitlab, using Gitlab CI/CD tools. We'll use the same name for the workspaces, except they will have a different ending, based on which environment we are using. E.g. my-new-workspace-dev and my-new-workspace-prod. 

I assume you have Terraform Cloud up and running, if not, Terraform Cloud shows how this can be set up.

Different Terraform Cloud workflows

Terraform Cloud offers three different workflows for managing Terraform runs.

• CLI-driven workflow
• VCS-driven workflow
• API-driven workflow

We will be using the CLI-driven workflow in this article. The CLI-driven workflow allows you to use the standard Terraform CLI commands to execute code in Terraform Cloud. When using VCS-driven workflow, Terraform Cloud registers webhooks directly to your VCS repository. The API-driven workflow is a bit more complicated to set up, but allows for greater flexibility when executing your Terraform code. You can read more about the different Terraform Cloud workflows here, Terraform Cloud Workflows. Normally, when we execute Terraform CLI commands from our terminal to Terraform Cloud, a login is required to authenticate with Terraform Cloud. We cannot do this using Gitlab CI/CD, but there is a work-around solution for this. We need to create an API Token so that we can access our Terraform Cloud from Gitlab CI/CD. We can create an API token by going to Terraform Cloud, User settings -> Tokens and press Create an API token. We'll use this later in our config.tf file.

Gitlab CI/CD

In Gitlab, we use runners that acts as agents for our CI/CD jobs. In this example, we'll not create our own runners, but instead we'll take use of Gitlab's shared runners. By default, shared runners are enabled on Gitlab repositories. If not, you can turn them on by going to Settings -> CI/CD, expand Runners and switch it on. You can read more about Gitlab Runners at Gitlab Runner Docs. Now, we want to set up our CI/CD. First we'll create a YAML file called, .gitlab-ci.yml, and add it to our Gitlab repository source code. With this file we can define scripts that deploy our code. To understand the structure of Gitlab CI/CD, we can look at our pipelines as the top-level component. Within each pipeline exists Stages and Jobs. Jobs define what to do, e.g. run Terraform commands, and stages define when we want to run each job, e.g. in which order the jobs are run. Add the following to our newly created file.

stages:
  - validate
  - plan
  - apply
  - destroy

validate:
  stage: validate
  script:
    - echo "### Validating terraform code..."
    - terraform validate
  when: always

plan:
  stage: plan
  script:
    - echo "### Running terraform plan..."
    - terraform plan
  dependencies:
    - validate
  when: manual

apply:
  stage: apply
  script:
    - echo "### Applying terraform code"
    - terraform apply -auto-approve
  dependencies:
    - plan
  when: manual
 
destroy:
  stage: destroy
  script:
    - echo "### Destroying infrastructure..."
    - terraform destroy -auto-approve
  dependencies:
    - apply
  rules:
    - if: '$ENV == "prod"'
      when: never
    - if: '$ENV == "dev"'
      when: manual

Here we first define the order of our jobs as stages, and then we define what each job should do. We use three commands during the Terraform life cycle, i.e. plan, apply and destroy. In our gitlab-ci.yaml we create three jobs that will perform each of these steps respectively. We have also added a validate job, to validate the Terraform configuration files. Each job consists of the following keywords.

• Stage relates each job to the corresponding stage.
• Script is where we write the commands we want to execute during the job.
• Dependencies tells us that the job will only run if the given job has already run. E.g. apply should only be run if plan has run.
• When defines when the job should be run. Here we have multiple different options to choose from. E.g. manual means job only runs if we've hit 'play' manually from the Gitlab Pipeline UI. Always means the job should run regardless, never tells the job that it should never run. You can read more about the different available option in the Gitlab docs. Here, I would recommend to use manual for any environment except maybe your dev environment.
• Rules are used to exclude or include jobs based on our condition. I will explain how to use rules later in this article.
• If we want our jobs to only run for specific branches, we could add the only keyword to our jobs, e.g. only: master.
  
  

Next, we need to add a docker image and environment variables. We'll add the following to our .gitlab-ci.yml file, right after we define our stages. 

image:
  name: hashicorp/terraform:0.15.0
  entrypoint:
    - '/usr/bin/env'
    - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

variables:
  ENV: "dev"

In order for us to run Terraform CLI commands, we use a Docker image that contains all the dependencies we need for Terraform CLI to work. Here we've also implemented variables, and created a variable called "ENV" and set the default value to "dev".This variable tells our Terraform configuration which Terraform Cloud workspace we should update. When we run the pipeline later, we can define our variables, meaning we can change the ENV variable when we run the pipeline. This makes it possible for us to use the same .gitlab-ci.yml file for different environments. The following shows exactly how this works and should be added above the jobs in our .gitlab-ci.yml file.

before_script:
  - apk update
  - apk add bash
  - rm -rf .terraform
  - |
    cat > config.tf <<EOF
    terraform {
      backend "remote" {
        organization = "<your-organization>"

        workspaces {
          name = "<your-workspace>-$ENV"
        }
        token = "$TERRAFORM_LOGIN_TOKEN"
      }
    }
    EOF
  - cat config.tf
  - terraform --version
  - terraform init -upgrade

Here we add a before_script to our CI/CD file. This defines what commands we want to run, prior to each job. Since we want to dynamically change which environment we execute our Terraform to, we have to update our config.tf file. The config.tf is what tells our Terraform code where it should execute. We therefore use string interpolation to update our workspace name with the chosen value for our ENV variable. We also run the following commands.

• cat config.tf, displays the state of the file when we run the pipeline. 
• terraform --version, displays the version of Terraform we're using. 
• terraform init -upgrade is executed to upgrade modules and plugins that we use. 

We have also the token argument. This is what will authenticate with Terraform Cloud. The reason we don't define the value of our TERRAFORM_LOGIN_TOKEN variable here, is because we don't want to do this in this file. Instead, we'll add this to our project's CI/CD settings variables. We can do this by going to our project’s Settings > CI/CD and expand the Variables section. Here we can add our TERRAFORM_LOGIN_TOKEN variable and it will be accessible within our .gitlab-ci.yml file. 

Run our pipeline

Now, we should have a Gitlab CI/CD configuration set up for us to deploy our Terraform code. Whenever we update our source code in our Gitlab repository, our pipeline should start. Let's try to manually start it. We'll go to our Gitlab repository and in the left hand menu, go to CI/CD -> Pipelines -> Run pipeline. Before we start our pipeline, we can change our ENV variable if we want to. By default we set the value to "dev".

Now we can manually hit 'play' on each job. Keep in mind that pipelines are supposed to be run from beginning to end. This means that that we should run all the jobs from start to end.

Conclusion

In this article I have showed how to use the CLI workflow of Terraform Cloud, and Gitlab CI/CD pipelines to provision infrastructure to multiple accounts in AWS, one for each environment. By setting the variable ENV in Gitlab we can easily switch target environment in our CI/CD job, thus utilizing the same pipeline for multiple environments.

I have showed examples of how a gitlab-ci.yaml file can look like, and explained some of the keywords used to configure jobs.