Terraform Backends

The following backends are supported directly in the plugin

Any other backend, can be supported by setting the appropriate tokens on GenericBackend.

Configuring the local backend

Most people using Terraform local state would probably just store terraform.tfstate in the same directory as the source and commit that to source control. In some cases this might not be what you want. For example you might want to apply terraform to a LocalStack instance and have no need for keeping the state. The state path can be configured.

import org.ysb33r.gradle.terraform.backends.LocalBackend

terraform {
  backends {
    local(LocalBackend) {
        path = 'my/other/path' (1)
    }
  }
  sourceSets {
    main {
      usedBackend('local')
    }
  }
}

1	Set a different path where to store local state.

main.tf

terraform {
  backend "local" {}
}

Configuring the S3 backend

import org.ysb33r.gradle.terraform.backends.S3Backend

terraform {
  backends {
    s3(S3Backend) {
      acl = 'private' (1)
      dynamoDbTable = 'arn:aws:dynamodb:us-east-1:123456789012:table/terraform-state-lock' (2)
      encrypt = true (3)
      forcePathStyle = true (4)
      kmsKeyId = '1234abcd-12ab-34cd-56ef-1234567890ab' (5)
      maxRetries = 20 (6)
      skipCredentialsValidation = false (7)
      skipMetadataApiCheck = false (8)
      skipRegionValidation = false (9)
      sseCustomerKey = '4xF8vMK8QtTn+6tgEoMdyf8xOYXzYqrE9mJVL3zFm3A=' (10)
      workspaceKeyPrefix = 'env:'  (11)

      s3BucketName = 'foo' (12)
      bucket = 'foo' (13)

      awsRegion = 'us-east-1' (14)
      region = 'use-east-1' (15)

      remoteStateName = 'my-source-set.tfState' (16)
      key = 'my-source-set.tfState' (17)

      dynamoDbEndpoint = 'http://127.0.0.1:4566' (18)
      iamEndpoint = 'http://127.0.0.1:4566' (19)
      s3Endpoint = 'http://127.0.0.1:4566' (20)
      ssoEndpoint = 'http://127.0.0.1:4566' (21)
      stsEndpoint = 'http://127.0.0.1:4566' (22)
    }
  }

  sourceSets {
    main {
      useBackend('s3')
    }
  }
}

1	Canned S3 ACL
2	Full ARN to DynamoDB lock table
3	Enable server-side encryption of the state file.
4	Force the URL to be `https://<HOST>/<BUCKET>;`.
5	KMS Key id.
6	Maximum number of retries before giving up.
7	Whether to skip validation of credentials.
8	Whether to skip the metadata API check.
9	Whether to skip region validation. Useful for Localstack.
10	SSE customer key.
11	Prefix to use for workspaces.
12	Name of the S3 bucket.
13	Short version of `s3BucketName`.
14	Region where bucket, DynamoDB is located in.
15	Alias for `awsRegion`.
16	Name of the file where the state will be stored in. An alias for `key`, but more descriptive.
17	Name of the file where the state will be stored in.
18	DynamoDB override. Useful for Localstack.
19	IAM endpoint override. Useful for Localstack.
20	Override the S3 endpoint. Useful when working with Localstack.
21	SSO endpoint override. Useful for Localstack.
22	STS endpoint override. Useful for Localstack.

main.tf

terraform {
  backend "s3" {}
}

Authentication can be set in two ways. If you only use S3 for state storage, and not anything else, set the authentication on the backend directly. If you need to have the same credentials avaiable to both the bucket and the actual source set use AwsSecrets instead.

Setting credentials on the backend

import org.ysb33r.gradle.terraform.backends.S3Backend

terraform {
  backends {
    s3(S3Backend) {
      accessKey = grolifantOps.resolveProperty('acme.aws.access.key') (1)
      accessKeyProperty = 'acme.aws.access.key' (2)
      secretKey = grolifantOps.resolveProperty('acme.aws.secret.key') (3)
      secretKeyProperty = 'acme.aws.secret.key' (4)
      profile = grolifantOps.resolveProperty('acme.aws.profile') (5)
      credentialsFile = '/path/to/aws/credentials' (6)
    }
  }
}

1	The AWS access key. Needs to be paired with `secretKey`.
2	A shorter way of setting the AWS access key via a property or environment. Looks for Gradle property, then system property, and finally an environment variable in the form `ACME_AWS_ACCESS_KEY`.
3	The AWS secret key. Needs to be paired with `accessKey`.
4	A shorter way of setting the AWS secret key via a property or environment. Looks for Gradle property, then system property, and finally an environment variable in the form `ACME_AWS_SECRET_KEY`.
5	AWS profile instead of access and secret key pairing
6	Location of the AWS credentials file.

Setting credentials on the backend

import org.ysb33r.gradle.terraform.backends.S3Backend
import org.ysb33r.gradle.iac.base.secrets.AwsSecrets

terraform {
  secrets {
    aws(AwsSecrets) { (1)
    }
  }
  backends {
    s3(S3Backend) {
      fromSecretsProvider(terraform.secrets.aws.secretVariables) (2)
    }
  }
}

1	See AWS Secrets for defining access keys, profiles etc.
2	Inherit secrets.

Configuring a generic backend

If you are using a backend that is not directly supported by this plugin suite, you can still use it by some manipulation of the local backend.

Let’s assume you want tho use the Consul backend and, it is not supported.

import org.ysb33r.gradle.terraform.backends.GenericBackend

terraform {
  backends {
    consul(GenericBackend) {
      tokens address : "consul.example.com", (1)
            scheme : "https",
            path : "full/path"
    }
  }
  sourceSets {
    main {
      usedBackend('consul')
    }
  }
}

1	Define the attributes that you need.

main.tf

terraform {
  backend "consul" {}
}

Configuring the legacy Terraform Cloud backend

This is for the legacy remote backend.

import org.ysb33r.gradle.terraform.backends.TerraformRemoteBackend

terraform {
  backends {
    tfRemote(TerraformRemoteBackend) {
        organization = 'company'
        authToken = grolifantOps.providerTools.resolveProperty('my.tf.cloud.token')
    }
  }
  sourceSets {
    main {
      useBackend('tfRemote')
    }
  }
}

main.tf

terraform {
  backend "remote" {}
}

Configuring the Gitlab backend

It is possible to store remote state in Gitlab and it utilises the http backend.

The Gitlab backend DOES NOT support workspaces. If you need something like workspaces, but want to use Gitlab tfstate storage, consider using multiple source sets and use local modules to share common IAC. Then configure a Gitlab backend for each source set and change the address.

build.gradle

import org.ysb33r.gradle.terraform.backends.GitlabBackend

terraform {
  backends {
    gitlab(GitlabBackend) {
        address = 'https://gitlab.com/api/v4/projects/123456/terraform/state/my-project' (1)
        username = grolifantOps.providerTools.resolveProperty('my.gitlab.username')  (2)
        accessToken = grolifantOps.providerTools.resolveProperty('my.gitlab.access.token') (3)

        retryMax = 2 (4)
        retryWaitMin= 1 (5)
        retryWaitMax = 30 (6)
    }
  }
  sourceSets {
    main {
      usedBackend('gitlab')
    }
  }
}

1	Set the location of your Gitlab endpoint. It takes the form `https://gitlab.com/api/v4/projects/<PROJECT_ID>/terraform/state/<STATE-NAME>;`, where `PROJECT_ID` is the project state will be stored in, and `STATE-NAME` is a name for the state.
2	Provide the Gitlab user name. This user needs to have at least a Gitlab `Maintainer` role to perform updates. This will be injected into the environment as `TF_USERNAME`
3	Provide the Gitlab access token. This will be injected into the environment as `TF_PASSWORD`
4	The number of HTTP request retries. Defaults to 2.
5	The minimum time in seconds to wait between HTTP request attempts. Defaults to 1.
6	The maximum time in seconds to wait between HTTP request attempts. Defaults to 30.

main.tf

terraform {
  backend "http" {}
}

Adding a new backend programmatically

See contributing/add-tofu-or-tf-backend.adoc if you would like to add a new backend.