OpenTofu Configuration

The opentofu project extension allows for the configuration of toolchains and source sets.

Toolchains

OpenTofu toolchains are configured in the toolchains block. On xref:platform-installation-support.adoc it offers the option of configuring a OpenTofu version, which Gradle will then download, cache and use.

By default, you do not need to configure anything as there will be a toolchain called standard which will have a version of OpenTofu defined. You can override this version.

opentofu {
  toolchains {
    standard {
      executableByVersion('1.8.8') (1)
      executableByPath('/here/is/my/installed/tofu') (2)
      executableBySearchPath('tofu') (3)
    }
  }
}
1 Use a specific version. If it is not available locally, then Gradle will bootstrap and cache it.
2 Use a fixed location of the executable.
3 Look in the system search path for an executable called tofu (or tofu.exe on Windows).

Backends

Backends is where remote state is stored. A configuration needs to have at least one backend defined which are then referenced by name from the source sets.

import org.ysb33r.gradle.opentofu.backends.S3Backend (1)

opentofu {
  backends {
    s3(S3Backend) { (2)
      bucket = 'my-bucket' (3)
    }
  }

  sourceSets {
    main {
      useBackend('s3') (4)
    }
  }
}
1 Import the backend that you need.
2 Name the backend and use the appropriate type that was imported. In this case the backend is named s3
3 Configure the backend appropriately depending on the backend type.
4 Link a source set to a specific backend. The following backend types are supported out of the box:

  • GenericBackend - For supporting any other backend that is not directly supported.

  • LocalBackend - Not really meant for real usage, but still has some benefit.

  • S3Backend - Store state in an S3 bucket.

Source Sets

Source sets are managed from opentofu.sourceSets. When the org.ysb33r.opentofu plugin is applied, it adds a main source set. Associated with this source set is the default folder of src/tf/main and a number of tasks including:

  • tofuApply

  • tofuCleanupWorkspaces

  • tofuDestroy

  • tofuDestroyPlan

  • tofuFmtCheck

  • tofuFmtApply

  • tofuImport

  • tofuInit

  • tofuOutput

  • tofuPlan

  • tofuShowState

  • tofuStateMv (tofu state mv)

  • tofuStatePush (tofu state push)

  • tofuStateRm (tofu state rm)

  • tofuValidate

If additional source sets are needed, they can be added by convention i.e.

opentofu {
    sourceSets {
        s3Buckets (1)
        cloudFront {
            srcDir = 'src/elsewhere/cf' (2)
        }
    }
}
1 Creates an OpenTofu source set named 's3Buckets' with default directory src/tf/s3Buckets
2 Creates an OpenTofu source set named 'cloudFront` and set the directory to src/elsewhere/cf.

Tasks for additional source sets follow the tofo<SourceSetName><OpenTofuCommand> format. For instance in the above example the initialisation task for the s3Buckets source set will be called tofuS3BucketsInit.

By convention all tasks that map OpenTofu commands start with tofu. Other non-commands tasks might start with opentofu or contain OpenTofu within the task name.

Configuring source sets

opentofu {
    sourceSets {
      main {
         srcDir = 'src/tf/main' (1)
         variables { (2)
           var 'aws_region', 'us-east-1'
         }
         secondarySources 'src/tf/modules' (3)

         workspaces 'alpha', 'beta' (4)

         useBackend 's3' (5)
      }
    }
}
1 Source directory, which will also be the working directory for terraform.
2 Configure any variables that are specific to the source set. See the Variables block for more details.
3 Additional sources that should affect re-running of tasks, but which are not directly part of the existing source set.
4 Adds additional workspaces to the source set.
5 Link this source set to a named backend.

Variables

A number of OpenTofu task types support variables and files containing variables. Any variables block support the following functionality

variables {
    var 'foo', 'bar' (1)
    map 'fooMap', foo: 'bar' (2)
    list 'fooList', 'foo', 'bar' (3)
    list 'fooList', [ 'foo', 'bar'] (4)
    files 'filename1.tfvars' , 'filename2.tfvars'(5)
    provider { VariablesSpec v -> v.var('foo2','bar2')} (6)
    remoteStateMap {
      injectVar = true (7)
      varName = 'my_alt_remote_state' (8)
    }
}
1 Adds one variable called foo or value bar. The provided value can be anything that can be lazy-evaluated to a string.
2 Adds a map called fooMap. The map keys have to be strings, but the map values can be anything that will lazy-evaluate to a string.
3 Adds a list called fooList with values foo and bar. The list entries can be anything that will lazy-evaluate to a string.
4 Alternative list format
5 Adds one or more files containing a list of terraform variables. The names can be anything that will lazy evaluate to a string and may be a relative path. The files will be resolved relative to the source set directory.
6 Adds an action that will be called on a VariablesSpec instance to provide more variables. These actions will be evaluated before any of the above methods are evaluated.
7 Inject the tokens from the backend into a map called remote_state.
8 Use an alternative variable name than remote_state.

Workspaces

This plugin suite adds naming conventions to easily deal with workspaces. If you add a workspace called alpha then the apply task for this workspace will be tofuApplyAlpha. If you add a workspace called beta to a source set called release, then the apply task will be tofuReleaseApplyBeta. These conventions only apply to tasks which are workstate/state-aware in terraform. For instance there will be no task named tofuInitAlpha or tofuFmtCheckAlpha. There is also no need to switch workspaces, as the plugin will do that under the hood automatically. If you run ./gradlew tofuApplyAlpha tofuApplyBeta tofuApplyGamma tofuOutput, the plugin will automatically perform a a tofu select before executing tofu apply or tofu output.

If you decide to remove workspaces, simply cleanup the state by running the appropriate tofuDestroy task(s) and then remove the workspaces from the source set DSL. Finally run tofuCleanupWorkspaces.

Override OpenTofu version for a source set

Sometimes you might need one of your source sets to run with a different version of OpenTofu. For instance, you might want to upgrade, but one of the source sets will take more work and leaving it at an older version for a period might be a good solution.

Assuming that you have sourceSet called monkey, this can be achieved via task actions

build.gradle
opentofu {
  toolchains {
    legacyTofu { (1)
      executableByVersion('1.8.3')
    }
  }

  sourceSets {
    monkey {
      useToolchain('legacyTofu') (2)
    }
  }
}
1 Create a toolchain for the legacy version
2 Tell your source set to use that toolchain.

Configure multiple source sets

build.gradle
opentofu {
  sourceSets.all { (1)
    useBackend('s3')
  }
}
1 Apply this configuration to all source sets.

Have relationships between source sets

Sometimes you need to ensure that the infrastructure of one source set is applied, before applying that of another source set. There is a simple DSL method to eliminate the need for you to add all the inter-tasks dependencies.

opentofu {
  sourceSets {
    database {
    }
    etl {
        mustRunAfter('database') (1)
    }
  }
}
1 Ensures that tofuEtlApply and tofuEtlPlan will run after tofuDatabaseApply* and tofuDatabasePlan*. The relationship is a simple mustRunAfter.

Have relationships between source sets of other subprojects

In certain cases, one might need to place dependencies between source sets and its workspaces to the source sets and workspaces of another project.

subprojectA/build.gradle
opentofu {
  sourceSets {
    database {
      workspaces 'anotherWs'
      producesVariants() (1)
    }
  }
}
1 Indicates that a source set produces context on which another subproject can depend on.
subprojectB/build.gradle
opentofu {
  sourceSets {
    etl {
      workspaces 'oneMore'
      consumes(':subprojectA', 'database') (1)
      consumes('oneMore', ':subprojectA', 'database', 'anotherWs') (2)
    }
  }
}
1 Indicates that the default workspace of the etl source set depends on the default workspace of the database source set in the subprojectA subproject. This is a shortcut for people who only use default workspaces.
2 Indicates that the oneMore workspace of the etl source set depends on the anotherWs workspace of the database source set in the subprojectA subproject.
These methods cannot be used in a single-project context and will result in a configuration error.

Configure the plugin cache directory

Be default the plugin cache directory from terraformrc in the root project is used.

It is possible to also use a custom plugin cache directory per source set.

build.gradle
terraform {
  sourceSets {
    main {
      useConfiguredPluginCache() (1)
      useCustomPluginCache() (2)
      useCustomPluginCache('path/to/my/cache', 100000) (3)
    }
  }
}
1 Use the plugin cache directory that is configured in terraformrc (Default behaviour).
2 Use a custom plugin cache directory for this source set. This cache dir is stored under the project cache directory

Secrets

You can create arbitrary secrets and share them with backends and source sets

build.gradle
import org.ysb33r.gradle.iac.base.secrets.AwsSecrets

opentofu {
  secrets {
    awsAcct1(AwsSecrets) { (1)
      useAccessKeyId('1234567890')
      useSecretAccessKey('abcdefghijklmn')
    }
  }

  backends {
    s3(S3Backend) {
      fromSecretsProvider(opentofu.secrets.awsAcct1) (2)
    }
  }

  sourceSets {
    main {
       fromSecretsProvider(opentofu.secrets.awsAcct1) (3)
    }
  }
}
1 Declare a collection of secrets. In this example we use org.ysb33r.gradle.iac.base.secrets.AwsSecrets for declaring AWS secrets like profile, credentials file, access keys etc.
2 Tell the S3 backend to use the same secrets that was defined in our secrets definition.

You can also use org.ysb33r.gradle.iac.base.secrets.GenericSecrets as a way of rolling your own secrets.