How to use Private Repos over HTTPS with Terraform Operator

Using Private Repo (SSH) Keys and (HTTPS) Tokens

Often Terraform modules are not stored in public git repos for any number of reasons. Fortunately, with the right keys or tokens, it’s not that hard to access them. In this article we’ll go over a few common examples of how to access terraform modules in private repos with Terraform Operator. In general, Terraform can accept either ssh keys and/or tokens to access modules in git repos.

Example 1: Private Repos over HTTPS on Github

To start, you’ll need an basic understanding of Git, and GitHub. Let’s pretend our private git repo is hosted on GitHub and we want to use HTTPS to download our module. First, get an “Access Token” from GitHub. Check the “repo” access checkbox. (Need help? Check out this article to help you Generate Access Tokens from Github Account.)

When you create the token, the token will only show up once. Copy it. We’ll save this to the cluster as a Secret.

Create a Secret in your cluster with the token.

GHTOKEN=$(printf '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' | base64)

cat << EOF | kubectl apply -f -
apiVersion: v1
data:
  mytoken: $GHTOKEN
kind: Secret
metadata:
  name: gh-access-token
  namespace: default
type: Opaque
EOF

In the above code, we saved our Github token to the cluster as a Secret called gh-access-token with data key mytoken in the default namespace. Next we’ll add these item to our terraform k8s-resource manifest.

Create the terraform k8s-resource manifest, let’s call it terraform.yaml. The following is an example of my manifest. Notice the scmAuthMethods. The scmAuthMethods is an array of objects.

1. In example below, we’ll fill in host: with github.com. That is the host our token is valid for.
2. Next, we’re using git: as the SCM (Source Control Management).
3. Under git:, we’re going with the https: protocol. The other protocol for git is ssh: which will be covered later.
4. And finally under https:, the secret that was created earlier is defined in tokenSecretRef:.

Take a look for the final manifest:

---
# terraform.yaml

apiVersion: tf.isaaguilar.com/v1alpha1
kind: Terraform
metadata:
  name: my-module
spec:
  terraformVersion: 1.1.9
  terraformModule: https://github.com/isaaguilar/terraform-do-something-awesome.git?ref=main
  customBackend: |-
    terraform {
      backend "kubernetes" {
        secret_suffix     = "my-module"
        namespace         = "default"
        in_cluster_config = true
      }
    }    

  # *-------------------------*
  scmAuthMethods:
  - host: github.com
    git:
      https:
        tokenSecretRef:
          name: gh-access-token
          key: mytoken
  # *-------------------------*

  keepLatestPodsOnly: true
  ignoreDelete: false
  writeOutputsToStatus: true

Apply the manifest using kubectl and we’re done and terraform-operator will handle the rest.

kubectl apply -f terraform.yaml

Watch this short terminal capture which follows the steps above of using a Github Token with Terraform Operator.