In this post we will explore how we can increase our security posture by leveraging Open ID Connect (OIDC) as authentication mechanism in CI/CD pipelines for deploying Terraform based infrastructure to Amazon Web Services (AWS).
The first design principle of the Security Pillar of the AWS Well-Architected Framework sounds as follows:
“Implement a strong identity foundation: Implement the principle of least privilege and enforce separation of duties with appropriate authorization for each interaction with your AWS resources. Centralize identity management, and aim to eliminate reliance on long-term static credentials.”
The technical implementation guidance for SEC02-BP02 Use temporary credentials advises to leverage temporary security credentials instead of long-term credentials for all AWS API and CLI requests.
“Avoiding the use of long-term credentials in favor of temporary credentials should go hand in hand with a strategy of reducing the usage of IAM users in favor of federation and IAM roles.
While IAM users have been used for both human and machine identities in the past, we now recommend not using them to avoid the risks in using long-term access keys.”
For automated deployments to AWS with CI/CD solutions such as GitHub, GitLab, Terraform Cloud and so on, IAM users with static access keys where traditionally the way to go, with the hassle of rotating them at a reasonable interval.
Lately, the advent of Open ID Connect (OIDC) has made this both simpler and more secure. This technology effectively eliminates the need for rotation of long-term access keys. However, there’s been some drawbacks for Terraform users using the AWS S3 state backend, which did not support assume role with web identity. Additionally, for the Terraform AWS provider, an IAM Identity Provider had to be configured in every target AWS account.
With the release of Terraform 1.6 assume role with web identity support for the S3 backend was finally included (even though this was not clearly communicated in the release announcement, it’s stated in the changelog: assume_role_with_web_identity
nested block for assuming a role with dynamic credentials such as a JSON Web Token. (#31244)).
This means that we now are able to deploy an Identity Provider configuration in a centralized CI/CD tooling account and assume IAM roles to each target workload account!
In this post I’ll take you through the steps on how you can deploy infrastructure with Terraform and GitHub Actions using Open ID Connect (OIDC) and AWS IAM AssumeRoleWithWebIdentity.
Each job requests an OIDC token from GitHub’s OIDC provider, which responds with an automatically generated JSON web token (JWT) that is unique for each workflow job where it is generated. When the job runs, the OIDC token is presented to the cloud provider. To validate the token, the cloud provider checks if the OIDC token’s subject and other claims are a match for the conditions that were preconfigured on the cloud role’s OIDC trust definition. For more information see https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#understanding-the-oidc-token.
A) Create Identity Provider in the tooling account
Normally I provision resources with Terraform, but for the sake of simplicity in this example we’re going to use the CLI, so in your tooling CI/CD account, open AWS Cloudshell.
You can add GitHub as an IdP in your account with a single AWS CLI command. The following code will create an Open ID Connect (OIDC) Provider for GitHub.com with their OIDC thumbprint 938fd4d98bab03faadb97b34396831e3780aea1.
aws iam create-open-id-connect-provider --url "https://token.actions.githubusercontent.com" --thumbprint-list "6938fd4d98bab03faadb97b34396831e3780aea1" --client-id-list 'sts.amazonaws.com'
B) Create IAM roles in the tooling account
In CloudShell: vim trustpolicyforGitHubOIDC.json
. Adapt AWS account ID on line 7 and GitHub organization and repo name on line 12 to your use-case. For more details see https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#configuring-the-role-and-trust-policy.
Copy the AWS tooling account ID and keep it available in a temporary text document, it’s going to be referenced a few times throughout this setup procedure.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<id-of-tooling-account>:oidc-provider/token.actions.githubusercontent.com" # Replace
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:sub": "repo:organization/repo-name:ref:refs/heads/main", # Replace
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
}
}
}
]
}
Then create the IAM role for AssumeRoleWithWebIdentity.
aws iam create-role --role-name GitHubAction-AssumeRoleWithAction --assume-role-policy-document file://trustpolicyforGitHubOIDC.json
Create a corresponding IAM role for cicd purposes, to be assumed by the previously created role in the tooling account.
In CloudShell: vim trustpolicyforcicd.json
. Adapt AWS account ID on line 8 to your tooling account ID.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Principal": {
"AWS": "arn:aws:iam::<id-of-tooling-account>:role/GitHubAction-AssumeRoleWithAction"
}
}
]
}
Then create the IAM role for AssumeRole.
aws iam create-role --role-name cicd --assume-role-policy-document file://trustpolicyforcicd.json
C) Create IAM role in workload accounts(s) with corresponding trust policy
This enables the IAM role in the tooling account to assume IAM roles in workload AWS accounts for CI/CD purposes.
In CloudShell in a sample worklod AWS account: vim trustpolicyforcicd.json
. Adapt account ID on line 7 to your tooling account ID.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<id-of-tooling-account>:role/GitHubAction-AssumeRoleWithAction"
},
"Action": "sts:AssumeRole"
}
]
}
Then create the IAM role for AssumeRole.
aws iam create-role --role-name cicd --assume-role-policy-document file://trustpolicyforcicd.json
The AWS configurations are now complete. Let’s jump to GitHub.com.
D) Set up GitHub Actions workflow
As with GitHub Action implementations, there are many roads that lead to Rome, but here is a simplified example should work as a starting point. Feel free to adopt to your use-case and change account ID on line 5 to your tooling account ID.
TF_VERSION is your desired Terraform version of choice, but do note that this functionality is only supported from version 1.6.0.
Step Get OIDC Token on line 29 issues a short-lived OIDC token, stores it at the path configured in AWS_WEB_IDENTITY_TOKEN_FILE, configures it for a profile called “cicd” and then issues a request to AWS STS to get-caller-id to verify the profile “cicd” is working.
Sample .github/workflows/tf-cicd-aws.yml
---
name: "Terraform CI/CD to AWS"
env:
AWS_REGION: "eu-central-1"
AWS_ROLE_ARN: "arn:aws:iam::<id-of-tooling-account>:role/GitHubAction-AssumeRoleWithAction" # Replace
AWS_WEB_IDENTITY_TOKEN_FILE: "/tmp/web_identity_token_file"
TF_VERSION: "1.6.0"
"on":
pull_request:
push:
branches:
- main
# Permission can be added at job level or workflow level
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
jobs:
Terraform:
name: Terraform
runs-on: ubuntu-latest
strategy:
matrix: { dir: ['aws-accounts/tooling', 'aws-accounts/workload-prod-01'] } # Replace, declare folder(s) to traverse
#env:
# working-directory: aws-accounts/tooling
steps:
- name: Get OIDC Token
id: get_oidc_token
run: |
curl -s -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sts.amazonaws.com" | jq -r '.value' > /tmp/web_identity_token_file
mkdir ~/.aws
echo -e "[profile cicd]\nrole_arn=${AWS_ROLE_ARN}\nweb_identity_token_file=${AWS_WEB_IDENTITY_TOKEN_FILE}" >> ~/.aws/config
aws sts get-caller-identity --profile cicd
- name: Checkout
uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v2
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Terraform format
id: fmt
run: terraform fmt -check
working-directory: ${{ matrix.dir }}
continue-on-error: true
- name: Terraform init
id: init
run: terraform init
working-directory: ${{ matrix.dir }}
- name: Terraform validate
id: validate
run: terraform validate
working-directory: ${{ matrix.dir }}
- name: Terraform plan
id: plan
run: terraform plan -no-color -input=false -refresh=true -detailed-exitcode
working-directory: ${{ matrix.dir }}
- name: Checkov GitHub Action
uses: bridgecrewio/checkov-action@v12.1347.0
with:
directory: ${{ matrix.dir }}
# Run scan on all checks but a specific check identifier (comma separated)
# skip_check: # optional
download_external_modules: false
quiet: true
soft_fail: true # Replace to false to break on errors
- name: Plan output
id: output
uses: actions/github-script@v3
if: github.event_name == 'pull_request'
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
### Workspace
\`${process.env.TF_WORKSPACE}\`
#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`hcl
${process.env.PLAN}
\`\`\`
</details>
**Pusher**: @${{ github.actor }}
**Action**: ${{ github.event_name }}
`;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
- name: Terraform apply
id: apply
run: terraform apply -input=false -auto-approve
working-directory: ${{ matrix.dir }}
E) Arrange your Terraform code
Sample for backend.tf.
Change account ID on line 13 to your tooling account ID. Change values for lines 8-10 as well.
If you are starting from scratch with Terraform state on AWS this module can help you get going: https://github.com/cloudposse/terraform-aws-tfstate-backend .
terraform {
backend "s3" {
profile = "cicd"
session_name = "SESSION_NAME"
external_id = "EXTERNAL_ID"
encrypt = true
acl = "bucket-owner-full-control"
dynamodb_table = "tf-build-terraform-state-lock" # Replace to your configuration
bucket = "tf-build-terraform-state" # Replace to your configuration
key = "workload-prod-01/terraform.tfstate" # Replace to your configuration
region = "eu-central-1"
assume_role_with_web_identity = {
role_arn = "arn:aws:iam::<id-of-tooling-account:role/GitHubAction-AssumeRoleWithAction" # Replace to your configuration
web_identity_token_file = "/tmp/web_identity_token_file"
}
}
}
Sample for provider.tf
provider "aws" {
region = var.aws_region
profile = var.profile_cicd
assume_role {
role_arn = "arn:aws:iam::${var.aws_account_id}:role/${var.profile_cicd}"
session_name = "SESSION_NAME"
external_id = "EXTERNAL_ID"
}
}
Sample for variables.tf
variable "aws_region" {
type = string
default = "eu-central-1"
}
variable "aws_account_id" {
type = number
}
variable "profile_cicd" {
type = string
default = "cicd"
}
Sample for terraform.tfvars
aws_region = "eu-central-1" # Replace to your region
profile_cicd = "cicd"
aws_account_id = 1234567890 # Replace to your destination AWS account ID
Sample for main.tf
module "s3_bucket" {
source = "terraform-aws-modules/s3-bucket/aws"
versioning = {
enabled = true
}
}
As always, adjust variables and desired configurations to your use-case.
Please let me know if you have any questions or feedback, you’ll find my contact details at the /about/ page.
Conclusion
In this article we saw how Open ID Connect (OIDC) can be leveraged to avoid configuring long-lived static access keys (IAM users) in CI/CD pipelines.
We configured an Identity Provider in an AWS account dedicated for tooling purposes with corresponding IAM roles in the tooling + workload accounts.
A GitHub Actions workflow was then configured and sample Terraform code was provided for demonstration.
You now no longer need to worry about rotating static secrets for this use-case (or introduce more complexity with HashiCorp Vault [disclaimer: might be valuable in certain situations]).
References
- https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/
- https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html
- https://github.com/hashicorp/terraform/pull/31276
- https://github.com/hashicorp/terraform/issues/31244
- https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services
- https://github.com/hashicorp/terraform/releases/tag/v1.6.0