feat(misconf): export raw Terraform data to Rego

[nikpivkin]

Description

Related issues

• Close feat(misconf): Support additional terraform attributes  #5460
• Close feat(misconf): export raw Terraform data to Rego #8877

Usage examples

Example config:

resource "aws_s3_bucket" "test" {
  bucket = "test"
  tags = {
    Environment = "Production"
    Project     = "ProjectX"
  }
}

variable "password" {}

resource "aws_db_instance" "test" {
  instance_class       = "db.t3.micro"
  password             = var.password
}

data "aws_secretsmanager_random_password" "test" {}

Output:

 (MEDIUM): The resource "aws_s3_bucket" is missing required tags: {"Owner"}
══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════


See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/resource-tagging
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 main.tf:3-6
   via main.tf:1-7 (aws_s3_bucket.test)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   resource "aws_s3_bucket" "test" {
   2     bucket = "test"
   3 ┌   tags = {
   4 │     Environment = "Production"
   5 │     Project     = "ProjectX"
   6 └   }
   7   }
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────


 (MEDIUM): The data source "aws_secretsmanager_random_password" has an ephemeral equivalent and should be replaced with it
══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════


See https://developer.hashicorp.com/terraform/language/resources/ephemeral#ephemeral-resources
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 main.tf:16
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  16 [ data "aws_secretsmanager_random_password" "test" {}
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────


 (HIGH): "password" is not a write-only attribute. Use the write-only alternative "password_wo"
══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════


See https://developer.hashicorp.com/terraform/language/resources/ephemeral#write-only-arguments
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 main.tf:13
   via main.tf:11-14 (aws_db_instance.test)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  11   resource "aws_db_instance" "test" {
  12     instance_class       = "db.t3.micro"
  13 [   password             = var.password
  14   }
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Checks:

required_tags.rego

# METADATA
# title: Resources should have required tags
# related_resources:
# - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/resource-tagging
# custom:
#   id: TF_001
#   short_code: required-tags
#   severity: MEDIUM
#   input:
#     selector:
#     - type: terraform-raw
package user. tf_required_tags

import rego.v1

required_tags := {"Environment", "Owner", "Project"}

resources_to_check := {"aws_s3_bucket"}

deny contains res if {
	some block in input.modules[_].blocks
	block.kind == "resource"
	block.type in resources_to_check
	tags := block.attributes.tags

	used_tags := {k | some k, _ in tags.value}
	missed_tags := required_tags - used_tags
	count(missed_tags) > 0
	res := result.new(
		sprintf("The resource %q is missing required tags: %v", [block.type, missed_tags]),
		tags,
	)
}

ephemeral_data_sources.rego

# METADATA
# title: Use ephemeral resources when data source has an equivalent
# related_resources:
# - https://developer.hashicorp.com/terraform/language/resources/ephemeral#ephemeral-resources
# custom:
#   id: TF_002
#   short_code: ephemeral_data_sources
#   severity: MEDIUM
#   input:
#     selector:
#     - type: terraform-raw
package user.tf_eph_data_sources

import rego.v1

data_sources_to_check := {
	"aws_secretsmanager_random_password",
	"aws_secretsmanager_secret_version",
}

deny contains res if {
	some block in input.modules[_].blocks
	block.kind == "data"
	block.type in data_sources_to_check
	res := result.new(
		sprintf("The data source %q has an ephemeral equivalent and should be replaced with it", [block.type]),
		block,
	)
}

write_only_arguments.rego

# METADATA
# title: Use write-only arguments
# related_resources:
# - https://developer.hashicorp.com/terraform/language/resources/ephemeral#write-only-arguments
# custom:
#   id: TF_003
#   short_code: wo-arguments
#   severity: HIGH
#   input:
#     selector:
#     - type: terraform-raw
package user.tf_wo_args

import rego.v1

resources_to_check := {
	"aws_db_instance": ["password", "password_wo"],
	"aws_docdb_cluster": ["master_password", "master_password_wo"],
}

deny contains res if {
	some block in input.modules[_].blocks
	block.kind == "resource"
	[original, alternative] := resources_to_check[block.type]
	attr := block.attributes[original]
	res := result.new(
		sprintf("%q is not a write-only attribute. Use the write-only alternative %q", [original, alternative]),
		attr,
	)
}

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[nikpivkin]

@simar7 This is a POC of an idea that has been discussed here.

What can be improved:

• Export data directly as OPA values, which will skip the step of marshaling to json and unmarshaling to OPA values and improve performance
• Export references from expressions to detect relationships between blocks. Example uses:
  • Find related blocks for complex checking.
  • Find non-ephemeral input variables that are used in write-only attributes.

Related:

• Support for Kubernetes terraform resources #8506

[simar7]

Can we somehow put this behind a flag? Doing this each time even when raw terraform is not needed might be a bit wasteful and may impact performance.

[nikpivkin]

Done abe2422

[simar7]

lgtm, left a couple of comments.