Add tp_index field to partition - this may specify a column name or transform used to populate the tp_index column

[kaidaguerre]

A user may optionally choose to use a column as an index on a per-partition basis, eg:

 partition "aws_cloudtrail_log" "s3_bucket_us_east_1" {
    source "aws_s3_bucket" {
      connection  = connection.aws.account_a
      bucket      = "aws-cloudtrail-logs-account-a"
      file_layout = `AWSLogs/(%{DATA:org_id}/)?%{NUMBER:account_id}/CloudTrail/us-east-1/%{DATA}.json.gz`   
    }
    tp_index = "account_id"
  }

The user may create a "composite index" by using a function instead. The syntax should be the same as for the transform column argument:

  partition "aws_cloudtrail_log" "s3_bucket_us_east_1" {
    source "aws_s3_bucket" {
      connection  = connection.aws.account_a
      bucket      = "aws-cloudtrail-logs-account-a"
      file_layout = `AWSLogs/(%{DATA:org_id}/)?%{NUMBER:account_id}/CloudTrail/us-east-1/%{DATA}.json.gz`   
    }
    tp_index = "concat(account_id, '_', region)"
  }