Creatiion of Bedrock Agent - Multi agent collaborators fails when multiple collaborators are added

[preak95]

Terraform and AWS Provider Version

Terraform v1.7.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/archive v2.7.0
+ provider registry.terraform.io/hashicorp/aws v5.94.1
+ provider registry.terraform.io/hashicorp/null v3.2.3
+ provider registry.terraform.io/hashicorp/time v0.13.0

Your version of Terraform is out of date! The latest version
is 1.11.3. You can update by downloading from https://www.terraform.io/downloads.html

Affected Resource(s) or Data Source(s)

• bedrockagent_agent_collaborator
• bedrockagent_agent

Expected Behavior

When associating multiple agent collaborators with a Bedrock Agent SUPERVISOR, there should be a mechanism to wait for the agent to go into a PREPARED state before attempting to associate another agent collaborator.

Actual Behavior

When associating a collaborator with a Bedrock Supervisor Agent, the agent goes into a preparing state. If multiple collaborators are associated with the agent at the same time, this causes the deployment to fail because the agent could still be in a PREPARING state.

When using for_each to deploy multiple such collaborators, this causes a failure. Upon, retrying apply, terraform isn't aware of the dangling collaborators.

Relevant Error/Panic Output

╷
│ Error: preparing Agent
│ 
│   with aws_bedrockagent_agent_collaborator.prompt_generation_agent_collaborator,
│   on main.tf line 300, in resource "aws_bedrockagent_agent_collaborator" "prompt_generation_agent_collaborator":
│  300: resource "aws_bedrockagent_agent_collaborator" "prompt_generation_agent_collaborator" {
│ 
│ preparing Bedrock Agent (RKZJM*****): operation error Bedrock Agent: PrepareAgent, https response error StatusCode: 400, RequestID: a8bcd1236e8-9930-, ValidationException: Prepare operation can't be performed on Agent when it is in Preparing state.
│ Retry the request when the agent is in a valid state.

Sample Terraform Configuration

Click to expand configuration

data "aws_caller_identity" "current" {}

data "aws_partition" "current" {}

data "aws_region" "current" {}

data "aws_iam_policy_document" "example_agent_trust" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      identifiers = ["bedrock.amazonaws.com"]
      type        = "Service"
    }
    condition {
      test     = "StringEquals"
      values   = [data.aws_caller_identity.current.account_id]
      variable = "aws:SourceAccount"
    }
    condition {
      test     = "ArnLike"
      values   = ["arn:${data.aws_partition.current.partition}:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:agent/*"]
      variable = "AWS:SourceArn"
    }
  }
}

data "aws_iam_policy_document" "example_agent_permissions" {
  statement {
    actions = ["bedrock:InvokeModel"]
    resources = [
      "arn:${data.aws_partition.current.partition}:bedrock:${data.aws_region.current.name}::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0",
    ]
  }
  statement {
    actions = ["bedrock:GetAgentAlias", "bedrock:InvokeAgent"]
    resources = [
      "arn:${data.aws_partition.current.partition}:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:agent/*",
      "arn:${data.aws_partition.current.partition}:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:agent-alias/*"
    ]
  }
}

resource "aws_iam_role" "example" {
  assume_role_policy = data.aws_iam_policy_document.example_agent_trust.json
  name_prefix        = "AmazonBedrockExecutionRoleForAgents_"
}

resource "aws_iam_role_policy" "example" {
  policy = data.aws_iam_policy_document.example_agent_permissions.json
  role   = aws_iam_role.example.id
}

resource "aws_bedrockagent_agent" "example_collaborator_1" {
  agent_name                  = "my-agent-collaborator-1"
  agent_resource_role_arn     = aws_iam_role.example.arn
  idle_session_ttl_in_seconds = 500
  foundation_model            = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  instruction                 = "do what the supervisor tells you to do. Think clearly"
}

resource "aws_bedrockagent_agent" "example_collaborator_2" {
  agent_name                  = "my-agent-collaborator-2"
  agent_resource_role_arn     = aws_iam_role.example.arn
  idle_session_ttl_in_seconds = 500
  foundation_model            = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  instruction                 = "do what the supervisor tells you to do. Think clearly"
}

resource "aws_bedrockagent_agent" "example_supervisor" {
  agent_name                  = "my-agent-supervisor"
  agent_resource_role_arn     = aws_iam_role.example.arn
  agent_collaboration         = "SUPERVISOR"
  idle_session_ttl_in_seconds = 500
  foundation_model            = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  instruction                 = "tell the sub agent what to do based on the query provided"
  prepare_agent               = false
}

resource "aws_bedrockagent_agent_alias" "example_1" {
  agent_alias_name = "my-agent-alias"
  agent_id         = aws_bedrockagent_agent.example_collaborator_1.agent_id
  description      = "Test Alias"
}

resource "aws_bedrockagent_agent_alias" "example_2" {
  agent_alias_name = "my-agent-alias"
  agent_id         = aws_bedrockagent_agent.example_collaborator_2.agent_id
  description      = "Test Alias"
}

resource "aws_bedrockagent_agent_collaborator" "example_1" {
  agent_id                   = aws_bedrockagent_agent.example_supervisor.agent_id
  collaboration_instruction  = "tell the other agent what to do"
  collaborator_name          = "my-collab-example_1"
  relay_conversation_history = "TO_COLLABORATOR"

  agent_descriptor {
    alias_arn = aws_bedrockagent_agent_alias.example_1.agent_alias_arn
  }
}

resource "aws_bedrockagent_agent_collaborator" "example_2" {
  agent_id                   = aws_bedrockagent_agent.example_supervisor.agent_id
  collaboration_instruction  = "tell the other agent what to do"
  collaborator_name          = "my-collab-example_2"
  relay_conversation_history = "TO_COLLABORATOR"

  agent_descriptor {
    alias_arn = aws_bedrockagent_agent_alias.example_2.agent_alias_arn
  }
}

Steps to Reproduce

1. Deploy the above terraform

Debug Logging

Click to expand log output

╷
│ Error: preparing Agent
│ 
│   with aws_bedrockagent_agent_collaborator.prompt_generation_agent_collaborator,
│   on main.tf line 300, in resource "aws_bedrockagent_agent_collaborator" "prompt_generation_agent_collaborator":
│  300: resource "aws_bedrockagent_agent_collaborator" "prompt_generation_agent_collaborator" {
│ 
│ preparing Bedrock Agent (RKZJ******): operation error Bedrock Agent: PrepareAgent, https response error StatusCode: 400, RequestID: aabcd123458-44298-besf1-521a6c878802aa, ValidationException: Prepare operation can't be performed on Agent when it is in Preparing state.
│ Retry the request when the agent is in a valid state.
╵

GenAI / LLM Assisted Development

n/a

Important Facts and References

Workaround:

When adding multiple collaborators, add one as a dependency for another. Doable for a few collaborators but not for multiple and hinders with module creation.

If this is expected behavior, would be great to have an example with multiple collaborators.

Would you like to implement a fix?

No

[github-actions]

Community Guidelines

This comment is added to every new Issue to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀

Voting for Prioritization

• Please vote on this Issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize it.
• Please see our prioritization guide for additional information on how the maintainers handle prioritization.
• Please do not leave +1 or other comments that do not add relevant new information or questions; they generate extra noise for others following the Issue and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.

[justinretzolk]

Hey @preak95 👋 Thank you for taking the time to raise this! This does seem like something that should be addressed, so I'm going to leave it open for prioritization. That said, I think I've figured out a workaround for you in the meantime.

Notably, you providing an applyable sample configuration was extremely helpful, so thank you very much for doing that. While I wasn't actually able to reproduce the issue (presumably, this configuration is simplified enough that the preparation happens fairly quickly), I think a slight modification of the configuration should help. There's a wait function that's used by the aws_bedrockagent_agent_collaborator resource to wait for the agent to be prepared. Due to that wait function, by making aws_bedrockagent_agent_collaborator.example_2 dependent on aws_bedrockagent_agent_collaborator.example_1 and forcing Terraform to wait for complete creation of one before starting creation of the other, you should be able to get around this. I did this by updating your configuration slightly to:

resource "aws_bedrockagent_agent_collaborator" "example_2" {
  agent_id                   = aws_bedrockagent_agent_collaborator.example_1.agent_id
  collaboration_instruction  = "tell the other agent what to do"
  collaborator_name          = "my-collab-example_2"
  relay_conversation_history = "TO_COLLABORATOR"

  agent_descriptor {
    alias_arn = aws_bedrockagent_agent_alias.example_2.agent_alias_arn
  }
}

By interpolating the agent_id value from aws_bedrockagent_agent_collaborator.example_1, an implicit dependency is created. This can be seen in these graphs (that I generated using a neat tool I've just found to easily generate mermaid graphs -- terramaid; thanks for being my guinea pig for my excitement to actually use it 🙂):

Before:

flowchart TD
    subgraph Terraform
        data_aws_caller_identity_current["data.aws_caller_identity.current"]
        data_aws_iam_policy_document_example_agent_permissions["data.aws_iam_policy_document.example_agent_permissions"]
        data_aws_iam_policy_document_example_agent_trust["data.aws_iam_policy_document.example_agent_trust"]
        data_aws_partition_current["data.aws_partition.current"]
        data_aws_region_current["data.aws_region.current"]
        aws_bedrockagent_agent_example_collaborator_1["aws_bedrockagent_agent.example_collaborator_1"]
        aws_bedrockagent_agent_example_collaborator_2["aws_bedrockagent_agent.example_collaborator_2"]
        aws_bedrockagent_agent_example_supervisor["aws_bedrockagent_agent.example_supervisor"]
        aws_bedrockagent_agent_alias_example_1["aws_bedrockagent_agent_alias.example_1"]
        aws_bedrockagent_agent_alias_example_2["aws_bedrockagent_agent_alias.example_2"]
        aws_bedrockagent_agent_collaborator_example_1["aws_bedrockagent_agent_collaborator.example_1"]
        aws_bedrockagent_agent_collaborator_example_2["aws_bedrockagent_agent_collaborator.example_2"]
        aws_iam_role_example["aws_iam_role.example"]
        aws_iam_role_policy_example["aws_iam_role_policy.example"]
    end
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_caller_identity_current
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_partition_current
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_region_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_caller_identity_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_partition_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_region_current
    aws_bedrockagent_agent_example_collaborator_1 --> aws_iam_role_example
    aws_bedrockagent_agent_example_collaborator_2 --> aws_iam_role_example
    aws_bedrockagent_agent_example_supervisor --> aws_iam_role_example
    aws_bedrockagent_agent_alias_example_1 --> aws_bedrockagent_agent_example_collaborator_1
    aws_bedrockagent_agent_alias_example_2 --> aws_bedrockagent_agent_example_collaborator_2
    aws_bedrockagent_agent_collaborator_example_1 --> aws_bedrockagent_agent_example_supervisor
    aws_bedrockagent_agent_collaborator_example_1 --> aws_bedrockagent_agent_alias_example_1
    aws_bedrockagent_agent_collaborator_example_2 --> aws_bedrockagent_agent_example_supervisor
    aws_bedrockagent_agent_collaborator_example_2 --> aws_bedrockagent_agent_alias_example_2
    aws_iam_role_example --> data_aws_iam_policy_document_example_agent_trust
    aws_iam_role_policy_example --> data_aws_iam_policy_document_example_agent_permissions
    aws_iam_role_policy_example --> aws_iam_role_example

Loading

After:

flowchart TD
    subgraph Terraform
        data_aws_caller_identity_current["data.aws_caller_identity.current"]
        data_aws_iam_policy_document_example_agent_permissions["data.aws_iam_policy_document.example_agent_permissions"]
        data_aws_iam_policy_document_example_agent_trust["data.aws_iam_policy_document.example_agent_trust"]
        data_aws_partition_current["data.aws_partition.current"]
        data_aws_region_current["data.aws_region.current"]
        aws_bedrockagent_agent_example_collaborator_1["aws_bedrockagent_agent.example_collaborator_1"]
        aws_bedrockagent_agent_example_collaborator_2["aws_bedrockagent_agent.example_collaborator_2"]
        aws_bedrockagent_agent_example_supervisor["aws_bedrockagent_agent.example_supervisor"]
        aws_bedrockagent_agent_alias_example_1["aws_bedrockagent_agent_alias.example_1"]
        aws_bedrockagent_agent_alias_example_2["aws_bedrockagent_agent_alias.example_2"]
        aws_bedrockagent_agent_collaborator_example_1["aws_bedrockagent_agent_collaborator.example_1"]
        aws_bedrockagent_agent_collaborator_example_2["aws_bedrockagent_agent_collaborator.example_2"]
        aws_iam_role_example["aws_iam_role.example"]
        aws_iam_role_policy_example["aws_iam_role_policy.example"]
    end
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_caller_identity_current
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_partition_current
    data_aws_iam_policy_document_example_agent_permissions --> data_aws_region_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_caller_identity_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_partition_current
    data_aws_iam_policy_document_example_agent_trust --> data_aws_region_current
    aws_bedrockagent_agent_example_collaborator_1 --> aws_iam_role_example
    aws_bedrockagent_agent_example_collaborator_2 --> aws_iam_role_example
    aws_bedrockagent_agent_example_supervisor --> aws_iam_role_example
    aws_bedrockagent_agent_alias_example_1 --> aws_bedrockagent_agent_example_collaborator_1
    aws_bedrockagent_agent_alias_example_2 --> aws_bedrockagent_agent_example_collaborator_2
    aws_bedrockagent_agent_collaborator_example_1 --> aws_bedrockagent_agent_example_supervisor
    aws_bedrockagent_agent_collaborator_example_1 --> aws_bedrockagent_agent_alias_example_1
    aws_bedrockagent_agent_collaborator_example_2 --> aws_bedrockagent_agent_alias_example_2
    aws_bedrockagent_agent_collaborator_example_2 --> aws_bedrockagent_agent_collaborator_example_1
    aws_iam_role_example --> data_aws_iam_policy_document_example_agent_trust
    aws_iam_role_policy_example --> data_aws_iam_policy_document_example_agent_permissions
    aws_iam_role_policy_example --> aws_iam_role_example

Loading

[preak95]

Hello @justinretzolk Thanks for the thorough investigation and the workaround. I will try and check if this can somehow be extended when using for_each to deploy multiple such resources.

[aqumus]

@preak95 Were you able to use the workaround with for_each to deploy multiple resources considering those multiple resources would be created sequentially where next collection items depends on earlier collection item?

[aqumus]

@justinretzolk The similar issue is observed for action group as well when declaring an aws_bedrockagent_agent_action_group resource using for_each, the subsequent collection items fails since the agent is in Preparing state due to preceding collection items.
How could we prioritise this issue?