Skip to content

Add Transparent HTTP Proxy Mode for Native Bedrock API Access #170

@LionsAd

Description

@LionsAd

Is your feature request related to a problem? Please describe.

Currently, the Bedrock Access Gateway requires complex request/response parsing and format conversion between different
API styles (OpenAI, Anthropic, native Bedrock). This approach has several limitations:

  • Format drift risk: Custom parsing can become outdated when AWS updates their APIs
  • Maintenance overhead: Need to manually implement every new Bedrock feature and parameter
  • Performance impact: JSON parsing and reconstruction adds latency
  • Limited compatibility: Headers, query parameters, and edge cases may not be handled correctly
  • Development complexity: Each new Bedrock endpoint requires custom implementation

Describe the feature you'd like

Add a transparent HTTP proxy mode for direct Bedrock API access that:

  1. Universal endpoint: Single route /bedrock/model/{model_id}/{endpoint_path:path} that handles ALL Bedrock
    endpoints automatically
  2. Pure passthrough: Forward requests to AWS Bedrock as-is with zero parsing or modification
  3. Authentication swap: Replace proxy API key with AWS bearer token transparently
  4. Header preservation: Forward all original headers (User-Agent, custom headers, etc.)
  5. Future-proof: New AWS Bedrock features work immediately without code changes
  6. Method agnostic: Support GET, POST, PUT, DELETE, PATCH automatically
  7. Streaming support: Detect and handle streaming responses automatically

Implementation approach:

  • Environment variable AWS_BEARER_TOKEN_BEDROCK enables transparent proxy mode
  • URL rewriting: localhost/api/v1/bedrock/model/X/invokebedrock-runtime.{region}.amazonaws.com/model/X/invoke
  • Client uses same proxy API key, backend uses AWS token (transparent to client)
  • Use async HTTP client (httpx) for optimal performance

Additional context

This feature would provide:

  • Perfect AWS compatibility - Always matches official AWS API exactly
  • Zero maintenance overhead - New AWS features work automatically
  • Better performance - No JSON parsing/reconstruction overhead
  • Broader compatibility - Handles all HTTP methods, headers, query params
  • Future-proof design - Works with any future Bedrock API changes

Example usage remains the same for clients:

curl -H "Authorization: Bearer proxy-api-key" \
  -H "Content-Type: application/json" \
  -d '{"messages": [...]}' \
  "http://localhost:8000/api/v1/bedrock/model/claude-3/invoke"

But internally it becomes a simple HTTP proxy instead of complex format conversion.

This would complement the existing OpenAI-compatible endpoints while providing a "native" path for maximum AWS Bedrock
compatibility.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions