-
Notifications
You must be signed in to change notification settings - Fork 226
Closed as not planned
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem? Please describe.
Currently, the Bedrock Access Gateway requires complex request/response parsing and format conversion between different
API styles (OpenAI, Anthropic, native Bedrock). This approach has several limitations:
- Format drift risk: Custom parsing can become outdated when AWS updates their APIs
- Maintenance overhead: Need to manually implement every new Bedrock feature and parameter
- Performance impact: JSON parsing and reconstruction adds latency
- Limited compatibility: Headers, query parameters, and edge cases may not be handled correctly
- Development complexity: Each new Bedrock endpoint requires custom implementation
Describe the feature you'd like
Add a transparent HTTP proxy mode for direct Bedrock API access that:
- Universal endpoint: Single route
/bedrock/model/{model_id}/{endpoint_path:path}
that handles ALL Bedrock
endpoints automatically - Pure passthrough: Forward requests to AWS Bedrock as-is with zero parsing or modification
- Authentication swap: Replace proxy API key with AWS bearer token transparently
- Header preservation: Forward all original headers (User-Agent, custom headers, etc.)
- Future-proof: New AWS Bedrock features work immediately without code changes
- Method agnostic: Support GET, POST, PUT, DELETE, PATCH automatically
- Streaming support: Detect and handle streaming responses automatically
Implementation approach:
- Environment variable
AWS_BEARER_TOKEN_BEDROCK
enables transparent proxy mode - URL rewriting:
localhost/api/v1/bedrock/model/X/invoke
→bedrock-runtime.{region}.amazonaws.com/model/X/invoke
- Client uses same proxy API key, backend uses AWS token (transparent to client)
- Use async HTTP client (httpx) for optimal performance
Additional context
This feature would provide:
- Perfect AWS compatibility - Always matches official AWS API exactly
- Zero maintenance overhead - New AWS features work automatically
- Better performance - No JSON parsing/reconstruction overhead
- Broader compatibility - Handles all HTTP methods, headers, query params
- Future-proof design - Works with any future Bedrock API changes
Example usage remains the same for clients:
curl -H "Authorization: Bearer proxy-api-key" \
-H "Content-Type: application/json" \
-d '{"messages": [...]}' \
"http://localhost:8000/api/v1/bedrock/model/claude-3/invoke"
But internally it becomes a simple HTTP proxy instead of complex format conversion.
This would complement the existing OpenAI-compatible endpoints while providing a "native" path for maximum AWS Bedrock
compatibility.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request