Support for Amazon Bedrock

[vkameswaran]

Is your feature request related to a problem? Please describe.
No current support for Amazon Bedrock

Describe the solution you'd like
Support added for Amazon Bedrock

Due to privacy concerns with OpenAI and it's sub products, I'd love to use this with Amazon Bedrock. Direct integration with the go SDK would be awesome but either way, love to see this added.

[greptile-apps]

To add support for Amazon Bedrock, follow these steps:

1. Extend Configuration:
   Update /config/models.go to include configuration fields for Amazon Bedrock, similar to how OpenAI and Anthropic are configured. This might include fields for API keys, endpoints, and any other necessary configuration.

   type LLM struct {
       ...
       BedrockAPIKey     string `mapstructure:"bedrock_api_key"`
       BedrockEndpoint   string `mapstructure:"bedrock_endpoint"`
   }

2. Implement Bedrock LLM Client:
   Create a new file in /pkg/llms, perhaps named llm_bedrock.go, to implement the LLM interface for Amazon Bedrock. This will involve creating a struct that adheres to the ZepLLM interface and implementing the necessary methods to interact with the Bedrock API.

   Example skeleton:

   type ZepBedrockLLM struct {
       client *bedrock.Client
   }
   
   func NewBedrockLLM(ctx context.Context, cfg *config.Config) (*ZepBedrockLLM, error) {
       // Initialize and return a new ZepBedrockLLM instance
   }
   
   // Implement other necessary methods here

3. Integration:
   Ensure that the new Bedrock LLM client is properly integrated within the Zep ecosystem, which includes being selectable based on configuration and used in places where LLMs are utilized.

By following these steps, you'll extend the Zep codebase to support Amazon Bedrock, aligning with the project's architecture and existing patterns for LLM integration.

References

greptileai/zep/config/models.go
greptileai/zep/pkg/llms/llm_base.go
greptileai/zep/pkg/llms/llm_openai_test.go
greptileai/zep/pkg/llms

Ask Greptile