Support Anthropic Claude model from GCP

[ankit-tyagi]

Why
I have access to GCP. And I want to pay only for the request I make rather than a constant monthly fee. GCP allows me to pay for the resource I use. use - https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude for more description.

Description
I want to use Claude through GCP.

Requirements
I want to be able to use my GCP credential to make inference request to Claude models on GCP Vertex AI.

[enricoros]

Hi @ankit-tyagi this is a good request. In the past I tried to support Google Vertex AI, but then Gemini came with a simpler API, which was best.

This ticket is open for anyone that would like to work on if for big-agi-2, but I won't be able to work on it.

[ankit-tyagi]

ok, I will try implement that. Let me start with some simpler tickets and understand the codebase

[rwb-truelime]

@ankit-tyagi Maybe this helps? I use https://github.com/paul-gauthier/aider which uses https://github.com/BerriAI/litellm that handles all LLM integrations (and more). Here you can find a VertexAI configuration and code example: https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/litellm/llms/vertex_ai_and_google_ai_studio/vertex_ai_anthropic.py . Hope this helps.

[fry69]

FYI: The JS/TypeScript equivalent to LiteLLM is the Vercel AI SDK -> https://github.com/vercel/ai which supports Sonnet-3.5 via Vertex AI through a community adapter -> https://sdk.vercel.ai/providers/community-providers/anthropic-vertex-ai