adding support for claude, azure, llama2 and cohere

[krrishdholakia]

Hi @adamyodinsky,

Noticed you're only calling OpenAI. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and was wondering if we could be helpful.

Added support for Claude, Cohere, Azure and Llama2 (via Replicate) by replacing the ChatOpenAI completion call with a litellm completion call. The code is pretty similar to the OpenAI class - as litellm follows the same pattern as the openai-python sdk.

Would love to know if this helps.

Happy to add additional tests / update documentation, if the initial PR looks good to you.

[adamyodinsky]

@krrishdholakia

First of all, really appreciate the PR and the initiative mate! this is awsome!

the PR looks ok, but it seems missing some stuff.

• token window control. as different models have different window size, diff token encoders, we will need to match a window token controller mechanism to each model.
• how do i choose which model I use? Which vendor?

P.S - there is another project I'm working on where litellm can be useful, I'll check how I can integrate it. also, the langchain guys may find it useful as well. maybe you should holla them as well.

[krrishdholakia]

@adamyodinsky

re: choosing which model - you can do that by the model parameter

E.g.

completion(model="gpt-3.5-turbo"...)

Could you explain more / point me to example of what you mean regarding token window control? I don't think i saw how you were doing it today

[krrishdholakia]

Hey @adamyodinsky, can you say more about the token window controller mechanism. What are you looking for here?

[kwuite]

@krrishdholakia I have an Azure hosted GPT4 model, will that work with your code out-of-the-box?

[krrishdholakia]

@kwuite yup - https://docs.litellm.ai/docs/providers/azure