Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore Alternative ML Model Providers #300

Open
1 task
davidgxue opened this issue Feb 16, 2024 · 0 comments
Open
1 task

Explore Alternative ML Model Providers #300

davidgxue opened this issue Feb 16, 2024 · 0 comments

Comments

@davidgxue
Copy link
Contributor

davidgxue commented Feb 16, 2024

Please describe the feature you'd like to see
During downtimes of services such as OpenAI, Cohere and Azure OpenAI, have hot swappable.

For cost reduction, we should also do A/B testing and see how other models perform compared to GPT-4 as GPT-4 itself is very expensive.

Describe the solution you'd like
Explore using OpenAI and Anthropic for LLM services as backup for Azure OpenAI's GPT-3.5 and GPT-4.

Are there any alternatives to this feature?
Explore open source LLMs

Additional context

Acceptance Criteria

  • Implement env variable based mechanism to hot swap an existing LLM to another provider during service downtime
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant