Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Foundry models don't work (Mistral Codestral 2501 with Azure) #3735

Open
BastinFlorian opened this issue Jan 15, 2025 · 1 comment
Open
Assignees
Labels
area:configuration Relates to configuration options area:docs Relates to documentation ide:vscode Relates specifically to VS Code extension kind:enhancement Indicates a new feature request, imrovement, or extension priority:high Indicates high priority priority:medium Indicates medium priority

Comments

@BastinFlorian
Copy link

BastinFlorian commented Jan 15, 2025

Issue Category

Undocumented feature or missing documentation

Affected Documentation Page URL

https://docs.continue.dev/customize/model-providers/azure

Issue Description

Using Codestral 2501 available in the Azure Hub through Azure Foundry or Azure ML products have a different target URI that OpenAI Azure models.

Azure Open AI Target URI:
https://just-an-example.openai.azure.com/openai/deployments/gpt-4o-july/chat/completions?api-version=2023-03-15-preview
Foundry Codestral URL example (same process for AzureML deployed models):
https://just-an-example.openai.azure.com/chat/completions?api-version=2023-03-15-preview

The difference is just : openai/deployments/gpt-4o-july

How to configure such models ?
Can you please update the Codestral documentation thats seems to be lights/wrong for Autocomplete Codestral in Azure ?

Adding this feature allows both Azure Foundry and Azure ML models to be deployed with continue dev because the URI are the same for both products (I know azure is messy).

RESOLUTION: just add a new value for ApiType = 'foundry' to set the goold URI.

Version: 0.8.66
IDE: VS Code

Thank you,

Expected Content

No response

@BastinFlorian BastinFlorian added the area:docs Relates to documentation label Jan 15, 2025
@sestinj sestinj self-assigned this Jan 15, 2025
@BastinFlorian BastinFlorian changed the title Azure Foundry Model models don't work (Mistral Codestral 2501 with Azure) Azure Foundry models don't work (Mistral Codestral 2501 with Azure) Jan 15, 2025
@dosubot dosubot bot added area:configuration Relates to configuration options kind:enhancement Indicates a new feature request, imrovement, or extension priority:medium Indicates medium priority ide:vscode Relates specifically to VS Code extension labels Jan 15, 2025
@Patrick-Erichsen
Copy link
Collaborator

Discord thread that this issue began in: https://discord.com/channels/1108621136150929458/1328759921512288316

@Patrick-Erichsen Patrick-Erichsen added priority:high Indicates high priority and removed "needs-triage" labels Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:configuration Relates to configuration options area:docs Relates to documentation ide:vscode Relates specifically to VS Code extension kind:enhancement Indicates a new feature request, imrovement, or extension priority:high Indicates high priority priority:medium Indicates medium priority
Projects
None yet
Development

No branches or pull requests

3 participants