Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for both Azure OpenAI and non-Azure OpenAI #3

Merged
merged 2 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@ Virgil also enriches chat messages by suggesting follow-up actions (e.g. opening

Virgil Agent is powered by .NET 8, using [.NET Aspire](https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview) to build and deploy the application, and the [Microsoft Bot Framework](https://dev.botframework.com/) to provide the chatbot functionality.

The AI capabilities are provided by the [Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service).
The AI capabilities can be provided by services such as:

- [Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
- [OpenAI](https://openai.com/)

The application consists of multiple components:

Expand All @@ -37,30 +40,34 @@ VirgilAgent does not currently include a frontend application, but, through the
- [Docker](https://www.docker.com/)
- [Bot Framework Emulator](https://github.com/microsoft/BotFramework-Emulator/blob/master/README.md)
- .NET Aspire workload
- Azure OpenAI Service endpoint and key
- Azure OpenAI Service endpoint and key (if using Azure OpenAI), or an OpenAI API key (if using OpenAI)

### Required configuration

Before running the application, you need to configure the Azure OpenAI endpoint and key that will be used by the chat service and the suggestions service.
Before running the application, you need to configure the OpenAI endpoint and key that will be used by the chat service and the suggestions service.

To do so, open the `appsettings.json` file in both the `VirgilAgent.ChatService` and `VirgilAgent.SuggestionsService` projects, and set the following values to the endpoint and key of your Azure OpenAI service:
To do so, open the `appsettings.json` file in both the `VirgilAgent.ChatService` and `VirgilAgent.SuggestionsService` projects, and set the following values:

```json
"AzureOpenAI": {
"OpenAI": {
"Endpoint": "YOUR_ENDPOINT_HERE",
"Key": "YOUR_KEY_HERE",
...
},
```

If you are using Azure OpenAI Service, you need to provide both the endpoint and the key; if you are using non-Azure OpenAI, leave the endpoint empty and provide only the API key.

Make sure to also update the `DeploymentName` value in the same section to match the name of a valid model or deployment.

### Additional configuration

The application has a few additional configuration options that you can set in the `appsettings.json` files, to customize the behavior of the application and tweak the performance of the AI service.

For instance, the `AzureOpenAI` section can be used to set the parameters of the Azure OpenAI service (e.g. the temperature of the generated responses, the maximum number of tokens, etc.).
For instance, the `OpenAI` section can be used to set the parameters of the OpenAI service (e.g. the temperature of the generated responses, the maximum number of tokens, etc.).
Instead, the `Chat` section can be used to set the maximum number of messages that are stored in the cache for a conversation (a greater number of messages will improve the quality of the responses, but will also increase the number of tokens used by the AI service).

> **Note**: be careful to set the parameters appropriately, to avoid exceeding the limits of the Azure OpenAI service (e.g. the maximum number of tokens for a conversation).
> **Note**: be careful to set the parameters appropriately, to avoid exceeding the limits of the OpenAI service (e.g. the maximum number of tokens for a conversation).

### Running the application

Expand Down
12 changes: 6 additions & 6 deletions src/VirgilAgent.ChatService/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@

var configuration = builder.Configuration;

// Bind AzureOpenAIOptions from configuration.
string azureOpenAIOptionsSectionName = "AzureOpenAI";
AzureOpenAIOptions azureOpenAIOptions = configuration.GetSection(azureOpenAIOptionsSectionName).Get<AzureOpenAIOptions>()
?? throw new InvalidOperationException($"Missing configuration section: {azureOpenAIOptionsSectionName}");
builder.Services.AddSingleton(azureOpenAIOptions);
// Bind OpenAIOptions from configuration.
string openAIOptionsSectionName = "OpenAI";
OpenAIOptions openAIOptions = configuration.GetSection(openAIOptionsSectionName).Get<OpenAIOptions>()
?? throw new InvalidOperationException($"Missing configuration section: {openAIOptionsSectionName}");
builder.Services.AddSingleton(openAIOptions);

// Add cache.
builder.AddCache("Cache");
Expand All @@ -33,7 +33,7 @@
ChatOptions chatOptions = configuration.GetSection(chatOptionsSectionName).Get<ChatOptions>()
?? throw new InvalidOperationException($"Missing configuration section: {chatOptionsSectionName}");
builder.Services.AddSingleton(chatOptions);
builder.Services.AddSingleton<IChatAIClient, AzureOpenAIChatAIClient>();
builder.Services.AddSingleton<IChatAIClient, OpenAIChatAIClient>();
builder.Services.AddSingleton<ChatService>();

var app = builder.Build();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
namespace VirgilAgent.ChatService.Services;

/// <summary>
/// Azure OpenAI implementation of the <see cref="IChatAIClient"/> interface.
/// OpenAI implementation of the <see cref="IChatAIClient"/> interface.
/// </summary>
internal class AzureOpenAIChatAIClient : IChatAIClient
internal class OpenAIChatAIClient : IChatAIClient
{
private const string SystemPrompt = @"
Your name is Virgil.
Expand All @@ -22,16 +22,25 @@ Give answers that are less than 200 words long.
Reply that you don't know if you don't know how to answer a question.";

private readonly OpenAIClient _openAIClient;
private readonly AzureOpenAIOptions _options;
private readonly OpenAIOptions _options;

public AzureOpenAIChatAIClient(AzureOpenAIOptions options)
public OpenAIChatAIClient(OpenAIOptions options)
{
_options = options;

Uri endpoint = new(_options.Endpoint);
AzureKeyCredential token = new(_options.Key);
if (string.IsNullOrWhiteSpace(_options.Endpoint))
{
// Use a non-Azure OpenAI endpoint.
_openAIClient = new(_options.Key);
}
else
{
// Use the specified Azure OpenAI Service endpoint.
Uri endpoint = new(_options.Endpoint);
AzureKeyCredential token = new(_options.Key);

_openAIClient = new(endpoint, token);
_openAIClient = new(endpoint, token);
}
}

/// <inheritdoc/>
Expand Down
2 changes: 1 addition & 1 deletion src/VirgilAgent.ChatService/appsettings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
}
},
"AllowedHosts": "*",
"AzureOpenAI": {
"OpenAI": {
"Endpoint": "YOUR_ENDPOINT_HERE",
"Key": "YOUR_KEY_HERE",
"DeploymentName": "gpt-35-turbo",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
namespace VirgilAgent.Core;

/// <summary>
/// Options for configuring the Azure OpenAI service.
/// Options for configuring the OpenAI service.
/// </summary>
public record AzureOpenAIOptions(
public record OpenAIOptions(
string Endpoint,
string Key,
string DeploymentName,
Expand Down
12 changes: 6 additions & 6 deletions src/VirgilAgent.SuggestionsService/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@

var configuration = builder.Configuration;

// Bind AzureOpenAIOptions from configuration.
string azureOpenAIOptionsSection = "AzureOpenAI";
var azureOpenAIOptions = configuration.GetSection(azureOpenAIOptionsSection).Get<AzureOpenAIOptions>()
?? throw new InvalidOperationException($"Missing configuration section: {azureOpenAIOptionsSection}");
builder.Services.AddSingleton(azureOpenAIOptions);
// Bind OpenAIOptions from configuration.
string openAIOptionsSection = "OpenAI";
var openAIOptions = configuration.GetSection(openAIOptionsSection).Get<OpenAIOptions>()
?? throw new InvalidOperationException($"Missing configuration section: {openAIOptionsSection}");
builder.Services.AddSingleton(openAIOptions);

// Add AI services.
builder.Services.AddSingleton<ISuggestionsAIClient, AzureOpenAISuggestionsAIClient>();
builder.Services.AddSingleton<ISuggestionsAIClient, OpenAISuggestionsAIClient>();
builder.Services.AddSingleton<SuggestionsService>();

var app = builder.Build();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
namespace VirgilAgent.SuggestionsService.Services;

/// <summary>
/// Azure OpenAI implementation of the <see cref="ISuggestionsAIClient"/> interface.
/// OpenAI implementation of the <see cref="ISuggestionsAIClient"/> interface.
/// </summary>
internal class AzureOpenAISuggestionsAIClient : ISuggestionsAIClient
internal class OpenAISuggestionsAIClient : ISuggestionsAIClient
{
private const string SystemPrompt = @"
You suggest possible follow-up actions and questions to a given input message.
Expand Down Expand Up @@ -41,18 +41,27 @@ What are some typical foods in Florence?|Reply|
";

private readonly OpenAIClient _openAIClient;
private readonly AzureOpenAIOptions _options;
private readonly ILogger<AzureOpenAISuggestionsAIClient> _logger;
private readonly OpenAIOptions _options;
private readonly ILogger<OpenAISuggestionsAIClient> _logger;

public AzureOpenAISuggestionsAIClient(AzureOpenAIOptions options, ILogger<AzureOpenAISuggestionsAIClient> logger)
public OpenAISuggestionsAIClient(OpenAIOptions options, ILogger<OpenAISuggestionsAIClient> logger)
{
_options = options;
_logger = logger;

Uri endpoint = new(_options.Endpoint);
AzureKeyCredential token = new(_options.Key);
if (string.IsNullOrWhiteSpace(_options.Endpoint))
{
// Use a non-Azure OpenAI endpoint.
_openAIClient = new(_options.Key);
}
else
{
// Use the specified Azure OpenAI Service endpoint.
Uri endpoint = new(_options.Endpoint);
AzureKeyCredential token = new(_options.Key);

_openAIClient = new(endpoint, token);
_logger = logger;
_openAIClient = new(endpoint, token);
}
}

/// <inheritdoc/>
Expand Down
2 changes: 1 addition & 1 deletion src/VirgilAgent.SuggestionsService/appsettings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
}
},
"AllowedHosts": "*",
"AzureOpenAI": {
"OpenAI": {
"Endpoint": "YOUR_ENDPOINT_HERE",
"Key": "YOUR_KEY_HERE",
"DeploymentName": "gpt-35-turbo",
Expand Down