Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add log_props support #155

Merged
merged 4 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/ChatGptNet.Models/ChatGptChoice.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ public class ChatGptChoice
| [FinishReason](ChatGptChoice/FinishReason.md) { get; set; } | Gets or sets a value specifying why the choice has been returned. |
| [Index](ChatGptChoice/Index.md) { get; set; } | Gets or sets the index of the choice in the list. |
| [IsFiltered](ChatGptChoice/IsFiltered.md) { get; } | Gets or sets a value indicating whether the this [`ChatGptChoice`](./ChatGptChoice.md) has been filtered by the content filtering system. |
| [LogProbabilities](ChatGptChoice/LogProbabilities.md) { get; set; } | Gets or sets the log probabilities associated with this [`ChatGptChoice`](./ChatGptChoice.md). |
| [Message](ChatGptChoice/Message.md) { get; set; } | Gets or sets the message associated with this [`ChatGptChoice`](./ChatGptChoice.md), if any. |

## See Also
Expand Down
16 changes: 16 additions & 0 deletions docs/ChatGptNet.Models/ChatGptChoice/LogProbabilities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# ChatGptChoice.LogProbabilities property

Gets or sets the log probabilities associated with this [`ChatGptChoice`](../ChatGptChoice.md).

```csharp
public ChatGptLogProbability? LogProbabilities { get; set; }
```

## See Also

* class [ChatGptLogProbability](../ChatGptLogProbability.md)
* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* class [ChatGptChoice](../ChatGptChoice.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
21 changes: 21 additions & 0 deletions docs/ChatGptNet.Models/ChatGptLogProbability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# ChatGptLogProbability class

Represents the log probability information of a [`completion choice`](./ChatGptChoice.md).

```csharp
public class ChatGptLogProbability
```

## Public Members

| name | description |
| --- | --- |
| [ChatGptLogProbability](ChatGptLogProbability/ChatGptLogProbability.md)() | The default constructor. |
| [Content](ChatGptLogProbability/Content.md) { get; set; } | Gets or sets the list of message content tokens with log probability information. |

## See Also

* namespace [ChatGptNet.Models](../ChatGptNet.md)
* [ChatGptLogProbability.cs](https://github.com/marcominerva/ChatGptNet/tree/master/src/ChatGptNet/Models/ChatGptLogProbability.cs)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# ChatGptLogProbability constructor

The default constructor.

```csharp
public ChatGptLogProbability()
```

## See Also

* class [ChatGptLogProbability](../ChatGptLogProbability.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
15 changes: 15 additions & 0 deletions docs/ChatGptNet.Models/ChatGptLogProbability/Content.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# ChatGptLogProbability.Content property

Gets or sets the list of message content tokens with log probability information.

```csharp
public IEnumerable<ChatGptLogProbabilityContent> Content { get; set; }
```

## See Also

* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* class [ChatGptLogProbability](../ChatGptLogProbability.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
24 changes: 24 additions & 0 deletions docs/ChatGptNet.Models/ChatGptLogProbabilityContent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ChatGptLogProbabilityContent class

Represents a message content token with log probability information.

```csharp
public class ChatGptLogProbabilityContent
```

## Public Members

| name | description |
| --- | --- |
| [ChatGptLogProbabilityContent](ChatGptLogProbabilityContent/ChatGptLogProbabilityContent.md)() | The default constructor. |
| [Bytes](ChatGptLogProbabilityContent/Bytes.md) { get; set; } | Gets or sets a list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be `null` if there is no bytes representation for the token. |
| [LogProbality](ChatGptLogProbabilityContent/LogProbality.md) { get; set; } | Gets or sets the log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely. |
| [Token](ChatGptLogProbabilityContent/Token.md) { get; set; } | Gets or sets the token. |
| [TopLogProbabilities](ChatGptLogProbabilityContent/TopLogProbabilities.md) { get; set; } | Gets or sets the list of the most likely tokens and their log probability, at this token position. In rare cases, there may be fewer than the number of requested [`TopLogProbabilities`](./ChatGptParameters/TopLogProbabilities.md) returned. |

## See Also

* namespace [ChatGptNet.Models](../ChatGptNet.md)
* [ChatGptLogProbabilityContent.cs](https://github.com/marcominerva/ChatGptNet/tree/master/src/ChatGptNet/Models/ChatGptLogProbabilityContent.cs)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
14 changes: 14 additions & 0 deletions docs/ChatGptNet.Models/ChatGptLogProbabilityContent/Bytes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# ChatGptLogProbabilityContent.Bytes property

Gets or sets a list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be `null` if there is no bytes representation for the token.

```csharp
public IEnumerable<byte>? Bytes { get; set; }
```

## See Also

* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# ChatGptLogProbabilityContent constructor

The default constructor.

```csharp
public ChatGptLogProbabilityContent()
```

## See Also

* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# ChatGptLogProbabilityContent.LogProbality property

Gets or sets the log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.

```csharp
public double LogProbality { get; set; }
```

## See Also

* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
14 changes: 14 additions & 0 deletions docs/ChatGptNet.Models/ChatGptLogProbabilityContent/Token.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# ChatGptLogProbabilityContent.Token property

Gets or sets the token.

```csharp
public string Token { get; set; }
```

## See Also

* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# ChatGptLogProbabilityContent.TopLogProbabilities property

Gets or sets the list of the most likely tokens and their log probability, at this token position. In rare cases, there may be fewer than the number of requested [`TopLogProbabilities`](../ChatGptParameters/TopLogProbabilities.md) returned.

```csharp
public IEnumerable<ChatGptLogProbabilityContent>? TopLogProbabilities { get; set; }
```

## See Also

* property [TopLogProbabilities](../ChatGptParameters/TopLogProbabilities.md)
* class [ChatGptLogProbabilityContent](../ChatGptLogProbabilityContent.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
4 changes: 3 additions & 1 deletion docs/ChatGptNet.Models/ChatGptParameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ public class ChatGptParameters
| --- | --- |
| [ChatGptParameters](ChatGptParameters/ChatGptParameters.md)() | The default constructor. |
| [FrequencyPenalty](ChatGptParameters/FrequencyPenalty.md) { get; set; } | Gets or sets the frequency penalties for chat completion. Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim (default: 0). |
| [LogProbabilities](ChatGptParameters/LogProbabilities.md) { get; set; } | Gets or set a value that determines whether to return log probabilities of the output tokens or not. If `true`, returns the log probabilities of each output token returned in the content of message (default: `false`). |
| [MaxTokens](ChatGptParameters/MaxTokens.md) { get; set; } | Gets or sets the maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. |
| [PresencePenalty](ChatGptParameters/PresencePenalty.md) { get; set; } | Gets or sets the presence penalties for chat completion. Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics (default: 0). |
| [ResponseFormat](ChatGptParameters/ResponseFormat.md) { get; set; } | An object specifying the format that the model must output. Used to enable JSON mode. |
| [Seed](ChatGptParameters/Seed.md) { get; set; } | If specified, the system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. |
| [Seed](ChatGptParameters/Seed.md) { get; set; } | Gets or sets a value such that, if specified, the system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. |
| [Temperature](ChatGptParameters/Temperature.md) { get; set; } | Gets or sets what sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic (default: 1). |
| [TopLogProbabilities](ChatGptParameters/TopLogProbabilities.md) { get; set; } | Gets or sets a value between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. |
| [TopP](ChatGptParameters/TopP.md) { get; set; } | Gets or sets an alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with [`TopP`](./ChatGptParameters/TopP.md) probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered (default: 1). |

## Remarks
Expand Down
15 changes: 15 additions & 0 deletions docs/ChatGptNet.Models/ChatGptParameters/LogProbabilities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# ChatGptParameters.LogProbabilities property

Gets or set a value that determines whether to return log probabilities of the output tokens or not. If `true`, returns the log probabilities of each output token returned in the content of message (default: `false`).

```csharp
public bool? LogProbabilities { get; set; }
```

## See Also

* property [TopLogProbabilities](./TopLogProbabilities.md)
* class [ChatGptParameters](../ChatGptParameters.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
2 changes: 1 addition & 1 deletion docs/ChatGptNet.Models/ChatGptParameters/Seed.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ChatGptParameters.Seed property

If specified, the system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.
Gets or sets a value such that, if specified, the system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.

```csharp
public int? Seed { get; set; }
Expand Down
19 changes: 19 additions & 0 deletions docs/ChatGptNet.Models/ChatGptParameters/TopLogProbabilities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ChatGptParameters.TopLogProbabilities property

Gets or sets a value between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability.

```csharp
public int? TopLogProbabilities { get; set; }
```

## Remarks

[`LogProbabilities`](./LogProbabilities.md)must be set to `true` if this parameter is used.

## See Also

* property [LogProbabilities](./LogProbabilities.md)
* class [ChatGptParameters](../ChatGptParameters.md)
* namespace [ChatGptNet.Models](../../ChatGptNet.md)

<!-- DO NOT EDIT: generated by xmldocmd for ChatGptNet.dll -->
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
| class [ChatGptFunction](./ChatGptNet.Models/ChatGptFunction.md) | Represents the description of a function available for ChatGPT. |
| class [ChatGptFunctionCall](./ChatGptNet.Models/ChatGptFunctionCall.md) | Represents a response function call. |
| class [ChatGptInnerError](./ChatGptNet.Models/ChatGptInnerError.md) | Contains further details about the error. |
| class [ChatGptLogProbability](./ChatGptNet.Models/ChatGptLogProbability.md) | Represents the log probability information of a [`completion choice`](./ChatGptNet.Models/ChatGptChoice.md). |
| class [ChatGptLogProbabilityContent](./ChatGptNet.Models/ChatGptLogProbabilityContent.md) | Represents a message content token with log probability information. |
| class [ChatGptMessage](./ChatGptNet.Models/ChatGptMessage.md) | Represents a single chat message. |
| class [ChatGptParameters](./ChatGptNet.Models/ChatGptParameters.md) | Represents chat completion parameters. |
| class [ChatGptPromptFilterResults](./ChatGptNet.Models/ChatGptPromptFilterResults.md) | Contains information about content filtering for input prompts. |
Expand Down
4 changes: 2 additions & 2 deletions samples/ChatGptApi/ChatGptApi.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.3" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.3.0" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.4" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.4.0" />
<PackageReference Include="Polly.Extensions" Version="8.3.1" />
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.5.0" />
<PackageReference Include="TinyHelpers.AspNetCore" Version="3.0.6" />
Expand Down
8 changes: 4 additions & 4 deletions samples/ChatGptBlazor.Wasm/ChatGptBlazor.Wasm.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Markdig" Version="0.35.0" />
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="8.0.3" />
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="8.0.3" PrivateAssets="all" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.3.0" />
<PackageReference Include="Markdig" Version="0.37.0" />
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="8.0.4" />
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="8.0.4" PrivateAssets="all" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.4.0" />
<PackageReference Include="Polly.Extensions" Version="8.3.1" />
</ItemGroup>

Expand Down
2 changes: 1 addition & 1 deletion samples/ChatGptConsole/ChatGptConsole.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.3.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.4.0" />
<PackageReference Include="Polly.Extensions" Version="8.3.1" />
</ItemGroup>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.3.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.4.0" />
<PackageReference Include="Polly.Extensions" Version="8.3.1" />
</ItemGroup>

Expand Down
2 changes: 1 addition & 1 deletion samples/ChatGptStreamConsole/ChatGptStreamConsole.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.3.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.4.0" />
<PackageReference Include="Polly.Extensions" Version="8.3.1" />
</ItemGroup>

Expand Down
10 changes: 6 additions & 4 deletions src/ChatGptNet/ChatGptClient.cs
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ public async Task<Guid> LoadConversationAsync(Guid conversationId, IEnumerable<C
if (!replaceHistory)
{
// Otherwise, retrieves the current history and adds the messages.
var conversationHistory = await cache.GetAsync(conversationId, cancellationToken) ?? Enumerable.Empty<ChatGptMessage>();
var conversationHistory = await cache.GetAsync(conversationId, cancellationToken) ?? [];
messages = conversationHistory.Union(messages);
}

Expand All @@ -250,7 +250,7 @@ public async Task AddInteractionAsync(Guid conversationId, string question, stri
ArgumentNullException.ThrowIfNull(question);
ArgumentNullException.ThrowIfNull(answer);

var messages = await cache.GetAsync(conversationId, cancellationToken) ?? Enumerable.Empty<ChatGptMessage>();
var messages = await cache.GetAsync(conversationId, cancellationToken) ?? [];
messages = messages.Union([
new()
{
Expand Down Expand Up @@ -356,8 +356,10 @@ private ChatGptRequest CreateChatGptRequest(IEnumerable<ChatGptMessage> messages
MaxTokens = parameters?.MaxTokens ?? options.DefaultParameters.MaxTokens,
PresencePenalty = parameters?.PresencePenalty ?? options.DefaultParameters.PresencePenalty,
FrequencyPenalty = parameters?.FrequencyPenalty ?? options.DefaultParameters.FrequencyPenalty,
User = options.User,
ResponseFormat = parameters?.ResponseFormat ?? options.DefaultParameters.ResponseFormat
ResponseFormat = parameters?.ResponseFormat ?? options.DefaultParameters.ResponseFormat,
LogProbabilities = parameters?.LogProbabilities ?? options.DefaultParameters.LogProbabilities,
TopLogProbabilities = parameters?.TopLogProbabilities ?? options.DefaultParameters.TopLogProbabilities,
User = options.User
};

private EmbeddingRequest CreateEmbeddingRequest(IEnumerable<string> messages, EmbeddingParameters? parameters, string? model)
Expand Down
4 changes: 3 additions & 1 deletion src/ChatGptNet/ChatGptOptionsBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,9 @@ internal ChatGptOptionsBuilder(ChatGptOptionsBuilder source)
ResponseFormat = source.DefaultParameters?.ResponseFormat,
TopP = source.DefaultParameters?.TopP,
Temperature = source.DefaultParameters?.Temperature,
Seed = source.DefaultParameters?.Seed
Seed = source.DefaultParameters?.Seed,
LogProbabilities = source.DefaultParameters?.LogProbabilities,
TopLogProbabilities = source.DefaultParameters?.TopLogProbabilities
};

DefaultEmbeddingParameters = new()
Expand Down
8 changes: 8 additions & 0 deletions src/ChatGptNet/Models/ChatGptChoice.cs
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,12 @@ public class ChatGptChoice
/// </remarks>
[JsonPropertyName("finish_reason")]
public string FinishReason { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the log probabilities associated with this <see cref="ChatGptChoice"/>.
/// </summary>
/// <seealso cref="ChatGptLogProbability"/>
/// <seealso cref="ChatGptLogProbabilityContent"/>
[JsonPropertyName("logprobs")]
public ChatGptLogProbability? LogProbabilities { get; set; }
}
12 changes: 12 additions & 0 deletions src/ChatGptNet/Models/ChatGptLogProbability.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
namespace ChatGptNet.Models;

/// <summary>
/// Represents the log probability information of a <see cref="ChatGptChoice">completion choice</see>.
/// </summary>
public class ChatGptLogProbability
{
/// <summary>
/// Gets or sets the list of message content tokens with log probability information.
/// </summary>
public IEnumerable<ChatGptLogProbabilityContent> Content { get; set; } = [];
}
Loading
Loading