Skip to content

Commit

Permalink
chore: cleanup convenience functions usage
Browse files Browse the repository at this point in the history
  • Loading branch information
KavithaSiva committed Oct 2, 2024
1 parent ecf5c01 commit 831b355
Showing 1 changed file with 18 additions and 29 deletions.
47 changes: 18 additions & 29 deletions packages/orchestration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ This package incorporates generative AI orchestration capabilities into your AI
- [Templating](#templating)
- [Content Filtering](#content-filtering)
- [Data Masking](#data-masking)
- [Retrieving Data from the Response](#retrieving-data-from-the-response)
- [Using Resource Groups](#using-resource-groups)
- [Local Testing](#local-testing)
- [Support, Feedback, Contribution](#support-feedback-contribution)
Expand Down Expand Up @@ -115,6 +114,24 @@ const responseContent = response.getContent();

`getContent()` is a convenience method that parses the response and returns the model's output as a string.

```ts
const finishReason = response.getFinishReason();
```

`getFinishReason()` is a convenience method that specifies the reason for stopping the chat completion request.

```ts
const tokenUsage = response.getTokenUsage();

console.log(
`Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` +
`Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` +
`Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n`
);
```

`getTokenUsage()` is a convenience method that retrieves the token usage details of the chat completion request.

#### Passing a Message History

It is possible to provide a history of a conversation to the model.
Expand Down Expand Up @@ -241,34 +258,6 @@ const response = await orchestrationClient.chatCompletion({
return response.getContent();
```

### Retrieving Data from the Response

In addition to `getContent()`, other available convenience methods can retrieve the finish reason and token usage.
Use `response.rawReason` to access the complete HTTP response from the orchestration service.

#### Finish Reason

The finish reason indicates the reason for stopping the chat completion request.
For example, when output is filtered based on the configuration, the finish reason is `content_filter`.

```ts
const finishReason = response.getFinishReason();
```

#### Token Usage

To retrieve the token usage details of the orchestration request, use the following snippet:

```ts
const tokenUsage = response.getTokenUsage();

console.log(
`Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` +
`Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` +
`Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n`
);
```

### Using Resource Groups

The resource group can be used as an additional parameter to pick the right orchestration deployment.
Expand Down

0 comments on commit 831b355

Please sign in to comment.