From 7239bbbffdf0d150af9c0eef15875766f015cc77 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 20 Sep 2024 16:19:08 +0200 Subject: [PATCH 01/52] chore: add local testing instructions --- README.md | 11 +++++++++++ packages/ai-api/README.md | 9 +++++++-- packages/foundation-models/README.md | 9 +++++++-- packages/langchain/README.md | 9 +++++++-- packages/orchestration/README.md | 9 +++++++-- 5 files changed, 39 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 11ec50b9..a332348b 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ Integrate chat completion into your business applications with SAP Cloud SDK for - [@sap-ai-sdk/orchestration](#sap-ai-sdkorchestration) - [@sap-ai-sdk/langchain](#sap-ai-sdklangchain) - [SAP Cloud SDK for AI Sample Project](#sap-cloud-sdk-for-ai-sample-project) +- [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) - [Security / Disclosure](#security--disclosure) - [Code of Conduct](#code-of-conduct) @@ -71,6 +72,16 @@ $ npm install @sap-ai-sdk/langchain We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](./sample-code/README.md) outlines the set-up needed to build and run it locally. +## Local Testing + +To test SAP Cloud SDK for AI features locally while building your application, follow these steps: + +- Download a service key for your AI Core service instance. +- Create a `.env` file in the root of your application. +- Add an entry `AICORE_SERVICE_KEY=''`. + +Adding a `.env` file ensures that the SDK uses the service key to interact with the AI Core service, allowing you to test all the clients like orchestration, OpenAI locally, provided you have the deployments for orchestration and OpenAI in SAP BTP. + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index 97390e27..cdac6a28 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -18,8 +18,9 @@ We maintain a list of [currently available and tested AI Core APIs](https://gith - [Create a Configuration](#create-a-configuration) - [Create a Deployment](#create-a-deployment) - [Delete a Deployment](#delete-a-deployment) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +4. [Local Testing](#local-testing) +5. [Support, Feedback, Contribution](#support-feedback-contribution) +6. [License](#license) ## Installation @@ -157,6 +158,10 @@ async function modifyDeployment() { } ``` +## Local Testing + +Refer [here](../../README.md#local-testing) for local testing instructions. + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index f94ab181..eba4eaad 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -11,8 +11,9 @@ This package incorporates generative AI foundation models into your AI activitie - [Azure OpenAI Client](#azure-openai-client) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +4. [Local Testing](#local-testing) +5. [Support, Feedback, Contribution](#support-feedback-contribution) +6. [License](#license) ## Installation @@ -140,6 +141,10 @@ const response = await embeddingClient.run({ const embedding = response.getEmbedding(); ``` +## Local Testing + +Refer [here](../../README.md#local-testing) for local testing instructions. + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 6834d552..cf046244 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -10,8 +10,9 @@ This package provides LangChain model clients built on top of the foundation mod - [Client Initialization](#client-initialization) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +4. [Local Testing](#local-testing) +5. [Support, Feedback, Contribution](#support-feedback-contribution) +6. [License](#license) ## Installation @@ -110,6 +111,10 @@ const embeddedDocument = await embeddingClient.embedDocuments([ ]); ``` +## Local Testing + +Refer [here](../../README.md#local-testing) for local testing instructions. + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 3d949b80..927375d9 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -10,8 +10,9 @@ This package incorporates generative AI orchestration capabilities into your AI 4. [Usage](#usage) - [Templating](#templating) - [Content Filtering](#content-filtering) -5. [Support, Feedback, Contribution](#support-feedback-contribution) -6. [License](#license) +5. [Local Testing](#local-testing) +6. [Support, Feedback, Contribution](#support-feedback-contribution) +7. [License](#license) ## Installation @@ -162,6 +163,10 @@ try { The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. Each category can be configured with severity levels of 0, 2, 4, or 6. +## Local Testing + +Refer [here](../../README.md#local-testing) for local testing instructions. + ## Support, Feedback, Contribution Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md). From f89fc2924ac0d0e0addd6fb9948f3553d58f3189 Mon Sep 17 00:00:00 2001 From: cloud-sdk-js Date: Fri, 20 Sep 2024 14:20:47 +0000 Subject: [PATCH 02/52] fix: Changes from lint --- packages/langchain/README.md | 2 +- packages/orchestration/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/langchain/README.md b/packages/langchain/README.md index cf046244..5c5d75bb 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -10,7 +10,7 @@ This package provides LangChain model clients built on top of the foundation mod - [Client Initialization](#client-initialization) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) -4. [Local Testing](#local-testing) +4. [Local Testing](#local-testing) 5. [Support, Feedback, Contribution](#support-feedback-contribution) 6. [License](#license) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 927375d9..2a272d35 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -10,7 +10,7 @@ This package incorporates generative AI orchestration capabilities into your AI 4. [Usage](#usage) - [Templating](#templating) - [Content Filtering](#content-filtering) -5. [Local Testing](#local-testing) +5. [Local Testing](#local-testing) 6. [Support, Feedback, Contribution](#support-feedback-contribution) 7. [License](#license) From 971883ea1fc09d603b8b435aa25e6e6646cacf2c Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 20 Sep 2024 16:25:40 +0200 Subject: [PATCH 03/52] chore: minor sentence fix --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a332348b..f57bb683 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ To test SAP Cloud SDK for AI features locally while building your application, f - Create a `.env` file in the root of your application. - Add an entry `AICORE_SERVICE_KEY=''`. -Adding a `.env` file ensures that the SDK uses the service key to interact with the AI Core service, allowing you to test all the clients like orchestration, OpenAI locally, provided you have the deployments for orchestration and OpenAI in SAP BTP. +By adding the `.env` file, the SDK uses the service key to interact with the AI Core service. This setup allows you to test clients such as orchestration and OpenAI locally, provided you have deployments for orchestration and OpenAI in SAP BTP. ## Support, Feedback, Contribution From 807212861ebe4eb11ee92b9a602f3589479bd8d4 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 20 Sep 2024 16:33:40 +0200 Subject: [PATCH 04/52] chore: fix sentence --- packages/ai-api/README.md | 2 +- packages/foundation-models/README.md | 2 +- packages/langchain/README.md | 2 +- packages/orchestration/README.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index cdac6a28..a43322ec 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -160,7 +160,7 @@ async function modifyDeployment() { ## Local Testing -Refer [here](../../README.md#local-testing) for local testing instructions. +For local testing instuctions, refer [here](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index eba4eaad..c3b102c5 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -143,7 +143,7 @@ const embedding = response.getEmbedding(); ## Local Testing -Refer [here](../../README.md#local-testing) for local testing instructions. +For local testing instuctions, refer [here](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 5c5d75bb..a53d4028 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -113,7 +113,7 @@ const embeddedDocument = await embeddingClient.embedDocuments([ ## Local Testing -Refer [here](../../README.md#local-testing) for local testing instructions. +For local testing instuctions, refer [here](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 2a272d35..23e69e18 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -165,7 +165,7 @@ Each category can be configured with severity levels of 0, 2, 4, or 6. ## Local Testing -Refer [here](../../README.md#local-testing) for local testing instructions. +For local testing instuctions, refer [here](../../README.md#local-testing). ## Support, Feedback, Contribution From f39f0447f65d69b487999d84f76b92be8012bdc3 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 20 Sep 2024 16:54:19 +0200 Subject: [PATCH 05/52] chore: add missing sections --- packages/orchestration/README.md | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 23e69e18..8a0fe9d2 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -69,6 +69,23 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` +#### Passing a custom request configuration + +To pass custom request configuration, for example additional headers to be passed to the client, use the following snippet: + +```ts +const response = await orchestrationClient.chatCompletion( + { + inputParams: { country: 'France' } + }, + { + headers: { 'x-custom-header': 'custom-value' } + } +); +``` + +#### Passing message history + It is possible to provide a history of a conversation to the model. Use the following snippet to send a chat completion request with history and a system message: @@ -106,6 +123,10 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` +#### Retrieving data from the response + +//Todo + #### Token Usage To retrieve the token usage details of the orchestration request, use the following snippet: @@ -122,6 +143,27 @@ logger.info( Remember to initialize a logger before using it. +#### Using Resource Groups + +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. + +The resource group can be used as an additional parameter to pick the right orchestration deployment. + +```ts +const orchestrationClient = new OrchestrationClient( + { + llm: { + model_name: 'gpt-4-32k', + model_params: { max_tokens: 50, temperature: 0.1 } + }, + templating: { + template: [{ role: 'user', content: 'What is my name?' }] + } + }, + { resourceGroup: 'rg1234' } +); +``` + ### Content Filtering Use the orchestration client with filtering to restrict content that is passed to and received from a generative AI model. From 9028c12045f20035fc54c7bdd9a8de8e336695d2 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:42:47 +0200 Subject: [PATCH 06/52] Update packages/orchestration/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 8a0fe9d2..1de3650d 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -207,7 +207,7 @@ Each category can be configured with severity levels of 0, 2, 4, or 6. ## Local Testing -For local testing instuctions, refer [here](../../README.md#local-testing). +For local testing, refer to [here](../../README.md#local-testing). ## Support, Feedback, Contribution From e0f2033dd96dce7063055a0404b9b7d951a36671 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:43:08 +0200 Subject: [PATCH 07/52] Update packages/orchestration/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 1de3650d..35795a38 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -123,7 +123,7 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` -#### Retrieving data from the response +#### Retrieving Data from the Response //Todo From df50cd12314864b48b91dbbcb8a0454e61eadaa2 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:43:17 +0200 Subject: [PATCH 08/52] Update packages/orchestration/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 35795a38..4c007ec8 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -84,7 +84,7 @@ const response = await orchestrationClient.chatCompletion( ); ``` -#### Passing message history +#### Passing a Message History It is possible to provide a history of a conversation to the model. Use the following snippet to send a chat completion request with history and a system message: From 7390dbee83052b21e1909cfba3d20e05e5c45a54 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:43:39 +0200 Subject: [PATCH 09/52] Update packages/orchestration/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 4c007ec8..c46fcb07 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -69,7 +69,7 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` -#### Passing a custom request configuration +#### Customizing the Request Configuration To pass custom request configuration, for example additional headers to be passed to the client, use the following snippet: From 8506bd65f056c796133dfa845123e9cbdb4a53ec Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:43:46 +0200 Subject: [PATCH 10/52] Update packages/langchain/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/langchain/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/langchain/README.md b/packages/langchain/README.md index a53d4028..224292be 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -113,7 +113,7 @@ const embeddedDocument = await embeddingClient.embedDocuments([ ## Local Testing -For local testing instuctions, refer [here](../../README.md#local-testing). +For local testing, refer to [here](../../README.md#local-testing). ## Support, Feedback, Contribution From 361940b6ae6bdb8dac947046f87790876b6c1c5c Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 15:44:37 +0200 Subject: [PATCH 11/52] Update packages/ai-api/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/ai-api/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index a43322ec..5907487c 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -160,7 +160,7 @@ async function modifyDeployment() { ## Local Testing -For local testing instuctions, refer [here](../../README.md#local-testing). +For local testing, refer to [here](../../README.md#local-testing). ## Support, Feedback, Contribution From d62d62d98dce2c1da053a0b0aa9ef659e95b9854 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Mon, 23 Sep 2024 15:57:40 +0200 Subject: [PATCH 12/52] chore: link client readmes in root --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index f57bb683..bb9c63c0 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,8 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc $ npm install @sap-ai-sdk/orchestration ``` +For details on Orchestration client usage, refer to [here](./packages/orchestration/README.md). + ### @sap-ai-sdk/ai-api This package provides tools to manage your scenarios and workflows in SAP AI Core. @@ -48,6 +50,8 @@ This package provides tools to manage your scenarios and workflows in SAP AI Cor $ npm install @sap-ai-sdk/ai-api ``` +For details on client usage, refer to [here](./packages/ai-api/README.md). + ### @sap-ai-sdk/foundation-models This package incorporates generative AI foundation models into your AI activities in SAP AI Core and SAP AI Launchpad. @@ -58,6 +62,8 @@ This package incorporates generative AI foundation models into your AI activitie $ npm install @sap-ai-sdk/foundation-models ``` +For details on Azure OpenAI client usage, refer to [here](./packages/foundation-models/README.md). + ### @sap-ai-sdk/langchain This package provides LangChain model clients, built on top of the foundation model clients of the SAP Cloud SDK for AI. @@ -68,6 +74,8 @@ This package provides LangChain model clients, built on top of the foundation mo $ npm install @sap-ai-sdk/langchain ``` +For details on LangChain model client usage, refer to [here](./packages/langchain/README.md). + ## SAP Cloud SDK for AI Sample Project We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](./sample-code/README.md) outlines the set-up needed to build and run it locally. From f1991b19632063ea2442cdb47cb064f411cd4ace Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Mon, 23 Sep 2024 16:33:47 +0200 Subject: [PATCH 13/52] Update packages/orchestration/README.md Co-authored-by: Marika Marszalkowski <868536+marikaner@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index c46fcb07..02804a8e 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -71,7 +71,7 @@ const responseContent = response.getContent(); #### Customizing the Request Configuration -To pass custom request configuration, for example additional headers to be passed to the client, use the following snippet: +To customize the request configuration, for example if you want to pass additional headers to the client, use the second parameter in the `chatCompletion()` method: ```ts const response = await orchestrationClient.chatCompletion( From 864f04199600ac5cf016de892b3e60344dca3ca4 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Mon, 23 Sep 2024 17:07:49 +0200 Subject: [PATCH 14/52] chore: Address review comments --- README.md | 3 +-- packages/foundation-models/README.md | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index bb9c63c0..0f5229ee 100644 --- a/README.md +++ b/README.md @@ -85,8 +85,7 @@ We have created a sample project demonstrating the different clients' usage of t To test SAP Cloud SDK for AI features locally while building your application, follow these steps: - Download a service key for your AI Core service instance. -- Create a `.env` file in the root of your application. -- Add an entry `AICORE_SERVICE_KEY=''`. +- Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in your local environment. By adding the `.env` file, the SDK uses the service key to interact with the AI Core service. This setup allows you to test clients such as orchestration and OpenAI locally, provided you have deployments for orchestration and OpenAI in SAP BTP. diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index c3b102c5..9bb11588 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -143,7 +143,7 @@ const embedding = response.getEmbedding(); ## Local Testing -For local testing instuctions, refer [here](../../README.md#local-testing). +For local testing instuctions, refer to [here](../../README.md#local-testing). ## Support, Feedback, Contribution From 7b94a230e1039f1ff3a74a0fc03d52bec9a1612c Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Tue, 24 Sep 2024 10:03:55 +0200 Subject: [PATCH 15/52] chore: address review comments --- packages/orchestration/README.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 02804a8e..9b8abbdc 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -69,6 +69,8 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` +`response.getContent()` is a convenience method that parses the response and returns the model's output as a string. + #### Customizing the Request Configuration To customize the request configuration, for example if you want to pass additional headers to the client, use the second parameter in the `chatCompletion()` method: @@ -125,9 +127,15 @@ const responseContent = response.getContent(); #### Retrieving Data from the Response -//Todo +In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish and token usage. +Use `response.rawReason` to access the complete HTTP response from the orchestration service. + +##### Finish Reason -#### Token Usage +```ts +const finishReason = response.getFinishReason(); +``` +##### Token Usage To retrieve the token usage details of the orchestration request, use the following snippet: From 93336454f58f238ebe08f9453d647c4d4750aea9 Mon Sep 17 00:00:00 2001 From: cloud-sdk-js Date: Tue, 24 Sep 2024 08:04:46 +0000 Subject: [PATCH 16/52] fix: Changes from lint --- packages/orchestration/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 9b8abbdc..af7d87ff 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -127,7 +127,7 @@ const responseContent = response.getContent(); #### Retrieving Data from the Response -In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish and token usage. +In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish and token usage. Use `response.rawReason` to access the complete HTTP response from the orchestration service. ##### Finish Reason @@ -135,6 +135,7 @@ Use `response.rawReason` to access the complete HTTP response from the orchestra ```ts const finishReason = response.getFinishReason(); ``` + ##### Token Usage To retrieve the token usage details of the orchestration request, use the following snippet: From 87a2ac0de8bb304b6530536d8b8b1bd71f8f6825 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 12:29:44 +0200 Subject: [PATCH 17/52] chore: address review comments --- README.md | 3 +- packages/orchestration/README.md | 99 +++++++++++++++++--------------- 2 files changed, 55 insertions(+), 47 deletions(-) diff --git a/README.md b/README.md index 0f5229ee..693cb016 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,8 @@ To test SAP Cloud SDK for AI features locally while building your application, f - Download a service key for your AI Core service instance. - Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in your local environment. -By adding the `.env` file, the SDK uses the service key to interact with the AI Core service. This setup allows you to test clients such as orchestration and OpenAI locally, provided you have deployments for orchestration and OpenAI in SAP BTP. +The SDK parses the service key from the environment variable to interact with the AI Core service. +This setup allows you to test clients such as orchestration and OpenAI locally, provided you have deployments for orchestration and OpenAI in SAP BTP. ## Support, Feedback, Contribution diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index af7d87ff..b706fb30 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -10,6 +10,10 @@ This package incorporates generative AI orchestration capabilities into your AI 4. [Usage](#usage) - [Templating](#templating) - [Content Filtering](#content-filtering) + - [Retrieving Data from the Response](#retrieving-data-from-the-response) + - [Finish Reason](#finish-reason) + - [Token Usage](#token-usage) + - [Using Resource Groups](#using-resource-groups) 5. [Local Testing](#local-testing) 6. [Support, Feedback, Contribution](#support-feedback-contribution) 7. [License](#license) @@ -125,18 +129,62 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` -#### Retrieving Data from the Response +### Content Filtering + +Use the orchestration client with filtering to restrict content that is passed to and received from a generative AI model. + +This feature allows filtering both the [input](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/consume-orchestration#content-filtering-on-input) and [output](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/consume-orchestration#content-filtering-on-input) of a model based on content safety criteria. + +```ts +import { + OrchestrationClient, + buildAzureContentFilter +} from '@sap-ai-sdk/orchestration'; -In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish and token usage. +const filter = buildAzureContentFilter({ Hate: 2, Violence: 4 }); +const orchestrationClient = new OrchestrationClient({ + llm: { + model_name: 'gpt-4-32k', + model_params: { max_tokens: 50, temperature: 0.1 } + }, + templating: { + template: [{ role: 'user', content: '{{?input}}' }] + }, + filtering: { + input: filter, + output: filter + } +}); + +try { + const response = await orchestrationClient.chatCompletion({ + inputParams: { input: 'I hate you!' } + }); + return response.getContent(); +} catch (error: any) { + return `Error: ${error.message} with underlying cause: ${error.response.data.message}`; +} +``` + +`buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs. +The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. +Each category can be configured with severity levels of 0, 2, 4, or 6. + +### Retrieving Data from the Response + +In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish reason and token usage. Use `response.rawReason` to access the complete HTTP response from the orchestration service. -##### Finish Reason +#### Finish Reason + +Finish Reason gives you the reason for stopping the chat completion request. +For example, when output gets filtered based on your configuration, the finish reason is `content_filter`. ```ts const finishReason = response.getFinishReason(); ``` -##### Token Usage +#### Token Usage To retrieve the token usage details of the orchestration request, use the following snippet: @@ -152,7 +200,7 @@ logger.info( Remember to initialize a logger before using it. -#### Using Resource Groups +### Using Resource Groups [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. @@ -173,47 +221,6 @@ const orchestrationClient = new OrchestrationClient( ); ``` -### Content Filtering - -Use the orchestration client with filtering to restrict content that is passed to and received from a generative AI model. - -This feature allows filtering both the [input](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/consume-orchestration#content-filtering-on-input) and [output](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/consume-orchestration#content-filtering-on-input) of a model based on content safety criteria. - -```ts -import { - OrchestrationClient, - buildAzureContentFilter -} from '@sap-ai-sdk/orchestration'; - -const filter = buildAzureContentFilter({ Hate: 2, Violence: 4 }); -const orchestrationClient = new OrchestrationClient({ - llm: { - model_name: 'gpt-4-32k', - model_params: { max_tokens: 50, temperature: 0.1 } - }, - templating: { - template: [{ role: 'user', content: '{{?input}}' }] - }, - filtering: { - input: filter, - output: filter - } -}); - -try { - const response = await orchestrationClient.chatCompletion({ - inputParams: { input: 'I hate you!' } - }); - return response.getContent(); -} catch (error: any) { - return `Error: ${error.message}`; -} -``` - -`buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs. -The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. -Each category can be configured with severity levels of 0, 2, 4, or 6. - ## Local Testing For local testing, refer to [here](../../README.md#local-testing). From 124a0d85d661050ab7b8b17035d690e60e0dbe32 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 15:25:30 +0200 Subject: [PATCH 18/52] chore: add deploymentID relationship --- packages/foundation-models/README.md | 24 ++++++++++++++------- packages/langchain/README.md | 31 ++++++++++++++++++++-------- packages/orchestration/README.md | 20 ++++++++++++------ 3 files changed, 53 insertions(+), 22 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 9bb11588..80e92b2a 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -6,14 +6,15 @@ This package incorporates generative AI foundation models into your AI activitie 1. [Installation](#installation) 2. [Prerequisites](#prerequisites) -3. [Usage](#usage) +3. [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) +4. [Usage](#usage) - [Client Initialization](#client-initialization) - [Azure OpenAI Client](#azure-openai-client) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) -4. [Local Testing](#local-testing) -5. [Support, Feedback, Contribution](#support-feedback-contribution) -6. [License](#license) +5. [Local Testing](#local-testing) +6. [Support, Feedback, Contribution](#support-feedback-contribution) +7. [License](#license) ## Installation @@ -28,7 +29,16 @@ $ npm install @sap-ai-sdk/foundation-models - A deployed OpenAI model in SAP Generative AI hub. - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - - Once a deployment is complete, the model can be accessed via the `deploymentUrl` + - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. + +## Relationship between Models and Deployment ID + +Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. +You can create a deployment for a model, only if you have access to the global AI scenario `foundation-models`. +Each model, model version, and resource group allows for a one-time deployment. +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. + +Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. ## Usage @@ -36,6 +46,8 @@ $ npm install @sap-ai-sdk/foundation-models You can pass the model name as a parameter to a client, the SDK will implicitly fetch the deployment ID for the model from the AI Core service and use it in the request. +A particular model it's version and resource group are tied to a deployment ID for the global AI scenario `foundation-models`. + By default, the SDK caches the deployment information, including the deployment ID, model name, and version, for 5 minutes to avoid performance issues from fetching this data with each request. ```ts @@ -50,8 +62,6 @@ const chatClient = new AzureOpenAiChatClient({ modelName: 'gpt-4o' }); const embeddingClient = new AzureOpenAiEmbeddingClient({ modelName: 'gpt-4o' }); ``` -[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. - The deployment ID and resource group can be used as an alternative to the model name for obtaining a model. ```ts diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 224292be..547c3430 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -6,13 +6,14 @@ This package provides LangChain model clients built on top of the foundation mod 1. [Installation](#installation) 2. [Prerequisites](#prerequisites) -3. [Usage](#usage) +3. [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) +4. [Usage](#usage) - [Client Initialization](#client-initialization) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) -4. [Local Testing](#local-testing) -5. [Support, Feedback, Contribution](#support-feedback-contribution) -6. [License](#license) +5. [Local Testing](#local-testing) +6. [Support, Feedback, Contribution](#support-feedback-contribution) +7. [License](#license) ## Installation @@ -23,12 +24,21 @@ $ npm install @sap-ai-sdk/langchain ## Prerequisites - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). -- Bind the service to your application. +- Bind the service to your application you are building. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. -- For testing your application locally: - - Download a service key for your AI Core service instance. - - Create a `.env` file in the root of your directory. - - Add an entry `AICORE_SERVICE_KEY=''`. +- A deployed model is available in SAP Generative AI hub. + - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. + - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. + +## Relationship between Models and Deployment ID + +Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. +You can create a deployment for a model, only if you have access to the global AI scenario `foundation-models`. +Each model, model version, and resource group allows for a one-time deployment. +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. + +Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. ## Usage @@ -61,6 +71,9 @@ const chatClient = new AzureOpenAiChatClient({ }); ``` +**Do not pass a `deployment ID` to initialize the client.** +For the LangChain model clients, initialization is done using the model name, model version and resource group. + ### Chat Client The chat client allows you to interact with Azure OpenAI chat models, accessible via the generative AI hub of SAP AI Core. diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index b706fb30..24036ece 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -7,16 +7,17 @@ This package incorporates generative AI orchestration capabilities into your AI 1. [Installation](#installation) 2. [Prerequisites](#prerequisites) 3. [Orchestration Service](#orchestration-service) -4. [Usage](#usage) +4. [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups) +5. [Usage](#usage) - [Templating](#templating) - [Content Filtering](#content-filtering) - [Retrieving Data from the Response](#retrieving-data-from-the-response) - [Finish Reason](#finish-reason) - [Token Usage](#token-usage) - [Using Resource Groups](#using-resource-groups) -5. [Local Testing](#local-testing) -6. [Support, Feedback, Contribution](#support-feedback-contribution) -7. [License](#license) +6. [Local Testing](#local-testing) +7. [Support, Feedback, Contribution](#support-feedback-contribution) +8. [License](#license) ## Installation @@ -41,6 +42,15 @@ The orchestration service provides essential features like templating and conten Find more details about orchestration workflow [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/orchestration-workflow). +## Relationship between Orchestration and Resource Groups + +Access to orchestration of generative AI models is provided under the global AI scenario `orchestration`, which is managed by SAP AI Core. +You can create a deployment to make orchestration capabilities available for use, only if you have access to the global AI scenario `orchestration`. +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. +Each resource group allows for a one-time orchestration deployment. + +Consequently, each orchestration deployment uniquely maps to a resource group within the `orchestration` scenario. + ## Usage Leverage the orchestration service capabilities by using the orchestration client. @@ -202,8 +212,6 @@ Remember to initialize a logger before using it. ### Using Resource Groups -[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. - The resource group can be used as an additional parameter to pick the right orchestration deployment. ```ts From 54f78159d1971f94f5100087b1721c4237e4f482 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 15:49:40 +0200 Subject: [PATCH 19/52] chore: address review comment --- packages/ai-api/README.md | 20 ++++++++++---------- packages/foundation-models/README.md | 22 +++++++++++----------- packages/langchain/README.md | 20 ++++++++++---------- packages/orchestration/README.md | 28 ++++++++++++++-------------- 4 files changed, 45 insertions(+), 45 deletions(-) diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index 5907487c..f70e7d81 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -11,16 +11,16 @@ We maintain a list of [currently available and tested AI Core APIs](https://gith ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Usage](#usage) - - [Create an Artifact](#create-an-artifact) - - [Create a Configuration](#create-a-configuration) - - [Create a Deployment](#create-a-deployment) - - [Delete a Deployment](#delete-a-deployment) -4. [Local Testing](#local-testing) -5. [Support, Feedback, Contribution](#support-feedback-contribution) -6. [License](#license) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Usage](#usage) + - [Create an Artifact](#create-an-artifact) + - [Create a Configuration](#create-a-configuration) + - [Create a Deployment](#create-a-deployment) + - [Delete a Deployment](#delete-a-deployment) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 80e92b2a..e028022d 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -4,17 +4,17 @@ This package incorporates generative AI foundation models into your AI activitie ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) -4. [Usage](#usage) - - [Client Initialization](#client-initialization) - - [Azure OpenAI Client](#azure-openai-client) - - [Chat Client](#chat-client) - - [Embedding Client](#embedding-client) -5. [Local Testing](#local-testing) -6. [Support, Feedback, Contribution](#support-feedback-contribution) -7. [License](#license) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) +- [Usage](#usage) + - [Client Initialization](#client-initialization) + - [Azure OpenAI Client](#azure-openai-client) + - [Chat Client](#chat-client) + - [Embedding Client](#embedding-client) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 547c3430..09d7739c 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -4,16 +4,16 @@ This package provides LangChain model clients built on top of the foundation mod ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) -4. [Usage](#usage) - - [Client Initialization](#client-initialization) - - [Chat Client](#chat-client) - - [Embedding Client](#embedding-client) -5. [Local Testing](#local-testing) -6. [Support, Feedback, Contribution](#support-feedback-contribution) -7. [License](#license) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) +- [Usage](#usage) + - [Client Initialization](#client-initialization) + - [Chat Client](#chat-client) + - [Embedding Client](#embedding-client) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 24036ece..808eb06e 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -4,20 +4,20 @@ This package incorporates generative AI orchestration capabilities into your AI ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Orchestration Service](#orchestration-service) -4. [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups) -5. [Usage](#usage) - - [Templating](#templating) - - [Content Filtering](#content-filtering) - - [Retrieving Data from the Response](#retrieving-data-from-the-response) - - [Finish Reason](#finish-reason) - - [Token Usage](#token-usage) - - [Using Resource Groups](#using-resource-groups) -6. [Local Testing](#local-testing) -7. [Support, Feedback, Contribution](#support-feedback-contribution) -8. [License](#license) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Orchestration Service](#orchestration-service) +- [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups) +- [Usage](#usage) + - [Templating](#templating) + - [Content Filtering](#content-filtering) + - [Retrieving Data from the Response](#retrieving-data-from-the-response) + - [Finish Reason](#finish-reason) + - [Token Usage](#token-usage) + - [Using Resource Groups](#using-resource-groups) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation From 0f34a2dbd0f5b72cd2c2e00cd3234471e809bb9a Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 16:20:43 +0200 Subject: [PATCH 20/52] chore: address review comments --- README.md | 8 ++++---- packages/ai-api/README.md | 2 +- packages/foundation-models/README.md | 2 +- packages/langchain/README.md | 2 +- packages/orchestration/README.md | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 693cb016..2d717f09 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc $ npm install @sap-ai-sdk/orchestration ``` -For details on Orchestration client usage, refer to [here](./packages/orchestration/README.md). +For details on Orchestration client, refer to this [document](./packages/orchestration/README.md). ### @sap-ai-sdk/ai-api @@ -50,7 +50,7 @@ This package provides tools to manage your scenarios and workflows in SAP AI Cor $ npm install @sap-ai-sdk/ai-api ``` -For details on client usage, refer to [here](./packages/ai-api/README.md). +For details on the client, refer to this [document](./packages/ai-api/README.md). ### @sap-ai-sdk/foundation-models @@ -62,7 +62,7 @@ This package incorporates generative AI foundation models into your AI activitie $ npm install @sap-ai-sdk/foundation-models ``` -For details on Azure OpenAI client usage, refer to [here](./packages/foundation-models/README.md). +For details on Azure OpenAI client, refer to this [document](./packages/foundation-models/README.md). ### @sap-ai-sdk/langchain @@ -74,7 +74,7 @@ This package provides LangChain model clients, built on top of the foundation mo $ npm install @sap-ai-sdk/langchain ``` -For details on LangChain model client usage, refer to [here](./packages/langchain/README.md). +For details on LangChain model client, refer to this [document](./packages/langchain/README.md). ## SAP Cloud SDK for AI Sample Project diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index f70e7d81..c45af2ec 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -160,7 +160,7 @@ async function modifyDeployment() { ## Local Testing -For local testing, refer to [here](../../README.md#local-testing). +For local testing instructions, refer to this [section](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index e028022d..488f8198 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -153,7 +153,7 @@ const embedding = response.getEmbedding(); ## Local Testing -For local testing instuctions, refer to [here](../../README.md#local-testing). +For local testing instuctions, refer to this [section](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 09d7739c..729bf8e7 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -126,7 +126,7 @@ const embeddedDocument = await embeddingClient.embedDocuments([ ## Local Testing -For local testing, refer to [here](../../README.md#local-testing). +For local testing instructions, refer to this [section](../../README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 808eb06e..d89479fd 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -231,7 +231,7 @@ const orchestrationClient = new OrchestrationClient( ## Local Testing -For local testing, refer to [here](../../README.md#local-testing). +For local testing instructions, refer to this [section](../../README.md#local-testing). ## Support, Feedback, Contribution From d5856df09eb61a54226b5595a2460cfb2ccd6277 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 16:25:05 +0200 Subject: [PATCH 21/52] chore: remove usage of you --- README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2d717f09..f74361d7 100644 --- a/README.md +++ b/README.md @@ -82,13 +82,12 @@ We have created a sample project demonstrating the different clients' usage of t ## Local Testing -To test SAP Cloud SDK for AI features locally while building your application, follow these steps: +To test SAP Cloud SDK for AI features locally during application development, follow these steps: -- Download a service key for your AI Core service instance. -- Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in your local environment. +- Download a service key for the AI Core service instance. +- Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in the local environment. -The SDK parses the service key from the environment variable to interact with the AI Core service. -This setup allows you to test clients such as orchestration and OpenAI locally, provided you have deployments for orchestration and OpenAI in SAP BTP. +The SDK parses the service key from the environment variable to interact with the AI Core service. This setup enables local testing of clients such as orchestration and OpenAI, provided that deployments for orchestration and OpenAI exist in SAP BTP. ## Support, Feedback, Contribution From 45b547e5214cbe9e5840983522dadafabd15f292 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 16:33:03 +0200 Subject: [PATCH 22/52] chore: remove usages of you. --- packages/foundation-models/README.md | 2 +- packages/langchain/README.md | 3 ++- packages/orchestration/README.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 488f8198..d38dee65 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -34,7 +34,7 @@ $ npm install @sap-ai-sdk/foundation-models ## Relationship between Models and Deployment ID Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. -You can create a deployment for a model, only if you have access to the global AI scenario `foundation-models`. +A deployment can be created for a model only with access to the global AI scenario `foundation-models`. Each model, model version, and resource group allows for a one-time deployment. [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 729bf8e7..1b435b33 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -34,12 +34,13 @@ $ npm install @sap-ai-sdk/langchain ## Relationship between Models and Deployment ID Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. -You can create a deployment for a model, only if you have access to the global AI scenario `foundation-models`. +A deployment can be created for a model only with access to the global AI scenario `foundation-models`. Each model, model version, and resource group allows for a one-time deployment. [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. + ## Usage This package offers both chat and embedding clients, currently supporting Azure OpenAI. diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index d89479fd..d2e4776c 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -45,7 +45,7 @@ Find more details about orchestration workflow [here](https://help.sap.com/docs/ ## Relationship between Orchestration and Resource Groups Access to orchestration of generative AI models is provided under the global AI scenario `orchestration`, which is managed by SAP AI Core. -You can create a deployment to make orchestration capabilities available for use, only if you have access to the global AI scenario `orchestration`. +A deployment can be created to enable orchestration capabilities only with access to the global AI scenario `orchestration`. [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Each resource group allows for a one-time orchestration deployment. From 2e29d748abdeff644c3579a50bd32084da0604a2 Mon Sep 17 00:00:00 2001 From: cloud-sdk-js Date: Wed, 25 Sep 2024 14:34:04 +0000 Subject: [PATCH 23/52] fix: Changes from lint --- packages/langchain/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 1b435b33..1ac4e77d 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -40,7 +40,6 @@ Each model, model version, and resource group allows for a one-time deployment. Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. - ## Usage This package offers both chat and embedding clients, currently supporting Azure OpenAI. From 8fd9186f00a0ca8c4622166cf92da5c4d514b3a8 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 16:37:37 +0200 Subject: [PATCH 24/52] chore: remove more usages of you --- packages/orchestration/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index d2e4776c..dcbd1b03 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -87,7 +87,7 @@ const responseContent = response.getContent(); #### Customizing the Request Configuration -To customize the request configuration, for example if you want to pass additional headers to the client, use the second parameter in the `chatCompletion()` method: +To customize the request configuration, for example to pass additional headers to the client, use the second parameter in the `chatCompletion()` method: ```ts const response = await orchestrationClient.chatCompletion( @@ -182,13 +182,13 @@ Each category can be configured with severity levels of 0, 2, 4, or 6. ### Retrieving Data from the Response -In addition to `response.getContent()`, you can use other available convenience methods to retrieve the finish reason and token usage. +In addition to `response.getContent()`, other available convenience methods can retrieve the finish reason and token usage. Use `response.rawReason` to access the complete HTTP response from the orchestration service. #### Finish Reason -Finish Reason gives you the reason for stopping the chat completion request. -For example, when output gets filtered based on your configuration, the finish reason is `content_filter`. +The Finish Reason indicates the reason for stopping the chat completion request. +For example, when output gets filtered based on the configuration, the finish reason is `content_filter`. ```ts const finishReason = response.getFinishReason(); From 408b0a145ebaf809335cd28b049759be669b74fb Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 17:16:11 +0200 Subject: [PATCH 25/52] chore: address review comment --- packages/orchestration/README.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index dcbd1b03..ea0e045a 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -172,17 +172,27 @@ try { }); return response.getContent(); } catch (error: any) { - return `Error: ${error.message} with underlying cause: ${error.response.data.message}`; + return `Error: ${error.message}`; } ``` +In the above code snippet, both `orchestrationClient.chatCompletion()` and `response.getContent()` can throw errors. + +- **Axios Errors**: + When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause. + +- **Output Content Filtered**: + The method `response.getContent()` can throw an error if the content filter configuration filters the output. This situation can occur even if the initial request succeeds. The `error.message` property indicates whether the output was filtered. + +Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors. + `buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs. The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. Each category can be configured with severity levels of 0, 2, 4, or 6. ### Retrieving Data from the Response -In addition to `response.getContent()`, other available convenience methods can retrieve the finish reason and token usage. +In addition to `getContent()`, other available convenience methods can retrieve the finish reason and token usage. Use `response.rawReason` to access the complete HTTP response from the orchestration service. #### Finish Reason From 756b5dff5296cb588c877c064ce26ba4c808c998 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 25 Sep 2024 17:22:19 +0200 Subject: [PATCH 26/52] chore: minor cleanup --- README.md | 3 ++- packages/foundation-models/README.md | 2 -- packages/langchain/README.md | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index f74361d7..2141e785 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,8 @@ To test SAP Cloud SDK for AI features locally during application development, fo - Download a service key for the AI Core service instance. - Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in the local environment. -The SDK parses the service key from the environment variable to interact with the AI Core service. This setup enables local testing of clients such as orchestration and OpenAI, provided that deployments for orchestration and OpenAI exist in SAP BTP. +The SDK parses the service key from the environment variable to interact with the AI Core service. +This setup enables local testing of clients such as orchestration and OpenAI, provided that deployments for orchestration and OpenAI exist in SAP BTP. ## Support, Feedback, Contribution diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index d38dee65..4d593182 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -46,8 +46,6 @@ Consequently, each deployment ID and resource group uniquely map to a combinatio You can pass the model name as a parameter to a client, the SDK will implicitly fetch the deployment ID for the model from the AI Core service and use it in the request. -A particular model it's version and resource group are tied to a deployment ID for the global AI scenario `foundation-models`. - By default, the SDK caches the deployment information, including the deployment ID, model name, and version, for 5 minutes to avoid performance issues from fetching this data with each request. ```ts diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 1ac4e77d..0b2e906a 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -24,7 +24,7 @@ $ npm install @sap-ai-sdk/langchain ## Prerequisites - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). -- Bind the service to your application you are building. +- Bind the service to your application. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. - A deployed model is available in SAP Generative AI hub. - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). From 129bdeb2feca0bedeb3fd6b574a092a66b236b85 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Thu, 26 Sep 2024 18:29:31 +0200 Subject: [PATCH 27/52] Update packages/orchestration/README.md Co-authored-by: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 6db6966d..123bd32f 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -234,7 +234,7 @@ Use `response.rawReason` to access the complete HTTP response from the orchestra #### Finish Reason -The Finish Reason indicates the reason for stopping the chat completion request. +The finish reason indicates the reason for stopping the chat completion request. For example, when output gets filtered based on the configuration, the finish reason is `content_filter`. ```ts From 0a2a76875799f7bfe91103358731d92d5dfac24d Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Fri, 27 Sep 2024 12:35:57 +0200 Subject: [PATCH 28/52] Update packages/orchestration/README.md Co-authored-by: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 123bd32f..43bcc5a0 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -235,7 +235,7 @@ Use `response.rawReason` to access the complete HTTP response from the orchestra #### Finish Reason The finish reason indicates the reason for stopping the chat completion request. -For example, when output gets filtered based on the configuration, the finish reason is `content_filter`. +For example, when output is filtered based on the configuration, the finish reason is `content_filter`. ```ts const finishReason = response.getFinishReason(); From 728abe29b2c5d2f480aef96e23a7ec286d7cf4e0 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Fri, 27 Sep 2024 12:36:11 +0200 Subject: [PATCH 29/52] Update packages/orchestration/README.md Co-authored-by: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 43bcc5a0..80147be5 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -88,7 +88,7 @@ const responseContent = response.getContent(); #### Customizing the Request Configuration -To customize the request configuration, for example to pass additional headers to the client, use the second parameter in the `chatCompletion()` method: +To pass custom request configuration parameters (e.g., headers) to the client, use the second parameter in the `chatCompletion()` method: ```ts const response = await orchestrationClient.chatCompletion( From 267a6f7d9a59d12b0d0ebf882a6fd3f8a73c49f8 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 15:38:58 +0200 Subject: [PATCH 30/52] chore: cleanup links --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 2141e785..3c4e3403 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc $ npm install @sap-ai-sdk/orchestration ``` -For details on Orchestration client, refer to this [document](./packages/orchestration/README.md). +For details on Orchestration client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/orchestration/README.md). ### @sap-ai-sdk/ai-api @@ -50,7 +50,7 @@ This package provides tools to manage your scenarios and workflows in SAP AI Cor $ npm install @sap-ai-sdk/ai-api ``` -For details on the client, refer to this [document](./packages/ai-api/README.md). +For details on the client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md). ### @sap-ai-sdk/foundation-models @@ -62,7 +62,7 @@ This package incorporates generative AI foundation models into your AI activitie $ npm install @sap-ai-sdk/foundation-models ``` -For details on Azure OpenAI client, refer to this [document](./packages/foundation-models/README.md). +For details on Azure OpenAI client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/foundation-models/README.md). ### @sap-ai-sdk/langchain @@ -74,11 +74,11 @@ This package provides LangChain model clients, built on top of the foundation mo $ npm install @sap-ai-sdk/langchain ``` -For details on LangChain model client, refer to this [document](./packages/langchain/README.md). +For details on LangChain model client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/langchain/README.md). ## SAP Cloud SDK for AI Sample Project -We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](./sample-code/README.md) outlines the set-up needed to build and run it locally. +We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](https://github.com/SAP/ai-sdk-js/blob/main/sample-code/README.md) outlines the set-up needed to build and run it locally. ## Local Testing @@ -94,7 +94,7 @@ This setup enables local testing of clients such as orchestration and OpenAI, pr This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). -Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](CONTRIBUTING.md). +Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md). ## Security / Disclosure From 53b14e2133504198de8fa6d37e774154b210090f Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 15:54:12 +0200 Subject: [PATCH 31/52] chore: cleanup links --- packages/foundation-models/README.md | 9 +++++---- packages/langchain/README.md | 9 +++++---- packages/orchestration/README.md | 9 +++++---- 3 files changed, 15 insertions(+), 12 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 4d593182..9ee5b110 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -27,14 +27,15 @@ $ npm install @sap-ai-sdk/foundation-models - [Enable the AI Core service in BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - A deployed OpenAI model in SAP Generative AI hub. - - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. ## Relationship between Models and Deployment ID -Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. -A deployment can be created for a model only with access to the global AI scenario `foundation-models`. +SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`. +Creating a deployment for a model requires access to this scenario. + Each model, model version, and resource group allows for a one-time deployment. [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. @@ -151,7 +152,7 @@ const embedding = response.getEmbedding(); ## Local Testing -For local testing instuctions, refer to this [section](../../README.md#local-testing). +For local testing instuctions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/langchain/README.md b/packages/langchain/README.md index fe3c5722..6e9b0bac 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -27,14 +27,15 @@ $ npm install @sap-ai-sdk/langchain - Bind the service to your application. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. - A deployed model is available in SAP Generative AI hub. - - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. ## Relationship between Models and Deployment ID -Access to generative AI models is provided under the global AI scenario `foundation-models`, which is managed by SAP AI Core. -A deployment can be created for a model only with access to the global AI scenario `foundation-models`. +SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`. +Creating a deployment for a model requires access to this scenario. + Each model, model version, and resource group allows for a one-time deployment. [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. @@ -171,7 +172,7 @@ const retriever = vectorStore.asRetriever(); ## Local Testing -For local testing instructions, refer to this [section](../../README.md#local-testing). +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 80147be5..f87f290d 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -31,7 +31,7 @@ $ npm install @sap-ai-sdk/orchestration - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - An orchestration deployment is running. - - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). - Once a deployment is complete, the orchestration service can be accessed via the `deploymentUrl`. ## Orchestration Service @@ -45,8 +45,9 @@ Find more details about orchestration workflow [here](https://help.sap.com/docs/ ## Relationship between Orchestration and Resource Groups -Access to orchestration of generative AI models is provided under the global AI scenario `orchestration`, which is managed by SAP AI Core. -A deployment can be created to enable orchestration capabilities only with access to the global AI scenario `orchestration`. +SAP AI Core manages access to orchestration of generative AI models through the global AI scenario `orchestration`. +Creating a deployment for enabling orchestration capabilities requires access to this scenario. + [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Each resource group allows for a one-time orchestration deployment. @@ -278,7 +279,7 @@ const orchestrationClient = new OrchestrationClient( ## Local Testing -For local testing instructions, refer to this [section](../../README.md#local-testing). +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution From cd54014319cff16c9276334809475b244b2d3a3d Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 16:08:11 +0200 Subject: [PATCH 32/52] chore: explain deployment id --- packages/foundation-models/README.md | 1 + packages/langchain/README.md | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 9ee5b110..050352c4 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -37,6 +37,7 @@ SAP AI Core manages access to generative AI models through the global AI scenari Creating a deployment for a model requires access to this scenario. Each model, model version, and resource group allows for a one-time deployment. +After deployment completion, the response includes a deploymentUrl and an 'id', which is the Deployment ID (id). For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 6e9b0bac..e9503d3b 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -28,7 +28,6 @@ $ npm install @sap-ai-sdk/langchain - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. - A deployed model is available in SAP Generative AI hub. - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). - Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. ## Relationship between Models and Deployment ID @@ -37,6 +36,8 @@ SAP AI Core manages access to generative AI models through the global AI scenari Creating a deployment for a model requires access to this scenario. Each model, model version, and resource group allows for a one-time deployment. +After deployment completion, the response includes a deploymentUrl and an 'id', which is the Deployment ID (id). For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. From b379d9ebe228d55d658f64f6b5e23788fce4db4e Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 16:10:48 +0200 Subject: [PATCH 33/52] chore: explain deployment ID --- packages/foundation-models/README.md | 2 +- packages/langchain/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 050352c4..573699e7 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -37,7 +37,7 @@ SAP AI Core manages access to generative AI models through the global AI scenari Creating a deployment for a model requires access to this scenario. Each model, model version, and resource group allows for a one-time deployment. -After deployment completion, the response includes a deploymentUrl and an 'id', which is the Deployment ID (id). For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). +After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. diff --git a/packages/langchain/README.md b/packages/langchain/README.md index e9503d3b..24430239 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -36,7 +36,7 @@ SAP AI Core manages access to generative AI models through the global AI scenari Creating a deployment for a model requires access to this scenario. Each model, model version, and resource group allows for a one-time deployment. -After deployment completion, the response includes a deploymentUrl and an 'id', which is the Deployment ID (id). For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). +After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). [Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. From 40e7409608ba0fae010a1f41f45263fdd06ba4d4 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 16:16:34 +0200 Subject: [PATCH 34/52] chore: fix toc --- packages/orchestration/README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index f87f290d..8b3e0849 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -13,9 +13,7 @@ This package incorporates generative AI orchestration capabilities into your AI - [Content Filtering](#content-filtering) - [Data Masking](#data-masking) - [Retrieving Data from the Response](#retrieving-data-from-the-response) - - [Finish Reason](#finish-reason) - - [Token Usage](#token-usage) - - [Using Resource Groups](#using-resource-groups) + - [Using Resource Groups](#using-resource-groups) - [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) - [License](#license) From 688af4193f4c5bbc013c286d53ffa72526e07270 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Fri, 27 Sep 2024 16:16:59 +0200 Subject: [PATCH 35/52] Update packages/orchestration/README.md Co-authored-by: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 8b3e0849..27390638 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -83,7 +83,7 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` -`response.getContent()` is a convenience method that parses the response and returns the model's output as a string. +`getContent()` is a convenience method that parses the response and returns the model's output as a string. #### Customizing the Request Configuration From 9e26d5aaf5116de968c094669c1aa1d826560c69 Mon Sep 17 00:00:00 2001 From: KavithaSiva <32287936+KavithaSiva@users.noreply.github.com> Date: Fri, 27 Sep 2024 16:17:19 +0200 Subject: [PATCH 36/52] Update packages/orchestration/README.md Co-authored-by: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 27390638..96c33ad1 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -176,7 +176,7 @@ try { } ``` -In the above code snippet, both `orchestrationClient.chatCompletion()` and `response.getContent()` can throw errors. +In the above code snippet, both `orchestrationClient.chatCompletion()` and `response.getContent()` methods can throw errors. - **Axios Errors**: When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause. From d7f952255040e9ba4a909f5e4cef1791c57cc6f0 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 17:20:02 +0200 Subject: [PATCH 37/52] chore: address review comments --- packages/orchestration/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 96c33ad1..bf0bf97b 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -176,13 +176,13 @@ try { } ``` -In the above code snippet, both `orchestrationClient.chatCompletion()` and `response.getContent()` methods can throw errors. +In the above code snippet, both `chatCompletion()` and `getContent()` methods can throw errors. - **Axios Errors**: When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause. - **Output Content Filtered**: - The method `response.getContent()` can throw an error if the content filter configuration filters the output. This situation can occur even if the initial request succeeds. The `error.message` property indicates whether the output was filtered. + The method `getContent()` can throw an error if the output filter filters the model output. This situation can occur even if the chat completion request succeeds with a `200` status code. The `error.message` property indicates whether the output was filtered. Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors. From 69afd57b97718ed9fbcca11f8616449e72fe01f3 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Fri, 27 Sep 2024 17:27:30 +0200 Subject: [PATCH 38/52] chore: final review comment --- packages/orchestration/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index bf0bf97b..44ee8a64 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -275,6 +275,8 @@ const orchestrationClient = new OrchestrationClient( ); ``` +The relationship between orchestration and resource groups is explained in this [section](#relationship-between-orchestration-and-resource-groups). + ## Local Testing For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). From 508fee5c7b4f524d6ef9228bf64a13c986da4807 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 2 Oct 2024 11:03:28 +0200 Subject: [PATCH 39/52] chore: address review comments --- packages/ai-api/README.md | 2 +- packages/foundation-models/README.md | 3 ++- packages/langchain/README.md | 3 ++- packages/orchestration/README.md | 7 +++---- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index 8cb3d7e9..4f7fcdb5 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -172,7 +172,7 @@ async function modifyDeployment() { ## Local Testing -For local testing instructions, refer to this [section](../../README.md#local-testing). +For local testing instuctions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 573699e7..72d0e4a1 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -27,7 +27,8 @@ $ npm install @sap-ai-sdk/foundation-models - [Enable the AI Core service in BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - A deployed OpenAI model in SAP Generative AI hub. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 24430239..6432c71c 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -27,7 +27,8 @@ $ npm install @sap-ai-sdk/langchain - Bind the service to your application. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. - A deployed model is available in SAP Generative AI hub. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. ## Relationship between Models and Deployment ID diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 44ee8a64..6ebd72f6 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -29,7 +29,8 @@ $ npm install @sap-ai-sdk/orchestration - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - An orchestration deployment is running. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). - Once a deployment is complete, the orchestration service can be accessed via the `deploymentUrl`. ## Orchestration Service @@ -247,15 +248,13 @@ To retrieve the token usage details of the orchestration request, use the follow ```ts const tokenUsage = response.getTokenUsage(); -logger.info( +console.log( `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` + `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` + `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n` ); ``` -Remember to initialize a logger before using it. - ### Using Resource Groups The resource group can be used as an additional parameter to pick the right orchestration deployment. From 697e2c9c9ce0b8d6490ce57f91e2bb34170c55e0 Mon Sep 17 00:00:00 2001 From: cloud-sdk-js Date: Wed, 2 Oct 2024 09:04:26 +0000 Subject: [PATCH 40/52] fix: Changes from lint --- packages/foundation-models/README.md | 4 ++-- packages/langchain/README.md | 4 ++-- packages/orchestration/README.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 72d0e4a1..3d99ab63 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -27,8 +27,8 @@ $ npm install @sap-ai-sdk/foundation-models - [Enable the AI Core service in BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - A deployed OpenAI model in SAP Generative AI hub. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. - For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. diff --git a/packages/langchain/README.md b/packages/langchain/README.md index 6432c71c..3884e270 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -27,8 +27,8 @@ $ npm install @sap-ai-sdk/langchain - Bind the service to your application. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. - A deployed model is available in SAP Generative AI hub. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. - For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. ## Relationship between Models and Deployment ID diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 6ebd72f6..ee8d9d64 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -29,8 +29,8 @@ $ npm install @sap-ai-sdk/orchestration - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - An orchestration deployment is running. - - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. - For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). - Once a deployment is complete, the orchestration service can be accessed via the `deploymentUrl`. ## Orchestration Service From 14259774d55470908e605eaa6c62c6a773e29c58 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 2 Oct 2024 11:25:56 +0200 Subject: [PATCH 41/52] chore: remove requestConfig section --- packages/orchestration/README.md | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 6ebd72f6..871dc394 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -86,21 +86,6 @@ const responseContent = response.getContent(); `getContent()` is a convenience method that parses the response and returns the model's output as a string. -#### Customizing the Request Configuration - -To pass custom request configuration parameters (e.g., headers) to the client, use the second parameter in the `chatCompletion()` method: - -```ts -const response = await orchestrationClient.chatCompletion( - { - inputParams: { country: 'France' } - }, - { - headers: { 'x-custom-header': 'custom-value' } - } -); -``` - #### Passing a Message History It is possible to provide a history of a conversation to the model. From 831b355ccb9fa7558766f7dd073a1a7d8be1095b Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 2 Oct 2024 15:27:37 +0200 Subject: [PATCH 42/52] chore: cleanup convenience functions usage --- packages/orchestration/README.md | 47 ++++++++++++-------------------- 1 file changed, 18 insertions(+), 29 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index f382de32..0d6161d0 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -12,7 +12,6 @@ This package incorporates generative AI orchestration capabilities into your AI - [Templating](#templating) - [Content Filtering](#content-filtering) - [Data Masking](#data-masking) - - [Retrieving Data from the Response](#retrieving-data-from-the-response) - [Using Resource Groups](#using-resource-groups) - [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) @@ -115,6 +114,24 @@ const responseContent = response.getContent(); `getContent()` is a convenience method that parses the response and returns the model's output as a string. +```ts +const finishReason = response.getFinishReason(); +``` + +`getFinishReason()` is a convenience method that specifies the reason for stopping the chat completion request. + +```ts +const tokenUsage = response.getTokenUsage(); + +console.log( + `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` + + `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` + + `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n` +); +``` + +`getTokenUsage()` is a convenience method that retrieves the token usage details of the chat completion request. + #### Passing a Message History It is possible to provide a history of a conversation to the model. @@ -241,34 +258,6 @@ const response = await orchestrationClient.chatCompletion({ return response.getContent(); ``` -### Retrieving Data from the Response - -In addition to `getContent()`, other available convenience methods can retrieve the finish reason and token usage. -Use `response.rawReason` to access the complete HTTP response from the orchestration service. - -#### Finish Reason - -The finish reason indicates the reason for stopping the chat completion request. -For example, when output is filtered based on the configuration, the finish reason is `content_filter`. - -```ts -const finishReason = response.getFinishReason(); -``` - -#### Token Usage - -To retrieve the token usage details of the orchestration request, use the following snippet: - -```ts -const tokenUsage = response.getTokenUsage(); - -console.log( - `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` + - `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` + - `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n` -); -``` - ### Using Resource Groups The resource group can be used as an additional parameter to pick the right orchestration deployment. From 5358ad1e363f8521cb59bae1f6ea76ea0f3428b0 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 2 Oct 2024 15:33:10 +0200 Subject: [PATCH 43/52] chore: fix toc --- packages/orchestration/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 0d6161d0..1f7cd972 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -5,7 +5,7 @@ This package incorporates generative AI orchestration capabilities into your AI ## Table of Contents - [Installation](#installation) - - [Prerequisites](#prerequisites) +- [Prerequisites](#prerequisites) - [Orchestration Service](#orchestration-service) - [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups) - [Usage](#usage) @@ -23,7 +23,7 @@ This package incorporates generative AI orchestration capabilities into your AI $ npm install @sap-ai-sdk/orchestration ``` -### Prerequisites +## Prerequisites - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. From bce751d6e3af1361954c93d9934b2c1b9023bee2 Mon Sep 17 00:00:00 2001 From: KavithaSiva Date: Wed, 2 Oct 2024 15:34:35 +0200 Subject: [PATCH 44/52] chore: fix toc --- packages/ai-api/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index 4f7fcdb5..f869329c 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -12,6 +12,7 @@ We maintain a list of [currently available and tested AI Core APIs](https://gith ## Table of Contents - [Installation](#installation) +- [Version Management](#version-management) - [Prerequisites](#prerequisites) - [Usage](#usage) - [Create an Artifact](#create-an-artifact) From 57c02a0ded37eabe203e9db1fc4b95364300fc1f Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:35:56 +0200 Subject: [PATCH 45/52] Update README.md Co-authored-by: Zhongpin Wang --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3c4e3403..86431254 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc $ npm install @sap-ai-sdk/orchestration ``` -For details on Orchestration client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/orchestration/README.md). +For details on orchestration client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/orchestration/README.md). ### @sap-ai-sdk/ai-api From 37f23a7f9c2e963ee3fbefd4e6e0f35c989c3d8d Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:38:21 +0200 Subject: [PATCH 46/52] Apply suggestions from code review Co-authored-by: Zhongpin Wang --- README.md | 2 +- packages/ai-api/README.md | 2 +- packages/foundation-models/README.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 86431254..26acfce1 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ This package incorporates generative AI foundation models into your AI activitie $ npm install @sap-ai-sdk/foundation-models ``` -For details on Azure OpenAI client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/foundation-models/README.md). +For details on foundation model clients, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/foundation-models/README.md). ### @sap-ai-sdk/langchain diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index f869329c..5e3c5d8b 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -173,7 +173,7 @@ async function modifyDeployment() { ## Local Testing -For local testing instuctions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index a03abca2..f9d6df64 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -167,7 +167,7 @@ Define optional request configuration when running the client, similar to the ch ## Local Testing -For local testing instuctions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). ## Support, Feedback, Contribution From 8188acc622cdc97880004e402d4995133a5bd455 Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:52:43 +0200 Subject: [PATCH 47/52] Apply suggestions from code review --- packages/orchestration/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 1f7cd972..d79f70d5 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -114,11 +114,12 @@ const responseContent = response.getContent(); `getContent()` is a convenience method that parses the response and returns the model's output as a string. +To retrieve the `finish_reason` for stopping the chat completion request, use the convenience method `getFinishReason()`: + ```ts const finishReason = response.getFinishReason(); ``` -`getFinishReason()` is a convenience method that specifies the reason for stopping the chat completion request. ```ts const tokenUsage = response.getTokenUsage(); @@ -130,7 +131,6 @@ console.log( ); ``` -`getTokenUsage()` is a convenience method that retrieves the token usage details of the chat completion request. #### Passing a Message History From 05d15b8cf733421fd10390b798665cf51378f7a5 Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:53:12 +0200 Subject: [PATCH 48/52] Update packages/orchestration/README.md --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index d79f70d5..68869528 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -208,7 +208,7 @@ try { } ``` -In the above code snippet, both `chatCompletion()` and `getContent()` methods can throw errors. +Both `chatCompletion()` and `getContent()` methods can throw errors. - **Axios Errors**: When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause. From 59cb45ad79cc1c77bd0e04699394d56a695144c1 Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:54:22 +0200 Subject: [PATCH 49/52] Update packages/orchestration/README.md --- packages/orchestration/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 68869528..c34c98c8 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -120,6 +120,7 @@ To retrieve the `finish_reason` for stopping the chat completion request, use th const finishReason = response.getFinishReason(); ``` +Use the `getTokenUsage()` convenience method to retrieve the token usage details of the chat completion request: ```ts const tokenUsage = response.getTokenUsage(); From 183827894f0034f0deec24d6a2097e618caa9b1f Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:54:49 +0200 Subject: [PATCH 50/52] Update packages/orchestration/README.md --- packages/orchestration/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index c34c98c8..0811db4f 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -132,7 +132,6 @@ console.log( ); ``` - #### Passing a Message History It is possible to provide a history of a conversation to the model. From 9eff8b0e8ef2ee617a273810d0e3fa8bc58370fe Mon Sep 17 00:00:00 2001 From: Deeksha Sinha <88374536+deekshas8@users.noreply.github.com> Date: Wed, 2 Oct 2024 16:57:30 +0200 Subject: [PATCH 51/52] Apply suggestions from code review --- packages/orchestration/README.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 0811db4f..93fb5d6f 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -211,10 +211,13 @@ try { Both `chatCompletion()` and `getContent()` methods can throw errors. - **Axios Errors**: - When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause. + When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. + The property `error.response.data.message` may provide additional details about the failure's cause. - **Output Content Filtered**: - The method `getContent()` can throw an error if the output filter filters the model output. This situation can occur even if the chat completion request succeeds with a `200` status code. The `error.message` property indicates whether the output was filtered. + The method `getContent()` can throw an error if the output filter filters the model output. + This can occur even if the chat completion request responds with a `200` HTTP status code. + The `error.message` property indicates if the output was filtered. Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors. From 3a70dc7ced5aaf6e59f3785e811eb654e49a6864 Mon Sep 17 00:00:00 2001 From: cloud-sdk-js Date: Fri, 4 Oct 2024 07:37:27 +0000 Subject: [PATCH 52/52] fix: Changes from lint --- packages/orchestration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 972618d8..ab3b76db 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -306,4 +306,4 @@ Contribution and feedback are encouraged and always welcome. For more informatio ## License -The SAP Cloud SDK for AI is released under the [Apache License Version 2.0](http://www.apache.org/licenses/). \ No newline at end of file +The SAP Cloud SDK for AI is released under the [Apache License Version 2.0](http://www.apache.org/licenses/).