Add agent framework/throttling/hidden model/OS assistant and update conversational search documentation

.github/vale/styles/Vocab/OpenSearch/Products/accept.txt

@@ -59,6 +59,7 @@ Open Distro
 OpenAI
 OpenID Connect
 OpenSearch
+OpenSearch Assistant
 OpenSearch Benchmark
 OpenSearch Dashboards
 OpenSearch Playground

_ml-commons-plugin/agents-tools/agents-tools-tutorial.md

@@ -0,0 +1,353 @@
+---
+layout: default
+title: Agents and tools tutorial
+parent: Agents and tools
+grand_parent: ML Commons APIs
+nav_order: 10
+---
+
+# Agents and tools tutorial
+**Introduced 2.12**
+{: .label .label-purple }
+
+This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/ml-commons/issues/1161).    
+{: .warning}
+
+The following tutorial illustrates creating a flow agent for retrieval-augmented generation (RAG). A flow agent runs its configured tools sequentially, in the order specified. In this example, you'll create an agent with two tools:
+
+1. `VectorDBTool`: The agent will use this tool to retrieve OpenSearch documents relevant to the user question. You'll ingest supplementary information to an OpenSearch index. To facilitate vector search, you'll deploy a text embedding model that translates text into vector embeddings. OpenSearch will translate the ingested documents into embeddings and store them in the index. When you provide a user question to the agent, the agent will construct a query from the question, run vector search on the OpenSearch index, and pass the retrieved relevant documents to the `MLModelTool`.
+1. `MLModelTool`: The agent will run this tool to connect to a large language model (LLM) and send the user query augmented with OpenSearch documents to the model. In this example, you'll use the Anthropic Claude model hosted on Amazon Bedrock. The LLM will then answer the question based on its knowledge and the provided documents.
+
+## Prerequisites
+
+To use the memory feature, fist configure the following cluster settings:
+
+```json
+PUT _cluster/settings
+{
+  "persistent": {
+    "plugins.ml_commons.only_run_on_ml_node": "false",
+    "plugins.ml_commons.memory_feature_enabled": "true"
+  }
+}
+```
+{% include copy-curl.html %}
+
+## Step 1: Register and deploy a text embedding model
+
+You need a text embedding model to facilitate vector search. For this tutorial, you'll use one of the OpenSearch-provided pretrained models. To register and deploy the model, send the following request:
+
+```json
+POST /_plugins/_ml/models/_register?deploy=true
+{
+  "name": "huggingface/sentence-transformers/all-MiniLM-L12-v2",
+  "version": "1.0.1",
+  "model_format": "TORCH_SCRIPT"
+}
+```
+{% include copy-curl.html %}
+
+Registering a model is an asynchronous task. OpenSearch sends back a task ID for this task:
+
+```json
+{
+  "task_id": "aFeif4oB5Vm0Tdw8yoN7",
+  "status": "CREATED"
+}
+```
+
+You can check the status of the task by calling the Tasks API:
+
+```json
+GET /_plugins/_ml/tasks/aFeif4oB5Vm0Tdw8yoN7
+```
+{% include copy-curl.html %}
+
+Once the task is complete, the task state changes to `COMPLETED` and the Tasks API response contains a model ID for the deployed model:
+
+```json
+{
+  "model_id": "aVeif4oB5Vm0Tdw8zYO2",
+  "task_type": "REGISTER_MODEL",
+  "function_name": "TEXT_EMBEDDING",
+  "state": "COMPLETED",
+  "worker_node": [
+    "4p6FVOmJRtu3wehDD74hzQ"
+  ],
+  "create_time": 1694358489722,
+  "last_update_time": 1694358499139,
+  "is_async": true
+}
+```
+
+## Step 2: Create an ingest pipeline
+
+To translate text into vector embeddings, you'll set up an ingest pipeline. The pipeline translates the `text` field and writes the resulting vector embeddings into the `embedding` field. Create the pipeline by specifying the `model_id` from the previous step in the following request:
+
+```json
+PUT /_ingest/pipeline/test-pipeline-local-model
+{
+  "description": "text embedding pipeline",
+  "processors": [
+    {
+      "text_embedding": {
+        "model_id": "aVeif4oB5Vm0Tdw8zYO2",
+        "field_map": {
+          "text": "embedding"
+        }
+      }
+    }
+  ]
+}
+```
+
+## Step 3: Create a k-NN index and ingest data
+
+Now you'll ingest supplementary data into an OpenSearch index. In OpenSearch, vectors are stored in a k-NN index. You can create a k-NN index by sending the following request:
+
+```json
+PUT my_test_data
+{
+  "mappings": {
+    "properties": {
+      "text": {
+        "type": "text"
+      },
+      "embedding": {
+        "type": "knn_vector",
+        "dimension": 384
+      }
+    }
+  },
+  "settings": {
+    "index": {
+      "knn.space_type": "cosinesimil",
+      "default_pipeline": "test-pipeline-local-model",
+      "knn": "true"
+    }
+  }
+}
+```
+{% include copy-curl.html %}
+
+Then, ingest data into the index using a bulk request:
+
+```json
+POST _bulk
+{"index": {"_index": "my_test_data", "_id": "1"}}
+{"text": "Chart and table of population level and growth rate for the Ogden-Layton metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Ogden-Layton in 2023 is 750,000, a 1.63% increase from 2022.\nThe metro area population of Ogden-Layton in 2022 was 738,000, a 1.79% increase from 2021.\nThe metro area population of Ogden-Layton in 2021 was 725,000, a 1.97% increase from 2020.\nThe metro area population of Ogden-Layton in 2020 was 711,000, a 2.16% increase from 2019."}
+{"index": {"_index": "my_test_data", "_id": "2"}}
+{"text": "Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019."}
+{"index": {"_index": "my_test_data", "_id": "3"}}
+{"text": "Chart and table of population level and growth rate for the Chicago metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Chicago in 2023 is 8,937,000, a 0.4% increase from 2022.\\nThe metro area population of Chicago in 2022 was 8,901,000, a 0.27% increase from 2021.\\nThe metro area population of Chicago in 2021 was 8,877,000, a 0.14% increase from 2020.\\nThe metro area population of Chicago in 2020 was 8,865,000, a 0.03% increase from 2019."}
+{"index": {"_index": "my_test_data", "_id": "4"}}
+{"text": "Chart and table of population level and growth rate for the Miami metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Miami in 2023 is 6,265,000, a 0.8% increase from 2022.\\nThe metro area population of Miami in 2022 was 6,215,000, a 0.78% increase from 2021.\\nThe metro area population of Miami in 2021 was 6,167,000, a 0.74% increase from 2020.\\nThe metro area population of Miami in 2020 was 6,122,000, a 0.71% increase from 2019."}
+{"index": {"_index": "my_test_data", "_id": "5"}}
+{"text": "Chart and table of population level and growth rate for the Austin metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Austin in 2023 is 2,228,000, a 2.39% increase from 2022.\\nThe metro area population of Austin in 2022 was 2,176,000, a 2.79% increase from 2021.\\nThe metro area population of Austin in 2021 was 2,117,000, a 3.12% increase from 2020.\\nThe metro area population of Austin in 2020 was 2,053,000, a 3.43% increase from 2019."}
+{"index": {"_index": "my_test_data", "_id": "6"}}
+{"text": "Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019."}
+```
+{% include copy-curl.html %}
+
+## Step 4: Create a connector to an externally hosted model
+
+You'll need an LLM to generate responses to user questions. The LLM is too large for the OpenSearch cluster, so you'll create a connector to connect to an externally hosted LLM. For this example, you'll create a connector to the Anthropic Claude model hosted on Amazon Bedrock:
+
+```json
+POST /_plugins/_ml/connectors/_create
+{
+  "name": "BedRock test claude Connector",
+  "description": "The connector to BedRock service for claude model",
+  "version": 1,
+  "protocol": "aws_sigv4",
+  "parameters": {
+      "region": "us-east-1",
+      "service_name": "bedrock",
+      "anthropic_version": "bedrock-2023-05-31",
+      "endpoint": "bedrock.us-east-1.amazonaws.com",
+      "auth": "Sig_V4",
+      "content_type": "application/json",
+      "max_tokens_to_sample": 8000,
+      "temperature": 0.0001,
+      "response_filter": "$.completion"
+  },
+  "credential": {
+      "access_key": "<bedrock_access_key>",
+      "secret_key": "<bedrock_secret_key>",
+      "session_token": "<bedrock_session_token>"
+  },
+  "actions": [
+    {
+      "action_type": "predict",
+      "method": "POST",
+      "url": "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
+      "headers": { 
+        "content-type": "application/json",
+        "x-amz-content-sha256": "required"
+      },
+      "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }"
+    }
+  ]
+}
+```
+{% include copy-curl.html %}
+
+The response contains the connector ID for the newly created connector:
+
+```json
+{
+  "connector_id": "a1eMb4kBJ1eYAeTMAljY"
+}
+```
+
+## Step 5: Register and deploy the externally hosted model
+
+Just like the text embedding model, an LLM needs to be registered and deployed to OpenSearch. To set up the externally hosted model, first create a model group for this model:
+
+```json
+POST /_plugins/_ml/model_groups/_register
+{
+    "name": "test_model_group_bedrock",
+    "description": "This is a public model group"
+}
+```
+{% include copy-curl.html %}
+
+The response contains the model group ID that you’ll use to register a model to this model group:
+
+```json
+{
+ "model_group_id": "wlcnb4kBJ1eYAeTMHlV6",
+ "status": "CREATED"
+}
+
+```
+
+Next, register and deploy the externally hosted Claude model:
+
+```json
+POST /_plugins/_ml/models/_register?deploy=true
+{
+    "name": "Bedrock Claude V2 model",
+    "function_name": "remote",
+    "model_group_id": "wlcnb4kBJ1eYAeTMHlV6",
+    "description": "test model",
+    "connector_id": "a1eMb4kBJ1eYAeTMAljY"
+}
+```
+{% include copy-curl.html %}
+
+Similarly to [Step 1](#step-1-register-and-deploy-a-text-embedding-model), the response contains a task ID that you can use to check the status of the deployment. Once the model is deployed, the status changes to `COMPLETED` and the response includes the model ID for the Claude model:
+
+```json
+{
+  "model_id": "NWR9YIsBUysqmzBdifVJ",
+  "task_type": "REGISTER_MODEL",
+  "function_name": "remote",
+  "state": "COMPLETED",
+  "worker_node": [
+    "4p6FVOmJRtu3wehDD74hzQ"
+  ],
+  "create_time": 1694358489722,
+  "last_update_time": 1694358499139,
+  "is_async": true
+}
+```
+
+To test the LLM, send the following predict request:
+
+```json
+POST /_plugins/_ml/models/NWR9YIsBUysqmzBdifVJ/_predict
+{
+  "parameters": {
+    "prompt": "\n\nHuman:hello\n\nnAssistant:"
+  }
+}
+```
+{% include copy-curl.html %}
+
+## Step 6: Register and execute an agent
+
+Finally, you'll use the text embedding model created in Step 1 and the Claude model created in Step 5 to create a flow agent. This flow agent will run a `VectorDBTool`, and then an `MLModelTool`. The `VectorDBTool` is configured with the model ID for the text embedding model created in Step 1 for vector search, and the `MLModelTool` is configured with the Claude model created in step 5:
+
+```json
+POST /_plugins/_ml/agents/_register
+{
+  "name": "Test_Agent_For_RAG",
+  "type": "flow",
+  "description": "this is a test agent",
+  "tools": [
+    {
+      "type": "VectorDBTool",
+      "parameters": {
+        "model_id": "aVeif4oB5Vm0Tdw8zYO2",
+        "index": "my_test_data",
+        "embedding_field": "embedding",
+        "source_field": ["text"],
+        "input": "${parameters.question}"
+      }
+    },
+    {
+      "type": "MLModelTool",
+      "description": "A general tool to answer any question",
+      "parameters": {
+        "model_id": "NWR9YIsBUysqmzBdifVJ",
+        "prompt": "\n\nHuman:You are a professional data analyst. You will always answer a question based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say you don't know. \n\n Context:\n${parameters.VectorDBTool.output}\n\nHuman:${parameters.question}\n\nAssistant:"
+      }
+    }
+  ]
+}
+```
+{% include copy-curl.html %}
+
+OpenSearch returns an agent ID for the newly created agent:
+
+```json
+{
+  "agent_id": "879v9YwBjWKCe6Kg12Tx"
+}
+```
+
+You can inspect the agent by sending a request to the `agents` endpoint and providing the agent ID:
+
+```json
+GET /_plugins/_ml/agents/879v9YwBjWKCe6Kg12Tx
+```
+{% include copy-curl.html %}
+
+To execute the agent, send the following request. When registering the agent, you configured it to take in `parameters.question`, so you need to provide this parameter in this request. This parameter represents a human-generated user question:
+
+```json
+POST /_plugins/_ml/agents/879v9YwBjWKCe6Kg12Tx/_execute
+{
+  "parameters": {
+    "question": "what's the population increase of Seattle from 2021 to 2023"
+  }
+}
+```
+{% include copy-curl.html %}
+
+The LLM does not have the recent information in its knowledge base so it infers the response to the question based on the ingested data, demonstrating RAG:
+
+```json
+{
+  "inference_results": [
+    {
+      "output": [
+        {
+          "result": """ Based on the given context, the key information is:
+
+The metro area population of Seattle in 2021 was 3,461,000.
+The metro area population of Seattle in 2023 is 3,519,000.
+
+To calculate the population increase from 2021 to 2023:
+
+Population in 2023 (3,519,000) - Population in 2021 (3,461,000) = 58,000
+
+Therefore, the population increase of Seattle from 2021 to 2023 is 58,000."""
+        }
+      ]
+    }
+  ]
+}
+```