Merge pull request #769 from orkes-io/liv-update

Doc updates

docs/developer-guides/caching-task-outputs.md

@@ -0,0 +1,61 @@
+---
+slug: "../faqs/task-cache-output"
+---
+# Caching Task Outputs
+
+Some task types support caching, which saves task outputs for reuse in subsequent tasks. This feature can be configured in the task configuration.
+
+**To cache task output:**
+1. In your Orkes Conductor cluster, go to **Definitions** > **Workflow** and select a workflow.
+2. In the visual workflow editor, select a task.
+3. Turn on the **Cache output** toggle and enter the **ttlInSecond** and **key**. Refer to [Configuration](#configuration) for more information on the parameters.
+
+<p align="center"><img src="/content/img/dev-guides/caching_task_outputs-UI_screen.png" alt="Configuring cache output in UI." width="100%" height="auto"></img></p>
+
+
+## Configuration
+
+Configure the following parameters to enable task caching.
+
+| Parameter | Description | Required/ Optional | 
+| --------- | ----------- | ------------------ |
+| cacheConfig. **ttlInSecond** | The time to live in seconds, which is the duration for the output to be cached. | Required. | 
+| cacheConfig. **key** | The cache key, which is a unique identifier for the cached output. <br/><br/> The key should be constructed as a string of dynamic task inputs, such as `${uri}-${method}`. | Required. | 
+
+**Example**
+
+``` json
+//task configuration
+
+  "cacheConfig": {
+    "ttlInSecond": 36000,
+    "key": "${uri}-${method}"
+  }
+```
+
+## Task behavior with caching
+
+Before a task is scheduled, the server checks if there is cached output for the given task definition name by matching the cache key. If a match is found, the task does not get scheduled, and the cached output is used to complete the task.
+
+If no cached output is found, the task is scheduled as usual. When the task is completed successfully, the output is cached against the cache key for the duration specified.
+
+
+## Supported tasks
+
+Caching is currently supported for the following task types:
+* [Worker (Simple)](../reference-docs/worker-task)
+* [HTTP](../reference-docs/system-tasks/http)
+* [HTTP Poll](../reference-docs/system-tasks/http-poll)
+* [Business Rule](../reference-docs/system-tasks/business-rule)
+* [JDBC](../reference-docs/system-tasks/jdbc)
+* [Get Signed JWT](../reference-docs/system-tasks/get-signed-jwt)
+* [Opsgenie](../reference-docs/system-tasks/opsgenie)
+* [Text Complete](../reference-docs/ai-tasks/llm-text-complete)
+* [Generate Embeddings](../reference-docs/ai-tasks/llm-generate-embeddings)
+* [Get Embeddings](../reference-docs/ai-tasks/llm-get-embeddings)
+* [Store Embeddings](../reference-docs/ai-tasks/llm-store-embeddings.md)
+* [Search Index](../reference-docs/ai-tasks/llm-search-index)
+* [Index Document](../reference-docs/ai-tasks/llm-index-document)
+* [Get Document](../reference-docs/ai-tasks/llm-get-document)
+* [Index Text](../reference-docs/ai-tasks/llm-index-text)
+* [Chat Complete](../reference-docs/ai-tasks/llm-chat-complete.md)

docs/developer-guides/passing-inputs-to-task-in-conductor.md

@@ -3,7 +3,7 @@ import TabItem from '@theme/TabItem';
 import Install from '@site/src/components/install.mdx';
 
 
-# Dynamic Task Inputs
+# Dynamic Inputs
 
 When you start a Conductor workflow, you can pass inputs to the workflow - which can be used as parameters for various steps in the workflow. As the tasks progress in a workflow - each task will create its own outputs. So how can this data from previous tasks be used in the steps succeeding it? In Conductor, data can be passed to a task in different ways, such as from workflow inputs/outputs, prior task inputs/outputs, variables, and secrets.
 

docs/developer-guides/schema-validation.md

@@ -0,0 +1,199 @@
+---
+sidebar_label: Schema Validation
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Input/Output Schema Validation
+
+Create schemas to define and enforce the payload structure of workflow or task inputs/outputs. 
+
+Once created, schemas can be added to workflow or task definitions:
+* A **workflow-level schema** allows you to specify what workflow inputs must be supplied.
+* A **task-level schema** allows you to specify a contract that the task worker should follow—in other words, the inputs/outputs that must be wired to/from the task configuration. The task-level schema can be general across workflows (specified in the *task definition*) or unique to a specific workflow (specified in the *task configuration*).
+
+The schema is enforced at runtime. The workflow or task will fail if the inputs/outputs do not pass the validation. This behavior is the main distinction between a *task-level schema* versus the *input keys, output keys, and input templates* in a task definition. The schema enforces validation on a payload, while the latter parameters are non-enforceable guides.
+
+
+## Schema formats
+
+Currently, schemas can be defined in the [JSON Schema](https://json-schema.org/specification) format. Like workflows, schemas can be versioned.
+
+**Example**
+``` json
+{
+  "createTime": 1727378396701,
+  "updateTime": 1727378396701,
+  "createdBy": "[email protected]",
+  "updatedBy": "[email protected]",
+  "name": "itemSchema",
+  "version": 1,
+  "type": "JSON", // set schema type as JSON Schema
+  "data": { // include the JSON Schema parameters here
+    "$schema": "https://json-schema.org/draft/2020-12/schema",
+    "$id": "https://example.com/product.schema.json",
+    "type": "object",
+    "properties": {
+      "productId": {
+        "description": "The unique identifier for a product",
+        "type": "integer"
+      },
+      "productName": {
+        "description": "Name of the product",
+        "type": "string"
+      }
+    },
+    "required": [
+      "productId"
+    ]
+  }
+}
+```
+
+## Using schemas
+
+To enforce validation for input/output payloads, schemas can be added to:
+* **A workflow definition**—Enforce workflow inputs.
+* **A task definition**—Enforce task inputs/outputs for all instances of the task.
+* **A task configuration**—Enforce task inputs/outputs for a specific workflow.
+
+Here is an overview of using schemas in Conductor workflows:
+1. Define a schema for a task’s or workflow’s inputs/outputs.
+2. Add the schema in a workflow definition, task definition, or task configuration.
+
+
+### Step 1: Define a schema
+
+The Orkes Platform can be used to define schemas.
+
+**To define a schema:**
+1. Log in to your Orkes Conductor cluster.
+2. Go to **Definitions** > **Schemas**.
+3. Select **(+) New schema** on the top right.
+    A schema editor opens.
+4. Add your schema definition in the `data` object. Make sure to fill out at least the `$schema`, `$id`,  `type`, and `properties` parameters as outlined in the JSON Schema.
+
+Learn more about creating a JSON Schema in [their documentation](https://json-schema.org/learn/getting-started-step-by-step).
+
+
+### Step 2: Add schema to workflow or task
+
+If the version is unspecified, the latest schema version will be used.
+
+**To add a schema:**
+
+<Tabs>
+<TabItem value="Task-definition" label="Task definition">
+
+1. Go to **Definitions** > **Task**.
+2. Select a task definition that you want to add a schema to.
+3. In the Schema section, select a schema to use as an **Input Schema** or **Output Schema**.
+4. Select a **Version** for the schema.
+5. Switch on the **Enforce schema** toggle.
+6. Select **Save** in the top right.
+
+``` json
+// task definition
+
+{
+...
+  "inputSchema": {
+    "name": "itemSchema",
+    "version": 1,
+    "type": "JSON"
+  },
+  "outputSchema": {
+    "name": "outputSchema",
+    "type": "JSON"
+  }
+  "enforceSchema": true
+}
+```
+
+</TabItem>
+
+<TabItem value="Task-configuration" label="Task configuration">
+
+1. Go to **Definitions** > **Workflow**.
+2. Select a workflow that you want to add a task-level schema to.
+3. In the visual workflow editor, select a task.
+4. In the Schema section, select a schema to use as an **Input Schema** or **Output Schema**.
+5. Select a **Version** for the schema.
+6. Switch on the **Enforce schema** toggle.
+7. Select **Save** > **Confirm** in the top right.
+
+``` json
+// workflow definition
+
+{
+...
+  "tasks": [
+    {
+      "name": "getItem",
+      "taskReferenceName": "getItem_ref",
+      "inputParameters": {
+        "productId": 1,
+        "productName": "toy"
+      },
+      "type": "SIMPLE",
+      "taskDefinition": {
+        "inputSchema": {
+          "createTime": 0,
+          "updateTime": 0,
+          "name": "itemSchema",
+          "type": "JSON"
+        },
+        "outputSchema": {
+          "name": "outputSchema",
+          "version": 1,
+          "type": "JSON"
+        },
+        "enforceSchema": true
+      }
+    }
+  ]
+}
+```
+
+</TabItem>
+
+<TabItem value="Workflow-definition" label="Workflow definition">
+
+1. Go to **Definitions** > **Workflow**.
+2. Select a workflow that you want to add a task-level schema to.
+3. In the Schema section of the Workflow tab, select a schema to use as an **Input Schema**.
+4. Select a **Version** for the schema.
+5. Switch on the **Enforce schema** toggle.
+6. Select **Save** > **Confirm** in the top right.
+
+``` json
+// workflow definition
+
+{
+...
+  "inputSchema": {
+    "name": "itemSchema",
+    "version": 1,
+    "type": "JSON"
+  },
+  "enforceSchema": true
+}
+```
+
+</TabItem>
+</Tabs>
+
+Once the schema is added, modify your workflow or task inputs/outputs to match the schema.
+
+
+## Workflow behavior with schemas
+
+If a workflow does not pass its schema validation, it will not be allowed to execute.
+
+<p align="center"><img src="/content/img/dev-guides/input_output_validation-execution_failure.png" alt="Screenshot of execution failure due to invalid inputs." width="100%" height="auto"></img></p>
+
+If a task does not pass its schema validation, it will fail with a terminal error.
+
+<p align="center"><img src="/content/img/dev-guides/input_output_validation-task_failure.png" alt="Screenshot of task failure due to invalid inputs or outputs." width="100%" height="auto"></img></p>
+

sidebars.js

@@ -74,13 +74,10 @@ const sidebars = {
                             label: 'Inputs/Outputs',
                             items: [
                                 'developer-guides/passing-inputs-to-task-in-conductor',
+                                'developer-guides/schema-validation',
                                 'developer-guides/secrets-in-conductor',
                                 'developer-guides/using-environment-variables',
-                                {
-                                    type: 'doc',
-                                    id: 'faqs/task-cache-output',
-                                    label: "Caching Task Outputs"
-                                },
+                                'developer-guides/caching-task-outputs',
                             ],
                             collapsible: false,
                         },