Merge branch 'main' into data-prepper-autogenerate-campaign-1

_data-prepper/pipelines/configuration/sources/kafka.md

@@ -120,7 +120,7 @@ Use the following options when setting SSL encryption.
 Option | Required | Type | Description
 :--- | :--- | :--- | :---
 `type` | No | String | The encryption type. Use `none` to disable encryption. Default is `ssl`.
-`Insecure` | No | Boolean | A Boolean flag used to turn off SSL certificate verification. If set to `true`, certificate authority (CA) certificate verification is turned off and insecure HTTP requests are sent. Default is `false`.
+`insecure` | No | Boolean | A Boolean flag used to turn off SSL certificate verification. If set to `true`, certificate authority (CA) certificate verification is turned off and insecure HTTP requests are sent. Default is `false`.
 
 
 #### AWS

_field-types/dynamic.md

@@ -0,0 +1,342 @@
+---
+layout: default
+title: Dynamic parameter
+nav_order: 10
+redirect_from:
+  - /opensearch/dynamic/
+---
+
+# Dynamic parameter
+
+The `dynamic` parameter specifies whether newly detected fields can be added dynamically to a mapping. It accepts the parameters listed in the following table.
+
+Parameter | Description 
+:--- | :--- 
+`true`  | Specfies that new fields can be added dynamically to the mapping. Default is `true`.
+`false` | Specifies that new fields cannot be added dynamically to the mapping. If a new field is detected, then it is not indexed or searchable but can be retrieved from the `_source` field.
+`strict` | Throws an exception. The indexing operation fails when new fields are detected.
+`strict_allow_templates` | Adds new fields if they match predefined dynamic templates in the mapping.
+
+--- 
+
+## Example: Create an index with `dynamic` set to `true`
+
+1. Create an index with `dynamic` set to `true` by sending the following request:
+
+```json
+PUT testindex1
+{
+  "mappings": {
+    "dynamic": true
+  }
+}
+```
+{% include copy-curl.html %}
+
+2. Index a document with an object field `patient` containing two string fields by sending the following request:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  } 
+}
+```
+{% include copy-curl.html %}
+
+3. Confirm the mapping works as expected by sending the following request:
+
+```json
+GET testindex1/_mapping
+```
+{% include copy-curl.html %}
+
+The object field `patient` and two subfields `name` and `id` are added to the mapping, as shown in the following response:
+
+```json
+{
+  "testindex1": {
+    "mappings": {
+      "dynamic": "true",
+      "properties": {
+        "patient": {
+          "properties": {
+            "id": {
+              "type": "text",
+              "fields": {
+                "keyword": {
+                  "type": "keyword",
+                  "ignore_above": 256
+                }
+              }
+            },
+            "name": {
+              "type": "text",
+              "fields": {
+                "keyword": {
+                  "type": "keyword",
+                  "ignore_above": 256
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+---
+
+## Example: Create an index with `dynamic` set to `false`
+
+1. Create an index with explicit mappings and `dynamic` set to `false` by sending the following request:
+
+```json
+PUT testindex1
+{
+  "mappings": {
+    "dynamic": false,
+    "properties": {
+      "patient": {
+        "properties": {
+          "id": {
+            "type": "keyword"
+          },
+          "name": {
+            "type": "keyword"
+          }
+        }
+      }
+    }
+  }
+}
+```
+{% include copy-curl.html %}
+
+2. Index a document with an object field `patient` containing two string fields and additional unmapped fields by sending the following request:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  },
+  "room": "room1",
+  "floor": "1"
+}
+```
+{% include copy-curl.html %}
+
+3.  Confirm the mapping works as expected by sending the following request:
+
+```json
+GET testindex1/_mapping
+```
+{% include copy-curl.html %}
+
+The following response shows that the new fields `room` and `floor` were not added to the mapping, which remained unchanged:
+
+```json
+{
+  "testindex1": {
+    "mappings": {
+      "dynamic": "false",
+      "properties": {
+        "patient": {
+          "properties": {
+            "id": {
+              "type": "keyword"
+            },
+            "name": {
+              "type": "keyword"
+            }
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+4. Get the unmapped fields `room` and `floor` from the document by sending the following request:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  },
+  "room": "room1",
+  "floor": "1"
+}
+```
+
+The following request searches for the fields `room` and `floor`:
+
+```json
+POST testindex1/_search
+{
+  "query": {
+    "term": {
+      "room": "room1"
+    }
+  }
+}
+```
+
+The response returns no results:
+
+```json
+{
+  "took": 3,
+  "timed_out": false,
+  "_shards": {
+    "total": 1,
+    "successful": 1,
+    "skipped": 0,
+    "failed": 0
+  },
+  "hits": {
+    "total": {
+      "value": 0,
+      "relation": "eq"
+    },
+    "max_score": null,
+    "hits": []
+  }
+}
+```
+
+---
+
+## Example: Create an index with `dynamic` set to `strict`
+
+1. Create an index with explicit mappings and `dynamic` set to `strict` by sending the following request:
+
+```json
+PUT testindex1
+{
+  "mappings": {
+    "dynamic": strict,
+    "properties": {
+      "patient": {
+        "properties": {
+          "id": {
+            "type": "keyword"
+          },
+          "name": {
+            "type": "keyword"
+          }
+        }
+      }
+    }
+  }
+}
+```
+{% include copy-curl.html %}
+
+2. Index a document with an object field `patient` containing two string fields and additional unmapped fields by sending the following request:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  },
+  "room": "room1",
+  "floor": "1"
+}
+```
+{% include copy-curl.html %}
+
+Note that an exception is thrown, as shown in the following response: 
+
+```json
+{
+  "error": {
+    "root_cause": [
+      {
+        "type": "strict_dynamic_mapping_exception",
+        "reason": "mapping set to strict, dynamic introduction of [room] within [_doc] is not allowed"
+      }
+    ],
+    "type": "strict_dynamic_mapping_exception",
+    "reason": "mapping set to strict, dynamic introduction of [room] within [_doc] is not allowed"
+  },
+  "status": 400
+}
+```
+
+---
+
+## Example: Create an index with `dynamic` set to `strict_allow_templates`
+
+1. Create an index with predefined dynamic templates and `dynamic` set to `strict_allow_templates` by sending the following request: 
+
+```json
+PUT testindex1
+{
+  "mappings": {
+    "dynamic": "strict_allow_templates",
+    "dynamic_templates": [
+      {
+        "strings": {
+          "match": "room*",
+          "match_mapping_type": "string",
+          "mapping": {
+            "type": "keyword"
+          }
+        }
+      }
+    ],
+    "properties": {
+      "patient": {
+        "properties": {
+          "id": {
+            "type": "keyword"
+          },
+          "name": {
+            "type": "keyword"
+          }
+        }
+      }
+    }
+  }
+}
+```
+{% include copy-curl.html %}
+
+2. Index a document with an object field `patient` containing two string fields and a new field `room` that matches one of the dynamic templates by sending the following request:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  },
+  "room": "room1"
+}
+```
+{% include copy-curl.html %}
+
+Indexing succeeds because the new field `room` matches the dynamic templates. However, indexing fails for the new field `floor` because it does not match one of the dynamic templates and is not explicitly mapped, as shown in the following response:
+
+```json
+PUT testindex1/_doc/1
+{ 
+  "patient": { 
+    "name" : "John Doe",
+    "id" : "123456"
+  },
+  "room": "room1",
+  "floor": "1"
+}
+```

_field-types/supported-field-types/nested.md

@@ -308,7 +308,7 @@ The following table lists the parameters accepted by object field types. All par
 
 Parameter | Description 
 :--- | :--- 
-[`dynamic`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to this object. Valid values are `true`, `false`, and `strict`. Default is `true`.
+[`dynamic`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to the object. Valid values are `true`, `false`, `strict`, and `strict_allow_templates`. Default is `true`.
 `include_in_parent` | A Boolean value that specifies whether all fields in the child nested object should also be added to the parent document in flattened form. Default is `false`.
 `include_in_root` | A Boolean value that specifies whether all fields in the child nested object should also be added to the root document in flattened form. Default is `false`.
 `properties` | Fields of this object, which can be of any supported type. New properties can be dynamically added to this object if `dynamic` is set to `true`.

_field-types/supported-field-types/object.md

@@ -73,7 +73,7 @@ The following table lists the parameters accepted by object field types. All par
 
 Parameter | Description 
 :--- | :--- 
-[`dynamic`](#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to this object. Valid values are `true`, `false`, and `strict`. Default is `true`.
+[`dynamic`](#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to the object. Valid values are `true`, `false`, `strict`, and `strict_allow_templates`. Default is `true`.
 `enabled` | A Boolean value that specifies whether the JSON contents of the object should be parsed. If `enabled` is set to `false`, the object's contents are not indexed or searchable, but they are still retrievable from the _source field. Default is `true`.
 `properties` | Fields of this object, which can be of any supported type. New properties can be dynamically added to this object if `dynamic` is set to `true`.
 
@@ -149,6 +149,7 @@ Value | Description
 `true` | New fields can be added to the mapping dynamically. This is the default.
 `false` | New fields cannot be added to the mapping dynamically. If a new field is detected, it is not indexed or searchable. However, it is still retrievable from the _source field. 
 `strict` | When new fields are added to the mapping dynamically, an exception is thrown. To add a new field to an object, you have to add it to the mapping first.
+`strict_allow_templates` | If the newly detected fields match any of the predefined dynamic templates in the mapping, then they are added to the mapping; if they do not match any of them, then an exception is thrown.
 
 Inner objects inherit the `dynamic` parameter value from their parent unless they declare their own `dynamic` parameter value.
 {: .note }