From e098cc0c04626f6e534481b564829c4928f822b8 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Tue, 2 Apr 2024 16:07:16 -0500 Subject: [PATCH 1/4] Fix Kafka table (#6845) Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _data-prepper/pipelines/configuration/buffers/kafka.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_data-prepper/pipelines/configuration/buffers/kafka.md b/_data-prepper/pipelines/configuration/buffers/kafka.md index f641874a919..87600601b4c 100644 --- a/_data-prepper/pipelines/configuration/buffers/kafka.md +++ b/_data-prepper/pipelines/configuration/buffers/kafka.md @@ -128,6 +128,7 @@ Option | Required | Type | Description #### producer_properties Use the following configuration options to configure a Kafka producer. + Option | Required | Type | Description :--- | :--- | :--- | :--- `max_request_size` | No | Integer | The maximum size of the request that the producer sends to Kafka. Default is 1 MB. From 89f9b5afb804196ec5a90135051c109d21eb4406 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 2 Apr 2024 17:09:34 -0400 Subject: [PATCH 2/4] Update opensearch-documentation-release-notes-2.13.0.md (#6844) Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- release-notes/opensearch-documentation-release-notes-2.13.0.md | 1 - 1 file changed, 1 deletion(-) diff --git a/release-notes/opensearch-documentation-release-notes-2.13.0.md b/release-notes/opensearch-documentation-release-notes-2.13.0.md index 49e753782cc..38f7678cdec 100644 --- a/release-notes/opensearch-documentation-release-notes-2.13.0.md +++ b/release-notes/opensearch-documentation-release-notes-2.13.0.md @@ -25,7 +25,6 @@ The OpenSearch 2.13.0 documentation includes the following additions and updates - Add post_filter is supported in hybrid search [#6724](https://github.com/opensearch-project/documentation-website/pull/6724) - Update deb/rpm autorestart service after upgrade documentation [#6720](https://github.com/opensearch-project/documentation-website/pull/6720) - Add documentation page for Vega Visualizations [#6711](https://github.com/opensearch-project/documentation-website/pull/6711) -- Add cache plugin and tiered cache documentation [#6708](https://github.com/opensearch-project/documentation-website/pull/6708) - Add documentation for text chunking processor [#6707](https://github.com/opensearch-project/documentation-website/pull/6707) - Update documentation to support InnerProduct with k-NN Lucene Engine [#6703](https://github.com/opensearch-project/documentation-website/pull/6703) - Add documentation for kuromoji_completion filter [#6699](https://github.com/opensearch-project/documentation-website/pull/6699) From 90ba1d6343cd0c9cd2d6396258f2a9873fcc66a8 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 2 Apr 2024 17:34:21 -0400 Subject: [PATCH 3/4] Add 2.13 to version history (#6836) Signed-off-by: Fanit Kolchina --- _about/version-history.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_about/version-history.md b/_about/version-history.md index 25e345568f3..9a57c1ef358 100644 --- a/_about/version-history.md +++ b/_about/version-history.md @@ -9,6 +9,7 @@ permalink: /version-history/ OpenSearch version | Release highlights | Release date :--- | :--- | :--- +[2.13.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.13.0.md) | Makes agents and tools and the OpenSearch Assistant Toolkit generally available. Introduces vector quantization within OpenSearch. Adds LLM guardrails and hybrid search with aggregations. Adds the Bloom filter skipping index for Apache Spark data sources, I/O-based admission control, and the ability to add an alerting cluster that manages all alerting tasks. For a full list of release highlights, see the Release Notes. | 2 April 2024 [2.12.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.12.0.md) | Makes concurrent segment search and conversational search generally available. Provides an experimental OpenSearch Assistant Toolkit, including agents and tools, workflow automation, and OpenSearch Assistant for OpenSearch Dashboards UI. Adds a new match-only text field, query insights to monitor top N queries, and k-NN search on nested fields. For a full list of release highlights, see the Release Notes. | 20 February 2024 [2.11.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.11.1.md) | Includes maintenance changes and bug fixes for cross-cluster replication, alerting, observability, OpenSearch Dashboards, index management, machine learning, security, and security analytics. For a full list of release highlights, see the Release Notes. | 30 November 2023 [2.11.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.11.0.md) | Adds multimodal and sparse neural search capability and the ability to take shallow snapshots that refer to data stored in remote-backed storage. Makes the search comparison tool generally available. Includes a simplified workflow to create threat detectors in Security Analytics and improved security in OpenSearch Dashboards. Experimental features include a new framework and toolset for distributed tracing and updates to conversational search. For a full list of release highlights, see the Release Notes. | 16 October 2023 From 9ff5db315c3353907996966fc5c34a5d01faae30 Mon Sep 17 00:00:00 2001 From: Yuki Imamura <40051885+imamuray@users.noreply.github.com> Date: Wed, 3 Apr 2024 06:48:58 +0900 Subject: [PATCH 4/4] Add or update boost prameter for term-level queries (#6700) * Add to boost parameter to the prefix query Signed-off-by: Yuki Imamura <40051885+imamuray@users.noreply.github.com> * Add or update boost prameter for term-level queries Signed-off-by: Yuki Imamura <40051885+imamuray@users.noreply.github.com> --------- Signed-off-by: Yuki Imamura <40051885+imamuray@users.noreply.github.com> --- _query-dsl/term/exists.md | 6 +++++- _query-dsl/term/fuzzy.md | 5 +++-- _query-dsl/term/ids.md | 1 + _query-dsl/term/prefix.md | 5 +++-- _query-dsl/term/range.md | 14 +++++++------- _query-dsl/term/regexp.md | 4 ++-- _query-dsl/term/term.md | 6 +++--- _query-dsl/term/terms-set.md | 5 +++-- _query-dsl/term/terms.md | 5 +++-- _query-dsl/term/wildcard.md | 2 +- 10 files changed, 31 insertions(+), 22 deletions(-) diff --git a/_query-dsl/term/exists.md b/_query-dsl/term/exists.md index a62dda981bc..1d52744c912 100644 --- a/_query-dsl/term/exists.md +++ b/_query-dsl/term/exists.md @@ -146,4 +146,8 @@ The response contains the matching document: ## Parameters -The query accepts the name of the field (``) as a top-level parameter. \ No newline at end of file +The query accepts the name of the field (``) as a top-level parameter. + +Parameter | Data type | Description +:--- | :--- | :--- +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. diff --git a/_query-dsl/term/fuzzy.md b/_query-dsl/term/fuzzy.md index f5a9773aeb5..9afa85ea93c 100644 --- a/_query-dsl/term/fuzzy.md +++ b/_query-dsl/term/fuzzy.md @@ -67,7 +67,7 @@ GET _search "fuzzy": { "": { "value": "sample", - ... + ... } } } @@ -80,11 +80,12 @@ The `` accepts the following parameters. All parameters except `value` ar Parameter | Data type | Description :--- | :--- | :--- `value` | String | The term to search for in the field specified in ``. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. `fuzziness` | `AUTO`, `0`, or a positive integer | The number of character edits (insert, delete, substitute) needed to change one word to another when determining whether a term matched a value. For example, the distance between `wined` and `wind` is 1. The default, `AUTO`, chooses a value based on the length of each term and is a good choice for most use cases. `max_expansions` | Positive integer | The maximum number of terms to which the query can expand. Fuzzy queries “expand to” a number of matching terms that are within the distance specified in `fuzziness`. Then OpenSearch tries to match those terms. Default is `50`. `prefix_length` | Non-negative integer | The number of leading characters that are not considered in fuzziness. Default is `0`. `rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`. -`transpositions` | Boolean | Specifies whether to allow transpositions of two adjacent characters (`ab` to `ba`) as edits. Default is `true`. +`transpositions` | Boolean | Specifies whether to allow transpositions of two adjacent characters (`ab` to `ba`) as edits. Default is `true`. Specifying a large value in `max_expansions` can lead to poor performance, especially if `prefix_length` is set to `0`, because of the large number of variations of the word that OpenSearch tries to match. {: .warning} diff --git a/_query-dsl/term/ids.md b/_query-dsl/term/ids.md index a1a098f5868..0c3b5393fb4 100644 --- a/_query-dsl/term/ids.md +++ b/_query-dsl/term/ids.md @@ -32,3 +32,4 @@ The query accepts the following parameter. Parameter | Data type | Description :--- | :--- | :--- `values` | Array of strings | The document IDs to search for. Required. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. diff --git a/_query-dsl/term/prefix.md b/_query-dsl/term/prefix.md index 14f208f3c5a..eda5307d142 100644 --- a/_query-dsl/term/prefix.md +++ b/_query-dsl/term/prefix.md @@ -50,7 +50,7 @@ GET _search "prefix": { "": { "value": "sample", - ... + ... } } } @@ -63,8 +63,9 @@ The `` accepts the following parameters. All parameters except `value` ar Parameter | Data type | Description :--- | :--- | :--- `value` | String | The term to search for in the field specified in ``. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. `case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping). `rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`. If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, prefix queries are not run. If `index_prefixes` is enabled, the `search.allow_expensive_queries` setting is ignored and an optimized query is built and run. -{: .important} \ No newline at end of file +{: .important} diff --git a/_query-dsl/term/range.md b/_query-dsl/term/range.md index 076ab5ad153..8a8f53c4806 100644 --- a/_query-dsl/term/range.md +++ b/_query-dsl/term/range.md @@ -90,7 +90,7 @@ OpenSearch populates missing date components with the following values: - `SECOND_OF_MINUTE`: `59` - `NANO_OF_SECOND`: `999_999_999` -If the year is missing, it is not populated. +If the year is missing, it is not populated. For example, consider the following request that specifies only the year in the start date: @@ -131,7 +131,7 @@ GET products/_search ``` {% include copy-curl.html %} -In the preceding example, `2019/01/01` is the anchor date (the starting point) for the date math. After the two pipe characters (`||`), you are specifying a mathematical expression relative to the anchor date. In this example, you are subtracting 1 year (`-1y`) and 1 day (`-1d`). +In the preceding example, `2019/01/01` is the anchor date (the starting point) for the date math. After the two pipe characters (`||`), you are specifying a mathematical expression relative to the anchor date. In this example, you are subtracting 1 year (`-1y`) and 1 day (`-1d`). You can also round off dates by adding a forward slash to the date or time unit. @@ -175,8 +175,8 @@ GET /products/_search "query": { "range": { "created": { - "time_zone": "-04:00", - "gte": "2022-04-17T06:00:00" + "time_zone": "-04:00", + "gte": "2022-04-17T06:00:00" } } } @@ -184,7 +184,7 @@ GET /products/_search ``` {% include copy-curl.html %} -The `gte` parameter in the preceding query is converted to `2022-04-17T10:00:00 UTC`, which is the UTC equivalent of `2022-04-17T06:00:00-04:00`. +The `gte` parameter in the preceding query is converted to `2022-04-17T10:00:00 UTC`, which is the UTC equivalent of `2022-04-17T06:00:00-04:00`. The `time_zone` parameter does not affect the `now` value because `now` always corresponds to the current system time in UTC. {: .note} @@ -200,7 +200,7 @@ GET _search "range": { "": { "gt": 10, - ... + ... } } } @@ -215,7 +215,7 @@ Parameter | Data type | Description :--- | :--- | :--- `format` | String | A [format]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/#formats) for dates in this query. Default is the field's mapped format. `relation` | String | Indicates how the range query matches values for [`range`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/range/) fields. Valid values are:
- `INTERSECTS` (default): Matches documents whose `range` field value intersects the range provided in the query.
- `CONTAINS`: Matches documents whose `range` field value contains the entire range provided in the query.
- `WITHIN`: Matches documents whose `range` field value is entirely within the range provided in the query. -`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. `time_zone` | String | The time zone used to convert [`date`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/) values to UTC in the query. Valid values are ISO 8601 [UTC offsets](https://en.wikipedia.org/wiki/List_of_UTC_offsets) and [IANA time zone IDs](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). For more information, see [Time zone](#time-zone). If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, range queries on [`text`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/text/) and [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/) fields are not run. diff --git a/_query-dsl/term/regexp.md b/_query-dsl/term/regexp.md index 31bc6460aac..65d69535163 100644 --- a/_query-dsl/term/regexp.md +++ b/_query-dsl/term/regexp.md @@ -43,7 +43,7 @@ GET _search "regexp": { "": { "value": "[Ss]ample", - ... + ... } } } @@ -56,6 +56,7 @@ The `` accepts the following parameters. All parameters except `value` ar Parameter | Data type | Description :--- | :--- | :--- `value` | String | The regular expression used for matching terms in the field specified in ``. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. `case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the regular expression value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping). `flags` | String | Enables optional operators for Lucene’s regular expression engine. `max_determinized_states` | Integer | Lucene converts a regular expression to an automaton with a number of determinized states. This parameter specifies the maximum number of automaton states the query requires. Use this parameter to prevent high resource consumption. To run complex regular expressions, you may need to increase the value of this parameter. Default is 10,000. @@ -63,4 +64,3 @@ Parameter | Data type | Description If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, `regexp` queries are not run. {: .important} - diff --git a/_query-dsl/term/term.md b/_query-dsl/term/term.md index 20694fb4558..c1c296b9a08 100644 --- a/_query-dsl/term/term.md +++ b/_query-dsl/term/term.md @@ -82,7 +82,7 @@ GET _search "term": { "": { "value": "sample", - ... + ... } } } @@ -95,5 +95,5 @@ The `` accepts the following parameters. All parameters except `value` ar Parameter | Data type | Description :--- | :--- | :--- `value` | String | The term to search for in the field specified in ``. A document is returned in the results only if its field value exactly matches the term, with the correct spacing and capitalization. -`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`. -`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping). \ No newline at end of file +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. +`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping). diff --git a/_query-dsl/term/terms-set.md b/_query-dsl/term/terms-set.md index 452cae66b20..ea0251ddffb 100644 --- a/_query-dsl/term/terms-set.md +++ b/_query-dsl/term/terms-set.md @@ -153,7 +153,7 @@ GET _search "terms_set": { "": { "terms": [ "term1", "term2" ], - ... + ... } } } @@ -167,4 +167,5 @@ Parameter | Data type | Description :--- | :--- | :--- `terms` | Array of strings | The array of terms to search for in the field specified in ``. A document is returned in the results only if the required number of terms matches the document's field values exactly, with the correct spacing and capitalization. `minimum_should_match_field` | String | The name of the [numeric]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/numeric/) field that specifies the number of matching terms required in order to return a document in the results. -`minimum_should_match_script` | String | A script that returns the number of matching terms required in order to return a document in the results. \ No newline at end of file +`minimum_should_match_script` | String | A script that returns the number of matching terms required in order to return a document in the results. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. diff --git a/_query-dsl/term/terms.md b/_query-dsl/term/terms.md index 5e5524e6a3c..fd151262557 100644 --- a/_query-dsl/term/terms.md +++ b/_query-dsl/term/terms.md @@ -39,7 +39,7 @@ The query accepts the following parameters. All parameters are optional. Parameter | Data type | Description :--- | :--- | :--- `` | String | The field in which to search. A document is returned in the results only if its field value exactly matches at least one term, with the correct spacing and capitalization. -`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. ## Terms lookup @@ -249,4 +249,5 @@ Parameter | Data type | Description `index` | String | The name of the index in which to fetch field values. Required. `id` | String | The document ID of the document from which to fetch field values. Required. `path` | String | The name of the field from which to fetch field values. Specify nested fields using dot path notation. Required. -`routing` | String | Custom routing value of the document from which to fetch field values. Optional. Required if a custom routing value was provided when the document was indexed. \ No newline at end of file +`routing` | String | Custom routing value of the document from which to fetch field values. Optional. Required if a custom routing value was provided when the document was indexed. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. diff --git a/_query-dsl/term/wildcard.md b/_query-dsl/term/wildcard.md index 897ab6ed0f8..06525819415 100644 --- a/_query-dsl/term/wildcard.md +++ b/_query-dsl/term/wildcard.md @@ -61,7 +61,7 @@ The `` accepts the following parameters. All parameters except `value` ar Parameter | Data type | Description :--- | :--- | :--- `value` | String | The wildcard pattern used for matching terms in the field specified in ``. -`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`. +`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0. `case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping). `rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`.