From 4a8360f2af2f61d940424442385142965e1af1d3 Mon Sep 17 00:00:00 2001 From: Chenyang Ji Date: Thu, 6 Jun 2024 13:52:20 -0700 Subject: [PATCH 1/7] Add document for top n queries by cpu and memory Signed-off-by: Chenyang Ji --- .../query-insights/top-n-queries.md | 77 +++++++++++++++++-- 1 file changed, 70 insertions(+), 7 deletions(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 44469fa64b..29abe58478 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -13,9 +13,15 @@ Monitoring the top N queries in query insights features can help you gain real-t To enable monitoring of the top N queries, configure the following [dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings): -- `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.latency.window_size`: [Configure the window size](#configuring-window-size). -- `search.insights.top_queries.latency.top_n_size`: [Specify the value of n](#configuring-the-value-of-n). +- `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries by latency](#enabling-the-top-n-queries-feature). +- `search.insights.top_queries.latency.window_size`: [Configure the window size for top n queries by latency](#configuring-window-size). +- `search.insights.top_queries.latency.top_n_size`: [Specify the value of n for top n queries by latency](#configuring-the-value-of-n). +- `search.insights.top_queries.cpu.enabled`: Set to `true` to [enable monitoring of the top N queries by cpu usage](#enabling-the-top-n-queries-feature). +- `search.insights.top_queries.cpu.window_size`: [Configure the window size for top n queries by cpu usage](#configuring-window-size). +- `search.insights.top_queries.cpu.top_n_size`: [Specify the value of n for top n queries by cpu usage](#configuring-the-value-of-n). +- `search.insights.top_queries.memory.enabled`: Set to `true` to [enable monitoring of the top N queries for top n queries by memory usage](#enabling-the-top-n-queries-feature). +- `search.insights.top_queries.memory.window_size`: [Configure the window size for top n queries by memory usage](#configuring-window-size). +- `search.insights.top_queries.memory.top_n_size`: [Specify the value of n for top n queries by memory usage](#configuring-the-value-of-n). It's important to exercise caution when enabling this feature because it can consume system resources. {: .important} @@ -25,7 +31,7 @@ For detailed information about enabling and configuring this feature, see the fo ## Enabling the top N queries feature -After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster: +After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster. For example, the below command enables top n queries by latency. ```json PUT _cluster/settings @@ -37,9 +43,27 @@ PUT _cluster/settings ``` {% include copy-curl.html %} +Other available metrics are cpu usage and memory usage. You can enable them with +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.cpu.enabled" : true + } +} + +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.memory.enabled" : true + } +} +``` +{% include copy-curl.html %} + ## Configuring window size -You can configure the window size for the top N queries by latency with `search.insights.top_queries.latency.window_size`. For example, a cluster with the following configuration will collect top N queries in a 60-minute window: +You can configure the window size for the top N queries by latency with `search.insights.top_queries.latency.window_size`. For example, a cluster with the following configuration will collect top N queries by latency in a 60-minute window: ```json PUT _cluster/settings @@ -51,11 +75,29 @@ PUT _cluster/settings ``` {% include copy-curl.html %} +Other available metrics are cpu usage and memory usage. You can configure the window size to 60 minutes for these metrics by: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.cpu.window_size" : "60m" + } +} + +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.memory.window_size" : "60m" + } +} +``` +{% include copy-curl.html %} + ## Configuring the value of N You can configure the value of N in the `search.insights.top_queries.latency.top_n_size` parameter. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size: -``` +```json PUT _cluster/settings { "persistent" : { @@ -65,6 +107,25 @@ PUT _cluster/settings ``` {% include copy-curl.html %} + +Other available metrics are cpu usage and memory usage. You can configure the top n size to 10 for these metrics by: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.cpu.top_n_size" : 10 + } +} + +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.memory.top_n_size" : 10 + } +} +``` +{% include copy-curl.html %} + ## Monitoring the top N queries You can use the Insights API endpoint to obtain top N queries by latency: @@ -74,9 +135,11 @@ GET /_insights/top_queries ``` {% include copy-curl.html %} -Specify a metric type to filter the response by metric type (latency is the only supported type as of 2.12): +Specify a metric type to filter the response by metric type: ```json GET /_insights/top_queries?type=latency +GET /_insights/top_queries?type=cpu +GET /_insights/top_queries?type=memory ``` {% include copy-curl.html %} \ No newline at end of file From 817334828572278c811e923e5562b610c693b563 Mon Sep 17 00:00:00 2001 From: Chenyang Ji Date: Mon, 10 Jun 2024 15:19:58 -0700 Subject: [PATCH 2/7] update document to fix style checks Signed-off-by: Chenyang Ji --- _observing-your-data/query-insights/top-n-queries.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 29abe58478..1045ea55c9 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -31,7 +31,7 @@ For detailed information about enabling and configuring this feature, see the fo ## Enabling the top N queries feature -After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster. For example, the below command enables top n queries by latency. +After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster. For example, the following command enables top n queries by latency. ```json PUT _cluster/settings @@ -43,7 +43,7 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -Other available metrics are cpu usage and memory usage. You can enable them with +Other available metrics are CPU usage and memory usage. You can enable them with ```json PUT _cluster/settings { @@ -75,7 +75,7 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -Other available metrics are cpu usage and memory usage. You can configure the window size to 60 minutes for these metrics by: +Other available metrics are CPU usage and memory usage. You can configure the window size to 60 minutes for these metrics by: ```json PUT _cluster/settings { @@ -108,7 +108,7 @@ PUT _cluster/settings {% include copy-curl.html %} -Other available metrics are cpu usage and memory usage. You can configure the top n size to 10 for these metrics by: +Other available metrics are CPU usage and memory usage. You can configure the top n size to 10 for these metrics by: ```json PUT _cluster/settings { From 314bc925e393eb6aeea2fd433e27621be263235d Mon Sep 17 00:00:00 2001 From: Chenyang Ji Date: Thu, 13 Jun 2024 13:01:46 -0700 Subject: [PATCH 3/7] combine exporter and metrics enhancements documents into one PR Signed-off-by: Chenyang Ji --- .../query-insights/top-n-queries.md | 57 ++++++++++++++++++- 1 file changed, 56 insertions(+), 1 deletion(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 1045ea55c9..691c223190 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -142,4 +142,59 @@ GET /_insights/top_queries?type=latency GET /_insights/top_queries?type=cpu GET /_insights/top_queries?type=memory ``` -{% include copy-curl.html %} \ No newline at end of file +{% include copy-curl.html %} + +## Export the top N queries data +You can configure your desired exporter to export the top N queries data to different sinks. Currently, supported exporters are the debug exporter and the local index exporter. + +### Configuring the debug exporter +To export the top queries by latency using the debug exporter, use the following configuration: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.latency.exporter.type" : "debug" + } +} +``` +{% include copy-curl.html %} + +To configure the debug exporter for other metrics such as `CPU`, use: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.cpu.exporter.type" : "debug" + } +} +``` +{% include copy-curl.html %} + +### Configuring the local index exporter +The local index exporter allows you to export the top N queries to local OpenSearch indexes. To configure the local index exporter for latency, use: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.latency.exporter.type" : "local_index", + "search.insights.top_queries.latency.exporter.config.index" : "YYYY.MM.dd" + } +} +``` +{% include copy-curl.html %} + +For reference on the date pattern format, see the [DateTimeFormat documentation](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). + +To configure the local index exporter for other metrics such as `CPU`, use: +```json +PUT _cluster/settings +{ + "persistent" : { + "search.insights.top_queries.cpu.exporter.type" : "local_index", + "search.insights.top_queries.cpu.exporter.config.index" : "YYYY.MM.dd" + } +} +``` +{% include copy-curl.html %} + +By configuring these settings, you can direct the query insights data to the appropriate sink, allowing for better monitoring and analysis of your OpenSearch queries. \ No newline at end of file From cc6626089469a665019b2bcb2756f397a07b5f93 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Fri, 14 Jun 2024 14:26:03 -0400 Subject: [PATCH 4/7] Doc review Signed-off-by: Fanit Kolchina --- .../query-insights/top-n-queries.md | 130 +++++------------- 1 file changed, 31 insertions(+), 99 deletions(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 691c223190..8848894bbd 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -14,14 +14,14 @@ Monitoring the top N queries in query insights features can help you gain real-t To enable monitoring of the top N queries, configure the following [dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings): - `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries by latency](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.latency.window_size`: [Configure the window size for top n queries by latency](#configuring-window-size). -- `search.insights.top_queries.latency.top_n_size`: [Specify the value of n for top n queries by latency](#configuring-the-value-of-n). -- `search.insights.top_queries.cpu.enabled`: Set to `true` to [enable monitoring of the top N queries by cpu usage](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.cpu.window_size`: [Configure the window size for top n queries by cpu usage](#configuring-window-size). -- `search.insights.top_queries.cpu.top_n_size`: [Specify the value of n for top n queries by cpu usage](#configuring-the-value-of-n). -- `search.insights.top_queries.memory.enabled`: Set to `true` to [enable monitoring of the top N queries for top n queries by memory usage](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.memory.window_size`: [Configure the window size for top n queries by memory usage](#configuring-window-size). -- `search.insights.top_queries.memory.top_n_size`: [Specify the value of n for top n queries by memory usage](#configuring-the-value-of-n). +- `search.insights.top_queries.latency.window_size`: [Configure the window size the top N queries by latency](#configuring-window-size). +- `search.insights.top_queries.latency.top_n_size`: [Specify the value of N for the top N queries by latency](#configuring-the-value-of-n). +- `search.insights.top_queries.cpu.enabled`: Set to `true` to [enable monitoring of the top N queries by CPU usage](#enabling-the-top-n-queries-feature). +- `search.insights.top_queries.cpu.window_size`: [Configure the window size for the top N queries by CPU usage](#configuring-window-size). +- `search.insights.top_queries.cpu.top_n_size`: [Specify the value of N for the top N queries by CPU usage](#configuring-the-value-of-n). +- `search.insights.top_queries.memory.enabled`: Set to `true` to [enable monitoring of the top N queries by memory usage](#enabling-the-top-n-queries-feature). +- `search.insights.top_queries.memory.window_size`: [Configure the window size for the top N queries by memory usage](#configuring-window-size). +- `search.insights.top_queries.memory.top_n_size`: [Specify the value of N for the top N queries by memory usage](#configuring-the-value-of-n). It's important to exercise caution when enabling this feature because it can consume system resources. {: .important} @@ -29,9 +29,9 @@ It's important to exercise caution when enabling this feature because it can con For detailed information about enabling and configuring this feature, see the following sections. -## Enabling the top N queries feature +## Enabling top N query monitoring -After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster. For example, the following command enables top n queries by latency. +When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired monitoring types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable monitoring top N queries by latency, update the `search.insights.top_queries.latency.enabled` setting: ```json PUT _cluster/settings @@ -43,27 +43,9 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -Other available metrics are CPU usage and memory usage. You can enable them with -```json -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.cpu.enabled" : true - } -} +## Configuring the window size -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.memory.enabled" : true - } -} -``` -{% include copy-curl.html %} - -## Configuring window size - -You can configure the window size for the top N queries by latency with `search.insights.top_queries.latency.window_size`. For example, a cluster with the following configuration will collect top N queries by latency in a 60-minute window: +To configure the monitoring window size, update the `window_size` setting for the desired monitoring type. For example, to collect the top N queries by latency in a 60-minute window, update the `search.insights.top_queries.latency.window_size` setting: ```json PUT _cluster/settings @@ -75,27 +57,9 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -Other available metrics are CPU usage and memory usage. You can configure the window size to 60 minutes for these metrics by: -```json -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.cpu.window_size" : "60m" - } -} - -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.memory.window_size" : "60m" - } -} -``` -{% include copy-curl.html %} - ## Configuring the value of N -You can configure the value of N in the `search.insights.top_queries.latency.top_n_size` parameter. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size: +To configure the value of N, update the `top_n_size` setting for the desired monitoring type. For example, to collect the top 10 queries by latency, update the `insights.top_queries.latency.top_n_size` setting: ```json PUT _cluster/settings @@ -107,35 +71,16 @@ PUT _cluster/settings ``` {% include copy-curl.html %} - -Other available metrics are CPU usage and memory usage. You can configure the top n size to 10 for these metrics by: -```json -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.cpu.top_n_size" : 10 - } -} - -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.memory.top_n_size" : 10 - } -} -``` -{% include copy-curl.html %} - ## Monitoring the top N queries -You can use the Insights API endpoint to obtain top N queries by latency: +You can use the Insights API endpoint to obtain top N queries for all monitoring types: ```json GET /_insights/top_queries ``` {% include copy-curl.html %} -Specify a metric type to filter the response by metric type: +Specify a monitoring type to filter the response by metric type: ```json GET /_insights/top_queries?type=latency @@ -144,11 +89,16 @@ GET /_insights/top_queries?type=memory ``` {% include copy-curl.html %} -## Export the top N queries data -You can configure your desired exporter to export the top N queries data to different sinks. Currently, supported exporters are the debug exporter and the local index exporter. +## Export top N query data + +You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the supported exporters are: +- [Debug exporter](#configuring-a-debug-exporter) +- [Local index exporter](#configuring-a-local-index-exporter) + +### Configuring a debug exporter + +To configure a debug exporter, update the exporter setting for the desired monitoring type. For example, to export the top N queries by latency using the debug exporter, send the following request: -### Configuring the debug exporter -To export the top queries by latency using the debug exporter, use the following configuration: ```json PUT _cluster/settings { @@ -159,19 +109,12 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -To configure the debug exporter for other metrics such as `CPU`, use: -```json -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.cpu.exporter.type" : "debug" - } -} -``` -{% include copy-curl.html %} +To configure a debug exporter for other monitoring types, use the `search.insights.top_queries.cpu.exporter.type` or `search.insights.top_queries.memory.exporter.type` settings. + +### Configuring a local index exporter + +A local index exporter allows you to export the top N queries to local OpenSearch indexes. To configure the local index exporter for the top N queiries by latency, send the following request: -### Configuring the local index exporter -The local index exporter allows you to export the top N queries to local OpenSearch indexes. To configure the local index exporter for latency, use: ```json PUT _cluster/settings { @@ -183,18 +126,7 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -For reference on the date pattern format, see the [DateTimeFormat documentation](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). +For more information about date formats, see the [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). -To configure the local index exporter for other metrics such as `CPU`, use: -```json -PUT _cluster/settings -{ - "persistent" : { - "search.insights.top_queries.cpu.exporter.type" : "local_index", - "search.insights.top_queries.cpu.exporter.config.index" : "YYYY.MM.dd" - } -} -``` -{% include copy-curl.html %} +To configure a local index exporter for other monitoring types, use the `search.insights.top_queries.cpu.exporter.type` or `search.insights.top_queries.memory.exporter.type` settings. -By configuring these settings, you can direct the query insights data to the appropriate sink, allowing for better monitoring and analysis of your OpenSearch queries. \ No newline at end of file From da637fb736d8caa482ab10f1bf7d3308492b23ec Mon Sep 17 00:00:00 2001 From: Chenyang Ji Date: Mon, 17 Jun 2024 13:20:13 -0700 Subject: [PATCH 5/7] add document for default exporter pattern Signed-off-by: Chenyang Ji --- _observing-your-data/query-insights/top-n-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 8848894bbd..44b2def263 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -126,7 +126,7 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -For more information about date formats, see the [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). +The default index pattern for top N queries is `top_queries-YYYY.MM.dd`. This means that all top queries for the same day will be saved to the same index, and a new index will be created for the following day. For more information on date formats, refer to the [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). To configure a local index exporter for other monitoring types, use the `search.insights.top_queries.cpu.exporter.type` or `search.insights.top_queries.memory.exporter.type` settings. From 3de47395c11065d0df571624f2c0f2346c272f9b Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Tue, 18 Jun 2024 12:57:34 -0400 Subject: [PATCH 6/7] Consolidating metric types Signed-off-by: Fanit Kolchina --- .../query-insights/top-n-queries.md | 60 ++++++++++--------- 1 file changed, 31 insertions(+), 29 deletions(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 44b2def263..21c2925dc3 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -9,29 +9,28 @@ nav_order: 65 Monitoring the top N queries in query insights features can help you gain real-time insights into the top queries with high latency within a certain time frame (for example, the last hour). -## Getting started +## Configuring top N query monitoring -To enable monitoring of the top N queries, configure the following [dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings): +You can configure monitoring top N queries by the following metric types: -- `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries by latency](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.latency.window_size`: [Configure the window size the top N queries by latency](#configuring-window-size). -- `search.insights.top_queries.latency.top_n_size`: [Specify the value of N for the top N queries by latency](#configuring-the-value-of-n). -- `search.insights.top_queries.cpu.enabled`: Set to `true` to [enable monitoring of the top N queries by CPU usage](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.cpu.window_size`: [Configure the window size for the top N queries by CPU usage](#configuring-window-size). -- `search.insights.top_queries.cpu.top_n_size`: [Specify the value of N for the top N queries by CPU usage](#configuring-the-value-of-n). -- `search.insights.top_queries.memory.enabled`: Set to `true` to [enable monitoring of the top N queries by memory usage](#enabling-the-top-n-queries-feature). -- `search.insights.top_queries.memory.window_size`: [Configure the window size for the top N queries by memory usage](#configuring-window-size). -- `search.insights.top_queries.memory.top_n_size`: [Specify the value of N for the top N queries by memory usage](#configuring-the-value-of-n). +- `latency` +- `cpu` +- `memory` -It's important to exercise caution when enabling this feature because it can consume system resources. -{: .important} +Each metric has a set of corresponding settings: + +- `search.insights.top_queries..enabled`: Set to `true` to [enable top N query monitoring by the metric](#enabling-top-n-query-monitoring). +- `search.insights.top_queries..window_size`: [Configure the window size the top N queries by the metric](#configuring-the-window-size). +- `search.insights.top_queries..top_n_size`: [Specify the value of N for the top N queries by the metric](#configuring-the-value-of-n). +For example, to enable top N query monitoring by CPU usage, set `search.insights.top_queries.cpu.enabled` to `true`. For more information about ways to specify dynamic settings, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings). -For detailed information about enabling and configuring this feature, see the following sections. +It's important to exercise caution when enabling this feature because it can consume system resources. +{: .important} ## Enabling top N query monitoring -When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired monitoring types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable monitoring top N queries by latency, update the `search.insights.top_queries.latency.enabled` setting: +When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired metric types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable monitoring top N queries by latency, update the `search.insights.top_queries.latency.enabled` setting: ```json PUT _cluster/settings @@ -45,7 +44,7 @@ PUT _cluster/settings ## Configuring the window size -To configure the monitoring window size, update the `window_size` setting for the desired monitoring type. For example, to collect the top N queries by latency in a 60-minute window, update the `search.insights.top_queries.latency.window_size` setting: +To configure the monitoring window size, update the `window_size` setting for the desired metric type. For example, to collect the top N queries by latency in a 60-minute window, update the `search.insights.top_queries.latency.window_size` setting: ```json PUT _cluster/settings @@ -59,7 +58,7 @@ PUT _cluster/settings ## Configuring the value of N -To configure the value of N, update the `top_n_size` setting for the desired monitoring type. For example, to collect the top 10 queries by latency, update the `insights.top_queries.latency.top_n_size` setting: +To configure the value of N, update the `top_n_size` setting for the desired metric type. For example, to collect the top 10 queries by latency, update the `insights.top_queries.latency.top_n_size` setting: ```json PUT _cluster/settings @@ -73,23 +72,31 @@ PUT _cluster/settings ## Monitoring the top N queries -You can use the Insights API endpoint to obtain top N queries for all monitoring types: +You can use the Insights API endpoint to obtain top N queries for all metric types: ```json GET /_insights/top_queries ``` {% include copy-curl.html %} -Specify a monitoring type to filter the response by metric type: +Specify a metric type to filter the response: ```json GET /_insights/top_queries?type=latency +``` +{% include copy-curl.html %} + +```json GET /_insights/top_queries?type=cpu +``` +{% include copy-curl.html %} + +```json GET /_insights/top_queries?type=memory ``` {% include copy-curl.html %} -## Export top N query data +## Exporting top N query data You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the supported exporters are: - [Debug exporter](#configuring-a-debug-exporter) @@ -97,7 +104,7 @@ You can configure your desired exporter to export top N query data to different ### Configuring a debug exporter -To configure a debug exporter, update the exporter setting for the desired monitoring type. For example, to export the top N queries by latency using the debug exporter, send the following request: +To configure a debug exporter, update the exporter setting for the desired metric type. For example, to export the top N queries by latency using the debug exporter, send the following request: ```json PUT _cluster/settings @@ -109,11 +116,11 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -To configure a debug exporter for other monitoring types, use the `search.insights.top_queries.cpu.exporter.type` or `search.insights.top_queries.memory.exporter.type` settings. - ### Configuring a local index exporter -A local index exporter allows you to export the top N queries to local OpenSearch indexes. To configure the local index exporter for the top N queiries by latency, send the following request: +A local index exporter allows you to export the top N queries to local OpenSearch indexes. The default index pattern for top N query indexes is `top_queries-YYYY.MM.dd`. All top queries for the same day are saved to the same index, and a new index is created every day. You can change the default index pattern to use other date formats. For more information about supported formats, see [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). + +To configure the local index exporter for the top N queiries by latency, send the following request: ```json PUT _cluster/settings @@ -125,8 +132,3 @@ PUT _cluster/settings } ``` {% include copy-curl.html %} - -The default index pattern for top N queries is `top_queries-YYYY.MM.dd`. This means that all top queries for the same day will be saved to the same index, and a new index will be created for the following day. For more information on date formats, refer to the [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). - -To configure a local index exporter for other monitoring types, use the `search.insights.top_queries.cpu.exporter.type` or `search.insights.top_queries.memory.exporter.type` settings. - From db2dd2b74cb85083e2d80cde9bf2c49f86f22ac8 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 18 Jun 2024 13:35:34 -0400 Subject: [PATCH 7/7] Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- .../query-insights/top-n-queries.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md index 21c2925dc3..e6dadf33c5 100644 --- a/_observing-your-data/query-insights/top-n-queries.md +++ b/_observing-your-data/query-insights/top-n-queries.md @@ -11,7 +11,7 @@ Monitoring the top N queries in query insights features can help you gain real-t ## Configuring top N query monitoring -You can configure monitoring top N queries by the following metric types: +You can configure top N query monitoring by the following metric types: - `latency` - `cpu` @@ -19,8 +19,8 @@ You can configure monitoring top N queries by the following metric types: Each metric has a set of corresponding settings: -- `search.insights.top_queries..enabled`: Set to `true` to [enable top N query monitoring by the metric](#enabling-top-n-query-monitoring). -- `search.insights.top_queries..window_size`: [Configure the window size the top N queries by the metric](#configuring-the-window-size). +- `search.insights.top_queries..enabled`: Set to `true` to [enable top N query monitoring](#enabling-top-n-query-monitoring) by the metric. +- `search.insights.top_queries..window_size`: [Configure the window size of the top N queries](#configuring-the-window-size) by the metric. - `search.insights.top_queries..top_n_size`: [Specify the value of N for the top N queries by the metric](#configuring-the-value-of-n). For example, to enable top N query monitoring by CPU usage, set `search.insights.top_queries.cpu.enabled` to `true`. For more information about ways to specify dynamic settings, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings). @@ -30,7 +30,7 @@ It's important to exercise caution when enabling this feature because it can con ## Enabling top N query monitoring -When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired metric types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable monitoring top N queries by latency, update the `search.insights.top_queries.latency.enabled` setting: +When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired metric types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable top N query monitoring by latency, update the `search.insights.top_queries.latency.enabled` setting: ```json PUT _cluster/settings @@ -72,7 +72,7 @@ PUT _cluster/settings ## Monitoring the top N queries -You can use the Insights API endpoint to obtain top N queries for all metric types: +You can use the Insights API endpoint to obtain the top N queries for all metric types: ```json GET /_insights/top_queries @@ -98,7 +98,7 @@ GET /_insights/top_queries?type=memory ## Exporting top N query data -You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the supported exporters are: +You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the following exporters are supported: - [Debug exporter](#configuring-a-debug-exporter) - [Local index exporter](#configuring-a-local-index-exporter) @@ -118,9 +118,9 @@ PUT _cluster/settings ### Configuring a local index exporter -A local index exporter allows you to export the top N queries to local OpenSearch indexes. The default index pattern for top N query indexes is `top_queries-YYYY.MM.dd`. All top queries for the same day are saved to the same index, and a new index is created every day. You can change the default index pattern to use other date formats. For more information about supported formats, see [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). +A local index exporter allows you to export the top N queries to local OpenSearch indexes. The default index pattern for top N query indexes is `top_queries-YYYY.MM.dd`. All top queries from the same day are saved to the same index, and a new index is created each day. You can change the default index pattern to use other date formats. For more information about supported formats, see [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). -To configure the local index exporter for the top N queiries by latency, send the following request: +To configure the local index exporter for the top N queries by latency, send the following request: ```json PUT _cluster/settings