Add document for top n queries improvements in 2.15 (opensearch-proje…

…ct#7326) * Add document for top n queries by cpu and memory Signed-off-by: Chenyang Ji <[email protected]> * update document to fix style checks Signed-off-by: Chenyang Ji <[email protected]> * combine exporter and metrics enhancements documents into one PR Signed-off-by: Chenyang Ji <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * add document for default exporter pattern Signed-off-by: Chenyang Ji <[email protected]> * Consolidating metric types Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Chenyang Ji <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: [email protected] <[email protected]>
leanneeliatra · Jul 24, 2024 · 1c305d8 · 1c305d8
1 parent c1ccb18
commit 1c305d8
Showing 1 changed file with 69 additions and 17 deletions.
diff --git a/_observing-your-data/query-insights/top-n-queries.md b/_observing-your-data/query-insights/top-n-queries.md
@@ -9,23 +9,28 @@ nav_order: 65
 
 Monitoring the top N queries in query insights features can help you gain real-time insights into the top queries with high latency within a certain time frame (for example, the last hour). 
 
-## Getting started
+## Configuring top N query monitoring
 
-To enable monitoring of the top N queries, configure the following [dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings):
+You can configure top N query monitoring by the following metric types:
 
-- `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries](#enabling-the-top-n-queries-feature).
-- `search.insights.top_queries.latency.window_size`: [Configure the window size](#configuring-window-size). 
-- `search.insights.top_queries.latency.top_n_size`: [Specify the value of n](#configuring-the-value-of-n).
+- `latency`
+- `cpu`
+- `memory`
 
-It's important to exercise caution when enabling this feature because it can consume system resources.
-{: .important}
+Each metric has a set of corresponding settings:
+
+- `search.insights.top_queries.<metric>.enabled`: Set to `true` to [enable top N query monitoring](#enabling-top-n-query-monitoring) by the metric.
+- `search.insights.top_queries.<metric>.window_size`: [Configure the window size of the top N queries](#configuring-the-window-size) by the metric. 
+- `search.insights.top_queries.<metric>.top_n_size`: [Specify the value of N for the top N queries by the metric](#configuring-the-value-of-n).
 
+For example, to enable top N query monitoring by CPU usage, set `search.insights.top_queries.cpu.enabled` to `true`. For more information about ways to specify dynamic settings, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings).
 
-For detailed information about enabling and configuring this feature, see the following sections.
+It's important to exercise caution when enabling this feature because it can consume system resources.
+{: .important}
 
-## Enabling the top N queries feature 
+## Enabling top N query monitoring 
 
-After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster:
+When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired metric types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable top N query monitoring by latency, update the `search.insights.top_queries.latency.enabled` setting:
 
 ```json
 PUT _cluster/settings
@@ -37,9 +42,9 @@ PUT _cluster/settings
 ```
 {% include copy-curl.html %}
 
-## Configuring window size
+## Configuring the window size
 
-You can configure the window size for the top N queries by latency with `search.insights.top_queries.latency.window_size`. For example, a cluster with the following configuration will collect top N queries in a 60-minute window:
+To configure the monitoring window size, update the `window_size` setting for the desired metric type. For example, to collect the top N queries by latency in a 60-minute window, update the `search.insights.top_queries.latency.window_size` setting:
 
 ```json
 PUT _cluster/settings
@@ -53,9 +58,9 @@ PUT _cluster/settings
 
 ## Configuring the value of N 
 
-You can configure the value of N in the `search.insights.top_queries.latency.top_n_size` parameter. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size:
+To configure the value of N, update the `top_n_size` setting for the desired metric type. For example, to collect the top 10 queries by latency, update the `insights.top_queries.latency.top_n_size` setting:
 
-```
+```json
 PUT _cluster/settings
 {
   "persistent" : {
@@ -67,16 +72,63 @@ PUT _cluster/settings
 
 ## Monitoring the top N queries 
 
-You can use the Insights API endpoint to obtain top N queries by latency:
+You can use the Insights API endpoint to obtain the top N queries for all metric types:
 
 ```json
 GET /_insights/top_queries
 ```
 {% include copy-curl.html %}
 
-Specify a metric type to filter the response by metric type (latency is the only supported type as of 2.12):
+Specify a metric type to filter the response:
 
 ```json
 GET /_insights/top_queries?type=latency
 ```
-{% include copy-curl.html %}
+{% include copy-curl.html %}
+
+```json
+GET /_insights/top_queries?type=cpu
+```
+{% include copy-curl.html %}
+
+```json
+GET /_insights/top_queries?type=memory
+```
+{% include copy-curl.html %}
+
+## Exporting top N query data
+
+You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the following exporters are supported:
+- [Debug exporter](#configuring-a-debug-exporter)
+- [Local index exporter](#configuring-a-local-index-exporter)
+
+### Configuring a debug exporter
+
+To configure a debug exporter, update the exporter setting for the desired metric type. For example, to export the top N queries by latency using the debug exporter, send the following request:
+
+```json
+PUT _cluster/settings
+{
+  "persistent" : {
+     "search.insights.top_queries.latency.exporter.type" : "debug"
+  }
+}
+```
+{% include copy-curl.html %}
+
+### Configuring a local index exporter
+
+A local index exporter allows you to export the top N queries to local OpenSearch indexes. The default index pattern for top N query indexes is `top_queries-YYYY.MM.dd`. All top queries from the same day are saved to the same index, and a new index is created each day. You can change the default index pattern to use other date formats. For more information about supported formats, see [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html).
+
+To configure the local index exporter for the top N queries by latency, send the following request:
+
+```json
+PUT _cluster/settings
+{
+  "persistent" : {
+    "search.insights.top_queries.latency.exporter.type" : "local_index",
+    "search.insights.top_queries.latency.exporter.config.index" : "YYYY.MM.dd"
+  }
+}
+```
+{% include copy-curl.html %}