-
Notifications
You must be signed in to change notification settings - Fork 508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add document for top n queries improvements in 2.15 #7326
Changes from all commits
4a8360f
8173348
314bc92
39bc477
cc66260
d05ab3c
da637fb
3de4739
db2dd2b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,23 +9,28 @@ nav_order: 65 | |
|
||
Monitoring the top N queries in query insights features can help you gain real-time insights into the top queries with high latency within a certain time frame (for example, the last hour). | ||
|
||
## Getting started | ||
## Configuring top N query monitoring | ||
|
||
To enable monitoring of the top N queries, configure the following [dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings): | ||
You can configure top N query monitoring by the following metric types: | ||
|
||
- `search.insights.top_queries.latency.enabled`: Set to `true` to [enable monitoring of the top N queries](#enabling-the-top-n-queries-feature). | ||
- `search.insights.top_queries.latency.window_size`: [Configure the window size](#configuring-window-size). | ||
- `search.insights.top_queries.latency.top_n_size`: [Specify the value of n](#configuring-the-value-of-n). | ||
- `latency` | ||
- `cpu` | ||
- `memory` | ||
|
||
It's important to exercise caution when enabling this feature because it can consume system resources. | ||
{: .important} | ||
Each metric has a set of corresponding settings: | ||
|
||
- `search.insights.top_queries.<metric>.enabled`: Set to `true` to [enable top N query monitoring](#enabling-top-n-query-monitoring) by the metric. | ||
- `search.insights.top_queries.<metric>.window_size`: [Configure the window size of the top N queries](#configuring-the-window-size) by the metric. | ||
- `search.insights.top_queries.<metric>.top_n_size`: [Specify the value of N for the top N queries by the metric](#configuring-the-value-of-n). | ||
|
||
For example, to enable top N query monitoring by CPU usage, set `search.insights.top_queries.cpu.enabled` to `true`. For more information about ways to specify dynamic settings, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "by CPU usage" => "of CPU usage"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, you monitor queries that use the most CPU ("by") |
||
|
||
For detailed information about enabling and configuring this feature, see the following sections. | ||
It's important to exercise caution when enabling this feature because it can consume system resources. | ||
{: .important} | ||
|
||
## Enabling the top N queries feature | ||
## Enabling top N query monitoring | ||
|
||
After installing the `query-insights` plugin, you can enable the top N queries feature (which is disabled by default) by using the following dynamic setting. This setting enables the corresponding collectors and aggregators in the running cluster: | ||
When you install the `query-insights` plugin, top N query monitoring is disabled by default. To enable top N query monitoring, update the dynamic settings for the desired metric types. These settings enable the corresponding collectors and aggregators in the running cluster. For example, to enable top N query monitoring by latency, update the `search.insights.top_queries.latency.enabled` setting: | ||
|
||
```json | ||
PUT _cluster/settings | ||
|
@@ -37,9 +42,9 @@ PUT _cluster/settings | |
``` | ||
{% include copy-curl.html %} | ||
|
||
## Configuring window size | ||
## Configuring the window size | ||
|
||
You can configure the window size for the top N queries by latency with `search.insights.top_queries.latency.window_size`. For example, a cluster with the following configuration will collect top N queries in a 60-minute window: | ||
To configure the monitoring window size, update the `window_size` setting for the desired metric type. For example, to collect the top N queries by latency in a 60-minute window, update the `search.insights.top_queries.latency.window_size` setting: | ||
|
||
```json | ||
PUT _cluster/settings | ||
|
@@ -53,9 +58,9 @@ PUT _cluster/settings | |
|
||
## Configuring the value of N | ||
|
||
You can configure the value of N in the `search.insights.top_queries.latency.top_n_size` parameter. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size: | ||
To configure the value of N, update the `top_n_size` setting for the desired metric type. For example, to collect the top 10 queries by latency, update the `insights.top_queries.latency.top_n_size` setting: | ||
|
||
``` | ||
```json | ||
PUT _cluster/settings | ||
{ | ||
"persistent" : { | ||
|
@@ -67,16 +72,63 @@ PUT _cluster/settings | |
|
||
## Monitoring the top N queries | ||
|
||
You can use the Insights API endpoint to obtain top N queries by latency: | ||
You can use the Insights API endpoint to obtain the top N queries for all metric types: | ||
|
||
```json | ||
GET /_insights/top_queries | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
Specify a metric type to filter the response by metric type (latency is the only supported type as of 2.12): | ||
Specify a metric type to filter the response: | ||
|
||
```json | ||
GET /_insights/top_queries?type=latency | ||
``` | ||
{% include copy-curl.html %} | ||
{% include copy-curl.html %} | ||
|
||
```json | ||
GET /_insights/top_queries?type=cpu | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
```json | ||
GET /_insights/top_queries?type=memory | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
## Exporting top N query data | ||
|
||
You can configure your desired exporter to export top N query data to different sinks, allowing for better monitoring and analysis of your OpenSearch queries. Currently, the following exporters are supported: | ||
- [Debug exporter](#configuring-a-debug-exporter) | ||
- [Local index exporter](#configuring-a-local-index-exporter) | ||
|
||
### Configuring a debug exporter | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not a full setting name so I'll leave without code font. |
||
To configure a debug exporter, update the exporter setting for the desired metric type. For example, to export the top N queries by latency using the debug exporter, send the following request: | ||
|
||
```json | ||
PUT _cluster/settings | ||
{ | ||
"persistent" : { | ||
"search.insights.top_queries.latency.exporter.type" : "debug" | ||
} | ||
} | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
### Configuring a local index exporter | ||
|
||
A local index exporter allows you to export the top N queries to local OpenSearch indexes. The default index pattern for top N query indexes is `top_queries-YYYY.MM.dd`. All top queries from the same day are saved to the same index, and a new index is created each day. You can change the default index pattern to use other date formats. For more information about supported formats, see [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html). | ||
|
||
To configure the local index exporter for the top N queries by latency, send the following request: | ||
|
||
```json | ||
PUT _cluster/settings | ||
{ | ||
"persistent" : { | ||
"search.insights.top_queries.latency.exporter.type" : "local_index", | ||
"search.insights.top_queries.latency.exporter.config.index" : "YYYY.MM.dd" | ||
} | ||
} | ||
``` | ||
{% include copy-curl.html %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.