From 27800cfb69058b2778a71575eb0662fe5be6812b Mon Sep 17 00:00:00 2001 From: Bharathwaj G Date: Fri, 27 Oct 2023 16:13:23 +0530 Subject: [PATCH 1/3] Add documentation for resource usage stats Signed-off-by: Bharathwaj G --- _api-reference/nodes-apis/nodes-stats.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index d620be000c..73b9649937 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -51,6 +51,7 @@ adaptive_selection | Statistics about adaptive replica selection, which selects script_cache | Statistics about script cache. indexing_pressure | Statistics about the node's indexing pressure. shard_indexing_pressure | Statistics about shard indexing pressure. +resource_usage_stats | Resource usage statistics of nodes, such as CPU and JVM To filter the information returned for the `indices` metric, you can use specific `index_metric` values. You can use these only when you use the following query types: @@ -729,6 +730,8 @@ http.total_opened | Integer | The total number of HTTP connections the node has [indexing_pressure](#indexing_pressure) | Object | Statistics related to the node's indexing pressure. [shard_indexing_pressure](#shard_indexing_pressure) | Object | Statistics related to indexing pressure at the shard level. [search_backpressure]({{site.url}}{{site.baseurl}}/opensearch/search-backpressure#search-backpressure-stats-api) | Object | Statistics related to search backpressure. +[resource_usage_stats](#resourceusagestats) | Object | Statistics related to resource usage stats for the node. + ### `indices` @@ -1143,6 +1146,15 @@ total_rejections_breakup_shadow_mode.throughput_degradation_limits | Integer | T enabled | Boolean | Specifies whether the shard indexing pressure feature is turned on for the node. enforced | Boolean | If true, the shard indexing pressure runs in enforced mode (there are rejections). If false, the shard indexing pressure runs in shadow mode (there are no rejections, but statistics are recorded and can be retrieved in the `total_rejections_breakup_shadow_mode` object). Only applicable if shard indexing pressure is enabled. +### `resource_usage_stats` +The `resource_usage_stats` object contains the resource usage statistics. Each entry is specified by the node ID and has the following properties. + +Field | Field type | Description +:--- |:-----------| :--- +timestamp | Integer | The last refresh time for the resource usage statistics, in milliseconds since the epoch. +cpu_utilization_percent | Float | Statistics for the average CPU usage of OpenSearch process in the period configured in the setting `node.resource.tracker.global_cpu_usage.window_duration`. +memory_utilization_percent | Float | Statistics for the JVM memory usage on the node in the period configured in the setting `node.resource.tracker.global_jvmmp.window_duration` + ## Concurrent segment search Starting in OpenSearch 2.10, [concurrent segment search]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search/) allows each shard-level request to search segments in parallel during the query phase. If you [enable the experimental concurrent segment search feature flag]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search#enabling-the-feature-flag), the Nodes Stats API response will contain several additional fields with statistics about slices (units of work executed by a thread). For the descriptions of those fields, see [Index Stats API]({{site.url}}{{site.baseurl}}/api-reference/index-apis/stats#concurrent-segment-search). From adf517306aceec130a4274fd06c860ede15ca434 Mon Sep 17 00:00:00 2001 From: Bharathwaj G Date: Fri, 27 Oct 2023 16:36:37 +0530 Subject: [PATCH 2/3] Documentation changes to include resource usage stats Signed-off-by: Bharathwaj G --- _api-reference/nodes-apis/nodes-stats.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index 73b9649937..f6fcfdce49 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -677,6 +677,13 @@ Select the arrow to view the example response. }, "enabled" : false, "enforced" : false + }, + "resource_usage_stats": { + "nxLWtMdXQmWA-ZBVWU8nwA": { + "timestamp": 1698401391000, + "cpu_utilization_percent": "0.1", + "memory_utilization_percent": "3.9" + } } } } @@ -730,7 +737,7 @@ http.total_opened | Integer | The total number of HTTP connections the node has [indexing_pressure](#indexing_pressure) | Object | Statistics related to the node's indexing pressure. [shard_indexing_pressure](#shard_indexing_pressure) | Object | Statistics related to indexing pressure at the shard level. [search_backpressure]({{site.url}}{{site.baseurl}}/opensearch/search-backpressure#search-backpressure-stats-api) | Object | Statistics related to search backpressure. -[resource_usage_stats](#resourceusagestats) | Object | Statistics related to resource usage stats for the node. +[resource_usage_stats](#resource_usage_stats) | Object | Statistics related to resource usage stats for the node. ### `indices` From 78020c275ea006b02d990add9c39795a63e823ee Mon Sep 17 00:00:00 2001 From: Bharathwaj G Date: Fri, 3 Nov 2023 11:48:34 +0530 Subject: [PATCH 3/3] addressinc omments Signed-off-by: Bharathwaj G --- _api-reference/nodes-apis/nodes-stats.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index f6fcfdce49..8c92a0574a 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -51,7 +51,7 @@ adaptive_selection | Statistics about adaptive replica selection, which selects script_cache | Statistics about script cache. indexing_pressure | Statistics about the node's indexing pressure. shard_indexing_pressure | Statistics about shard indexing pressure. -resource_usage_stats | Resource usage statistics of nodes, such as CPU and JVM +resource_usage_stats | Node-level resource usage statistics, such as CPU and JVM memory. To filter the information returned for the `indices` metric, you can use specific `index_metric` values. You can use these only when you use the following query types: @@ -737,7 +737,7 @@ http.total_opened | Integer | The total number of HTTP connections the node has [indexing_pressure](#indexing_pressure) | Object | Statistics related to the node's indexing pressure. [shard_indexing_pressure](#shard_indexing_pressure) | Object | Statistics related to indexing pressure at the shard level. [search_backpressure]({{site.url}}{{site.baseurl}}/opensearch/search-backpressure#search-backpressure-stats-api) | Object | Statistics related to search backpressure. -[resource_usage_stats](#resource_usage_stats) | Object | Statistics related to resource usage stats for the node. +[resource_usage_stats](#resource_usage_stats) | Object | Statistics related to resource usage for the node. ### `indices` @@ -1159,8 +1159,8 @@ The `resource_usage_stats` object contains the resource usage statistics. Each e Field | Field type | Description :--- |:-----------| :--- timestamp | Integer | The last refresh time for the resource usage statistics, in milliseconds since the epoch. -cpu_utilization_percent | Float | Statistics for the average CPU usage of OpenSearch process in the period configured in the setting `node.resource.tracker.global_cpu_usage.window_duration`. -memory_utilization_percent | Float | Statistics for the JVM memory usage on the node in the period configured in the setting `node.resource.tracker.global_jvmmp.window_duration` +cpu_utilization_percent | Float | Statistics for the average CPU usage of OpenSearch process within the time period configured in the `node.resource.tracker.global_cpu_usage.window_duration` setting. +memory_utilization_percent | Float | The node JVM memory usage statistics within the time period configured in the `node.resource.tracker.global_jvmmp.window_duration` setting. ## Concurrent segment search