Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Change async query default setting #2563

Merged
merged 1 commit into from
Mar 19, 2024
Merged

[Backport 2.x] Change async query default setting #2563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/user/admin/settings.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,9 +318,9 @@ plugins.query.executionengine.spark.session.limit
Description
-----------

Each cluster can have maximum 100 sessions running in parallel by default. You can increase limit by this setting.
Each cluster can have maximum 10 sessions running in parallel by default. You can increase limit by this setting.

1. The default value is 100.
1. The default value is 10.
2. This setting is node scope.
3. This setting can be updated dynamically.

Expand Down Expand Up @@ -355,9 +355,9 @@ plugins.query.executionengine.spark.refresh_job.limit
Description
-----------

Each cluster can have maximum 20 datasources. You can increase limit by this setting.
Each cluster can have maximum 5 refresh job running concurrently. You can increase limit by this setting.

1. The default value is 20.
1. The default value is 5.
2. This setting is node scope.
3. This setting can be updated dynamically.

Expand Down Expand Up @@ -499,14 +499,14 @@ Description
This setting defines the time-to-live (TTL) for request indices when plugins.query.executionengine.spark.auto_index_management.enabled
is true. By default, request indices older than 14 days are deleted.

* Default Value: 14 days
* Default Value: 30 days

To change the TTL to 30 days for example, execute the following command:
To change the TTL to 60 days for example, execute the following command:

SQL query::

sh$ curl -sS -H 'Content-Type: application/json' -X PUT localhost:9200/_cluster/settings \
... -d '{"transient":{"plugins.query.executionengine.spark.session.index.ttl":"30d"}}'
... -d '{"transient":{"plugins.query.executionengine.spark.session.index.ttl":"60d"}}'
{
"acknowledged": true,
"persistent": {},
Expand All @@ -517,7 +517,7 @@ SQL query::
"spark": {
"session": {
"index": {
"ttl": "30d"
"ttl": "60d"
}
}
}
Expand Down
6 changes: 3 additions & 3 deletions opensearch/src/main/java/org/opensearch/sql/opensearch/setting/OpenSearchSettings.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,21 +147,21 @@ public class OpenSearchSettings extends Settings {
public static final Setting<?> SPARK_EXECUTION_SESSION_LIMIT_SETTING =
Setting.intSetting(
Key.SPARK_EXECUTION_SESSION_LIMIT.getKeyValue(),
100,
10,
Setting.Property.NodeScope,
Setting.Property.Dynamic);

public static final Setting<?> SPARK_EXECUTION_REFRESH_JOB_LIMIT_SETTING =
Setting.intSetting(
Key.SPARK_EXECUTION_REFRESH_JOB_LIMIT.getKeyValue(),
50,
5,
Setting.Property.NodeScope,
Setting.Property.Dynamic);

public static final Setting<TimeValue> SESSION_INDEX_TTL_SETTING =
Setting.positiveTimeSetting(
Key.SESSION_INDEX_TTL.getKeyValue(),
timeValueDays(14),
timeValueDays(30),
Setting.Property.NodeScope,
Setting.Property.Dynamic);

Expand Down
Loading