Merge remote-tracking branch 'upstream/main' into collapse_with_rescore

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerCli.java

@@ -237,6 +237,11 @@ public void close() {
         }
     }
 
+    // allow subclasses to access the started process
+    protected ServerProcess getServer() {
+        return server;
+    }
+
     // protected to allow tests to override
     protected Command loadTool(String toolname, String libs) {
         return CliToolProvider.load(toolname, libs).create();

docs/changelog/107435.yaml

@@ -0,0 +1,6 @@
+pr: 107435
+summary: '`NoSuchRemoteClusterException` should not be thrown when a remote is configured'
+area: Network
+type: bug
+issues:
+ - 107381

docs/changelog/107598.yaml

@@ -0,0 +1,5 @@
+pr: 107598
+summary: Fix bulk NPE when retrying failure redirect after cluster block
+area: Data streams
+type: bug
+issues: []

docs/changelog/107739.yaml

@@ -0,0 +1,6 @@
+pr: 107739
+summary: Binary field enables doc values by default for index mode with synthe…
+area: Mapping
+type: enhancement
+issues:
+ - 107554

docs/changelog/107764.yaml

@@ -0,0 +1,5 @@
+pr: 107764
+summary: Increase size of big arrays only when there is an actual value in the aggregators
+area: Aggregations
+type: enhancement
+issues: []

docs/changelog/107785.yaml

@@ -0,0 +1,5 @@
+pr: 107785
+summary: Fix `minimized_round_trips` in lookup runtime fields
+area: Search
+type: bug
+issues: []

docs/reference/connector/apis/cancel-connector-sync-job-api.asciidoc

@@ -15,7 +15,7 @@ Cancels a connector sync job.
 [[cancel-connector-sync-job-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_sync_job_id` parameter should reference an existing connector sync job.
 
 [[cancel-connector-sync-job-api-desc]]
@@ -49,4 +49,3 @@ The following example cancels the connector sync job with ID `my-connector-sync-
 PUT _connector/_sync_job/my-connector-sync-job-id/_cancel
 ----
 // TEST[skip:there's no way to clean up after creating a connector sync job, as we don't know the id ahead of time. Therefore, skip this test.]
-

docs/reference/connector/apis/check-in-connector-api.asciidoc

@@ -16,7 +16,7 @@ Updates the `last_seen` field of a connector with current timestamp.
 [[check-in-connector-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[check-in-connector-api-path-params]]

docs/reference/connector/apis/check-in-connector-sync-job-api.asciidoc

@@ -15,7 +15,7 @@ Checks in a connector sync job (updates `last_seen` to the current time).
 [[check-in-connector-sync-job-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_sync_job_id` parameter should reference an existing connector sync job.
 
 [[check-in-connector-sync-job-api-desc]]

docs/reference/connector/apis/connector-apis.asciidoc

@@ -5,7 +5,7 @@ preview::[]
 
 The connector and sync jobs APIs provide a convenient way to create and manage Elastic {enterprise-search-ref}/connectors.html[connectors^] and sync jobs in an internal index.
 
-Connectors are third-party {es} integrations which can be deployed on {ecloud} or hosted on your own infrastructure:
+Connectors are {es} integrations that bring content from third-party data sources, which can be deployed on {ecloud} or hosted on your own infrastructure:
 
 * *Native connectors* are a managed service on {ecloud}
 * *Connector clients* are self-managed on your infrastructure

docs/reference/connector/apis/create-connector-api.asciidoc

@@ -7,7 +7,7 @@
 preview::[]
 
 Creates an Elastic connector.
-Connectors are third-party {es} integrations which can be deployed on {ecloud} or hosted on your own infrastructure:
+Connectors are {es} integrations that bring content from third-party data sources, which can be deployed on {ecloud} or hosted on your own infrastructure:
 
 * *Native connectors* are a managed service on {ecloud}
 * *Connector clients* are self-managed on your infrastructure
@@ -41,8 +41,8 @@ DELETE _connector/my-connector
 [[create-connector-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
-* The `service_type` parameter should reference an existing connector service type.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
+* The `service_type` parameter should reference a supported third-party service. See the available service types for {enterprise-search-ref}/native-connectors.html[native] and {enterprise-search-ref}/build-connector.html[self-managed] connectors. This can also reference the service type of your custom connector.
 
 
 [[create-connector-api-desc]]
@@ -65,10 +65,10 @@ Creates a connector document in the internal index and initializes its configura
 (Optional, string) The description of the connector.
 
 `index_name`::
-(Required, string) The target index for syncing data by the connector.
+(Optional, string) The target index to sync data. If the index doesn't exist, it will be created upon the first sync.
 
 `name`::
-(Optional, string) The name of the connector.
+(Optional, string) The name of the connector. Setting the connector name is recommended when managing connectors in {kib}.
 
 `is_native`::
 (Optional, boolean) Indicates if it's a native connector. Defaults to `false`.
@@ -77,7 +77,7 @@ Creates a connector document in the internal index and initializes its configura
 (Optional, string) Language analyzer for the data. Limited to supported languages.
 
 `service_type`::
-(Optional, string) Connector service type. Can reference Elastic-supported connector types or a custom connector type.
+(Optional, string) Connector service type. Can reference Elastic-supported third-party services or a custom connector type. See the available service types for {enterprise-search-ref}/native-connectors.html[native] and {enterprise-search-ref}/build-connector.html[self-managed] connectors.
 
 
 [role="child_attributes"]

docs/reference/connector/apis/create-connector-sync-job-api.asciidoc

@@ -29,7 +29,7 @@ POST _connector/_sync_job
 [[create-connector-sync-job-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `id` parameter should reference an existing connector.
 
 [[create-connector-sync-job-api-desc]]

docs/reference/connector/apis/delete-connector-api.asciidoc

@@ -6,9 +6,11 @@
 
 preview::[]
 
-Removes a connector and its associated data.
+Removes a connector and associated sync jobs.
 This is a destructive action that is not recoverable.
 
+Note: this action doesn't delete any API key, ingest pipeline or data index associated with the connector. These need to be removed manually.
+
 [[delete-connector-api-request]]
 ==== {api-request-title}
 
@@ -17,7 +19,7 @@ This is a destructive action that is not recoverable.
 [[delete-connector-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[delete-connector-api-path-params]]

docs/reference/connector/apis/delete-connector-sync-job-api.asciidoc

@@ -17,7 +17,7 @@ This is a destructive action that is not recoverable.
 [[delete-connector-sync-job-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 
 [[delete-connector-sync-job-api-path-params]]
 ==== {api-path-parms-title}

docs/reference/connector/apis/get-connector-api.asciidoc

@@ -16,7 +16,7 @@ Retrieves the details about a connector.
 [[get-connector-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 
 [[get-connector-api-path-params]]
 ==== {api-path-parms-title}

docs/reference/connector/apis/get-connector-sync-job-api.asciidoc

@@ -16,7 +16,7 @@ Retrieves the details about a connector sync job.
 [[get-connector-sync-job-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 
 [[get-connector-sync-job-api-path-params]]
 ==== {api-path-parms-title}

docs/reference/connector/apis/list-connector-sync-jobs-api.asciidoc

@@ -18,7 +18,7 @@ Returns information about all stored connector sync jobs ordered by their creati
 [[list-connector-sync-jobs-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 
 [[list-connector-sync-jobs-api-path-params]]
 ==== {api-path-parms-title}

docs/reference/connector/apis/list-connectors-api.asciidoc

@@ -7,7 +7,7 @@
 
 preview::[]
 
-Returns information about all stored connectors.
+Returns information about all created connectors.
 
 
 [[list-connector-api-request]]
@@ -18,19 +18,19 @@ Returns information about all stored connectors.
 [[list-connector-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 
 [[list-connector-api-path-params]]
 ==== {api-path-parms-title}
 
 `size`::
-(Optional, integer) Maximum number of results to retrieve.
+(Optional, integer) Maximum number of results to retrieve. Defaults to `100`.
 
 `from`::
-(Optional, integer) The offset from the first result to fetch.
+(Optional, integer) The offset from the first result to fetch. Defaults to `0`.
 
 `index_name`::
-(Optional, string) A comma-separated list of data index names associated with connectors, used to filter search results.
+(Optional, string) A comma-separated list of index names associated with connectors, used to filter search results.
 
 `connector_name`::
 (Optional, string) A comma-separated list of connector names, used to filter search results.
@@ -81,27 +81,27 @@ The following example lists the first two connectors:
 
 [source,console]
 ----
-GET _connector/?from=0&size=2
+GET _connector?from=0&size=2
 ----
 
-An example to list connectors associated with `search-google-drive` data index name:
+An example to list a connector associated with the `search-google-drive` Elasticsearch index:
 
 [source,console]
 ----
-GET _connector/?index_name=search-google-drive
+GET _connector?index_name=search-google-drive
 ----
 
 
 An example to list all connectors with `sharepoint_online` service type:
 
 [source,console]
 ----
-GET _connector/?service_type=sharepoint_online
+GET _connector?service_type=sharepoint_online
 ----
 
 An example to list all connectors with `sharepoint_online` or `google_drive` service type:
 
 [source,console]
 ----
-GET _connector/?service_type=sharepoint_online,google_drive
+GET _connector?service_type=sharepoint_online,google_drive
 ----

docs/reference/connector/apis/set-connector-sync-job-error-api.asciidoc

@@ -15,7 +15,7 @@ Sets a connector sync job error.
 [[set-connector-sync-job-error-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_sync_job_id` parameter should reference an existing connector sync job.
 
 [[set-connector-sync-job-error-api-desc]]

docs/reference/connector/apis/set-connector-sync-job-stats-api.asciidoc

@@ -15,7 +15,7 @@ Sets connector sync job stats.
 [[set-connector-sync-job-stats-api-prereqs]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_sync_job_id` parameter should reference an existing connector sync job.
 
 [[set-connector-sync-job-stats-api-desc]]

docs/reference/connector/apis/update-connector-api-key-id-api.asciidoc

@@ -23,7 +23,7 @@ See the documentation for {enterprise-search-ref}/native-connectors.html#native-
 [[update-connector-api-key-id-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 * The `api_key_id` parameter should reference an existing API key.
 * The `api_key_secret_id` parameter should reference an existing Connector Secret containing an encoded API key value.

docs/reference/connector/apis/update-connector-configuration-api.asciidoc

@@ -17,7 +17,7 @@ Updates a connector's `configuration`, allowing for complete schema modification
 [[update-connector-configuration-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 * The configuration fields definition must be compatible with the specific connector type being used.
 

docs/reference/connector/apis/update-connector-error-api.asciidoc

@@ -16,7 +16,7 @@ Updates the `error` field of a connector.
 [[update-connector-error-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-error-api-path-params]]

docs/reference/connector/apis/update-connector-filtering-api.asciidoc

@@ -17,7 +17,7 @@ Updates the `filtering` configuration of a connector. Learn more about filtering
 [[update-connector-filtering-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-filtering-api-path-params]]

docs/reference/connector/apis/update-connector-index-name-api.asciidoc

@@ -16,7 +16,7 @@ Updates the `index_name` field of a connector, specifying the index where the da
 [[update-connector-index-name-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-index-name-api-path-params]]

docs/reference/connector/apis/update-connector-last-sync-api.asciidoc

@@ -18,7 +18,7 @@ This action is used for analytics and monitoring.
 [[update-connector-last-sync-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-last-sync-api-path-params]]

docs/reference/connector/apis/update-connector-name-description-api.asciidoc

@@ -17,7 +17,7 @@ Updates the `name` and `description` fields of a connector.
 [[update-connector-name-description-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-name-description-api-path-params]]

docs/reference/connector/apis/update-connector-pipeline-api.asciidoc

@@ -18,7 +18,7 @@ When you create a new connector, the configuration of an <<ingest-pipeline-searc
 [[update-connector-pipeline-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-pipeline-api-path-params]]

docs/reference/connector/apis/update-connector-scheduling-api.asciidoc

@@ -16,7 +16,7 @@ Updates the `scheduling` configuration of a connector.
 [[update-connector-scheduling-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 
 [[update-connector-scheduling-api-path-params]]

docs/reference/connector/apis/update-connector-service-type-api.asciidoc

@@ -16,7 +16,7 @@ Updates the `service_type` of a connector.
 [[update-connector-service-type-api-prereq]]
 ==== {api-prereq-title}
 
-* To sync data using connectors, it's essential to have the Elastic connectors service running.
+* To sync data using self-managed connectors, you need to deploy the {enterprise-search-ref}/build-connector.html[Elastic connector service] on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
 * The `connector_id` parameter should reference an existing connector.
 * The `service_type` must be a valid type as defined by the Connector framework.
 ** When you change a configured connector's `service_type`, you'll also need to reset its configuration to ensure compatibility.