emphasize the folllowing cluster need to have correct version #117739
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… testProcessFileChanges #115280
…=reference/rest-api/usage/line_38} #113694
This reverts commit 4c15cc0. This commit introduced an orders of magnitude regression when searching many shards. Co-authored-by: Elastic Machine <[email protected]>
This PR exposes operator-defined, cluster-state role mappings in the [Get role mappings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-get-role-mapping.html). Cluster-state role mappings are returned with a reserved suffix `-read-only-operator-mapping`, to disambiguate with native role mappings stored in the security index. CS role mappings are also marked with a `_read_only` metadata flag. It's possible to query a CS role mapping using its name both with and without the suffix. CS role mappings can be viewed via the API, but cannot be modified. To clarify this, the PUT and DELETE role mapping endpoints return header warnings if native role mappings that name-clash with CS role mappings are created, modified, or deleted. The PR also prevents the creation or role mappings with names ending in `-read-only-operator-mapping` to ensure that CS role mappings and native role mappings can always be fully disambiguated. Finally, the PR changes how CS role mappings are persisted in cluster-state. CS role mappings are written (and read from disk) in the `XContent` format. This format omits the role mapping's name. This means that if CS role mappings are ever recovered from disk (e.g., during a master-node restart), their names are erased. To address this, this PR changes CS role mapping serialization to persist the name of a mapping in a reserved metadata field, and recover it from metadata during serialization. This allows us to persist the name without BWC-breaks in role mapping `XContent` format. It also allows us to ensure that role mappings are re-written to cluster state in the new, name-preserving format the first time operator file settings are processed. Depends on: #114295 Relates: ES-9628
Fix #115281 Let's disable the faulty optimization for now and re-introduce it later, correctly. (cherry picked from commit 98d5335) Co-authored-by: Alexander Spies <[email protected]>
(cherry picked from commit f3572f4)
Co-authored-by: Elastic Machine <[email protected]>
…ndIncompleteContent with JDK23 (#115197) (#115441) Resolves #115172 Co-authored-by: Pooya Salehi <[email protected]>
Long GC disruption relies on Thread.resume, which is removed in JDK 23. Tests that use it predate more modern disruption tests. This commit removes gc disruption and the master disruption tests. Note that tests relying on this scheme have already not been running since JDK 20 first deprecated Thread.resume.
Fix the test muting on the test for grapheme clusters - it should only allow the test if we're on the 20+ jvm. Closes #114536
In LogsDB we would like to use a default value of `8191` for the index-level setting `index.mapping.ignore_above`. The value for `ignore_above` is the _character count_, but Lucene counts bytes. Here we set the limit to `32766 / 4 = 8191` since UTF-8 characters may occupy at most 4 bytes. (cherry picked from commit 521e434) # Conflicts: # server/src/main/java/org/elasticsearch/common/settings/Setting.java Co-authored-by: Salvatore Campagna <[email protected]>
This fixes filtered aggs when they are grouped on a field with ordinals. This looks like: ``` | STATS max = max(salary) WHERE salary > 0 BY job_positions ``` when the `job_positions` field is a keyword field with doc values. In that case we use a faster group-by-segment-ordinals algorithm that needs to be able to merge the results of aggregators from multiple segments. This previously failed with a `ClassCastException` because of a mistake. Also! the group-by-segment-ordinals algorithm wasn't properly releasing the closure used to add inputs, causing a breaker size leak. This wasn't really leaking memory, but leaking *tracking* of memory. Closes #114897
Fixes some faulty assertions in an upgrade test. Test failures only manifest on the 8.16 branch since 9.x does not qualify for these upgrade tests, and the change is not backported to 8.17 yet (unrelated CI failures). I validated this works by running it locally from the 8.16 branch. Resolves: #115410 Resolves: #115411
If a node has been removed from the cluster and the trained model assignment has not been updated the GET stats action can have an inconsistent view where it thinks a model is deployed on the removed node. The bug only affected nodes with failed deployments.
(cherry picked from commit e581ae3) Co-authored-by: shainaraskas <[email protected]>
Co-authored-by: Liam Thompson <[email protected]>
* [Gradle] Remove static use of BuildParams (#115122) Static fields dont do well in Gradle with configuration cache enabled. - Use buildParams extension in build scripts - Keep BuildParams.ci for now for easy serverless migration - Tweak testing doc (cherry picked from commit 13c8aae) # Conflicts: # TESTING.asciidoc # build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionBwcSetupPlugin.java # build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/RestTestBasePlugin.java # build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/AbstractYamlRestCompatTestPlugin.java # build.gradle # modules/ingest-geoip/qa/full-cluster-restart/build.gradle # qa/mixed-cluster/build.gradle # x-pack/plugin/ent-search/qa/full-cluster-restart/build.gradle # x-pack/plugin/eql/qa/rest/build.gradle # x-pack/plugin/fleet/qa/rest/build.gradle # x-pack/plugin/kql/build.gradle # x-pack/plugin/mapper-unsigned-long/build.gradle # x-pack/plugin/ml/qa/multi-cluster-tests-with-security/build.gradle # x-pack/plugin/security/qa/multi-cluster/build.gradle # x-pack/plugin/sql/qa/jdbc/build.gradle # x-pack/plugin/transform/qa/multi-cluster-tests-with-security/build.gradle * Fix merge * [Build] Fix fips testing after buildparams rework (#116934) * More Cleanup * [Build] Fix checkstyle exclusions on windows (#115185) * More merge fixes * Delete x-pack/plugin/kql/build.gradle
* ESQL: fix COUNT filter pushdown (#117503) If `COUNT` agg has a filter applied, this must also be push down to source. This currently does not happen, but this issue is masked currently by two factors: * a logical optimisation, `ExtractAggregateCommonFilter` that extracts the filter out of the STATS entirely (and pushes it to source then from a `WHERE`); * the phisical plan optimisation implementing the push down, `PushStatsToSource`, currently only applies if there's just one agg function to push down. However, this fix needs to be applied since: * it's still present in versions prior to `ExtractAggregateCommonFilter` introduction; * the defect might resurface when the restriction in `PushStatsToSource` is lifted. Fixes #115522. (cherry picked from commit 560e0c5) * 8.16 adaptation
I do believe a new version compatibility matrix should be made for following index. But it might involve other issues that support not thinking about. The current matrix here would likely mislead the customer to think the following cluster can be older than the remote cluster.
|
Documentation preview: |
|
@Camilleli please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation. |
elasticsearchmachine
added
needs:triage
PeteGillinElastic
added
:Distributed Indexing/CCR
elasticsearchmachine
added
the
Team:Distributed Indexing
|
Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing) |
|
@Camilleli is there a reason this is targetted only for 8.16 and not |
|
@kingherc, my bad, I do like to merge it to the main. Properly some wrong clicks during opening this PR. I have updated it. Please let me know if it works. |
|
@Camilleli Probably need to update the PR and merge main to it, or rebase it on top of main. Because now it has >5000 files changed. Feel free to open a new PR if it's more straightforward. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I do believe a new version compatibility matrix should be made for following index. But it might involve other issues that support not thinking about. The current matrix here would likely mislead the customer to think the following cluster can be older than the remote cluster.