Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x] Allow archive and searchable snapshots indices in N-2 version #118923

Merged
merged 6 commits into from
Dec 19, 2024

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Dec 18, 2024

This is the backport of #118941 for 8.18.

This change relaxes the index compatibility version checks to allow archive and searchable snapshot indices in version N-2 to exist on a 9.x cluster. It uses the min. read-only index compatible version added in #118884 to accept join requests from 9.x nodes that can read indices created in version 7.x (N-2) as long as they have a write block and are either archive or searchable snapshot indices.

Relates ES-10274

@tlrx tlrx force-pushed the 2024/12/18/ES-10274-8.x branch from 8ecac81 to a637679 Compare December 18, 2024 09:06
@tlrx tlrx changed the title [8.x] Allow read-only compatible indices in N-2 version [8.x] Allow archive and searchable snapshots indices in N-2 version Dec 18, 2024
@tlrx tlrx added :Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. backport labels Dec 18, 2024
@tlrx tlrx marked this pull request as ready for review December 18, 2024 11:16
@@ -124,7 +124,7 @@ public static IndexVersion current() {
}

public boolean isLegacyIndexVersion() {
return before(IndexVersions.MINIMUM_COMPATIBLE);
return before(IndexVersions.MINIMUM_READONLY_COMPATIBLE);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On 8.18, MINIMUM_COMPATIBLE is equal to MINIMUM_READONLY_COMPATIBLE as there is no N-2 support, but I changed it here to make it uniform with 9.x.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder about this. Why do we need MINIMUM_READONLY_COMPATIBLE in 8.x if it's the same as MINIMUM_COMPATIBLE? It seems like the code in 8.x could be much simpler, although it would diverge from main? Anything I am missing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you're not missing anything. Keeping the code similar on both branches helps for backporting changes and I suspect we will have more things to backport in a short future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, up to you!

@@ -1327,12 +1328,16 @@ public ClusterState execute(ClusterState currentState) {
request.indexSettings(),
request.ignoreIndexSettings()
);
if (snapshotIndexMetadata.getCompatibilityVersion().before(minIndexCompatibilityVersion)) {
if (snapshotIndexMetadata.getCompatibilityVersion().isLegacyIndexVersion()) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, it does not have an impact on 8.x but makes the code look similar to main.

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I think this can go in independently of the 9.0 change?

.settings(
Settings.builder()
.put(INDEX_STORE_TYPE_SETTING.getKey(), SearchableSnapshotsSettings.SEARCHABLE_SNAPSHOT_STORE_TYPE)
.put(IndexMetadata.SETTING_VERSION_CREATED, IndexVersions.MINIMUM_READONLY_COMPATIBLE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got fooled here, I wonder if you should comment (or assert) that readonly==regular compatibility to explain to reader?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertion added

@tlrx
Copy link
Member Author

tlrx commented Dec 18, 2024

Thanks for the quick review!

I think this can go in independently of the 9.0 change?

I think so, if we are OK leaving a time window (until the 9.0 change is merged) during which indices can be accepted without being really supported on 9.0.

@henningandersen
Copy link
Contributor

I think so, if we are OK leaving a time window (until the 9.0 change is merged) during which indices can be accepted without being really supported on 9.0.

Ah, right, let us hold off on it then until both are ready - unless you need to build on top of it?

@tlrx tlrx merged commit a716ede into elastic:8.x Dec 19, 2024
15 checks passed
@tlrx tlrx deleted the 2024/12/18/ES-10274-8.x branch December 19, 2024 13:07
@tlrx
Copy link
Member Author

tlrx commented Dec 19, 2024

Thanks Henning & Luca!

tlrx added a commit that referenced this pull request Dec 19, 2024
This change (along with the required change #118923 for 8.18) 
relaxes the index compatibility version checks to allow archive 
and searchable snapshot indices in version N-2 to exist on a 9.x 
cluster.

Relates ES-10274
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport :Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. v8.18.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants