Archive-Index upgrade compatibility #118599
Conversation
drempapis
added
>test
…:drempapis/elasticsearch into test/archive-index-version-compatibility
server/src/main/java/org/elasticsearch/cluster/coordination/NodeJoinExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/cluster/metadata/IndexMetadataVerifier.java
Outdated
Show resolved
Hide resolved
...rc/javaRestTest/java/org/elasticsearch/lucene/AbstractArchiveIndexCompatibilityTestCase.java
Outdated
Show resolved
Hide resolved
...rc/javaRestTest/java/org/elasticsearch/lucene/AbstractArchiveIndexCompatibilityTestCase.java
Outdated
Show resolved
Hide resolved
...ompatibility/src/javaRestTest/java/org/elasticsearch/lucene/ArchiveIndexCompatibilityIT.java
Outdated
Show resolved
Hide resolved
...ompatibility/src/javaRestTest/java/org/elasticsearch/lucene/ArchiveIndexCompatibilityIT.java
Outdated
Show resolved
Hide resolved
drempapis
changed the title
|
|
||
| import org.elasticsearch.test.cluster.util.Version; | ||
|
|
||
| public class RestoreFromVersion5IT extends ArchiveIndexTestCase { |
There was a problem hiding this comment.
Because the Gradle setup is not flexible, I needed to divide the different base versions into separate classes. For each case, we need a fresh cluster with version Current-1.
| } | ||
| } | ||
|
|
||
| public final void verifyArchiveIndexCompatibility(String version) throws Exception { |
There was a problem hiding this comment.
The Searchable snapshot test will also use this template and implement the abstract recovery method to mount a snapshot.
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
| * Restores snapshots from old-clusters (version 5/6) and upgrades it to the current version. | ||
| * Test methods are executed after each upgrade. | ||
| */ | ||
| public class ArchiveIndexTestCase extends AbstractUpgradeCompatibilityTestCase { |
There was a problem hiding this comment.
I think I could use some upgrade mention in this class name as well given it's what it tests? You probably wanted to make a distinction between archive indices restored from snapshot and those mounted as searchable snapshots?
There was a problem hiding this comment.
Correct, that was my intention. I considered that the Upgrade functionality was implicitly implied by the parent class, where this one denotes that the test is for ArchiveIndex. The same logic has been implemented for SearchableSnapshot in this pr
| * Test suite for Archive indices backward compatibility with N-2 versions. | ||
| * The test suite creates a cluster in the N-1 version, where N is the current version. | ||
| * Restores snapshots from old-clusters (version 5/6) and upgrades it to the current version. | ||
| * Test methods are executed after each upgrade. |
There was a problem hiding this comment.
I think it would help to provide an example with concrete versions as well., for clarity
| * Overrides the snapshot-restore operation for archive-indices scenario. | ||
| */ | ||
| @Override | ||
| public void recover(RestClient client, String repository, String snapshot, String index) throws Exception { |
There was a problem hiding this comment.
I would call this method importArchiveIndex or something along those lines.
There was a problem hiding this comment.
Using the template pattern, this template placeholder will be used for the archive-index and searchable-snapshots tests. This method implements a restore operation for archive-index tests and a mount operation for searchable snapshots. Yes, by changing the name to importIndex fits better.
|
@cbuescher would you like to take a look as well? |
| && indexMetadata.getCreationVersion().isLegacyIndexVersion() | ||
| && indexMetadata.getCompatibilityVersion().onOrAfter(IndexVersions.MINIMUM_READONLY_COMPATIBLE)) { | ||
| return; | ||
| } |
There was a problem hiding this comment.
@tlrx are you good with this special case here and the one above that targets archive indices? We are addressing the need to upgrade the compatibility version for archive indices imported first in 8.x, that don't allow to upgrade to 9.x otherwise.
There was a problem hiding this comment.
I don't think this is going to work during rolling upgrades, because the master may still be a 8.x node and therefore will reject join requests from 9.x, because a 8.x master will still think that 9.x nodes cannot support indices with a 7.x compatibility version.
I think it does work in the case of a full cluster restart though, since the node will be directly be 9.x with the new logic here.
We have to change that logic to also support searchable snapshot N-2 indices, maybe I can do it for both types of indices at once.
There was a problem hiding this comment.
good point Tanguy, thanks for raising this. I guess we are better off waiting for your changes then.
There was a problem hiding this comment.
I replicated the same test under the qa/rolling-upgrade project to verify the behavior. It fails in the line,
NodeJoinExecutor.ensureIndexCompatibility(NodeJoinExecutor.java:386)
when upgrading any of the nodes.
There was a problem hiding this comment.
I also got stuck in this area (IndexMetadataVerifier#checkSupportedVersion) when trying to revert one of the deleted FullClusterRestartIT tests. My quick fix of changing IndexVersions.MINIMUM_COMPATIBLE to IndexVersions.MINIMUM_READ_ONLY_COMPATIBLE in
led to other startup issues on full cluster restart, I haven't succeeded in exactly debugging those yet but will continue to investigate.There was a problem hiding this comment.
Sounds like we may need distrib help with some of these issues @cbuescher , or at least some of the changes they are working on.
There was a problem hiding this comment.
I opened #118941 (with #118923 for 8.18) that should allow archive indices in 7.x to exist on a 9.x cluster. I will try to test the change for archive too but in the case you're interested here is the change for IndexMetadataVerifier.
In this pr, archive-indices upgrade compatibility is tested.
The test suite creates a cluster in the N-1 version, where N is the current version. Restores snapshots from old clusters (version 5/6) and upgrades it to the current version. Test methods are executed after each upgrade.
This is based on #118363 and closes ES-10200