Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] InternalEngine to always use soft deletes #1933

Merged
merged 7 commits into from
Jan 26, 2022
Merged

[Refactor] InternalEngine to always use soft deletes #1933

merged 7 commits into from
Jan 26, 2022

Conversation

nknize
Copy link
Collaborator

@nknize nknize commented Jan 18, 2022

Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This PR refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

@nknize nknize added enhancement Enhancement or improvement to existing feature or request v2.0.0 Version 2.0.0 Indexing & Search labels Jan 18, 2022
@nknize nknize requested a review from a team as a code owner January 18, 2022 21:01
@opensearch-ci-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure bcfa7a9f598b5a9809e58d6f5328c51ae209128f
Log 1974

Reports 1974

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure a59fd51caa159a30a92aeb77c7daf57d26089ee5
Log 1976

Reports 1976

nknize added 3 commits January 25, 2022 10:02
@nknize
[Refactor] InternalEngine to always use soft deletes
be3b5b1 
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

Signed-off-by: Nicholas Walter Knize <[email protected]>
@nknize
spotless
a1b6ebc 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@nknize
fix failing tests add random w/o source testing
4665aa2 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@nknize nknize force-pushed the refactor/EngineAlwaysUsesSoftDeletes branch from a59fd51 to 4665aa2 Compare January 25, 2022 16:43
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4665aa2
Log 2046

Reports 2046

@nknize
spotless
384f836 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 384f836
Log 2047

Reports 2047

@nknize
fix failing tests
0303cdf 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 0303cdf
Log 2048

Reports 2048

@nknize
Copy link
Collaborator Author

nknize commented Jan 25, 2022

Repeat timeout failure reported in #1957

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation" -Dtests.seed=2DE1E5F63CA0EA53 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=el-GR -Dtests.timezone=America/Yakutat -Druntime.java=17

org.opensearch.cluster.routing.MovePrimaryFirstTests > testClusterGreenAfterPartialRelocation FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([2DE1E5F63CA0EA53:7E2B819317E49547]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:986)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:925)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:914)
        at org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation(MovePrimaryFirstTests.java:115)

@nknize
Copy link
Collaborator Author

nknize commented Jan 25, 2022

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 0303cdf
Log 2049

Reports 2049

@nknize
PR tweaks
7f04a9e 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 7f04a9e
Log 2051

Reports 2051

@nknize
revert random source testing
165ef98 
Signed-off-by: Nicholas Walter Knize <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 165ef98
Log 2057

Reports 2057

@nknize
Copy link
Collaborator Author

nknize commented Jan 26, 2022

Another #1908 failure

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery" -Dtests.seed=DBB99468CB97443F -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-Latn-ME -Dtests.timezone=Atlantic/Stanley -Druntime.java=17

org.opensearch.gateway.QuorumGatewayIT > testQuorumRecovery FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([DBB99468CB97443F:3E86A8BB6AEA9324]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:986)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:925)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:914)
        at org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery(QuorumGatewayIT.java:111)

@nknize
Copy link
Collaborator Author

nknize commented Jan 26, 2022

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 165ef98
Log 2059

Reports 2059

@nknize nknize merged commit 57ac788 into opensearch-project:main Jan 26, 2022
@nknize nknize mentioned this pull request Feb 9, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrross andrross andrross approved these changes

@reta reta reta approved these changes

Assignees
No one assigned
Labels
enhancement Enhancement or improvement to existing feature or request Indexing & Search v2.0.0 Version 2.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nknize @opensearch-ci-bot @reta @andrross