[Refactor] OpenSearchException streamables to a registry #7646

nknize · 2023-05-21T00:35:52Z

OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module.

This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively.

relates #5910

github-actions · 2023-05-21T01:02:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/15811/
CommitID: c955ba9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-05-21T01:08:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/15812/
CommitID: 57ea42f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

nknize · 2023-05-22T15:17:40Z

Not reproducible, rebasing and refiring

./gradlew ':server:test' --tests "org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex" -Dtests.seed=4212AD3BAE883BE9 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-BH -Dtests.timezone=CST6CDT -Druntime.java=19

github-actions · 2023-05-22T15:49:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/15895/
CommitID: f58e8d8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

nknize · 2023-05-22T16:30:18Z

another #6287, this was reproducible locally. Looks related to #6143, @RS146BIJAY can you take a look?

./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.shards.ClusterShardLimitIT.testCreateIndexWithMaxClusterShardSetting" -Dtests.seed=3094586F159455F2 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=lt-LT -Dtests.timezone=PLT -Druntime.java=19

org.opensearch.cluster.shards.ClusterShardLimitIT > testCreateIndexWithMaxClusterShardSetting FAILED
    java.lang.AssertionError: MaxShardPerCluster 2 should be greater than or equal to MaxShardPerNode 1000
        at __randomizedtesting.SeedInfo.seed([3094586F159455F2:7A72A9D1B8CA934A]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.cluster.shards.ClusterShardLimitIT.setMaxShardLimit(ClusterShardLimitIT.java:750)
        at org.opensearch.cluster.shards.ClusterShardLimitIT.testCreateIndexWithMaxClusterShardSetting(ClusterShardLimitIT.java:259)

github-actions · 2023-05-22T16:54:01Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/15906/
CommitID: f58e8d8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

nknize · 2023-05-22T17:18:13Z

Oye!! Another unrelated failure? Looks like PRs are becoming awfully volatile due to test failures not being regularly fixed.

This one was not reproducible but has already been documented as a "flaky test" and caught on multiple PRs.

./gradlew ':qa:smoke-test-http:integTest' --tests "org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase" -Dtests.seed=82B60F546F2663DE -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ru-RU -Dtests.timezone=Asia/Hebron -Druntime.java=19

github-actions · 2023-05-22T17:52:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/15921/
CommitID: f58e8d8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

nknize · 2023-05-22T18:03:40Z

Unreal.. two more (#3590)

Neither are reproducible locally.

./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards" -Dtests.seed=86C4F0E32ABC37F8 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=cs -Dtests.timezone=Atlantic/Jan_Mayen -Druntime.java=19

./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled" -Dtests.seed=86C4F0E32ABC37F8 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-CR -Dtests.timezone=Pacific/Pitcairn -Druntime.java=19

RS146BIJAY · 2023-05-22T18:06:00Z

testDelayWithALargeAmountOfShards

Sure will mute this test.

nknize · 2023-05-22T18:09:06Z

testDelayWithALargeAmountOfShards

Sure will mute this test.

That works.. let's try to resolve soon, though so @AwaitsFix doesn't turn into an alias of /dev/null :)

github-actions · 2023-05-22T18:39:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/15935/
CommitID: f58e8d8

codecov · 2023-05-22T18:42:43Z

Codecov Report

Merging #7646 (2bac9a6) into main (aa21585) will increase coverage by 0.13%.
The diff coverage is 96.88%.

@@             Coverage Diff              @@
##               main    #7646      +/-   ##
============================================
+ Coverage     70.78%   70.92%   +0.13%     
- Complexity    56251    56316      +65     
============================================
  Files          4689     4689              
  Lines        266304   266315      +11     
  Branches      39086    39086              
============================================
+ Hits         188505   188871     +366     
+ Misses        61847    61432     -415     
- Partials      15952    16012      +60

Impacted Files	Coverage Δ
.../org/opensearch/core/common/lease/Releasables.java	`78.78% <ø> (ø)`
...gregations/matrix/stats/MatrixStatsAggregator.java	`75.00% <ø> (-7.50%)`	⬇️
...aggregations/bucket/geogrid/GeoGridAggregator.java	`81.25% <ø> (ø)`
...regations/metrics/AbstractGeoBoundsAggregator.java	`56.52% <ø> (ø)`
...search/join/aggregations/ParentJoinAggregator.java	`73.91% <ø> (ø)`
.../transport/netty4/Netty4MessageChannelHandler.java	`61.90% <ø> (ø)`
...g/opensearch/transport/netty4/Netty4Transport.java	`73.65% <ø> (ø)`
.../opensearch/transport/nio/TcpReadWriteHandler.java	`100.00% <ø> (ø)`
...s/close/TransportVerifyShardBeforeCloseAction.java	`68.75% <ø> (ø)`
...readonly/TransportVerifyShardIndexBlockAction.java	`68.29% <ø> (+58.53%)`	⬆️
... and 136 more

... and 414 files with indirect coverage changes

OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module. This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively. Signed-off-by: Nicholas Walter Knize <[email protected]>

Signed-off-by: Nicholas Walter Knize <[email protected]>

github-actions · 2023-06-05T16:23:28Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.backpressure.SearchBackpressureIT.testSearchShardTaskCancellationWithHighCpu

URL: https://build.ci.opensearch.org/job/gradle-check/16857/
CommitID: 2bac9a6
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

dblock · 2023-06-06T19:20:00Z

Do you want this in 2.x?

opensearch-trigger-bot · 2023-06-06T19:21:11Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7646-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 db0933b58d10291e5cfe6a95badbe6810d86ffed
# Push it to GitHub
git push --set-upstream origin backport/backport-7646-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7646-to-2.x.

…project#7646) * [Refactor] OpenSearchException streamables to a registry OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module. This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively. Signed-off-by: Nicholas Walter Knize <[email protected]> * revert Project_Default Signed-off-by: Nicholas Walter Knize <[email protected]> --------- Signed-off-by: Nicholas Walter Knize <[email protected]>

…project#7646) OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module. This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively. Signed-off-by: Nicholas Walter Knize <[email protected]>

OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module. This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively. Signed-off-by: Nicholas Walter Knize <[email protected]>

…project#7646) * [Refactor] OpenSearchException streamables to a registry OpenSearchException uses an enumerator to create a handler for each serializable Exception implementation. The enumerator is to make it easy to store in an immutable map. The problem with this Hack is that it makes it difficult to unwind the tight coupling of StreamInput with the OpenSearchException class. This commit switches from using an enumerator to a registry in the opensearch-core library in order to support serverless and cloud native implementations outside of the server module. This commit also refactors base primitive serialization readers and writers from StreamInput and StreamOutput to BaseStreamInput and BaseStreamOutput, respectively. Signed-off-by: Nicholas Walter Knize <[email protected]> * revert Project_Default Signed-off-by: Nicholas Walter Knize <[email protected]> --------- Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

nknize added skip-changelog v2.8.0 'Issues and PRs related to version v2.8.0' labels May 21, 2023

nknize requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, owaiskazi19, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng and VachaShah as code owners May 21, 2023 00:35

nknize force-pushed the refactor/OpenSearchExceptionRegistryPrimitiveStreams branch from 57ea42f to f58e8d8 Compare May 22, 2023 15:18

nknize mentioned this pull request May 26, 2023

[Refactor] StreamInput, StreamOutput, Writeable to core library #7783

Closed

nknize added 2 commits June 5, 2023 10:48

revert Project_Default

2bac9a6

Signed-off-by: Nicholas Walter Knize <[email protected]>

nknize force-pushed the refactor/OpenSearchExceptionRegistryPrimitiveStreams branch from f58e8d8 to 2bac9a6 Compare June 5, 2023 15:51

nknize requested a review from dbwiddis as a code owner June 5, 2023 15:51

dblock approved these changes Jun 6, 2023

View reviewed changes

dblock merged commit db0933b into opensearch-project:main Jun 6, 2023

dblock added backport 2.x Backport to 2.x branch and removed v2.8.0 'Issues and PRs related to version v2.8.0' labels Jun 6, 2023

kaituo mentioned this pull request Jun 6, 2023

Add Forecaster class opensearch-project/anomaly-detection#920

Merged

stephen-crawford mentioned this pull request Jun 9, 2023

Fix JwtAuthenticationTests.shouldAuthenticateWithJwtToken_positive assertion opensearch-project/security#2844

Merged

3 tasks

gaiksaya mentioned this pull request Jun 13, 2023

Add a gradle task to check compability with child components #7998

Merged

6 tasks

vmmusings mentioned this pull request Jun 20, 2023

Improved error codes in case of security exception opensearch-project/sql#1753

Merged

6 tasks

nknize mentioned this pull request Jul 7, 2023

[Backport 2.x] [Refactor] OpenSearchException streamables to a registry (#7646) #8525

Merged

nknize added the v2.9.0 'Issues and PRs related to version v2.9.0' label Jul 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Refactor] OpenSearchException streamables to a registry #7646

[Refactor] OpenSearchException streamables to a registry #7646

nknize commented May 21, 2023

github-actions bot commented May 21, 2023

github-actions bot commented May 21, 2023

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

nknize commented May 22, 2023

RS146BIJAY commented May 22, 2023

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

codecov bot commented May 22, 2023 •

edited

Loading

github-actions bot commented Jun 5, 2023

dblock commented Jun 6, 2023

opensearch-trigger-bot bot commented Jun 6, 2023

[Refactor] OpenSearchException streamables to a registry #7646

[Refactor] OpenSearchException streamables to a registry #7646

Conversation

nknize commented May 21, 2023

github-actions bot commented May 21, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented May 21, 2023

Gradle Check (Jenkins) Run Completed with:

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

Gradle Check (Jenkins) Run Completed with:

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

Gradle Check (Jenkins) Run Completed with:

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

Gradle Check (Jenkins) Run Completed with:

nknize commented May 22, 2023

RS146BIJAY commented May 22, 2023

nknize commented May 22, 2023

github-actions bot commented May 22, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented May 22, 2023 • edited Loading

Codecov Report

github-actions bot commented Jun 5, 2023

Gradle Check (Jenkins) Run Completed with:

dblock commented Jun 6, 2023

opensearch-trigger-bot bot commented Jun 6, 2023

codecov bot commented May 22, 2023 •

edited

Loading