diff viewing - poc-search-protobuf #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…ream related functionality Signed-off-by: Vacha Shah <[email protected]>
…tegration Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…sAction Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…- mainly TransportService to try to fix multi node problem Signed-off-by: Vacha Shah <[email protected]>
…s commit Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…rch-project#9073) This commit refactors the following network and transport libraries to the opensearch common and core libraries respectively: * o.o.common.network.Cidrs -> :libs:opensearch-common * o.o.common.network.InetAddresses -> :libs:opensearch-common * o.o.common.network.NetworkAddress -> :libs:opensearch-common * o.o.common.transport.NetworkExceptionHelper -> :libs:opensearch-common * o.o.common.transport.PortsRange -> :libs:opensearch-common * o.o.common.transport.TransportAddress -> :libs:opensearch-core * o.o.common.transport.BoundTransportAddress -> :libs:opensearch-core * o.o.transport.TransportMessage -> :libs:opensearch-core * o.o.transport.TransportResponse -> :libs:opensearch-core The purpose is to reduce the change surface area of the core APIs to minimize impact to downstream consumers while moving toward establishing a formal API for cloud native or serverless implementations. Signed-off-by: Nicholas Walter Knize <[email protected]>
…st index deletion. (opensearch-project#8472) Signed-off-by: Harish Bhakuni <[email protected]>
…replay (opensearch-project#8578) Signed-off-by: Gaurav Bafna <[email protected]>
…search-project#9057) --------- Signed-off-by: Ashish Singh <[email protected]>
…ckpoint validation (opensearch-project#8889) * Fix test testDropPrimaryDuringReplication and clean up ReplicationCheckpoint validation. This test is now occasionally failing with replicas having 0 documents. This occurs in a couple of ways: 1. After dropping the old primary the new primary is not publishing a checkpoint to replicas unless it indexes docs from translog after flipping to primary mode. If there is nothing to index, it will not publish a checkpoint, but the other replica could have never sync'd with the original primary and be left out of date. - This PR fixes this by force publishing a checkpoint after the new primary flips to primary mode. 2. The replica receives a checkpoint post failover and cancels its sync with the former primary that is still active, recognizing a primary term bump. However this cancellation is async and immediately starting a new replication event could fail as its still replicating. - This PR fixes this by attempting to process the latest received checkpoint on failure, if the shard is not failed and still behind. This PR also introduces a few changes to ensure the accuracy of the ReplicationCheckpoint tracked on primary & replicas. - Ensure the checkpoint stored in SegmentReplicationTarget is the checkpoint passed from the primary and not locally computed. This ensures checks for primary term are accurate and not using a locally compued operationPrimaryTerm. - Introduces a refresh listener for both primary & replica to update the ReplicationCheckpoint and store it in replicationTracker post refresh rather than redundantly computing when accessed. - Removes unnecessary onCheckpointPublished method used to start replication timers manually. This will happen automatically on primaries once its local cp is updated. Signed-off-by: Marc Handalian <[email protected]> * Handle NoSuchFileException when attempting to delete decref'd files. To avoid divergent logic with remote store, we always incref/decref the segmentinfos.files(true) which includes the segments_n file. Decref to 0 will attempt to delete the file from the store and its possible this _n file does not yet exist. This change will ignore if we get a noSuchFile while attempting to delete. Signed-off-by: Marc Handalian <[email protected]> * Add more unit tests. Signed-off-by: Marc Handalian <[email protected]> * Clean up IndexShardTests.testCheckpointReffreshListenerWithNull Signed-off-by: Marc Handalian <[email protected]> * Remove unnecessary catch for NoSuchFileException. Signed-off-by: Marc Handalian <[email protected]> * Add another test for non segrep. Signed-off-by: Marc Handalian <[email protected]> * PR Feedback. Signed-off-by: Marc Handalian <[email protected]> * re-compute replication checkpoint on primary promotion. Signed-off-by: Marc Handalian <[email protected]> --------- Signed-off-by: Marc Handalian <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…otobuf Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
…hose requests Signed-off-by: Vacha Shah <[email protected]>
…node-to-node communication in protobuf Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
Signed-off-by: Vacha Shah <[email protected]>
finnegancarroll
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Diff viewing.