-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Remote Store] Support to emit multiple streams for a file content each responsible for processing a specific part of the file #6977
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## main #6977 +/- ##
============================================
- Coverage 70.71% 70.65% -0.06%
+ Complexity 59261 59249 -12
============================================
Files 4813 4822 +9
Lines 283767 283974 +207
Branches 40919 40935 +16
============================================
- Hits 200656 200637 -19
- Misses 66645 66882 +237
+ Partials 16466 16455 -11
... and 479 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Gradle Check (Jenkins) Run Completed with:
|
f6dcb01
to
68fef72
Compare
Gradle Check (Jenkins) Run Completed with:
|
68fef72
to
f1f03c7
Compare
Gradle Check (Jenkins) Run Completed with:
|
…for processing a specific part of the file Signed-off-by: Raghuvansh Raj <[email protected]>
Signed-off-by: Raghuvansh Raj <[email protected]>
f1f03c7
to
9dbf037
Compare
Gradle Check (Jenkins) Run Completed with:
|
Description
InputStream
extensions to emit stream from a specific part of a file, which starts reading from a specific position and ensures that maximum length of content read doesn't exceed a specified limit.OffsetRangeFileInputStream
achieves this for File objects, whileOffsetRangeIndexInputStream
achieves this for lucene's IndexInput constructRemoteTransferContainer
has utilities to open streams to specific parts of the file based on the type as mentioned in the previous point. It also manages post upload tasks by implementing anUploadFinalizer
.ResettableCheckedInputStream
allows for individual parts to be reset through mark and reset in the event of upload failuresIssues Resolved
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.