3.8.0 2024 07 30 #59
Draft
3.8.0 2024 07 30 #59
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pache#16556) Reviewers: Matthias J. Sax <[email protected]>
…try (apache#16555) When a active tasks are revoked they land as suspended tasks in the task registry. If they are then reassigned, the tasks are resumed and put into restoration. On assignment, we first handle the tasks in the task registry and then the tasks in the state updater. That means that if a task is re-assigned after a revocation, we remove the suspended task from the task registry, resume it, add it to the state updater, and then remove it from the list of tasks to create. After that we iterate over the tasks in the state updater and remove from there the tasks that are not in the list of tasks to create. However, now the state updater contains the resumed tasks that we removed from the task registry before but are no more in the list of tasks to create. In other words, we remove the resumed tasks from the state updater and close them although we just got them assigned. This commit ensures that we first handle the tasks in the state updater and then the tasks in the task registry. Reviewer: Lucas Brutschy <[email protected]>
… and JsonDeserializer (apache#16565) Reviewers: Mickael Maison <[email protected]>, Josep Prat <[email protected]>, Greg Harris <[email protected]>
…apache#16570) When Streams tries to transit a restored active task to RUNNING, the first thing it does is getting the committed offsets for this task. If getting the offsets expires a timeout, Streams does not re-throw the error initially, but tries to get the committed offsets later until a Streams-specific timeout is hit. Restored active tasks from the state updater are removed from the output queue of the restored tasks in the state updater. If a timeout occurs, the restored task is neither added to the task registry nor re-added to the state updater. The task is lost since it is not maintained anywhere. This means the task is also not closed. When the same task is created again on the same stream thread since the stream thread does not know about this lost task, the state stores are opened again and RocksDB will throw the "No locks available" error. This commit re-adds the task to the state updater if the committed request times out. Reviewers: Lucas Brutschy <[email protected]>
…eTopics (apache#16217) Reviewers: Andrew Schofield <[email protected]>, Lianet Magrans <[email protected]>, David Arthur <[email protected]>
Fix to complete Future which was stuck due the exception.getCause() throws an error. The fix in the apache#16217 unblocked blocking thread but exception in catch block from blocking get call was wrapped in ExecutionException which is not the case when moved to async workflow hence getCause is not needed. Reviewers: Luke Chen <[email protected]>, Chia-Ping Tsai <[email protected]>
…aliases correctly (apache#16608) Signed-off-by: Greg Harris <[email protected]> Reviewers: Chris Egerton <[email protected]>, Chia-Ping Tsai <[email protected]>, Josep Prat <[email protected]>
…ool (apache#16607) (3.8) (apache#16617) Reviewers: Chia-Ping Tsai <[email protected]>, Greg Harris <[email protected]>, Chris Egerton <[email protected]> Co-authored-by: Dmitry Werner <[email protected]>
…ics (apache#16614) Reviewers: Luke Chen <[email protected]>, Chia-Ping Tsai <[email protected]>
…ryMetrics (apache#16641) Reviewers: Okada Haruki <[email protected]>, Chia-Ping Tsai <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)