Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Pubsub fixes: Avro serialization caching and retries #3610

Merged
merged 7 commits into from
Nov 26, 2024

Conversation

passuied
Copy link
Contributor

@passuied passuied commented Nov 22, 2024

Description

  • When leveraging Kafka Pubsub Avro serialization, caching should be enabled by default with a TTL of 5min
  • Also, addresses another issue where poison pill messages could be skipped in some weird circumstances after a context cancelation, which doesn't in turn seem to come from session.Context()
    • In that specific case, we're forcing the processing back into the retry policy for the same message

Issue reference

#3609
#3529

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

@passuied passuied requested review from a team as code owners November 22, 2024 05:14
@passuied passuied changed the title When leveraging Kafka Pubsub Avro serialization, caching should be enabled by default with a TTL of 5min Kafka Pubsub fixes: Avro serialization caching and retries Nov 23, 2024
…abled by default with a TTL of 5min

Signed-off-by: Patrick Assuied <[email protected]>
…n some weird circumstances

  - To avoid complexity, always exiting the for loop in case the RetryNotify fails...
  - Let dapr re-initiate the session in those cases
- Address PR feedback and fix breaking tests

Signed-off-by: Patrick Assuied <[email protected]>
…ges in some weird circumstances

Signed-off-by: Patrick Assuied <[email protected]>
Signed-off-by: Patrick Assuied <[email protected]>
Signed-off-by: Patrick Assuied <[email protected]>
@passuied passuied force-pushed the feature/kafka-pubsub-avro-caching branch from 375876d to e6c2d4b Compare November 23, 2024 19:22
@yaron2 yaron2 added this pull request to the merge queue Nov 26, 2024
Merged via the queue into dapr:main with commit 8c02ff3 Nov 26, 2024
90 checks passed
passuied added a commit to passuied/components-contrib that referenced this pull request Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants