-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix race conditions in the concatenate engine streaming #16640
Merged
GuptaManan100
merged 4 commits into
vitessio:main
from
planetscale:concatenate-engine-fix
Aug 26, 2024
Merged
Fix race conditions in the concatenate engine streaming #16640
GuptaManan100
merged 4 commits into
vitessio:main
from
planetscale:concatenate-engine-fix
Aug 26, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
…coerced Signed-off-by: Manan Gupta <[email protected]>
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
vitess-bot
bot
added
NeedsBackportReason
If backport labels have been applied to a PR, a justification is required
NeedsDescriptionUpdate
The description is not clear or comprehensive enough, and needs work
NeedsIssue
A linked issue is missing for this Pull Request
NeedsWebsiteDocsUpdate
What it says
labels
Aug 23, 2024
GuptaManan100
added
Type: Bug
Component: Query Serving
and removed
NeedsDescriptionUpdate
The description is not clear or comprehensive enough, and needs work
NeedsWebsiteDocsUpdate
What it says
NeedsIssue
A linked issue is missing for this Pull Request
NeedsBackportReason
If backport labels have been applied to a PR, a justification is required
labels
Aug 23, 2024
systay
approved these changes
Aug 23, 2024
GuptaManan100
requested review from
harshit-gangal and
frouioui
as code owners
August 23, 2024 08:37
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16640 +/- ##
==========================================
- Coverage 68.98% 68.98% -0.01%
==========================================
Files 1562 1562
Lines 200690 200740 +50
==========================================
+ Hits 138449 138476 +27
- Misses 62241 62264 +23 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Manan Gupta <[email protected]>
GuptaManan100
added
the
Backport to: release-20.0
Needs to be backport to release-20.0
label
Aug 26, 2024
harshit-gangal
approved these changes
Aug 26, 2024
vitess-bot
pushed a commit
that referenced
this pull request
Aug 26, 2024
Signed-off-by: Manan Gupta <[email protected]>
GuptaManan100
added
Backport to: release-20.0
Needs to be backport to release-20.0
and removed
Backport to: release-20.0
Needs to be backport to release-20.0
labels
Aug 26, 2024
GuptaManan100
pushed a commit
that referenced
this pull request
Aug 26, 2024
#16640) (#16648) Signed-off-by: Manan Gupta <[email protected]> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
While looking at the flaky test pointed out in #16641 we found several issues in concatenate engine primitive in the streaming mode. The following are the issues and the fixes in this PR -
getFieldTypes
, we were overriding the fields of one of the result packets that we got. If this was a packet from a source that needed coercion, because we overwrote the fields, with what we expect them to be, when we check if coercion is required we end up with false even when it requires coercion. This is fixed in the PR by storing the result fields separately and checking for needing coercion with the original fields of the result.The problem was that we were checking for needing coercion per result packet. But in the streaming mode, if a source is sending multiple packets, then only the first packet will have the fields and the others won't. This leads to the concatenate engine only coercing values in the first packet and not in the subsequent ones since there are no fields. The fix was to check for needing coercion once for a given source and use it for all its packets.
Related Issue(s)
Checklist
Deployment Notes