Materializer: normalize schema via schemadiff on --atomic-copy #14636
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Shlomi Noach <[email protected]>
shlomi-noach
requested review from
deepthi,
mattlord,
rohit-nayak-ps and
ajm188
as code owners
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
vitess-bot
bot
added
NeedsDescriptionUpdate
shlomi-noach
commented
Nov 29, 2023
| @@ -88,7 +88,7 @@ func TestFKWorkflow(t *testing.T) { | |||
| } | |||
| targetKeyspace := "fktarget" | |||
| targetTabletId := 200 | |||
| vc.AddKeyspace(t, []*Cell{cell}, targetKeyspace, shardName, initialFKTargetVSchema, initialFKSchema, 0, 0, targetTabletId, sourceKsOpts) | |||
| vc.AddKeyspace(t, []*Cell{cell}, targetKeyspace, shardName, initialFKTargetVSchema, "", 0, 0, targetTabletId, sourceKsOpts) | |||
There was a problem hiding this comment.
This change to the test ensures the new behavior is applied. The schema in the test is composed of parent and child tables. Since child < parent lexicographically, the test fails without the new normalization.
shlomi-noach
removed
NeedsDescriptionUpdate
dbussink
reviewed
Nov 29, 2023
go/vt/vtctl/workflow/materializer.go
Outdated
| log.Error(vterrors.Wrapf(err, "AtomicCopy: failed to normalize schema via schemadiff")) | ||
| } else { | ||
| applyDDLs = schema.ToQueries() | ||
| log.Infof("AtomicCopy used, and schema was normalized via schemadiff. New queries: %v", applyDDLs) |
There was a problem hiding this comment.
Do we really want to always log this? As it can be a giant schema and thus a huge log entry?
dbussink
reviewed
Nov 29, 2023
go/vt/vtctl/workflow/materializer.go
Outdated
| sql := strings.Join(applyDDLs, ";\n") | ||
| log.Infof("materializer.deploySchema: sql=%v", sql) |
There was a problem hiding this comment.
Same here, this can be very big and also would duplicate the logging.
Signed-off-by: Shlomi Noach <[email protected]>
dbussink
approved these changes
Nov 29, 2023
rohit-nayak-ps
approved these changes
Nov 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In materializer, and right now scoped to
AtomicCopyoperations only (just to keep the blast radius smaller for the time being), we normalize the source schema viaschemadiffwhere possible. This ensures ordering of tables is compliant with any foreign key constraints. See #14635 for details.Related Issue(s)
Fixes #14635
Checklist
Deployment Notes