Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport fix for Vitess Online DDL atomic cut-over #106

Conversation

austenLacy
Copy link

@austenLacy austenLacy commented Jun 23, 2023

This PR backports an upstream fix for online DDL table rename cutover. We need to be very careful with this back port and should definitely be tested before being merged and deployed. Unfortunately unit/e2e testing is hard for this because the failure case is inherently flaky in a codified test. However, we have a reproducible example of the error it is meant to fix and we can confirm that works and that other schema changes continue to work.

ref: https://github.com/Shopify/shardlifter-project/issues/103

* cut-over with sentry table

Signed-off-by: Shlomi Noach <[email protected]>

* wait for rename via channel; write extra transaction post LOCK

Signed-off-by: Shlomi Noach <[email protected]>

* add stage info

Signed-off-by: Shlomi Noach <[email protected]>

* reduced wait-for-pos timeout. Improved stage logic

Signed-off-by: Shlomi Noach <[email protected]>

* cleanup

Signed-off-by: Shlomi Noach <[email protected]>

* more cleanup

Signed-off-by: Shlomi Noach <[email protected]>

* even more cleanup

Signed-off-by: Shlomi Noach <[email protected]>

* context timeout

Signed-off-by: Shlomi Noach <[email protected]>

* preserve stage by disabling deferred stage changes

Signed-off-by: Shlomi Noach <[email protected]>

* killing rename query upon error

Signed-off-by: Shlomi Noach <[email protected]>

* increase test timeout

Signed-off-by: Shlomi Noach <[email protected]>

* fix defer ordering

Signed-off-by: Shlomi Noach <[email protected]>

* log.info

Signed-off-by: Shlomi Noach <[email protected]>

* add and populate cutover_attempts column

Signed-off-by: Shlomi Noach <[email protected]>

* search PROCESSLIST with LIKE

Signed-off-by: Shlomi Noach <[email protected]>

* code comment

Signed-off-by: Shlomi Noach <[email protected]>

* making a variable more local

Signed-off-by: Shlomi Noach <[email protected]>

* literal string

Signed-off-by: Shlomi Noach <[email protected]>

Signed-off-by: Shlomi Noach <[email protected]>
@austenLacy austenLacy requested a review from brendar June 23, 2023 17:55
@austenLacy austenLacy changed the title [BACK-PORT]: Vitess Online DDL atomic cut-over [DO_NOT_MERGE] Backport fix for Vitess Online DDL atomic cut-over Jun 23, 2023
@austenLacy austenLacy changed the base branch from v15.0.3-shopify-3 to v15.0.3-shopify-4 June 30, 2023 18:49
@austenLacy austenLacy changed the title [DO_NOT_MERGE] Backport fix for Vitess Online DDL atomic cut-over Backport fix for Vitess Online DDL atomic cut-over Jun 30, 2023
@austenLacy
Copy link
Author

@brendar The upstream PR was merged and I confirmed these changes all match. I updated the base of this PR to be the new v15.0.3-shopify-4 branch.

@austenLacy austenLacy requested a review from toria-shopify July 5, 2023 18:39
Copy link

@toria-shopify toria-shopify left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@austenLacy austenLacy merged commit 67543ee into v15.0.3-shopify-4 Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants