Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vtctldclient: --strict rejects unknown vindex params in ApplyVSchema #14862

Merged

Conversation

maxenglander
Copy link
Collaborator

@maxenglander maxenglander commented Dec 26, 2023

Description

Currently, unknown vindex params found in ApplyVSchema are either logged as warnings (in the case of cli command) or returned in the gRPC response (in the case of the gRPC command).

This PR adds a --strict flag to the vtctldclient ApplyVSchema command that returns an error if any unknown params are found. An equivalent strict field is added to the gRPC command.

Based on feedback from @mattlord no changes are made to the deprecated vtctlclient binary.

Related Issue(s)

Fixes #13041

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added

Deployment Notes

Copy link
Contributor

vitess-bot bot commented Dec 26, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@vitess-bot vitess-bot bot added NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Dec 26, 2023
@github-actions github-actions bot added this to the v19.0.0 milestone Dec 26, 2023
@maxenglander maxenglander added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: Cluster management labels Dec 26, 2023
@maxenglander maxenglander changed the title vtctl(d): vschema unknown params strict vtctl: vschema unknown params strict Dec 26, 2023
@maxenglander maxenglander removed NeedsWebsiteDocsUpdate What it says NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Dec 26, 2023
@maxenglander maxenglander marked this pull request as ready for review December 26, 2023 23:48
Copy link
Contributor

@mattlord mattlord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New features have to be added to vtctldclient, in this case here: https://github.com/vitessio/vitess/blob/main/go/cmd/vtctldclient/command/vschemas.go

And IMO we should not add new things to vtctlclient since it's deprecated in v18: https://github.com/vitessio/vitess/releases#legacy-client-binaries

Comment on lines +22 to +23
- **[Apply VSchema](#apply-vschema)**
- [`--strict` sub-flag and `strict` gRPC field](#strict-flag-and-field)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this is a major change and needs to be here. That being said, I'm also fine leaving it here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create a new Minor section and move it there

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not against that really, but at last in my view the summary is for major changes that every Vitess user should be aware of. Typically a new flag like this is not included.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead and added to minor section for now

go/vt/vtctl/grpcvtctldserver/server.go Outdated Show resolved Hide resolved
go/vt/vtctl/vtctl.go Outdated Show resolved Hide resolved
go/vt/vtctl/grpcvtctldserver/server_test.go Show resolved Hide resolved

if *dryRun {
if *dryRun || (*strict && len(unknownParams) > 0) {
wr.Logger().Printf("Dry run: Skipping update of VSchema\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can keep the strict/unknown params logic separate from dry run? For example, if the user didn't specify dry run, but did specify strict, this log line about "Dry run: Skipping update of VSchema\n" would still be output.

VSchema: vs,
UnknownVindexParams: unknownVindexParams,
}, nil
if req.DryRun || (req.Strict && len(unknownVindexParams) > 0) { // return early if dry run or unknown params in strict mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit comment, it would be nice from a code-reading standpoint if the different "end states" (dry run vs strict with unknown params) could be separated 😃

@mattlord mattlord removed the NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work label Dec 27, 2023
Copy link
Contributor

@notfelineit notfelineit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for doing this 😄

@mattlord mattlord self-requested a review December 27, 2023 19:23
Signed-off-by: Max Englander <[email protected]>
Copy link
Contributor

@mattlord mattlord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I double checked the client quickly:

❯ git diff | cat
diff --git a/examples/local/vschema_customer_sharded.json b/examples/local/vschema_customer_sharded.json
index 3109e2a2f3..cef527ec26 100644
--- a/examples/local/vschema_customer_sharded.json
+++ b/examples/local/vschema_customer_sharded.json
@@ -2,7 +2,10 @@
     "sharded": true,
     "vindexes": {
         "hash": {
-            "type": "hash"
+            "type": "hash",
+            "params": {
+                "hello": "moto"
+            }
         }
     },
     "tables": {
     
❯ vtctldclient ApplyVSchema --strict --vschema-file vschema_customer_sharded.json commerce
E1227 14:38:50.866216   70996 main.go:56] rpc error: code = Unknown desc = unknown vindex params: hash (hello)

❯ vtctldclient ApplyVSchema --dry-run --strict --vschema-file vschema_customer_sharded.json commerce
E1227 14:38:58.601566   71017 main.go:56] rpc error: code = Unknown desc = unknown vindex params: hash (hello)

@maxenglander maxenglander changed the title vtctl: vschema unknown params strict vtctldclient: vschema unknown params strict Dec 28, 2023
@maxenglander maxenglander changed the title vtctldclient: vschema unknown params strict vtctldclient: add --strict argument to reject unknown params in ApplyVSchema vindexes Dec 28, 2023
@maxenglander maxenglander changed the title vtctldclient: add --strict argument to reject unknown params in ApplyVSchema vindexes vtctldclient: --strict rejects unknown vindex params in ApplyVSchema Dec 28, 2023
Signed-off-by: Max Englander <[email protected]>
@rohit-nayak-ps rohit-nayak-ps merged commit 260bf14 into vitessio:main Dec 29, 2023
102 checks passed
@rohit-nayak-ps rohit-nayak-ps deleted the maxeng-vschema-unknown-params-strict branch December 29, 2023 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Cluster management Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFC: surface warnings and/or errors for unknown vindex params
4 participants