Multi-tenant MoveTables: Create vreplication streams only on specified shards

[rohit-nayak-ps]

Description

Adds an option to provide shards for multi-tenant migrations. This will only create the workflow streams on the specified shard(s). It is expected that the user correctly maps the tenant id to the shard as per the configured vindex.

This PR also fixes a couple of places where we had either missed out or regressed on filtering our target shards for all MoveTables commands where shards were specified.

Related Issue(s)

#15748

Checklist

• "Backport to:" labels have been added if this change should be back-ported to release branches
• If this change is to be back-ported to previous releases, a justification is included in the PR description
• Tests were added or are not required
• Did the new or modified tests pass consistently locally and on CI?
• Documentation was added or is not required

Deployment Notes

[vitess-bot]

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

• Ensure that the Pull Request has a descriptive title.
• Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

• Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

• Apply the release notes (needs details) label if users need to know about this change.
• New features should be documented.
• There should be some code comments as to why things are implemented the way they are.
• There should be a comment at the top of each new or modified test to explain what the test does.

New flags

• Is this flag really necessary?
• Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

• Each item in Jobs should be named in order to mark it as required.
• If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

• Protobuf changes should be wire-compatible.
• Changes to _vt tables and RPCs need to be backward compatible.
• RPC changes should be compatible with vitess-operator
• If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
• vtctl command output order should be stable and awk-able.

[codecov]

Codecov Report

Attention: Patch coverage is 55.55556% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 68.43%. Comparing base (ca2659d) to head (4a1db5d).

Files	Patch %	Lines
...ldclient/command/vreplication/movetables/create.go	0.00%	5 Missing ⚠️
go/vt/vtctl/workflow/materializer.go	64.28%	5 Missing ⚠️
go/vt/vtctl/workflow/traffic_switcher.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15746      +/-   ##
==========================================
- Coverage   68.43%   68.43%   -0.01%     
==========================================
  Files        1558     1558              
  Lines      196186   196201      +15     
==========================================
+ Hits       134269   134270       +1     
- Misses      61917    61931      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mattlord]

LGTM! I only had some minor comments that you can address as you feel best.

[deepthi]

Suggested change

create.Flags().StringSliceVar(&createOptions.WorkflowOptions.Shards, "shards", nil, "(Multi-tenant migrations only) Specify that vreplication streams should only be created on this subset of target shards. Warning: you should first ensure that all rows on the source route to the specified subset of target shards using your VIndex of choice or you could lose data during the migration.")

create.Flags().StringSliceVar(&createOptions.WorkflowOptions.Shards, "shards", nil, "(EXPERIMENTAL: Multi-tenant migrations only) Specify that vreplication streams should only be created on this subset of target shards. Warning: you should first ensure that all rows on the source route to the specified subset of target shards using your Vindex of choice or you could lose data during the migration.")

[deepthi]

It feels like this should be called target-shards given that we already have a source-shards option. I wonder what happens if someone specifies both of these options. Do we want to prevent that? Or ignore source-shards if tenant-id is specified?

[deepthi]

Also, may as well expand EXPERIMENTAL in the flag help for tenant-id (line 50) to EXPERIMENTAL: Multi-tenant migrations only.

[rohit-nayak-ps]

I had started with --target-shards, but it wasn't uniform with our implementation for shard-by-shard migrations where shard subsets are specified with --shards.

Added a validation to ensure that both --source-shards and --tenant-id cannot be specified.

[deepthi]

That's unfortunate. I guess --shards made sense for shard-by-shard because it's very clear which shard you are currently migrating. That is not the case for multi-tenant. What do you think about adding --target-shards for multi-tenant migrations, and "migrating" shard-by-shard migrations to use that in the next release (deprecate old flag, add new flag, respect both, delete in next release etc.)? Or we can even wait to do that until multi-tenant is no longer experimental. I'll approve anyhow, it's not worth holding up this PR, but I feel the UX will be better and clearer if we call it --target-shards.

[rohit-nayak-ps]

I thought of changing just the `create option, but it looks inconsistent while trying to document it since it is different for the other actions. Merging as-is for now.