You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently vdiff runs at the vtctld layer, which is responsible for contacting tablets involved in a workflow, streaming the rows into memory, doing the diff, and output the results. This works but it does not scale well for running many vdiffs, or large vdiffs at once on vtctlds (which often are much less beefy than vttablets, hardware-wise as well).
If we were able to push down the work to just the tablets involved in the workflow, and leave the vtctld to only coordinate and collect results from the tablets, we could achieve much better vdiff throughput.
Design
This issue is a placeholder to start the discussion around how best to achieve this. We should eventually write up a design before doing significant work here.
The text was updated successfully, but these errors were encountered:
Feature Description
Currently vdiff runs at the vtctld layer, which is responsible for contacting tablets involved in a workflow, streaming the rows into memory, doing the diff, and output the results. This works but it does not scale well for running many vdiffs, or large vdiffs at once on vtctlds (which often are much less beefy than vttablets, hardware-wise as well).
If we were able to push down the work to just the tablets involved in the workflow, and leave the vtctld to only coordinate and collect results from the tablets, we could achieve much better vdiff throughput.
Design
This issue is a placeholder to start the discussion around how best to achieve this. We should eventually write up a design before doing significant work here.
The text was updated successfully, but these errors were encountered: