Skip to content

Commit

Permalink
Replace all vtctlclient usage for vreplication commands (#1619)
Browse files Browse the repository at this point in the history
* Replace all vtctlclient usage

Signed-off-by: Matt Lord <[email protected]>

* Revert OnlineDDL changes

Those will have to be done separately

Signed-off-by: Matt Lord <[email protected]>

* Changes after self review

Signed-off-by: Matt Lord <[email protected]>

* The the

Signed-off-by: Matt Lord <[email protected]>

* Overview and deprecated page removals

Signed-off-by: Matt Lord <[email protected]>

* MoveTables

Signed-off-by: Matt Lord <[email protected]>

* Picking away.

And incorporating changes from: vitessio/vitess#14280

Signed-off-by: Matt Lord <[email protected]>

* Reshard page

Signed-off-by: Matt Lord <[email protected]>

* Update reference docs from main

Signed-off-by: Matt Lord <[email protected]>

* Materialize

Signed-off-by: Matt Lord <[email protected]>

* LookupVindex

Signed-off-by: Matt Lord <[email protected]>

* Tablet selection

Signed-off-by: Matt Lord <[email protected]>

* Throttling

Signed-off-by: Matt Lord <[email protected]>

* Migrate

Signed-off-by: Matt Lord <[email protected]>

* Shard level migrations

Signed-off-by: Matt Lord <[email protected]>

* Mount page

Signed-off-by: Matt Lord <[email protected]>

* Remaining user guide changes

Signed-off-by: Matt Lord <[email protected]>

* Update transition page

Signed-off-by: Matt Lord <[email protected]>

* Revert docs rebuild. I did this to verify the links as
the Materialize, Mount, and Migrate commands are not
documented yet. I will open a separate PR to rebuild
the reference docs.

Signed-off-by: Matt Lord <[email protected]>

* Small correction

Signed-off-by: Matt Lord <[email protected]>

* More changes from review

Signed-off-by: Matt Lord <[email protected]>

* Changes from self review

Signed-off-by: Matt Lord <[email protected]>

* Copy changes to v18 docs

Signed-off-by: Matt Lord <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>
  • Loading branch information
mattlord authored Oct 16, 2023
1 parent 66fb422 commit 94ddf23
Show file tree
Hide file tree
Showing 68 changed files with 2,105 additions and 3,293 deletions.
2 changes: 1 addition & 1 deletion content/en/docs/18.0/concepts/vtctld.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ title: vtctld

**vtctld** is an HTTP server that lets you browse the information stored in the Topology Service. It is useful for troubleshooting or for getting a high-level overview of the servers and their current states.

`vtctld` also acts as the server for `vtctlclient` and `vtctldclient` connections.
`vtctld` also acts as the server for `vtctldclient` connections.
4 changes: 2 additions & 2 deletions content/en/docs/18.0/get-started/local-brew.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,13 +191,13 @@ rm -rf /usr/local/Cellar/vitess/9.0.0/share/vitess/examples/local/vtdataroot

## Setup Aliases

For ease-of-use, Vitess provides aliases for `mysql` and `vtctlclient`:
For ease-of-use, Vitess provides aliases for `mysql` and `vtctldclient`:

```bash
source ../common/env.sh
```

Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctlclient` or close your session.
Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctldclient` or close your session.

## Connect to your cluster

Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/18.0/get-started/local.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,13 +222,13 @@ rm -rf vtdataroot

## Setup Aliases

For ease-of-use, Vitess provides aliases for `mysql`, `vtctlclient` and `vtcltdclient`:
For ease-of-use, Vitess provides aliases for `mysql` and `vtcltdclient`:

```bash
source ../common/env.sh
```

Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctlclient && unalias vtctldclient` or close your session.
Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctldclient` or close your session.

## Connect to your cluster

Expand Down
5 changes: 2 additions & 3 deletions content/en/docs/18.0/get-started/operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,16 +72,15 @@ vitess-operator-5f47c6c45d-bgqp2 1/1 Running 0
The port-forward will only forward to a specific pod. Currently, `kubectl` does not automatically terminate a port-forward as the pod disappears due to apply/upgrade operations. You will need to manually restart the port-forward.
{{</ warning >}}

For ease-of-use, Vitess provides a script to port-forward from Kubernetes to your local machine. This script also recommends setting up aliases for `mysql` and `vtctlclient`:
For ease-of-use, Vitess provides a script to port-forward from Kubernetes to your local machine. This script also recommends setting up aliases for `mysql` and `vtctldclient`:

```bash
./pf.sh &
alias vtctldclient="vtctldclient --server=localhost:15999"
alias vtctlclient="vtctlclient --server=localhost:15999"
alias mysql="mysql -h 127.0.0.1 -P 15306 -u user"
```

Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctlclient && unalias vtctldclient` or close your session.
Setting up aliases changes `mysql` to always connect to Vitess for your current session. To revert this, type `unalias mysql && unalias vtctldclient` or close your session.

Once the port-forward starts running, the VTAdmin UI will be available at [http://localhost:14000/](http://localhost:14000/)

Expand Down
5 changes: 2 additions & 3 deletions content/en/docs/18.0/reference/features/connection-pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,15 @@ As a result, connection pools should be sized mindful of the capacity of the und
* Max size controlled by: `--dba_pool_size` (default 20)
* metric: `vttablet_dba_conn_pool_capacity`
* vttablet user flag: `--db_dba_user` (default 'vt_dba')
* Used by vttablet `ExecuteFetchAsDba` RPC. This is used when using `vtctlclient ExecuteFetchAsDba`
* Used by vttablet `ExecuteFetchAsDBA` RPC. This is used when using `vtctldclient ExecuteFetchAsDBA`
Also used implicitly for various internal Vitess maintenance tasks (e.g. schema reloads, etc.)

### app_conn_pool

* Max size controlled by: `--app_pool_size` (default 40)
* metric: `vttablet_app_conn_pool_capacity`
* vttablet user flag: `--db_app_user` default 'vt_app')
* Used by vttablet `ExecuteFetchAsApp` RPC. This is used when using `vtctlclient ExecuteFetchAsApp`
* Used by vttablet `ExecuteFetchAsApp` RPC. This is used when using `vtctldclient ExecuteFetchAsApp`

### tx_read_pool

Expand Down Expand Up @@ -89,7 +89,6 @@ As a result, connection pools should be sized mindful of the capacity of the und
#### `--db_allprivs_user`

* (default 'vt_allprivs')
* Created on demand by `vtctlclient ExecuteFetchAsAllPrivs`

#### `--db_erepl_user`
Expand Down
6 changes: 3 additions & 3 deletions content/en/docs/18.0/reference/features/tablet-throttler.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,12 +123,12 @@ The following flag was removed:

- `--enable_lag_throttler`

Updating the throttler config is done via `vtctlclient` or `vtctldclient`. For example:
Updating the throttler config is done via `vtctldclient`. For example:

```sh
$ vtctlclient -- UpdateThrottlerConfig --enable --threshold 3.0 commerce
$ vtctldclient UpdateThrottlerConfig --enable --threshold 3.0 commerce
$ vtctldclient UpdateThrottlerConfig --disable commerce
$ vtctldclient UpdateThrottlerConfig --throttle-app="vreplication" --throttle-app-ratio 0.5 --throttle-app-duration "30m" commerce
$ vtctldclient UpdateThrottlerConfig --throttle-app "vreplication" --throttle-app-ratio 0.5 --throttle-app-duration "30m" commerce
```

See [vtctl UpdateThrottlerConfig](../../programs/vtctl/throttler#updatethrottlerconfig).
Expand Down
12 changes: 6 additions & 6 deletions content/en/docs/18.0/reference/features/topology-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -516,13 +516,13 @@ information about the local topology service:
* `${CELL_NAME}` the name of our local cell in this case `US_EAST`

``` sh
# vtctlclient uses the IP address of the vtctld daemon with the `--server` flag
# vtctldclient uses the IP address of the vtctld daemon with the `--server` flag
# The daemon already has the global topology information, therefore, we do not
# need to explicitly provide these details.

vtctlclient --server ${VTCTLD_IP}:15999 AddCellInfo -- \
vtctldclient --server ${VTCTLD_IP}:15999 AddCellInfo -- \
--root=/vitess/${CELL_NAME} \
--server_address=${CELL_ETCD_SERVER} \
--server-address=${CELL_ETCD_SERVER} \
${CELL_NAME}
```

Expand Down Expand Up @@ -573,8 +573,8 @@ propagate information from the global topology service back to the local topolog
services

```sh
vtctlclient --server ${VTCTLD_IP}:15999 RebuildKeyspaceGraph ${KEYSPACE_NAME}
vtctlclient --server ${VTCTLD_IP}:15999 RebuildVSchemaGraph
vtctldclient --server ${VTCTLD_IP}:15999 RebuildKeyspaceGraph ${KEYSPACE_NAME}
vtctldclient --server ${VTCTLD_IP}:15999 RebuildVSchemaGraph
```


Expand Down Expand Up @@ -604,7 +604,7 @@ run 5 global servers across all cells (for global topology data) and 3 local
servers per cell (for per-cell topology data).

To migrate to such a setup, start by adding the 3 local servers in the second
cell and run `vtctlclient AddCellinfo` as was done for the first cell. Tablets and
cell and run `vtctldclient AddCellInfo` as was done for the first cell. Tablets and
vtgates can now be started in the second cell, and used normally.

vtgate can then be configured with a list of cells to watch for tablets using
Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/18.0/reference/features/vindexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ You can query Vindex functions to see the resulting `keyspace_id` it produces (t

First, a snippet of the VSchema:
``` shell
$ vtctlclient -server=localhost:15999 GetVSchema customer | jq '.vindexes'
$ vtctldclient --server=localhost:15999 GetVSchema customer | jq '.vindexes'
{
"binary_md5_vdx": {
"type": "binary_md5"
Expand Down
48 changes: 0 additions & 48 deletions content/en/docs/18.0/reference/vreplication/createlookupvindex.md

This file was deleted.

44 changes: 32 additions & 12 deletions content/en/docs/18.0/reference/vreplication/internal/cutover.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ cached locally, the processes involved will refresh their topo data throughout t
tablet on the source and target shards that are involved in a [VReplication](../../) workflow
will refresh their topo data multiple times as the state of things transition during the cutover. If we are *not* able
to confirm that all tablets involved in a VReplication worfklow are able to refresh their topo data then the cutover
command — e.g. [`vtctlclient SwitchTraffic`](../../switchtraffic) — will cancel the operation
and return an error indicating which tablet(s) are unhealthy (including for `--dry_run` executions).
command — e.g. [`vtctldclient SwitchTraffic`](../../switchtraffic) — will cancel the operation
and return an error indicating which tablet(s) are unhealthy (including for `--dry-run` executions).
{{< /info >}}

## VSchema
Expand All @@ -28,11 +28,16 @@ one [`Shard`](../../../features/topology-service/#shard) key per keyspace which
shard that has been created within the keyspace. For each shard that is healthy there is an
attribute `is_primary_serving` which is set to true. The other shards which have been created but are still not healthy
and serving within the keyspace will not have this attribute set. Here is an example shard info record from an unsharded
keyspace named commerce (without the `--cell` flag being passed the `global` topo base path is used):
keyspace named commerce:

```bash
$ vtctlclient --server=localhost:15999 TopoCat -- --decode_proto '/keyspaces/commerce/shards/0/Shard'
primary_alias:{cell:"zone1" uid:100} primary_term_start_time:{seconds:1650341417 nanoseconds:374817485} is_primary_serving:true
$ vtctldclient --server=localhost:15999 GetTopologyPath /global/keyspaces/commerce/shards/0/Shard
{
"name": "Shard",
"path": "/global//keyspaces/commerce/shards/0/Shard",
"data": "primary_alias:{cell:\"zone1\" uid:101} primary_term_start_time:{seconds:1697055542 nanoseconds:430056000} is_primary_serving:true",
"children": []
}
```

## SrvKeyspace
Expand All @@ -50,11 +55,21 @@ same unsharded commerce keyspace and here we specify the `--cell` flag so that c
its `CellInfo` record in the `global` topo — is used:

```bash
$ vtctlclient --server=localhost:15999 TopoCat -- --decode_proto '/cells/zone1/CellInfo'
server_address:"localhost:2379" root:"/vitess/zone1"
$ vtctldclient --server=localhost:15999 GetTopologyPath /global/cells/zone1/CellInfo
{
"name": "CellInfo",
"path": "/global/cells/zone1/CellInfo",
"data": "server_address:\"localhost:2379\" root:\"/vitess/zone1\"",
"children": []
}

$ vtctlclient --server=localhost:15999 TopoCat -- --decode_proto --cell=zone1 '/keyspaces/commerce/SrvKeyspace'
partitions:{served_type:PRIMARY shard_references:{name:"0"}} partitions:{served_type:REPLICA shard_references:{name:"0"}} partitions:{served_type:RDONLY shard_references:{name:"0"}}
$ vtctldclient --server=localhost:15999 GetTopologyPath /zone1/keyspaces/commerce/SrvKeyspace
{
"name": "SrvKeyspace",
"path": "/zone1/keyspaces/commerce/SrvKeyspace",
"data": "partitions:{served_type:PRIMARY shard_references:{name:\"0\"}} partitions:{served_type:REPLICA shard_references:{name:\"0\"}} partitions:{served_type:RDONLY shard_references:{name:\"0\"}}",
"children": []
}
```

## Routing Rules
Expand All @@ -66,15 +81,20 @@ where we have an active [`MoveTables`](../../../vreplication/movetables/) workfl
customer keyspace but we have not switched any traffic yet:

```bash
$ vtctlclient --server=localhost:15999 TopoCat -- --decode_proto '/RoutingRules'
rules:{from_table:"corder@rdonly" to_tables:"commerce.corder"} rules:{from_table:"customer.corder" to_tables:"commerce.corder"} rules:{from_table:"customer.corder@replica" to_tables:"commerce.corder"} rules:{from_table:"customer@rdonly" to_tables:"commerce.customer"} rules:{from_table:"customer.customer@rdonly" to_tables:"commerce.customer"} rules:{from_table:"customer.corder@rdonly" to_tables:"commerce.corder"} rules:{from_table:"customer@replica" to_tables:"commerce.customer"} rules:{from_table:"corder@replica" to_tables:"commerce.corder"} rules:{from_table:"commerce.corder@replica" to_tables:"commerce.corder"} rules:{from_table:"commerce.corder@rdonly" to_tables:"commerce.corder"} rules:{from_table:"commerce.customer@rdonly" to_tables:"commerce.customer"} rules:{from_table:"corder" to_tables:"commerce.corder"} rules:{from_table:"customer.customer@replica" to_tables:"commerce.customer"} rules:{from_table:"commerce.customer@replica" to_tables:"commerce.customer"} rules:{from_table:"customer" to_tables:"commerce.customer"} rules:{from_table:"customer.customer" to_tables:"commerce.customer"}
$ vtctldclient --server=localhost:15999 GetTopologyPath /global/RoutingRules
{
"name": "RoutingRules",
"path": "/global/RoutingRules",
"data": "rules:{from_table:\"customer.customer\" to_tables:\"commerce.customer\"} rules:{from_table:\"commerce.customer@rdonly\" to_tables:\"commerce.customer\"} rules:{from_table:\"corder\" to_tables:\"commerce.corder\"} rules:{from_table:\"corder@replica\" to_tables:\"commerce.corder\"} rules:{from_table:\"corder@rdonly\" to_tables:\"commerce.corder\"} rules:{from_table:\"customer.corder\" to_tables:\"commerce.corder\"} rules:{from_table:\"customer\" to_tables:\"commerce.customer\"} rules:{from_table:\"customer@rdonly\" to_tables:\"commerce.customer\"} rules:{from_table:\"customer.corder@replica\" to_tables:\"commerce.corder\"} rules:{from_table:\"commerce.customer@replica\" to_tables:\"commerce.customer\"} rules:{from_table:\"customer.customer@replica\" to_tables:\"commerce.customer\"} rules:{from_table:\"customer.customer@rdonly\" to_tables:\"commerce.customer\"} rules:{from_table:\"commerce.corder@rdonly\" to_tables:\"commerce.corder\"} rules:{from_table:\"customer@replica\" to_tables:\"commerce.customer\"} rules:{from_table:\"commerce.corder@replica\" to_tables:\"commerce.corder\"} rules:{from_table:\"customer.corder@rdonly\" to_tables:\"commerce.corder\"}",
"children": []
}
```

</br>

{{< info >}}
In practice you would instead typically view the routing rules via the
dedicated [`GetRoutingRules`](../../../programs/vtctl/schema-version-permissions/#getroutingrules)
dedicated [`GetRoutingRules`](../../../programs/vtctldclient/vtctldclient_getroutingrules/)
vtctl client command which will return the rules for all keyspaces in the topo.
{{< /info >}}

Expand Down
23 changes: 23 additions & 0 deletions content/en/docs/18.0/reference/vreplication/lookupvindex.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: LookupVindex
description: Create, backfill, and externalize Lookup Vindexes
weight: 60
---

[`LookupVindex`](../../../reference/programs/vtctldclient/vtctldclient_lookupvindex/) is a command used to create **and** backfill
a [Lookup Vindex](../../../reference/features/vindexes/#lookup-vindex-types) automatically for a table that already
exists, and may have a significant amount of data in it already.

Internally, the [`LookupVindex create`](../../../reference/programs/vtctldclient/vtctldclient_lookupvindex/vtctldclient_lookupvindex_create/) command uses
VReplication for the backfill process, until the lookup Vindex is "in sync". Then the normal process for
adding/deleting/updating rows in the lookup Vindex via the usual
[transactional flow when updating the "owner" table for the Vindex](../../../reference/features/vindexes/#lookup-vindex-types)
takes over.

## Command

Please see the [`LookupVindex` command reference](../../../reference/programs/vtctldclient/vtctldclient_lookupvindex/) for a full list of sub-commands and their flags.

## Additional Information

Please see the user guide for examples of [Creating a Lookup Vindex](../../../user-guides/configuration-advanced/createlookupvindex/) for more information on how to use this command.
Loading

0 comments on commit 94ddf23

Please sign in to comment.