Skip to content

Commit

Permalink
[release-18.0] release notes: edit summary for consistency (#14319) (#…
Browse files Browse the repository at this point in the history 
…14320)

Co-authored-by: Deepthi Sigireddi <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>
  • Loading branch information
@vitess-bot @deepthi @GuptaManan100
3 people authored Oct 20, 2023
1 parent d8a5dc0 commit 5f0df9a
Showing 1 changed file with 52 additions and 51 deletions.
103 changes: 52 additions & 51 deletions changelog/18.0/18.0.0/summary.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,68 +44,69 @@
In previous releases the [local examples](https://github.com/vitessio/vitess/tree/main/examples/local) were
explicitly using etcd v2 storage (`etcd --enable-v2=true`) and API (`ETCDCTL_API=2`) mode. We have now
removed this legacy etcd usage and instead use the new (default) etcd v3 storage and API. Please see
[PR #13791](https://github.com/vitessio/vitess/pull/13791) for additional info. If you are using the local
[PR #13791](https://github.com/vitessio/vitess/pull/13791) for details. If you are using the local
examples in any sort of long-term non-testing capacity, then you will need to explicitly use the v2 storage
and API mode or [migrate your existing data from v2 to v3](https://etcd.io/docs/v3.5/tutorials/how-to-migrate/).

### <a id="new-flag"/>New command line flags and behavior

#### <a id="new-flag-toggle-ers"/>VTOrc flag `--allow-emergency-reparent`

VTOrc has a new flag `--allow-emergency-reparent` that allows the users to toggle the ability of VTOrc to run emergency
reparent operations. Users that want VTOrc to fix the replication issues, but don't want it to run any reparents
should start using this flag. By default, VTOrc will be able to run `EmergencyReparentShard`. Users must specify the
flag to `false` to change the behaviour.
VTOrc has a new flag `--allow-emergency-reparent` that specifies whether VTOrc is allowed to run emergency
failover operations. Users that want VTOrc to fix replication issues, but don't want it to run any failovers
should use this flag. This flag defaults to `true` which corresponds to the default behavior from prior releases.

#### <a id="new-flag-errant-gtid-convert"/>VTOrc flag `--change-tablets-with-errant-gtid-to-drained`

VTOrc has a new flag `--change-tablets-with-errant-gtid-to-drained` that allows users to choose whether VTOrc should change the
tablet type of tablets with errant GTIDs to `DRAINED`. By default, it is disabled.
tablet type of tablets with errant GTIDs to `DRAINED`. By default, this flag is disabled.

This feature allows users to configure VTOrc such that any tablet that encounters errant GTIDs is automatically taken out of the
serving graph. These tablets can then be inspected for what the errant GTIDs are, and once fixed, they can rejoin the cluster.

#### <a id="new-ers-subflag"/>ERS sub flag `--wait-for-all-tablets`

Running `EmergencyReparentShard` from the vtctldclient has a new sub-flag `--wait-for-all-tablets` that makes `EmergencyReparentShard` wait
vtctldclient command `EmergencyReparentShard` has a new sub-flag `--wait-for-all-tablets` that makes `EmergencyReparentShard` wait
for a response from all the tablets. Originally `EmergencyReparentShard` was meant only to be run when a primary tablet is unreachable.
We have realized now that there are cases when the replication is broken but all the tablets are reachable. In these cases, it is advisable to
call `EmergencyReparentShard` with `--wait-for-all-tablets` so that it does not ignore one of the tablets.
We have realized now that there are cases when replication is broken but all tablets are reachable. In these cases, it is advisable to
call `EmergencyReparentShard` with `--wait-for-all-tablets` so that it does not ignore any of the tablets.

#### <a id="new-vtgate-streaming-sesion"/>VTGate GRPC stream execute session flag `--grpc-send-session-in-streaming`

This flag enables transaction support on `StreamExecute` api.
Once enabled, VTGate `StreamExecute` gRPC api will send session as the last packet in the response.
The client should enable it only when they have made the required changes to expect such a packet.
This flag enables transaction support on VTGate's `StreamExecute` gRPC API.
When this is enabled, `StreamExecute` will return the session in the last packet of the response.
Users should enable this flag only after client code has been changed to expect such a packet.

It is disabled by default.

### <a id="foreign-keys"/>Experimental Foreign Key Support

A new field `foreignKeyMode` has been added to the VSchema. This field can be provided for each keyspace. The VTGate flag `--foreign_key_mode` has been deprecated in favour of this field.
A new optional field `foreignKeyMode` has been added to the VSchema. This field can be provided for each keyspace. The VTGate flag `--foreign_key_mode` has been deprecated in favor of this field.

There are 3 foreign key modes now supported in Vitess -
1. `unmanaged` -
This mode represents the default behaviour in Vitess, where it does not manage foreign keys column references. Users are responsible for configuring foreign keys in MySQL in such a way that related rows, as determined by foreign keys, reside within the same shard.
This mode represents the default behavior in Vitess, where it does not manage foreign key column references. Users are responsible for configuring foreign keys in MySQL in such a way that related rows, as determined by foreign keys, reside within the same shard.
2. `managed` [EXPERIMENTAL] -
In this experimental mode, Vitess is fully aware of foreign key relationships and actively tracks foreign key constraints using the schema tracker. Vitess takes charge of handling DML operations with foreign keys cascading updates, deletes and verifying restrict. It will also validate parent row existence.
This ensures that all the operations are logged in binary logs, unlike MySQL implementation of foreign keys.
This enables seamless integration of VReplication with foreign keys.
For more details on what operations Vitess takes please refer to the [design document for foreign keys](https://github.com/vitessio/vitess/issues/12967).
In this experimental mode, Vitess is fully aware of foreign key relationships and actively tracks foreign key constraints using the schema tracker. VTGate will handle DML operations with foreign keys and correctly cascade updates and deletes.
It will also verify `restrict` constraints and validate the existence of parent rows before inserting child rows.
This ensures that all child operations are logged in binary logs, unlike the InnoDB implementation of foreign keys.
This allows the usage of VReplication workflows with foreign keys.
Implementation details are documented in the [RFC for foreign keys](https://github.com/vitessio/vitess/issues/12967).
3. `disallow` -
In this mode Vitess explicitly disallows any DDL statements that try to create a foreign key constraint. This mode is equivalent to running VTGate with the flag `--foreign_key_mode=disallow`.

#### Upgrade process

After upgrading from v17 to v18, the users should specify the correct foreign key mode for all their keyspaces in the VSchema using the new property.
Once this change has taken effect, the deprecated flag `--foreign_key_mode` can be dropped from all the VTGates.
After upgrading from v17 to v18, users should specify the correct foreign key mode for all their keyspaces in the VSchema using the new property.
Once this change has taken effect, the deprecated flag `--foreign_key_mode` can be dropped from all VTGates. Note that this is only required if running in `disallow` mode.
No action is needed to use `unmanaged` mode.

### <a id="vtadmin"/>VTAdmin

#### <a id="updated-node"/>vtadmin-web updated to node v18.16.0 (LTS)

Building `vtadmin-web` now requires node >= v18.16.0 (LTS). Breaking changes from v16 to v18 are listed
in https://nodejs.org/en/blog/release/v18.0.0, but none apply to VTAdmin. Full details on v18.16.0 are listed
in https://nodejs.org/en/blog/release/v18, but none apply to VTAdmin. Full details on node v18.16.0 are listed
on https://nodejs.org/en/blog/release/v18.16.0.

### <a id="deprecations-and-deletions"/>Deprecations and Deletions
Expand All @@ -124,58 +125,58 @@ develop new features without impacting other commands. This also presents an [AP
This makes development easier while also offering a better UX. For example, this offers a way to use
[configuration files](https://vitess.io/docs/18.0/reference/viper/config-files/) with support for
[dynamic configuration](https://vitess.io/docs/18.0/reference/viper/dynamic-values/) ([see also](https://github.com/vitessio/vitess/blob/release-18.0/doc/viper/viper.md)).
- The [reference documentation](https://vitess.io/docs/reference/programs/vtctldclient/) is now built through code. This
- The [reference documentation](https://vitess.io/docs/18.0/reference/programs/vtctldclient/) is now built through code. This
removes a burden from developers while helping users by ensuring the docs are always correct and up-to-date.

In Vitess 18.0 we have completed migrating all client commands to `vtctldclient` – the last ones being the [OnlineDDL](https://github.com/vitessio/vitess/issues/13712)
In Vitess 18 we have completed migrating all client commands to `vtctldclient` – the last ones being the [OnlineDDL](https://github.com/vitessio/vitess/issues/13712)
and [VReplication](https://github.com/vitessio/vitess/issues/12152) commands. With this work now completed, the
legacy `vtctl`/`vtctlclient` binaries are now fully deprecated and we plan to remove them in Vitess 19.0. You should
[begin your transition](https://vitess.io/docs/18.0/reference/vtctldclient-transition/) before upgrading to 18.0.
legacy `vtctl`/`vtctlclient` binaries are now fully deprecated and we plan to remove them in Vitess 19. You should
[begin your transition](https://vitess.io/docs/18.0/reference/vtctldclient-transition/) before upgrading to Vitess 18.

#### <a id="deprecated-flags"/>Deprecated Command Line Flags

Throttler related `vttablet` flags:

- `--throttle_threshold` is deprecated and will be removed in `v19.0`
- `--throttle_metrics_query` is deprecated and will be removed in `v19.0`
- `--throttle_metrics_threshold` is deprecated and will be removed in `v19.0`
- `--throttle_check_as_check_self` is deprecated and will be removed in `v19.0`
- `--throttler-config-via-topo` is deprecated after assumed `true` in `v17.0`. It will be removed in a future version.
- `--throttle_threshold` is deprecated and will be removed in `v19`
- `--throttle_metrics_query` is deprecated and will be removed in `v19`
- `--throttle_metrics_threshold` is deprecated and will be removed in `v19`
- `--throttle_check_as_check_self` is deprecated and will be removed in `v19`
- `--throttler-config-via-topo` is deprecated after assumed `true` in `v17`. It will be removed in a future version.

Cache related `vttablet` flags:

- `--queryserver-config-query-cache-lfu` is deprecated and will be removed in `v19.0`. The query cache always uses a LFU implementation now.
- `--queryserver-config-query-cache-size` is deprecated and will be removed in `v19.0`. This option only applied to LRU caches, which are now unsupported.
- `--queryserver-config-query-cache-lfu` is deprecated and will be removed in `v19`. The query cache always uses a LFU implementation now.
- `--queryserver-config-query-cache-size` is deprecated and will be removed in `v19`. This option only applied to LRU caches, which are now unsupported.

Buffering related `vtgate` flags:

- `--buffer_implementation` is deprecated and will be removed in `v19.0`
- `--buffer_implementation` is deprecated and will be removed in `v19`

Cache related `vtgate` flags:

- `--gate_query_cache_lfu` is deprecated and will be removed in `v19.0`. The query cache always uses a LFU implementation now.
- `--gate_query_cache_size` is deprecated and will be removed in `v19.0`. This option only applied to LRU caches, which are now unsupported.
- `--gate_query_cache_lfu` is deprecated and will be removed in `v19`. The query cache always uses a LFU implementation now.
- `--gate_query_cache_size` is deprecated and will be removed in `v19`. This option only applied to LRU caches, which are now unsupported.

VTGate flags:

- `--schema_change_signal_user` is deprecated and will be removed in `v19.0`
- `--foreign_key_mode` is deprecated and will be removed in `v19.0`. For more detail read the [foreign keys](#foreign-keys) section.
- `--schema_change_signal_user` is deprecated and will be removed in `v19`
- `--foreign_key_mode` is deprecated and will be removed in `v19`. For more detail read the [foreign keys](#foreign-keys) section.

VDiff v1:

[VDiff v2 was added in Vitess 15.0](https://vitess.io/blog/2022-11-22-vdiff-v2/) and marked as GA in 16.0.
The [legacy v1 client command](https://vitess.io/docs/18.0/reference/vreplication/vdiffv1/) is now deprecated in Vitess 18.0 and will be **removed** in 19.0.
[VDiff v2 was added in Vitess 15](https://vitess.io/blog/2022-11-22-vdiff-v2/) and marked as GA in 16.
The [legacy v1 client command](https://vitess.io/docs/18.0/reference/vreplication/vdiffv1/) is now deprecated in Vitess 18 and will be **removed** in Vitess 19.
Please switch all of your usage to the [new VDiff client](https://vitess.io/docs/18.0/reference/vreplication/vdiff/) command ASAP.


#### <a id="deprecated-stats"/>Deprecated Stats

The following `EmergencyReparentShard` stats are deprecated in `v18.0` and will be removed in `v19.0`:
The following `EmergencyReparentShard` stats are deprecated in `v18` and will be removed in `v19`:
- `ers_counter`
- `ers_success_counter`
- `ers_failure_counter`

These metrics are replaced by [new reparenting stats introduced in `v18.0`](#vtctld-and-vtorc-reparenting-stats).
These metrics are replaced by [new reparenting stats introduced in `v18`](#vtctld-and-vtorc-reparenting-stats).

VTBackup stat `DurationByPhase` is deprecated. Use the binary-valued `Phase` stat instead.

Expand Down Expand Up @@ -214,16 +215,15 @@ Flags in `vtorc`:

#### <a id="deleted-v3"/>Deleted `v3` planner

The `Gen4` planner has been the default planner since Vitess 14. The `v3` planner was deprecated in Vitess 15 and has now been removed in this release.
The `Gen4` planner has been the default planner since Vitess 14. The `v3` planner was deprecated in Vitess 15 and has been removed in Vitess 18.

#### <a id="deleted-k8stopo"/>Deleted `k8stopo`

The `k8stopo` has been deprecated in Vitess 17, also see https://github.com/vitessio/vitess/issues/13298. With Vitess 18
the `k8stopo` has been removed.
`k8stopo` was deprecated in Vitess 17, see https://github.com/vitessio/vitess/issues/13298. It has now been removed.

#### <a id="deleted-vtgr"/>Deleted `vtgr`

The `vtgr` has been deprecated in Vitess 17, also see https://github.com/vitessio/vitess/issues/13300. With Vitess 18 `vtgr` has been removed.
`vtgr` was deprecated in Vitess 17, see https://github.com/vitessio/vitess/issues/13300. It has now been removed.

#### <a id="deleted-query_analyzer"/>Deleted `query_analyzer`

Expand All @@ -237,7 +237,7 @@ The VTGate stat `VindexUnknownParameters` gauges unknown Vindex parameters found

#### <a id="vtbackup-stat-phase"/>VTBackup `Phase` stat

In v17, the `vtbackup` stat `DurationByPhase` stat was added measuring the time spent by `vtbackup` in each phase. This stat turned out to be awkward to use in production, and has been replaced in v18 by a binary-valued `Phase` stat.
In v17, the `vtbackup` stat `DurationByPhase` stat was added to measure the time spent by `vtbackup` in each phase. This stat turned out to be awkward to use in production, and has been replaced in v18 by a binary-valued `Phase` stat.

`Phase` reports a 1 (active) or a 0 (inactive) for each of the following phases:

Expand Down Expand Up @@ -272,7 +272,7 @@ vtbackup_restore_count{component="BackupStorage",implementation="S3",operation="

#### <a id="vtctld-and-vtorc-reparenting-stats"/>VTCtld and VTOrc reparenting stats

New VTCtld and VTorc stats were added to measure frequency of reparents by keyspace/shard:
New VTCtld and VTOrc stats were added to measure frequency of reparents by keyspace/shard:
- `emergency_reparent_counts` - Number of times `EmergencyReparentShard` has been run. It is further subdivided by the keyspace, shard and the result of the operation.
- `planned_reparent_counts` - Number of times `PlannedReparentShard` has been run. It is further subdivided by the keyspace, shard and the result of the operation.

Expand All @@ -297,7 +297,7 @@ tablet will not actually throttle any transactions; however, it will increase th
(`vttablet_transaction_throttler_throttled`). This allows users to deploy the transaction throttler in production and
gain observability on how much throttling would take place, without actually throttling any requests.

### <a id="docker"/>Docker
### <a id="docker"/>Docker Builds

#### <a id="debian-bookworm"/>Bookworm added and made default

Expand All @@ -306,12 +306,13 @@ Bullseye images will still be built and available as long as the OS build is cur

#### <a id="debian-buster"/>Buster removed

Buster LTS supports will stop in June 2024, and Vitess v18.0 will be supported through October 2024.
Buster LTS supports will stop in June 2024, and Vitess 18 will be supported through October 2024.
To prevent supporting a deprecated buster build for several months after June 2024, we are preemptively
removing Vitess support.
removing Vitess support for Buster.

### <a id="durability-policies"/>Durability Policies

#### <a id="new-durability-policies"/>New Durability Policies

2 new inbuilt durability policies have been added to Vitess in this release namely `semi_sync_with_rdonly_ack` and `cross_cell_with_rdonly_ack`. These policies are exactly like `semi_sync` and `cross_cell` respectively, and differ just in the part where the rdonly tablets can also send semi-sync ACKs.
Two new built-in durability policies have been added in Vitess 18: `semi_sync_with_rdonly_ack` and `cross_cell_with_rdonly_ack`.
These policies are similar to `semi_sync` and `cross_cell` respectively, the only difference is that `rdonly` tablets can also send semi-sync ACKs.

0 comments on commit 5f0df9a

Please sign in to comment.