Skip to content

Commit

Permalink
[BOP-1572] Add etcd configuration and custom flag migration info (#243)
Browse files Browse the repository at this point in the history
Co-authored-by: Kory <[email protected]>
Co-authored-by: Magdalena Dziadosz <[email protected]>
  • Loading branch information
3 people authored Jan 13, 2025
1 parent 97400ee commit e9ff65b
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 0 deletions.
1 change: 1 addition & 0 deletions content/docs/configuration/kubernetes/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ MKE 4 uses K0s to deploy core Kubernetes components, including:
- [kube-apiserver](../../configuration/kubernetes/kube-apiserver)
- [kube-controller-manager](../../configuration/kubernetes/kube-controller-manager)
- [kube-scheduler](../../configuration/kubernetes/kube-scheduler)
- [etcd](../../configuration/kubernetes/etcd)

You can configure all of the Kubernetes components through the MKE
configuration file.
35 changes: 35 additions & 0 deletions content/docs/configuration/kubernetes/etcd.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: etcd
weight: 6
---

etcd is a consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It handles leader elections during network partitions and can tolerate machine failure, even in the leader node.

For MKE, etcd serves as the Kubernetes backing store for all cluster data, with an etcd replica deployed on each MKE manager node. This is a primary reason why Mirantis recommends that you deploy an odd number of MKE manager nodes, as etcd uses the Raft consensus algorithm and thus requires that a quorum of nodes agrees on any updates to the cluster state.

For detailed information, refer to the official [etcd documentation](https://etcd.io/docs/).

## Configure etcd

You can configure etcd through the `etcd` section of the MKE configuration file, an example of which follows:

```yaml
spec:
etcd:
storageQuota: 2GB
tlsCipherSuites: TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
```
## Configure etcd storage quota
You can control the etcd distributed key-value storage quota using the `etcd.storageQuota` parameter in the MKE configuration file. By default, the value of the parameter is 2GB.

If you choose to increase the etcd quota, be aware that this quota has a limit and such action should be used in conjunction with other strategies, such as decreasing events TTL to ensure that the etcd database does not run out of space.

{{< callout type="warning" >}} If a manager node virtual machine runs out of disk space, or if all of its system memory is depleted, etcd can cause the MKE cluster to move into an irrecoverable state. To prevent this from happening, configure the disk space and the memory of the manager node VMs to levels that are well in excess of the set etcd storage quota. {{< /callout >}}

{{< callout type="info" >}}

For additional information, refer to the etcd official documentation, [How to debug large db size issue?](https://etcd.io/blog/2023/how_to_debug_large_db_size_issue/)

{{< /callout >}}
40 changes: 40 additions & 0 deletions content/docs/migrate-from-MKE-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,46 @@ mkectl init --mke3-config </path/to/mke3-config.toml>
ensure you complete the [GPU prerequisites](../configuration/nvidia-gpu/#prerequisites) before
starting the upgrade process. {{< /callout >}}

### Kubernetes Custom Flags

MKE 3 and MKE 4 both support the application of additional flags to Kubernetes components that have the following fields in the MKE configuration file, each specified as a list of strings:
```
custom_kube_api_server_flags
custom_kube_controller_manager_flags
custom_kubelet_flags
custom_kube_scheduler_flags
custom_kube_proxy_flags
```

MKE 4 supports an `extraArgs` field for each of these components, though, which accepts a map of key-value pairs. During upgrade from MKE 3, MKE 4 converts these custom flags to the corresponding `extraArgs` field. Any flags that cannot be automatically converted are listed in the upgrade summary.

Example of custom flags conversion:

- MKE 3 configuration file:

```
[cluster_config.custom_kube_api_server_flags] = ["--enable-garbage-collector=false"]
```

- MKE 4 configuration file:

```
spec:
apiServer:
extraArgs:
enable-garbage-collector: false
```

### Kubelet Custom Flag Profiles

MKE 3 supports a map of kubelet flag profiles to specific nodes using the `custom_kubelet_flags_profiles` setting in the toml configuration file.

MKE 4 does not support kubelet flag profiles, but you can use [Kubelet custom profiles](../configuration/kubernetes/kubelet.md#kubelet-custom-profiles) to map `KubeletConfiguration` values to specific nodes. MKE 4 does support the migration of MKE 3 kubelet flag profiles to kubelet custom profiles.

The conversion of flags to `KubeletConfiguration` values is best-effort, and any flags that cannot be
converted are listed in the upgrade summary. Hosts with a custom flag profile label are marked for the
corresponding kubelet custom profile.

## Perform the migration

An upgrade from MKE 3 to MKE 4 consists of the following steps, all of which
Expand Down

0 comments on commit e9ff65b

Please sign in to comment.