From e9ff65b00dbb966f4f3fdc2e75842080536c6cb8 Mon Sep 17 00:00:00 2001 From: Noah Merlis-Stephens Date: Mon, 13 Jan 2025 05:54:31 -0800 Subject: [PATCH] [BOP-1572] Add etcd configuration and custom flag migration info (#243) Co-authored-by: Kory <57411706+KoryKessel-Mirantis@users.noreply.github.com> Co-authored-by: Magdalena Dziadosz <160592158+MagdaDziadosz@users.noreply.github.com> --- .../docs/configuration/kubernetes/_index.md | 1 + content/docs/configuration/kubernetes/etcd.md | 35 ++++++++++++++++ content/docs/migrate-from-MKE-3.md | 40 +++++++++++++++++++ 3 files changed, 76 insertions(+) create mode 100644 content/docs/configuration/kubernetes/etcd.md diff --git a/content/docs/configuration/kubernetes/_index.md b/content/docs/configuration/kubernetes/_index.md index 830ca568..c658e7d7 100644 --- a/content/docs/configuration/kubernetes/_index.md +++ b/content/docs/configuration/kubernetes/_index.md @@ -9,6 +9,7 @@ MKE 4 uses K0s to deploy core Kubernetes components, including: - [kube-apiserver](../../configuration/kubernetes/kube-apiserver) - [kube-controller-manager](../../configuration/kubernetes/kube-controller-manager) - [kube-scheduler](../../configuration/kubernetes/kube-scheduler) +- [etcd](../../configuration/kubernetes/etcd) You can configure all of the Kubernetes components through the MKE configuration file. \ No newline at end of file diff --git a/content/docs/configuration/kubernetes/etcd.md b/content/docs/configuration/kubernetes/etcd.md new file mode 100644 index 00000000..17a9e24d --- /dev/null +++ b/content/docs/configuration/kubernetes/etcd.md @@ -0,0 +1,35 @@ +--- +title: etcd +weight: 6 +--- + +etcd is a consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It handles leader elections during network partitions and can tolerate machine failure, even in the leader node. + +For MKE, etcd serves as the Kubernetes backing store for all cluster data, with an etcd replica deployed on each MKE manager node. This is a primary reason why Mirantis recommends that you deploy an odd number of MKE manager nodes, as etcd uses the Raft consensus algorithm and thus requires that a quorum of nodes agrees on any updates to the cluster state. + +For detailed information, refer to the official [etcd documentation](https://etcd.io/docs/). + +## Configure etcd + +You can configure etcd through the `etcd` section of the MKE configuration file, an example of which follows: + +```yaml +spec: + etcd: + storageQuota: 2GB + tlsCipherSuites: TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 +``` + +## Configure etcd storage quota + +You can control the etcd distributed key-value storage quota using the `etcd.storageQuota` parameter in the MKE configuration file. By default, the value of the parameter is 2GB. + +If you choose to increase the etcd quota, be aware that this quota has a limit and such action should be used in conjunction with other strategies, such as decreasing events TTL to ensure that the etcd database does not run out of space. + +{{< callout type="warning" >}} If a manager node virtual machine runs out of disk space, or if all of its system memory is depleted, etcd can cause the MKE cluster to move into an irrecoverable state. To prevent this from happening, configure the disk space and the memory of the manager node VMs to levels that are well in excess of the set etcd storage quota. {{< /callout >}} + +{{< callout type="info" >}} + +For additional information, refer to the etcd official documentation, [How to debug large db size issue?](https://etcd.io/blog/2023/how_to_debug_large_db_size_issue/) + +{{< /callout >}} diff --git a/content/docs/migrate-from-MKE-3.md b/content/docs/migrate-from-MKE-3.md index 69375e19..3ef5746e 100644 --- a/content/docs/migrate-from-MKE-3.md +++ b/content/docs/migrate-from-MKE-3.md @@ -126,6 +126,46 @@ mkectl init --mke3-config ensure you complete the [GPU prerequisites](../configuration/nvidia-gpu/#prerequisites) before starting the upgrade process. {{< /callout >}} +### Kubernetes Custom Flags + +MKE 3 and MKE 4 both support the application of additional flags to Kubernetes components that have the following fields in the MKE configuration file, each specified as a list of strings: +``` +custom_kube_api_server_flags +custom_kube_controller_manager_flags +custom_kubelet_flags +custom_kube_scheduler_flags +custom_kube_proxy_flags +``` + +MKE 4 supports an `extraArgs` field for each of these components, though, which accepts a map of key-value pairs. During upgrade from MKE 3, MKE 4 converts these custom flags to the corresponding `extraArgs` field. Any flags that cannot be automatically converted are listed in the upgrade summary. + +Example of custom flags conversion: + +- MKE 3 configuration file: + + ``` + [cluster_config.custom_kube_api_server_flags] = ["--enable-garbage-collector=false"] + ``` + +- MKE 4 configuration file: + + ``` + spec: + apiServer: + extraArgs: + enable-garbage-collector: false + ``` + +### Kubelet Custom Flag Profiles + +MKE 3 supports a map of kubelet flag profiles to specific nodes using the `custom_kubelet_flags_profiles` setting in the toml configuration file. + +MKE 4 does not support kubelet flag profiles, but you can use [Kubelet custom profiles](../configuration/kubernetes/kubelet.md#kubelet-custom-profiles) to map `KubeletConfiguration` values to specific nodes. MKE 4 does support the migration of MKE 3 kubelet flag profiles to kubelet custom profiles. + +The conversion of flags to `KubeletConfiguration` values is best-effort, and any flags that cannot be +converted are listed in the upgrade summary. Hosts with a custom flag profile label are marked for the +corresponding kubelet custom profile. + ## Perform the migration An upgrade from MKE 3 to MKE 4 consists of the following steps, all of which