Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update R5->R6 upgrade guide #745

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update R5->R6 upgrade guide #745

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 53 additions & 41 deletions doc/Upgrade-Guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,12 @@ into the `k8s-cluster-api-provider` directory. Note that the checked out
branch will be the one that has been used when creating the management host,
and you might want to change branches from `maintained/v3.x` to `maintained/v4.x`
in case of R2 to R3 upgrade, `maintained/v5.x` for R3 to R4 upgrade, `maintained/v6.x`
for R4 to R5 upgrade or `maintained/v7.x` for R5 to R6 upgrade.
for R4 to R5 upgrade or the tag `v7.0.0` for R5 to R6 upgrade.
Use `git branch -rl` to see available branches in the k8s-cluster-api-provider
repository.

You can update the scripts and templates by checking out the relevant branch
`main`, `maintained/v4.x`, `maintained/v5.x`, `maintained/v6.x` or `maintained/v7.x`
You can update the scripts and templates by checking out the relevant branch/tag
`main`, `maintained/v4.x`, `maintained/v5.x`, `maintained/v6.x` or `v7.0.0`
and using a `git pull` to ensure the latest content is retrieved.
Once you do that, the cluster-management scripts will be up-to-date.
(The `~/bin` directory in your search `PATH` is symlinked to the check-ed out scripts.)
Expand Down Expand Up @@ -140,11 +140,9 @@ You can then upgrade the components:

1. `export CLUSTER_TOPOLOGY=true` - this is needed from R5 to R6 upgrade due to ClusterClass feature #600
2. Upgrade components
- You can do them one-by-one, e.g.:
```bash
clusterctl upgrade apply --infrastructure capo-system/openstack:v0.9.0 --core capi-system/cluster-api:v1.6.2 -b capi-kubeadm-bootstrap-system/kubeadm:v1.6.2 -c capi-kubeadm-control-plane-system/kubeadm:v1.6.2
```
- Or simply do `clusterctl upgrade apply --contract v1beta1`
```bash
clusterctl upgrade apply --infrastructure capo-system/openstack:v0.9.0 --core capi-system/cluster-api:v1.6.2 -b capi-kubeadm-bootstrap-system/kubeadm:v1.6.2 -c capi-kubeadm-control-plane-system/kubeadm:v1.6.2
```

#### New templates

Expand Down Expand Up @@ -220,7 +218,13 @@ You can use script `update-R4-to-R5.sh` to update the cluster's `cluster-templat
R4 to R5. This script could update an existing Kubernetes cluster configuration files
as well as `cluster-defaults` files that could be used for spawning new R5 clusters.

If you want to update an existing cluster configuration files from R4 to R5, just use script as follows:
> Note: If you didn't change `cluster-template.yaml` or `clusterctl.yaml` before, the script should work
> without any problems. If you customized these files (e.g. when you are upgrading from the main branch, instead of maintained/v5.x),
> you should review the [script](https://github.com/SovereignCloudStack/k8s-cluster-api-provider/blob/main/terraform/files/bin/update-R4-to-R5.sh)
> and apply necessary changes manually. E.g. use `git diff ~/${CLUSTER_NAME}/cluster-template.yaml ~/k8s-cluster-api-provider/terraform/files/template/cluster-template.yaml`,
> revisit changes between releases and your customizations and apply them.

If you want to update an existing cluster configuration files from R4 to R5, just use script as follows (backup recommended):
```bash
update-R4-to-R5.sh <CLUSTER_NAME>
```
Expand Down Expand Up @@ -251,7 +255,13 @@ You can use script `update-R5-to-R6.sh` to update the cluster's `cluster-templat
R5 to R6. This script could update an existing Kubernetes cluster configuration files
as well as `cluster-defaults` files that could be used for spawning new R6 clusters.

If you want to update an existing cluster configuration files from R5 to R6, just use script as follows:
> Note: If you didn't change `cluster-template.yaml` or `clusterctl.yaml` before, the script should work
> without any problems. If you customized these files (e.g. when you are upgrading from the main branch, instead of maintained/v6.x),
> you should review the [script](https://github.com/SovereignCloudStack/k8s-cluster-api-provider/blob/main/terraform/files/bin/update-R5-to-R6.sh)
> and apply necessary changes manually. E.g. use `git diff ~/${CLUSTER_NAME}/cluster-template.yaml ~/k8s-cluster-api-provider/terraform/files/template/cluster-template.yaml`,
> revisit changes between releases and your customizations and apply them.

If you want to update an existing cluster configuration files from R5 to R6, just use script as follows (backup recommended):
```bash
update-R5-to-R6.sh <CLUSTER_NAME>
```
Expand Down Expand Up @@ -371,54 +381,55 @@ If you decide to migrate your existing Kubernetes cluster from R4 to R5 be aware

Follow the below steps if you want to migrate an existing cluster from R4 to R5:
1. Access your management node
2. Checkout R5 branch
2. Make sure there are no changes waiting to be applied (`create_cluster.sh <CLUSTER_NAME>`)
3. Checkout R5 branch
```bash
cd k8s-cluster-api-provider
git pull
git checkout maintained/v6.x
```
3. Backup an existing cluster configuration files (recommended)
4. Backup an existing cluster configuration files (recommended)
```bash
cd ..
cp -R <CLUSTER_NAME> <CLUSTER_NAME>-backup
```
4. Update an existing cluster configuration files from R4 to R5
5. Update an existing cluster configuration files from R4 to R5, see [related](#r4-to-r5) section for details before
```bash
update-R4-to-R5.sh <CLUSTER_NAME>
```
5. Validate updated cluster configuration files. You will find that e.g. Calico version
6. Validate updated cluster configuration files. You will find that e.g. Calico version
has been bumped from v3.25.0 to v3.26.1. Note that some software versions are not configurable
and are not directly mentioned in the cluster configuration files, but they are hardcoded
in R5 scripts (e.g. ingress nginx controller, metrics server). Hence, read carefully the
R5 release notes too. Also see that Kubernetes version was not updated, and it is still v1.25.6.
6. Update an existing cluster (except Kubernetes version)
7. Update an existing cluster (except Kubernetes version)
```bash
create_cluster.sh <CLUSTER_NAME>
```
7. Update cluster-API and openstack cluster-API provider, see [related](#updating-cluster-api-and-openstack-cluster-api-provider) section for details
8. Update cluster-API and openstack cluster-API provider, see [related](#updating-cluster-api-and-openstack-cluster-api-provider) section for details
```bash
clusterctl upgrade plan
clusterctl upgrade apply --contract v1beta1
clusterctl upgrade apply --infrastructure capo-system/openstack:v0.7.3 --core capi-system/cluster-api:v1.5.1 -b capi-kubeadm-bootstrap-system/kubeadm:v1.5.1 -c capi-kubeadm-control-plane-system/kubeadm:v1.5.1
```
8. Bump Kubernetes version +1 minor release (to v1.26.8) and increase the generation counter for node and control plane nodes
9. Bump Kubernetes version +1 minor release (to v1.26.8) and increase the generation counter for node and control plane nodes
```bash
sed -i 's/^KUBERNETES_VERSION: v1.25.6/KUBERNETES_VERSION: v1.26.8/' <CLUSTER_NAME>/clusterctl.yaml
sed -i 's/^OPENSTACK_IMAGE_NAME: ubuntu-capi-image-v1.25.6/OPENSTACK_IMAGE_NAME: ubuntu-capi-image-v1.26.8/' <CLUSTER_NAME>/clusterctl.yaml
sed -r 's/(^CONTROL_PLANE_MACHINE_GEN: genc)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
sed -r 's/(^WORKER_MACHINE_GEN: genw)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
```
9. Update an existing cluster Kubernetes version to v1.26.8
```bash
create_cluster.sh <CLUSTER_NAME>
```
10. Bump Kubernetes version to R5 v1.27.5 and increase the generation counter for node and control plane nodes
10. Update an existing cluster Kubernetes version to v1.26.8
```bash
create_cluster.sh <CLUSTER_NAME>
```
11. Bump Kubernetes version to R5 v1.27.5 and increase the generation counter for node and control plane nodes
```bash
sed -i 's/^KUBERNETES_VERSION: v1.26.8/KUBERNETES_VERSION: v1.27.5/' <CLUSTER_NAME>/clusterctl.yaml
sed -i 's/^OPENSTACK_IMAGE_NAME: ubuntu-capi-image-v1.26.8/OPENSTACK_IMAGE_NAME: ubuntu-capi-image-v1.27.5/' <CLUSTER_NAME>/clusterctl.yaml
sed -r 's/(^CONTROL_PLANE_MACHINE_GEN: genc)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
sed -r 's/(^WORKER_MACHINE_GEN: genw)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
```
11. Update an existing cluster to the R5 Kubernetes version v1.27.5
12. Update an existing cluster to the R5 Kubernetes version v1.27.5
```bash
create_cluster.sh <CLUSTER_NAME>
```
Expand All @@ -432,49 +443,50 @@ If you decide to migrate your existing Kubernetes cluster from R5 to R6 be aware

Follow the below steps if you want to migrate an existing cluster from R5 to R6:
1. Access your management node
2. Checkout R6 branch
2. Make sure there are no changes waiting to be applied (`create_cluster.sh <CLUSTER_NAME>`)
3. Checkout R6 release
```bash
cd k8s-cluster-api-provider
git pull
git checkout maintained/v7.x
git checkout v7.0.0
```
3. Backup an existing cluster configuration files (recommended)
4. Backup an existing cluster configuration files (recommended)
```bash
cd ..
cp -R <CLUSTER_NAME> <CLUSTER_NAME>-backup
```
4. Update an existing cluster configuration files from R5 to R6
5. Update an existing cluster configuration files from R5 to R6, see [related](#r5-to-r6) section for details before
```bash
update-R5-to-R6.sh <CLUSTER_NAME>
```
5. Validate updated cluster configuration files. You will find that e.g. Calico version
6. Validate updated cluster configuration files. You will find that e.g. Calico version
has been bumped from v3.26.1 to v3.27.2 or Kubernetes version bumped from v1.27.5 to v1.28.7.
Note that some software versions are not configurable and are not directly mentioned in the
cluster configuration files, but they are hardcoded in R6 scripts (e.g. ingress nginx controller,
metrics server, cilium). Hence, read carefully the R6 release notes too.
6. Update cluster-API and openstack cluster-API provider, see [related](#updating-cluster-api-and-openstack-cluster-api-provider) section for details
7. Update cluster-API and openstack cluster-API provider, see [related](#updating-cluster-api-and-openstack-cluster-api-provider) section for details
```bash
clusterctl upgrade plan
export CLUSTER_TOPOLOGY=true
clusterctl upgrade apply --contract v1beta1
clusterctl upgrade apply --infrastructure capo-system/openstack:v0.9.0 --core capi-system/cluster-api:v1.6.2 -b capi-kubeadm-bootstrap-system/kubeadm:v1.6.2 -c capi-kubeadm-control-plane-system/kubeadm:v1.6.2
```
7. Migrate to ClusterClass
8. Migrate to ClusterClass
```bash
migrate-to-cluster-class.sh <CLUSTER_NAME>
```
8. Increase the generation counter for worker and control plane nodes
9. Increase the generation counter for worker and control plane nodes
```bash
sed -r 's/(^CONTROL_PLANE_MACHINE_GEN: genc)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
sed -r 's/(^WORKER_MACHINE_GEN: genw)([0-9][0-9])/printf "\1%02d" $((\2+1))/ge' -i <CLUSTER_NAME>/clusterctl.yaml
```
9. Update an existing cluster to the R6
```bash
create_cluster.sh <CLUSTER_NAME>
```
> Note: You will probably experience a double rollout of nodes because
> the k8s version and templates are changed concurrently here.
> See [https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/operate-cluster#effects-of-concurrent-changes](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/operate-cluster#effects-of-concurrent-changes)
10. Upgrade cilium (for clusters with `USE_CILIUM: true`)
10. Update an existing cluster to the R6
```bash
create_cluster.sh <CLUSTER_NAME>
```
> Note: You will probably experience a double rollout of nodes because
> the k8s version and templates are changed concurrently here.
> See [https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/operate-cluster#effects-of-concurrent-changes](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/operate-cluster#effects-of-concurrent-changes)
11. Upgrade cilium (for clusters with `USE_CILIUM: true`)
```bash
KUBECONFIG=<CLUSTER_NAME>/<CLUSTER_NAME>.yaml bash -c 'helm get values cilium -n kube-system -o yaml | cilium upgrade --version v1.15.1 -f -'
```
Expand Down