From 609712c58f2e7509434406f6491fa2a0cef6e3a7 Mon Sep 17 00:00:00 2001 From: tomas-balaz Date: Mon, 12 Sep 2022 13:20:14 +0200 Subject: [PATCH 1/3] docs: StorageClass documentation --- README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/README.md b/README.md index 800ff5e..c4363df 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,22 @@ Create helm release resource and deploy it as argo application (set `enabled = t To disable of creation IRSA role and IRSA policy, set `irsa_role_create = false` and `irsa_policy_enabled = false`, respectively +## Storage Class + +Creation of defined Storage Classes is enabled by default. Set `storage_classes_create = false` to disable it. + +A list of Storage Classes is defined in `storage_classes`. One Storage Class is defined by default. + +[kubernetes-specific Storage Class parameters](https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource) (`provisioner` is set to `ebs.csi.aws.com` in [EBS-CSI-driver chart](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/storageclass.yaml) statically) + +[EBS-CSI-specific Storage Class parameters](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md) + +### Potential issues after enabling StorageClass creation +`gp2` Storage Class is created along with EKS cluster creation and it is annotated as [default Storage Class](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html). +Default configuration of Storage Class in this module contains annotation `"storageclass.kubernetes.io/is-default-class" : "true"`, so an error may occur while creating Persistent Volume Claim objects without specifying Storage Class name, because of more than 1 Storage Classes are annotated as default. [Annotate only one Storage Class as default](https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass) to fix this issue. + + + ## Examples See [Basic example](examples/basic/README.md) for further information. From bfb3622478f656cf3bbbc725edfe948a0c04374c Mon Sep 17 00:00:00 2001 From: tomas-balaz Date: Mon, 12 Sep 2022 16:00:12 +0200 Subject: [PATCH 2/3] docs: update StorageClass documentation --- README.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index c4363df..beb46c6 100644 --- a/README.md +++ b/README.md @@ -40,15 +40,21 @@ To disable of creation IRSA role and IRSA policy, set `irsa_role_create = false` Creation of defined Storage Classes is enabled by default. Set `storage_classes_create = false` to disable it. -A list of Storage Classes is defined in `storage_classes`. One Storage Class is defined by default. +One Storage Class is defined by default: -[kubernetes-specific Storage Class parameters](https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource) (`provisioner` is set to `ebs.csi.aws.com` in [EBS-CSI-driver chart](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/storageclass.yaml) statically) +https://github.com/lablabs/terraform-aws-eks-ebs-csi-driver/blob/b5c05b76e2664dc235130791a958ab0409f84837/variables.tf#L111-L128 -[EBS-CSI-specific Storage Class parameters](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md) +To extend or modify this definition use variable `storage_classes`. +`provisioner` attribute of defined Storage Classes will be set to `ebs.csi.aws.com` ([EBS-CSI-driver chart](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/storageclass.yaml)). To define other Storage Class parameters use [EBS-CSI Storage Class parameters](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md) ### Potential issues after enabling StorageClass creation `gp2` Storage Class is created along with EKS cluster creation and it is annotated as [default Storage Class](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html). -Default configuration of Storage Class in this module contains annotation `"storageclass.kubernetes.io/is-default-class" : "true"`, so an error may occur while creating Persistent Volume Claim objects without specifying Storage Class name, because of more than 1 Storage Classes are annotated as default. [Annotate only one Storage Class as default](https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass) to fix this issue. +Default configuration of Storage Class in this module contains annotation `"storageclass.kubernetes.io/is-default-class" : "true"`, so an error may occur while creating Persistent Volume Claim objects without specifying Storage Class name, because of more than 1 Storage Classes are annotated as default. + +To prevent this error: + +1. [change annotation on gp2 Storage Class](https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass) or +2. modify `storage_classes` to not annotate Storage Classes as default From 902d96d1029997c6ed6e994ea5876ee6fbfaab33 Mon Sep 17 00:00:00 2001 From: tomas-balaz Date: Mon, 12 Sep 2022 16:52:58 +0200 Subject: [PATCH 3/3] docs: update Storage Class documentation --- README.md | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index beb46c6..b6b4164 100644 --- a/README.md +++ b/README.md @@ -38,23 +38,34 @@ To disable of creation IRSA role and IRSA policy, set `irsa_role_create = false` ## Storage Class -Creation of defined Storage Classes is enabled by default. Set `storage_classes_create = false` to disable it. +Default StorageClass is enabled by default with following definition: + + { + "name" : "ebs-csi-gp3" + "annotations" : { + "storageclass.kubernetes.io/is-default-class" : "true" + } + "allowVolumeExpansion" : true + "volumeBindingMode" : "WaitForFirstConsumer" + "reclaimPolicy" : "Delete" + "parameters" : { + "type" : "gp3" + "encrypted" : "true" + } + } + +To override default module StorageClass behaviour use variable `storage_classes`. Provisioner attribute is hardcoded in [EBS-CSI-driver chart](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/storageclass.yaml) and cannot be changed. For other parameters attributes please see [EBS-CSI Storage Class parameters](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md). -One Storage Class is defined by default: - -https://github.com/lablabs/terraform-aws-eks-ebs-csi-driver/blob/b5c05b76e2664dc235130791a958ab0409f84837/variables.tf#L111-L128 +### Potential issues after enabling StorageClass creation -To extend or modify this definition use variable `storage_classes`. -`provisioner` attribute of defined Storage Classes will be set to `ebs.csi.aws.com` ([EBS-CSI-driver chart](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/storageclass.yaml)). To define other Storage Class parameters use [EBS-CSI Storage Class parameters](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md) +With each new cluster, Amazon EKS applies a `kubernetes.io/aws-ebs` based StorageClass named `gp2` with defined `“storageclass.kubernetes.io/is-default-class” : “true”` and by default module also creates StorageClass which contains same `storageclass.kubernetes.io/is-default-class` annotations. This may lead to errors while creating Persistent Volume Claim objects without specifying StorageClass name as described in the [documentation](https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass). -### Potential issues after enabling StorageClass creation -`gp2` Storage Class is created along with EKS cluster creation and it is annotated as [default Storage Class](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html). -Default configuration of Storage Class in this module contains annotation `"storageclass.kubernetes.io/is-default-class" : "true"`, so an error may occur while creating Persistent Volume Claim objects without specifying Storage Class name, because of more than 1 Storage Classes are annotated as default. +There are several ways to prevent this error: +1. delete EKS default StorageClass gp2 before PVC usage +2. change annotation on EKS default storage class +3. disable or modify default module StorageClass definition -To prevent this error: -1. [change annotation on gp2 Storage Class](https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass) or -2. modify `storage_classes` to not annotate Storage Classes as default