customize-apps-with-configmaps (#461)

* customize-apps-with-configmaps
giantswarm · Dec 21, 2023 · 1d8e145 · 1d8e145
1 parent 2014313
commit 1d8e145
Show file tree

Hide file tree

Showing 9 changed files with 130 additions and 11 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### **Breaking change**
+
+- Added option to customize app via configmap or secret with `global.apps.{app_name}.extraConfigs`. 
+- In-line custom values for app moved from `global.apps.{app_name}` to `global.apps.{app_name}.values`.
+
 ## [0.53.0] - 2023-12-13
 
 ### Changed

diff --git a/helm/cluster-aws/README.md b/helm/cluster-aws/README.md
@@ -30,11 +30,36 @@ Configuration of apps that are part of the cluster.
 
 | **Property** | **Description** | **More Details** |
 | :----------- | :-------------- | :--------------- |
-| `global.apps.awsCloudControllerManager` | **AWS cloud controller manager** - Configuration of aws-cloud-controller-manager-app. For all available values see https://github.com/giantswarm/aws-cloud-controller-manager-app.|**Type:** `object`<br/>|
-| `global.apps.awsEbsCsiDriver` | **AWS EBS CSI driver** - Configuration of aws-ebs-csi-driver-app. For all available values see https://github.com/giantswarm/aws-ebs-csi-driver-app.|**Type:** `object`<br/>|
-| `global.apps.cilium` | **Cilium** - Configuration of cilium-app. For all available values see https://github.com/giantswarm/cilium-app.|**Type:** `object`<br/>|
-| `global.apps.coreDns` | **CoreDNS** - Configuration of coredns-app. For all available values see https://github.com/giantswarm/coredns-app.|**Type:** `object`<br/>|
-| `global.apps.verticalPodAutoscalerCrd` | **Vertical-pod-autoscaler CRD** - Configuration of vertical-pod-autoscaler-crd-app. For all available values see https://github.com/giantswarm/vertical-pod-autoscaler-crd.|**Type:** `object`<br/>|
+| `global.apps.awsCloudControllerManager` | **App** - Configuration of an default app that is part of the cluster.|**Type:** `object`<br/>|
+| `global.apps.awsCloudControllerManager.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.awsCloudControllerManager.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.awsCloudControllerManager.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.awsCloudControllerManager.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.awsCloudControllerManager.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
+| `global.apps.awsEbsCsiDriver` | **App** - Configuration of an default app that is part of the cluster.|**Type:** `object`<br/>|
+| `global.apps.awsEbsCsiDriver.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.awsEbsCsiDriver.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.awsEbsCsiDriver.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.awsEbsCsiDriver.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.awsEbsCsiDriver.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
+| `global.apps.cilium` | **App** - Configuration of an default app that is part of the cluster.|**Type:** `object`<br/>|
+| `global.apps.cilium.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.cilium.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.cilium.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.cilium.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.cilium.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
+| `global.apps.coreDns` | **App** - Configuration of an default app that is part of the cluster.|**Type:** `object`<br/>|
+| `global.apps.coreDns.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.coreDns.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.coreDns.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.coreDns.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.coreDns.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
+| `global.apps.verticalPodAutoscalerCrd` | **App** - Configuration of an default app that is part of the cluster.|**Type:** `object`<br/>|
+| `global.apps.verticalPodAutoscalerCrd.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.verticalPodAutoscalerCrd.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.verticalPodAutoscalerCrd.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.verticalPodAutoscalerCrd.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.verticalPodAutoscalerCrd.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
 
 ### Components
 Properties within the `.global.components` object

diff --git a/helm/cluster-aws/templates/aws-ebs-csi-driver-helmrelease.yaml b/helm/cluster-aws/templates/aws-ebs-csi-driver-helmrelease.yaml
@@ -43,10 +43,18 @@ spec:
     remediation:
       retries: 30
   {{- $awsEbsCsiDriverHelmValues := (include "defaultAwsEbsCsiDriverHelmValues" .) | fromYaml -}}
-  {{- $customAwsEbsCsiDriverHelmValues := $.Values.global.apps.awsEbsCsiDriver -}}
+  {{- $customAwsEbsCsiDriverHelmValues := $.Values.global.apps.awsEbsCsiDriver.values -}}
   {{- if $customAwsEbsCsiDriverHelmValues }}
   {{- $awsEbsCsiDriverHelmValues = merge (deepCopy $customAwsEbsCsiDriverHelmValues) $awsEbsCsiDriverHelmValues -}}
   {{- end }}
   {{- if $awsEbsCsiDriverHelmValues }}
   values: {{- $awsEbsCsiDriverHelmValues | toYaml | nindent 4 }}
   {{- end }}
+  {{- if $.Values.global.apps.awsEbsCsiDriver.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.awsEbsCsiDriver.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+    {{- end }}
+  {{- end }}
diff --git a/helm/cluster-aws/templates/cilium-helmrelease.yaml b/helm/cluster-aws/templates/cilium-helmrelease.yaml
@@ -66,10 +66,18 @@ spec:
     remediation:
       retries: 30
   {{- $ciliumHelmValues := (include "defaultCiliumHelmValues" .) | fromYaml -}}
-  {{- $customCiliumHelmValues := $.Values.global.apps.cilium -}}
+  {{- $customCiliumHelmValues := $.Values.global.apps.cilium.values -}}
   {{- if $customCiliumHelmValues }}
   {{- $ciliumHelmValues = merge (deepCopy $customCiliumHelmValues) $ciliumHelmValues -}}
   {{- end }}
   {{- if $ciliumHelmValues }}
   values: {{- $ciliumHelmValues | toYaml | nindent 4 }}
   {{- end }}
+  {{- if $.Values.global.apps.cilium.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.cilium.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+    {{- end }}
+  {{- end }}
diff --git a/helm/cluster-aws/templates/cloud-provider-aws-helmrelease.yaml b/helm/cluster-aws/templates/cloud-provider-aws-helmrelease.yaml
@@ -43,10 +43,18 @@ spec:
     remediation:
       retries: 30
   {{- $awsCloudControllerManagerHelmValues := (include "defaultAwsCloudControllerManagerHelmValues" .) | fromYaml -}}
-  {{- $customAwsCloudControllerManagerHelmValues := $.Values.global.apps.awsCloudControllerManager -}}
+  {{- $customAwsCloudControllerManagerHelmValues := $.Values.global.apps.awsCloudControllerManager.values -}}
   {{- if $customAwsCloudControllerManagerHelmValues }}
   {{- $awsCloudControllerManagerHelmValues = merge (deepCopy $customAwsCloudControllerManagerHelmValues) $awsCloudControllerManagerHelmValues -}}
   {{- end }}
   {{- if $awsCloudControllerManagerHelmValues }}
   values: {{- $awsCloudControllerManagerHelmValues | toYaml | nindent 4 }}
   {{- end }}
+  {{- if $.Values.global.apps.awsCloudControllerManager.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.awsCloudControllerManager.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+    {{- end }}
+  {{- end }}
diff --git a/helm/cluster-aws/templates/coredns-helmrelease.yaml b/helm/cluster-aws/templates/coredns-helmrelease.yaml
@@ -43,10 +43,18 @@ spec:
     remediation:
       retries: 30
   {{- $coreDnsHelmValues := (include "defaultCoreDnsHelmValues" .) | fromYaml -}}
-  {{- $customCoreDnsHelmValues := $.Values.global.apps.coreDns -}}
+  {{- $customCoreDnsHelmValues := $.Values.global.apps.coreDns.values -}}
   {{- if $customCoreDnsHelmValues }}
   {{- $coreDnsHelmValues = merge (deepCopy $customCoreDnsHelmValues) $coreDnsHelmValues -}}
   {{- end }}
   {{- if $coreDnsHelmValues }}
   values: {{- $coreDnsHelmValues | toYaml | nindent 4 }}
   {{- end }}
+  {{- if $.Values.global.apps.coreDns.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.coreDns.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+    {{- end }}
+  {{- end }}
diff --git a/helm/cluster-aws/templates/vpa-crd-helmrelease.yaml b/helm/cluster-aws/templates/vpa-crd-helmrelease.yaml
@@ -27,7 +27,15 @@ spec:
   install:
     remediation:
       retries: 30
-  {{- $customVerticalPodAutoscalerCrdHelmValues := $.Values.global.apps.verticalPodAutoscalerCrd -}}
+  {{- $customVerticalPodAutoscalerCrdHelmValues := $.Values.global.apps.verticalPodAutoscalerCrd.values -}}
   {{- if $customVerticalPodAutoscalerCrdHelmValues }}
   values: {{- $customVerticalPodAutoscalerCrdHelmValues | toYaml | nindent 4 }}
   {{- end }}
+  {{- if $.Values.global.apps.verticalPodAutoscalerCrd.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.verticalPodAutoscalerCrd.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+    {{- end }}
+  {{- end }}
diff --git a/helm/cluster-aws/values.schema.json b/helm/cluster-aws/values.schema.json
@@ -1,6 +1,45 @@
 {
     "$schema": "https://json-schema.org/draft/2020-12/schema",
     "$defs": {
+        "app": {
+            "type": "object",
+            "title": "App",
+            "description": "Configuration of an default app that is part of the cluster.",
+            "additionalProperties": false,
+            "properties": {
+                "extraConfigs": {
+                    "type": "array",
+                    "title": "Extra config maps or secrets",
+                    "description": "Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.",
+                    "items": {
+                        "type": "object",
+                        "title": "Config map or secret",
+                        "properties": {
+                            "kind": {
+                                "type": "string",
+                                "title": "Kind",
+                                "description": "Specifies whether the resource is a config map or a secret.",
+                                "enum": [
+                                    "ConfigMap",
+                                    "Secret"
+                                ]
+                            },
+                            "name": {
+                                "type": "string",
+                                "title": "Name",
+                                "description": "Name of the config map or secret. The object must exist in the same namespace as the cluster App."
+                            }
+                        }
+                    }
+                },
+                "values": {
+                    "type": "object",
+                    "title": "Values",
+                    "description": "Values to be passed to the app. Values will have higher priority than values from configmaps.",
+                    "additionalProperties": true
+                }
+            }
+        },
         "awsResourceTagValue": {
             "type": "string",
             "title": "Tag value",
@@ -160,26 +199,31 @@
                     "additionalProperties": false,
                     "properties": {
                         "awsCloudControllerManager": {
+                            "$ref": "#/$defs/app",
                             "type": "object",
                             "title": "AWS cloud controller manager",
                             "description": "Configuration of aws-cloud-controller-manager-app. For all available values see https://github.com/giantswarm/aws-cloud-controller-manager-app."
                         },
                         "awsEbsCsiDriver": {
+                            "$ref": "#/$defs/app",
                             "type": "object",
                             "title": "AWS EBS CSI driver",
                             "description": "Configuration of aws-ebs-csi-driver-app. For all available values see https://github.com/giantswarm/aws-ebs-csi-driver-app."
                         },
                         "cilium": {
+                            "$ref": "#/$defs/app",
                             "type": "object",
                             "title": "Cilium",
                             "description": "Configuration of cilium-app. For all available values see https://github.com/giantswarm/cilium-app."
                         },
                         "coreDns": {
+                            "$ref": "#/$defs/app",
                             "type": "object",
                             "title": "CoreDNS",
                             "description": "Configuration of coredns-app. For all available values see https://github.com/giantswarm/coredns-app."
                         },
                         "verticalPodAutoscalerCrd": {
+                            "$ref": "#/$defs/app",
                             "type": "object",
                             "title": "Vertical-pod-autoscaler CRD",
                             "description": "Configuration of vertical-pod-autoscaler-crd-app. For all available values see https://github.com/giantswarm/vertical-pod-autoscaler-crd."

diff --git a/helm/cluster-aws/values.yaml b/helm/cluster-aws/values.yaml
@@ -1,7 +1,12 @@
 # Auto-generated by helm-values-gen, do not edit!
 
 global:
-  apps: {}
+  apps:
+    awsCloudControllerManager: {}
+    awsEbsCsiDriver: {}
+    cilium: {}
+    coreDns: {}
+    verticalPodAutoscalerCrd: {}
   components:
     containerd:
       containerRegistries: