From 748ac516a8b09852d6f43e62b1636f12f53dd0f1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Oliver=20B=C3=A4hler?= Date: Sun, 24 Jan 2021 17:09:37 +0100 Subject: [PATCH] [manifests]: Deployment structure inheritance (#28) Fix Ingress inheritance & Environment Secrets --- CONTRIBUTING.md | 4 +- charts/manifests/Chart.yaml | 4 +- charts/manifests/README.md | 4 +- charts/manifests/README.md.gotmpl | 2 +- charts/manifests/templates/Quickstart.md | 8 +-- charts/manifests/templates/README.md | 10 ++-- .../manifests/templates/manifests/README.md | 38 +++++++++++--- .../templates/manifests/_deployment.tpl | 2 +- .../manifests/templates/_containerTpl.tpl | 23 ++++++-- charts/manifests/templates/presets/README.md | 2 +- charts/manifests/templates/utils/README.md | 52 +------------------ .../templates/utils/_environment.tpl | 32 ------------ 12 files changed, 69 insertions(+), 112 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 87012fe8..afb8f3bc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -72,7 +72,7 @@ By adding `WIP: *` as prefix for your pull request title, your pull request is c ## Review -When creating a Pull Request is automatically assigned. If your Pull Request does not have any activity after certain days, feel free to comment a reminder (it might happen that we forget about it, since we maintain this repository part time). Your Pull Request requires approve to mergedable. +When creating a Pull Request is automatically assigned. If your Pull Request does not have any activity after certain days, feel free to comment a reminder (it might happen that we forget about it, since we maintain this repository part time). Your Pull Request requires approve to be mergeable. # Chart Requirements @@ -194,7 +194,7 @@ On each Push [Helm-Docs](#documentation) will executed (fails on protected branc On creating a Pull Request the following workflows will be executed: 1. Chart Linting - All Charts are linted using the [ct tool](https://github.com/helm/chart-testing). - 2. Chart Installation - All Charts are installed to KinD isntance using the [ct tool](https://github.com/helm/chart-testing). + 2. Chart Installation - All Charts are installed to KinD instance using the [ct tool](https://github.com/helm/chart-testing). 3. Chart Release Dry-Run - Only charts which had changes to their **Chart.yaml** file are considered for the Release Dry-Run. No Release will be made during Dry-Run. The following checks must pass: * Passed [Kube-Linter](https://github.com/stackrox/kube-linter) Tests (Required). * Passed [Helm Unit-Tests](https://github.com/quintush/helm-unittest) if any are defined (Optional). diff --git a/charts/manifests/Chart.yaml b/charts/manifests/Chart.yaml index f8597e82..caa8f6c9 100644 --- a/charts/manifests/Chart.yaml +++ b/charts/manifests/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: manifests description: "Bedag's Manifest chart. Library full of basic kubernetes manifests." type: library -version: 0.4.7 +version: 0.4.8 icon: "https://www.bedag.ch/wGlobal/wGlobal/layout/images/logo.svg" keywords: - Bedag @@ -22,4 +22,4 @@ annotations: artifacthub.io/prerelease: "true" artifacthub.io/license: Apache-2.0 artifacthub.io/changes: | - - [Fixes] Ingress port name/number validation + - Fixes Deployment inheritance diff --git a/charts/manifests/README.md b/charts/manifests/README.md index b5d6e159..c13893ae 100644 --- a/charts/manifests/README.md +++ b/charts/manifests/README.md @@ -1,6 +1,6 @@ # Manifests Library -![Version: 0.4.7](https://img.shields.io/badge/Version-0.4.7-informational?style=flat-square) ![Type: library](https://img.shields.io/badge/Type-library-informational?style=flat-square) +![Version: 0.4.8](https://img.shields.io/badge/Version-0.4.8-informational?style=flat-square) ![Type: library](https://img.shields.io/badge/Type-library-informational?style=flat-square) This library's purpose is to have more flexibility as chart author but at the same time have kubernetes manifests managed in a central library. This way you can avoid big surprises when Kubernetes has breaking changes in any of their APIs. Currently we support a base set of resources. Resources may be added as soon as we see or get a request that there's a need for it. This chart is still under development and testing, since it's rather complex. Feel free to use it. Our goal is to get it as reliable as possible. @@ -52,7 +52,7 @@ Major Changes to functions are documented with the version affected. **Before up For Artifacthub Users: The entire documentation can be found in the GitHub Repository. -We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-hopw of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :) +We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-how of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :) We recommend looking thirst through the **Kubernetes Manifests** section. diff --git a/charts/manifests/README.md.gotmpl b/charts/manifests/README.md.gotmpl index fdd85875..aa9bcc45 100644 --- a/charts/manifests/README.md.gotmpl +++ b/charts/manifests/README.md.gotmpl @@ -56,7 +56,7 @@ Major Changes to functions are documented with the version affected. **Before up For Artifacthub Users: The entire documentation can be found in the GitHub Repository. -We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-hopw of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :) +We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-how of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :) We recommend looking thirst through the **Kubernetes Manifests** section. diff --git a/charts/manifests/templates/Quickstart.md b/charts/manifests/templates/Quickstart.md index 6fd8ed81..7c9d581d 100644 --- a/charts/manifests/templates/Quickstart.md +++ b/charts/manifests/templates/Quickstart.md @@ -35,7 +35,7 @@ It's difficult to quickstart with this project unless you already have a deeper ... ``` - We like to have our dependencies fixed over all bugfix versions of a minor release (which is implied by `~`). Use your prefered dependency strategie. + We like to have our dependencies fixed over all bugfix versions of a minor release (which is implied by `~`). Use your preferred dependency strategie. 5. **Initialize Bundle**
Now it's time to get started with actually using the Manifests library. Let's create a new file `new-charts/templates/bundle.yaml` and add the basic Bundle structure ([Read More on Bundles](./manifests/README.md#bundles)). Let's also add our first bundle resource of type `statefulset` @@ -98,7 +98,7 @@ It's difficult to quickstart with this project unless you already have a deeper There's your service! :) 8. **Before you go**
- 1. Don't forget that you have all the power of go sprif in your hands within your bundles file. Use it whisely: + 1. Don't forget that you have all the power of go sprig in your hands within your bundles file. Use it wisely: * [http://masterminds.github.io/sprig/](http://masterminds.github.io/sprig/) @@ -125,7 +125,7 @@ It's difficult to quickstart with this project unless you already have a deeper {{- end }} ``` - Eventough we are overwriting the port, we are still using the ports the user might input. + Even though we are overwriting the port, we are still using the ports the user might input. 3. ExtraResources are always welcome :) @@ -144,7 +144,7 @@ It's difficult to quickstart with this project unless you already have a deeper {{- end -}} ``` - 4. Sometimes you might encounter errors. Most of the time the source of errors will be a mallformed bundle YAML. To check how your YAML looks, you can do something like this and then `helm template` it: + 4. Sometimes you might encounter errors. Most of the time the source of errors will be a malformed bundle YAML. To check how your YAML looks, you can do something like this and then `helm template` it: ``` test: | {{- include "bedag-lib.manifest.bundle" (dict "bundle" (fromYaml (include "new-chart.bundle" $)) "context" $) | nindent 2 }} diff --git a/charts/manifests/templates/README.md b/charts/manifests/templates/README.md index 1a9076ec..88d73b4c 100644 --- a/charts/manifests/templates/README.md +++ b/charts/manifests/templates/README.md @@ -35,7 +35,7 @@ A preset is an implementation of a manifest/template for a very specific use cas {{/* Preset Say - Overwrite Values - These values will overwrite all the other values given as input. Effectivily implementing + These values will overwrite all the other values given as input. Effectively implementing the logic for the use case.The given values (defaults merged with user input) are available under the .values key. @@ -125,7 +125,7 @@ First i am creating the main file (do whatever structure you like, just make sur {{/* We need a dedicated resource for the values for this manifests. This is mainly to ensure to functionality of the manifest within a bundle. If you want to implement it the same way as - shown is up to. The function should just return the default values for the manifest, overwritien + shown is up to. The function should just return the default values for the manifest, overwritten by the .values field and superoverwritten by the .overwrites field. Make Sure: * Has **bedag-lib.manifest** as prefix @@ -209,7 +209,7 @@ We recommend adding default values for each manifest, these can be used as refer apiVersion: "" ## ServiceMonitor Namespace - ## {{ $_.path }}namespace -- Define the namespace to deploy the srviceMonitor in + ## {{ $_.path }}namespace -- Define the namespace to deploy the serviceMonitor in namespace: "" ## ServiceMonitor NamespaceSelector @@ -218,8 +218,8 @@ We recommend adding default values for each manifest, these can be used as refer namespaceSelector: [] ## ServiceMonitor Additional Fields - # {{ $_.path }}additonalFields -- Define additional fields, which aren't available as seperat key (e.g. `sampleLimit`) - additonalFields: {} + # {{ $_.path }}additionalFields -- Define additional fields, which aren't available as separat key (e.g. `sampleLimit`) + additionalFields: {} ## ServiceMonitor Endpoint Configuration # {{ $_.path }}endpoints -- Configure Prometheus ServiceMonitor [Endpoints](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#endpoint) diff --git a/charts/manifests/templates/manifests/README.md b/charts/manifests/templates/manifests/README.md index a1d1da2d..82c75a83 100644 --- a/charts/manifests/templates/manifests/README.md +++ b/charts/manifests/templates/manifests/README.md @@ -39,7 +39,7 @@ Explanation of the above keys: ## Usage -We recommend creating a dedicated template within your chart just for the bundle strucure. This structure should then be used to call the bundle template, like so: +We recommend creating a dedicated template within your chart just for the bundle structure. This structure should then be used to call the bundle template, like so: ``` {{/* @@ -124,19 +124,43 @@ Allows to directly create a serviceAccount without having to declare it as dedic #### Affects -Affects the following resources: +All resources -`Pod`, `Deployment` and `Statefulset` +#### Structure + +See ServiceAccount Values. ### Environment Secrets -Environment secrets allows to directly add secrets from the environment declaration. Secret Key's values will be encoded into a secret and mounted to the environment. This function is implemented by the `bedag-lib.environment.keyList` template [See here](). +Environment secrets allows to directly add secrets from the environment declaration. Secret Key's values will be encoded into a secret and mounted to the environment. #### Affects -Affects the following resources: +All resources + +#### Structure + +This template supports the following key structure: + +``` +environment: + +# Environment Variables +- name: "MY_ENV" + value: "someValue" +- name: "MY_SPEC" + valueFrom: + fieldRef: + fieldPath: spec.* + +# Secret Environment Variables +- name: "MY_SECRET" + value: "S3CRET" + secret: true + +``` -`Pod`, `Deployment` and `Statefulset` +**Note:** Secret Environment variables are only supported/rendered if the resource is part of a bundle. If the resource is not part of a bundle the secrets are removed, to prevent exposing them. # Manifest Templates @@ -884,7 +908,7 @@ resources: #### Multiple Bundles -Using multiple bundles is very straightforward, just do the same thing you did with a single bundle but multiple times - that's it. In this example we seperate two microservices within the chart (Frontend/Backend). +Using multiple bundles is very straightforward, just do the same thing you did with a single bundle but multiple times - that's it. In this example we separate two microservices within the chart (Frontend/Backend). **templates/frontend.yaml** ``` diff --git a/charts/manifests/templates/manifests/_deployment.tpl b/charts/manifests/templates/manifests/_deployment.tpl index d2e16a90..616fbecd 100644 --- a/charts/manifests/templates/manifests/_deployment.tpl +++ b/charts/manifests/templates/manifests/_deployment.tpl @@ -42,6 +42,6 @@ spec: {{- if $deployment.deploymentExtras }} {{- toYaml $deployment.deploymentExtras | nindent 2 }} {{- end }} - template: {{- include "bedag-lib.template.pod" (dict "pod" $deployment "context" $context) | nindent 4 }} + template: {{- include "bedag-lib.template.pod" (set . "pod" $deployment) | nindent 4 }} {{- end }} {{- end -}} diff --git a/charts/manifests/templates/manifests/templates/_containerTpl.tpl b/charts/manifests/templates/manifests/templates/_containerTpl.tpl index 09bae7ee..6488b734 100644 --- a/charts/manifests/templates/manifests/templates/_containerTpl.tpl +++ b/charts/manifests/templates/manifests/templates/_containerTpl.tpl @@ -16,7 +16,7 @@ limitations under the License. */}} {{- define "bedag-lib.template.container" -}} - {{- $values := mergeOverwrite (fromYaml (include "bedag-lib.values.template.container" .)) .container -}} + {{- $values := (mergeOverwrite (fromYaml (include "bedag-lib.values.template.container" .)) .container) -}} {{- if and $values .context -}} {{- $context := .context -}} name: {{ default $context.Chart.Name $values.containerName }} @@ -31,11 +31,24 @@ resources: {{- include "lib.utils.strings.template" (dict "value" . "context" $c {{- if $values.containerFields }} {{- include "lib.utils.strings.template" (dict "value" $values.containerFields "context" $context) | nindent 0 }} {{- end }} +env: {{- include "lib.utils.extras.environment" $context | nindent 2 }} {{- if and $values.environment (kindIs "slice" $values.environment) }} - {{- if $context.Bundle }} -env: {{- include "bedag-lib.utils.environment.keyList" (dict "environment" $values.environment "allowSecrets" true "context" $context) | nindent 2 }} - {{- else }} -env: {{- include "bedag-lib.utils.environment.keyList" (dict "environment" $values.environment "context" $context) | nindent 2 }} + {{- $filteredList := list -}} + {{- range $values.environment }} + {{- if .secret }} + {{- if $context.Bundle }} + - name: {{ required "Field .name is required for environment item!" .name | quote }} + valueFrom: + secretKeyRef: + name: {{ include "bedag-lib.utils.common.fullname" $ }}-env + key: {{ .name | quote }} + {{- end }} + {{- else }} + {{- $filteredList = append $filteredList . -}} + {{- end }} + {{- end }} + {{- if $filteredList }} + {{- include "lib.utils.strings.template" (dict "value" $filteredList "context" $context) | nindent 2 }} {{- end }} {{- end }} {{- if $values.command }} diff --git a/charts/manifests/templates/presets/README.md b/charts/manifests/templates/presets/README.md index a9412740..f3db4df3 100644 --- a/charts/manifests/templates/presets/README.md +++ b/charts/manifests/templates/presets/README.md @@ -106,7 +106,7 @@ extraResources: - type: "servicemonitor" name: jmx values: - additonalFields: {} + additionalFields: {} apiVersion: "" enabled: true endpoints: diff --git a/charts/manifests/templates/utils/README.md b/charts/manifests/templates/utils/README.md index 1d930e14..64ca0520 100644 --- a/charts/manifests/templates/utils/README.md +++ b/charts/manifests/templates/utils/README.md @@ -13,7 +13,6 @@ Description and Definition of all available Go Sprig Templates. Base functionali * **[Helpers](#helpers)** * [javaProxies](#javaproxies) * **[Environment](#environment)** - * [keyList](#keylist) * [hasSecrets](#hassecrets) * **[Presets](#presets)** * **[Values](#values)** @@ -152,53 +151,6 @@ String ## [Environment](./_environment.tpl) -### KeyList ---- - -This template adds to functionality, to add secret values to environment variables. When creating a secret environment variable, the value is written into a secret and the secret is referenced in the environment variable key. In addition it contains some useful default environment variables. - -#### Arguments - -If an as required marked argument is missing, the template engine will intentionally. - - * `.environment` - Supported environment key structure (Optional). If not set default environment variables are returned. - * `.context` - Inherited Root Context (Required). - * `.allowSecrets` - If set, allows the rendering of secret environment variables. - -#### Structure - -This template supports the following key structure: - -``` -environment: - -# Environment Variables -- name: "MY_ENV" - value: "someValue" -- name: "MY_SPEC" - valueFrom: - fieldRef: - fieldPath: spec.* - -# Secret Environment Variables -- name: "MY_SECRET" - value: "S3CRET" - secret: true - -``` - -**Note:** Secret Environment variables are only supported/rendered if the resource is part of a bundle. If the resource is not part of a bundle the secrets are removed, to prevent exposing them. - -#### Returns - -String, YAML Structure - -#### Usage - -``` -{{- include "bedag-lib.utils.environment.keyList" (dict "environment" $.Values.environment "context" $) }} -``` - ### HasSecrets --- @@ -269,7 +221,7 @@ This template supports the following key structure: ## Enabled Values Generator doc: - ## Define which manifest to generate (e.g. "statfulset") + ## Define which manifest to generate (e.g. "statefulset") ## On of .manifest or .preset has to be set, otherwise the function returns empty manifest: "statefulset" @@ -312,7 +264,7 @@ To use this template we need to be able to execute `helm template`. Since this i Now we can start using the generator: ``` -## Generate Values for the Statefulset manifest with "frontend" as topkey instead of statefulset. +## Generate Values for the Statefulset manifest with "frontend" as top key instead of statefulset. helm template gen . --set doc.manifest="statefulset" --set doc.key="frontend" diff --git a/charts/manifests/templates/utils/_environment.tpl b/charts/manifests/templates/utils/_environment.tpl index e0ceff39..0d23830a 100644 --- a/charts/manifests/templates/utils/_environment.tpl +++ b/charts/manifests/templates/utils/_environment.tpl @@ -15,38 +15,6 @@ See the License for the specific language governing permissions and limitations under the License. */ -{{/* - Sprig Template - KeyList -*/}} -{{- define "bedag-lib.utils.environment.keyList" -}} - {{- if .context }} - {{- $context := .context }} - {{- include "lib.utils.extras.environment" $context | indent 0 }} - {{- if .environment }} - {{- $filteredList := list -}} - {{- range .environment }} - {{- if .secret }} - {{- if $.allowSecrets }} -- name: {{ required "Field .name is required for environment item!" .name | quote }} - valueFrom: - secretKeyRef: - name: {{ include "bedag-lib.utils.common.fullname" $context }}-env - key: {{ .name | quote }} - {{- end }} - {{- else }} - {{- $filteredList = append $filteredList . -}} - {{- end }} - {{- end }} - {{- if $filteredList }} - {{- include "lib.utils.strings.template" (dict "value" $filteredList "context" $context) | nindent 0 }} - {{- end }} - {{- end }} - {{- else }} - {{- fail "Template requires '.context' as argument" }} - {{- end }} -{{- end -}} - - {{/* Sprig Template - HasSecrets */}}