Skip to content

Commit

Permalink
[manifests]: Deployment structure inheritance (#28)
Browse files Browse the repository at this point in the history
Fix Ingress inheritance & Environment Secrets
  • Loading branch information
oliverbaehler authored Jan 24, 2021
1 parent 3468140 commit 748ac51
Show file tree
Hide file tree
Showing 12 changed files with 69 additions and 112 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ By adding `WIP: *` as prefix for your pull request title, your pull request is c

## Review

When creating a Pull Request is automatically assigned. If your Pull Request does not have any activity after certain days, feel free to comment a reminder (it might happen that we forget about it, since we maintain this repository part time). Your Pull Request requires approve to mergedable.
When creating a Pull Request is automatically assigned. If your Pull Request does not have any activity after certain days, feel free to comment a reminder (it might happen that we forget about it, since we maintain this repository part time). Your Pull Request requires approve to be mergeable.

# Chart Requirements

Expand Down Expand Up @@ -194,7 +194,7 @@ On each Push [Helm-Docs](#documentation) will executed (fails on protected branc
On creating a Pull Request the following workflows will be executed:

1. Chart Linting - All Charts are linted using the [ct tool](https://github.com/helm/chart-testing).
2. Chart Installation - All Charts are installed to KinD isntance using the [ct tool](https://github.com/helm/chart-testing).
2. Chart Installation - All Charts are installed to KinD instance using the [ct tool](https://github.com/helm/chart-testing).
3. Chart Release Dry-Run - Only charts which had changes to their **Chart.yaml** file are considered for the Release Dry-Run. No Release will be made during Dry-Run. The following checks must pass:
* Passed [Kube-Linter](https://github.com/stackrox/kube-linter) Tests (Required).
* Passed [Helm Unit-Tests](https://github.com/quintush/helm-unittest) if any are defined (Optional).
Expand Down
4 changes: 2 additions & 2 deletions charts/manifests/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: manifests
description: "Bedag's Manifest chart. Library full of basic kubernetes manifests."
type: library
version: 0.4.7
version: 0.4.8
icon: "https://www.bedag.ch/wGlobal/wGlobal/layout/images/logo.svg"
keywords:
- Bedag
Expand All @@ -22,4 +22,4 @@ annotations:
artifacthub.io/prerelease: "true"
artifacthub.io/license: Apache-2.0
artifacthub.io/changes: |
- [Fixes] Ingress port name/number validation
- Fixes Deployment inheritance
4 changes: 2 additions & 2 deletions charts/manifests/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Manifests Library

![Version: 0.4.7](https://img.shields.io/badge/Version-0.4.7-informational?style=flat-square) ![Type: library](https://img.shields.io/badge/Type-library-informational?style=flat-square)
![Version: 0.4.8](https://img.shields.io/badge/Version-0.4.8-informational?style=flat-square) ![Type: library](https://img.shields.io/badge/Type-library-informational?style=flat-square)

This library's purpose is to have more flexibility as chart author but at the same time have kubernetes manifests managed in a central library. This way you can avoid big surprises when Kubernetes has breaking changes in any of their APIs. Currently we support a base set of resources. Resources may be added as soon as we see or get a request that there's a need for it. This chart is still under development and testing, since it's rather complex. Feel free to use it. Our goal is to get it as reliable as possible.

Expand Down Expand Up @@ -52,7 +52,7 @@ Major Changes to functions are documented with the version affected. **Before up

For Artifacthub Users: The entire documentation can be found in the GitHub Repository.

We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-hopw of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :)
We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-how of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :)

We recommend looking thirst through the **Kubernetes Manifests** section.

Expand Down
2 changes: 1 addition & 1 deletion charts/manifests/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Major Changes to functions are documented with the version affected. **Before up

For Artifacthub Users: The entire documentation can be found in the GitHub Repository.

We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-hopw of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :)
We tried our best with the documentation. Since it's a very different approach on writing helm charts it's hard to explain. Therefor we suggest taking a look at other charts in this chart repository, since they (might) be written with the help of this library. We expect users of this library to have a deep know-how of Helm und Go Sprig. If that's not the case yet, we recommend coming back later, because the usage might frustrate you more than you actually benefit from it. We are planing to expand the documentation in the future. Your contribution is welcome, if you are a fan of the project! :)

We recommend looking thirst through the **Kubernetes Manifests** section.

Expand Down
8 changes: 4 additions & 4 deletions charts/manifests/templates/Quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ It's difficult to quickstart with this project unless you already have a deeper
...
```
We like to have our dependencies fixed over all bugfix versions of a minor release (which is implied by `~`). Use your prefered dependency strategie.
We like to have our dependencies fixed over all bugfix versions of a minor release (which is implied by `~`). Use your preferred dependency strategie.
5. **Initialize Bundle**</br>
Now it's time to get started with actually using the Manifests library. Let's create a new file `new-charts/templates/bundle.yaml` and add the basic Bundle structure ([Read More on Bundles](./manifests/README.md#bundles)). Let's also add our first bundle resource of type `statefulset`
Expand Down Expand Up @@ -98,7 +98,7 @@ It's difficult to quickstart with this project unless you already have a deeper
There's your service! :)
8. **Before you go**</br>
1. Don't forget that you have all the power of go sprif in your hands within your bundles file. Use it whisely:
1. Don't forget that you have all the power of go sprig in your hands within your bundles file. Use it wisely:
* [http://masterminds.github.io/sprig/](http://masterminds.github.io/sprig/)
Expand All @@ -125,7 +125,7 @@ It's difficult to quickstart with this project unless you already have a deeper
{{- end }}
```
Eventough we are overwriting the port, we are still using the ports the user might input.
Even though we are overwriting the port, we are still using the ports the user might input.
3. ExtraResources are always welcome :)
Expand All @@ -144,7 +144,7 @@ It's difficult to quickstart with this project unless you already have a deeper
{{- end -}}
```
4. Sometimes you might encounter errors. Most of the time the source of errors will be a mallformed bundle YAML. To check how your YAML looks, you can do something like this and then `helm template` it:
4. Sometimes you might encounter errors. Most of the time the source of errors will be a malformed bundle YAML. To check how your YAML looks, you can do something like this and then `helm template` it:
```
test: | {{- include "bedag-lib.manifest.bundle" (dict "bundle" (fromYaml (include "new-chart.bundle" $)) "context" $) | nindent 2 }}
Expand Down
10 changes: 5 additions & 5 deletions charts/manifests/templates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ A preset is an implementation of a manifest/template for a very specific use cas
{{/*
Preset Say - Overwrite Values
These values will overwrite all the other values given as input. Effectivily implementing
These values will overwrite all the other values given as input. Effectively implementing
the logic for the use case.The given values (defaults merged with user input) are available
under the .values key.
Expand Down Expand Up @@ -125,7 +125,7 @@ First i am creating the main file (do whatever structure you like, just make sur
{{/*
We need a dedicated resource for the values for this manifests. This is mainly to ensure
to functionality of the manifest within a bundle. If you want to implement it the same way as
shown is up to. The function should just return the default values for the manifest, overwritien
shown is up to. The function should just return the default values for the manifest, overwritten
by the .values field and superoverwritten by the .overwrites field.
Make Sure:
* Has **bedag-lib.manifest** as prefix
Expand Down Expand Up @@ -209,7 +209,7 @@ We recommend adding default values for each manifest, these can be used as refer
apiVersion: ""
## ServiceMonitor Namespace
## {{ $_.path }}namespace -- Define the namespace to deploy the srviceMonitor in
## {{ $_.path }}namespace -- Define the namespace to deploy the serviceMonitor in
namespace: ""
## ServiceMonitor NamespaceSelector
Expand All @@ -218,8 +218,8 @@ We recommend adding default values for each manifest, these can be used as refer
namespaceSelector: []
## ServiceMonitor Additional Fields
# {{ $_.path }}additonalFields -- Define additional fields, which aren't available as seperat key (e.g. `sampleLimit`)
additonalFields: {}
# {{ $_.path }}additionalFields -- Define additional fields, which aren't available as separat key (e.g. `sampleLimit`)
additionalFields: {}
## ServiceMonitor Endpoint Configuration
# {{ $_.path }}endpoints -- Configure Prometheus ServiceMonitor [Endpoints](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#endpoint)
Expand Down
38 changes: 31 additions & 7 deletions charts/manifests/templates/manifests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Explanation of the above keys:

## Usage

We recommend creating a dedicated template within your chart just for the bundle strucure. This structure should then be used to call the bundle template, like so:
We recommend creating a dedicated template within your chart just for the bundle structure. This structure should then be used to call the bundle template, like so:

```
{{/*
Expand Down Expand Up @@ -124,19 +124,43 @@ Allows to directly create a serviceAccount without having to declare it as dedic

#### Affects

Affects the following resources:
All resources

`Pod`, `Deployment` and `Statefulset`
#### Structure

See ServiceAccount Values.

### Environment Secrets

Environment secrets allows to directly add secrets from the environment declaration. Secret Key's values will be encoded into a secret and mounted to the environment. This function is implemented by the `bedag-lib.environment.keyList` template [See here]().
Environment secrets allows to directly add secrets from the environment declaration. Secret Key's values will be encoded into a secret and mounted to the environment.

#### Affects

Affects the following resources:
All resources

#### Structure

This template supports the following key structure:

```
environment:
# Environment Variables
- name: "MY_ENV"
value: "someValue"
- name: "MY_SPEC"
valueFrom:
fieldRef:
fieldPath: spec.*
# Secret Environment Variables
- name: "MY_SECRET"
value: "S3CRET"
secret: true
```

`Pod`, `Deployment` and `Statefulset`
**Note:** Secret Environment variables are only supported/rendered if the resource is part of a bundle. If the resource is not part of a bundle the secrets are removed, to prevent exposing them.

# Manifest Templates

Expand Down Expand Up @@ -884,7 +908,7 @@ resources:

#### Multiple Bundles

Using multiple bundles is very straightforward, just do the same thing you did with a single bundle but multiple times - that's it. In this example we seperate two microservices within the chart (Frontend/Backend).
Using multiple bundles is very straightforward, just do the same thing you did with a single bundle but multiple times - that's it. In this example we separate two microservices within the chart (Frontend/Backend).

**templates/frontend.yaml**
```
Expand Down
2 changes: 1 addition & 1 deletion charts/manifests/templates/manifests/_deployment.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,6 @@ spec:
{{- if $deployment.deploymentExtras }}
{{- toYaml $deployment.deploymentExtras | nindent 2 }}
{{- end }}
template: {{- include "bedag-lib.template.pod" (dict "pod" $deployment "context" $context) | nindent 4 }}
template: {{- include "bedag-lib.template.pod" (set . "pod" $deployment) | nindent 4 }}
{{- end }}
{{- end -}}
23 changes: 18 additions & 5 deletions charts/manifests/templates/manifests/templates/_containerTpl.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ limitations under the License.
*/}}
{{- define "bedag-lib.template.container" -}}
{{- $values := mergeOverwrite (fromYaml (include "bedag-lib.values.template.container" .)) .container -}}
{{- $values := (mergeOverwrite (fromYaml (include "bedag-lib.values.template.container" .)) .container) -}}
{{- if and $values .context -}}
{{- $context := .context -}}
name: {{ default $context.Chart.Name $values.containerName }}
Expand All @@ -31,11 +31,24 @@ resources: {{- include "lib.utils.strings.template" (dict "value" . "context" $c
{{- if $values.containerFields }}
{{- include "lib.utils.strings.template" (dict "value" $values.containerFields "context" $context) | nindent 0 }}
{{- end }}
env: {{- include "lib.utils.extras.environment" $context | nindent 2 }}
{{- if and $values.environment (kindIs "slice" $values.environment) }}
{{- if $context.Bundle }}
env: {{- include "bedag-lib.utils.environment.keyList" (dict "environment" $values.environment "allowSecrets" true "context" $context) | nindent 2 }}
{{- else }}
env: {{- include "bedag-lib.utils.environment.keyList" (dict "environment" $values.environment "context" $context) | nindent 2 }}
{{- $filteredList := list -}}
{{- range $values.environment }}
{{- if .secret }}
{{- if $context.Bundle }}
- name: {{ required "Field .name is required for environment item!" .name | quote }}
valueFrom:
secretKeyRef:
name: {{ include "bedag-lib.utils.common.fullname" $ }}-env
key: {{ .name | quote }}
{{- end }}
{{- else }}
{{- $filteredList = append $filteredList . -}}
{{- end }}
{{- end }}
{{- if $filteredList }}
{{- include "lib.utils.strings.template" (dict "value" $filteredList "context" $context) | nindent 2 }}
{{- end }}
{{- end }}
{{- if $values.command }}
Expand Down
2 changes: 1 addition & 1 deletion charts/manifests/templates/presets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ extraResources:
- type: "servicemonitor"
name: jmx
values:
additonalFields: {}
additionalFields: {}
apiVersion: ""
enabled: true
endpoints:
Expand Down
52 changes: 2 additions & 50 deletions charts/manifests/templates/utils/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ Description and Definition of all available Go Sprig Templates. Base functionali
* **[Helpers](#helpers)**
* [javaProxies](#javaproxies)
* **[Environment](#environment)**
* [keyList](#keylist)
* [hasSecrets](#hassecrets)
* **[Presets](#presets)**
* **[Values](#values)**
Expand Down Expand Up @@ -152,53 +151,6 @@ String

## [Environment](./_environment.tpl)

### KeyList
---

This template adds to functionality, to add secret values to environment variables. When creating a secret environment variable, the value is written into a secret and the secret is referenced in the environment variable key. In addition it contains some useful default environment variables.

#### Arguments

If an as required marked argument is missing, the template engine will intentionally.

* `.environment` - Supported environment key structure (Optional). If not set default environment variables are returned.
* `.context` - Inherited Root Context (Required).
* `.allowSecrets` - If set, allows the rendering of secret environment variables.

#### Structure

This template supports the following key structure:

```
environment:
# Environment Variables
- name: "MY_ENV"
value: "someValue"
- name: "MY_SPEC"
valueFrom:
fieldRef:
fieldPath: spec.*
# Secret Environment Variables
- name: "MY_SECRET"
value: "S3CRET"
secret: true
```

**Note:** Secret Environment variables are only supported/rendered if the resource is part of a bundle. If the resource is not part of a bundle the secrets are removed, to prevent exposing them.

#### Returns

String, YAML Structure

#### Usage

```
{{- include "bedag-lib.utils.environment.keyList" (dict "environment" $.Values.environment "context" $) }}
```

### HasSecrets
---

Expand Down Expand Up @@ -269,7 +221,7 @@ This template supports the following key structure:
## Enabled Values Generator
doc:
## Define which manifest to generate (e.g. "statfulset")
## Define which manifest to generate (e.g. "statefulset")
## On of .manifest or .preset has to be set, otherwise the function returns empty
manifest: "statefulset"
Expand Down Expand Up @@ -312,7 +264,7 @@ To use this template we need to be able to execute `helm template`. Since this i
Now we can start using the generator:

```
## Generate Values for the Statefulset manifest with "frontend" as topkey instead of statefulset.
## Generate Values for the Statefulset manifest with "frontend" as top key instead of statefulset.
helm template gen . --set doc.manifest="statefulset" --set doc.key="frontend"
Expand Down
32 changes: 0 additions & 32 deletions charts/manifests/templates/utils/_environment.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -15,38 +15,6 @@ See the License for the specific language governing permissions and
limitations under the License.

*/
{{/*
Sprig Template - KeyList
*/}}
{{- define "bedag-lib.utils.environment.keyList" -}}
{{- if .context }}
{{- $context := .context }}
{{- include "lib.utils.extras.environment" $context | indent 0 }}
{{- if .environment }}
{{- $filteredList := list -}}
{{- range .environment }}
{{- if .secret }}
{{- if $.allowSecrets }}
- name: {{ required "Field .name is required for environment item!" .name | quote }}
valueFrom:
secretKeyRef:
name: {{ include "bedag-lib.utils.common.fullname" $context }}-env
key: {{ .name | quote }}
{{- end }}
{{- else }}
{{- $filteredList = append $filteredList . -}}
{{- end }}
{{- end }}
{{- if $filteredList }}
{{- include "lib.utils.strings.template" (dict "value" $filteredList "context" $context) | nindent 0 }}
{{- end }}
{{- end }}
{{- else }}
{{- fail "Template requires '.context' as argument" }}
{{- end }}
{{- end -}}


{{/*
Sprig Template - HasSecrets
*/}}
Expand Down

0 comments on commit 748ac51

Please sign in to comment.