forked from rancher/fleet
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial proposal of the new Helmops controller.
It adds a new custom resource `HelmApp` (resource name open to debate) that describes a helm chart to be deployed. The resource contains all the fealds from the classic `fleet.yaml` file plus a few new from the `GitRepo` resource. `HelmApp` yaml example: ```yaml apiVersion: fleet.cattle.io/v1alpha1 kind: HelmApp metadata: name: sample1 namespace: fleet-local spec: helm: releaseName: testhelm repo: https://charts.bitnami.com/bitnami chart: postgresql version: 16.2.1 insecureSkipTLSVerify: true ``` The implementation tries to share as much as possible from a `Bundle` spec inside the new resource, because it helps to "transform" the `HelmApp` into a deployment (no coversion is needed for most of the spec). The new controller was also implemented splitting the functionality into 2 controllers (similar to what we did for the `GitRepo` controller). This allows us to reuse most of the status handling code, as display fields in the status of the new resource are as similar as possible to have consistent user experience and to integrate with the UI in the same way the `GitRepo` does. When a new `HelmApp` resource is applied it is transformed into a single `Bundle`, adding some extra fields to let the `Bundle` reconciler know that this is not a regular `Bundle` coming from a `GitRepo`. Similar as we did for OCI storage, the `Bundle` created from a `HelmApp` does not contain resources. The helm chart to be deployed is downloaded by the agent. Code for downloading the helm chart is reused from gitops, so the same formats are supported. Insecure TLS skipping was added the the ChartURL and LoadDirectory functions in order to support this for gitops and helmops. If we need a secret to access the helm repository we can use the `helmSecretName` field. This secret will be cloned to secrets under the `BundleDeployment` namespace (same as we did for the OCI storage secret handling). The PR includes unit, integration (most of code is tested this way) and just one single e2e test so far just to test the whole feature together in a real cluster. Note: This is an experimental feature. In order to activate the `HelmApp` reconciling and `Bundle` deployment you need to the the environment variable: `EXPERIMENTAL_HELM_OPS=true` Refers to: rancher#2962
- Loading branch information
Showing
63 changed files
with
6,896 additions
and
341 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
# Fleet CRD Helm Chart | ||
|
||
Fleet CustomResourceDefinitions Helm chart is a requirement for the Fleet Helm Chart. | ||
Fleet CustomResourceDefinitions Helm chart is a requirement for the Helm App Chart. | ||
|
||
The Fleet documentation is centralized in the [doc website](https://fleet.rancher.io/). |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
{{- $shards := list (dict "id" "" "nodeSelector" dict) -}} | ||
{{- $uniqueShards := list -}} | ||
{{- if .Values.shards -}} | ||
{{- range .Values.shards -}} | ||
{{- if not (has .id $uniqueShards) -}} | ||
{{- $shards = append $shards . -}} | ||
{{- $uniqueShards = append $uniqueShards .id -}} | ||
{{- end -}} | ||
{{- end -}} | ||
{{- end -}} | ||
|
||
{{ range $shard := $shards }} | ||
{{- if $.Values.gitops.enabled }} | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: "helmops{{if $shard.id }}-shard-{{ $shard.id }}{{end}}" | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: "helmops" | ||
template: | ||
metadata: | ||
labels: | ||
app: "helmops" | ||
fleet.cattle.io/shard-id: "{{ $shard.id }}" | ||
{{- if empty $shard.id }} | ||
fleet.cattle.io/shard-default: "true" | ||
{{- end }} | ||
spec: | ||
serviceAccountName: helmops | ||
containers: | ||
- image: "{{ template "system_default_registry" $ }}{{ $.Values.image.repository }}:{{ $.Values.image.tag }}" | ||
name: helmops | ||
{{- if $.Values.metrics.enabled }} | ||
ports: | ||
- containerPort: 8081 | ||
name: metrics | ||
{{- end }} | ||
args: | ||
- fleetcontroller | ||
- helmops | ||
{{- if $.Values.debug }} | ||
- --debug | ||
- --debug-level | ||
- {{ quote $.Values.debugLevel }} | ||
{{- end }} | ||
{{- if $shard.id }} | ||
- --shard-id | ||
- {{ quote $shard.id }} | ||
{{- end }} | ||
{{- if not $.Values.metrics.enabled }} | ||
- --disable-metrics | ||
{{- end }} | ||
env: | ||
- name: NAMESPACE | ||
valueFrom: | ||
fieldRef: | ||
fieldPath: metadata.namespace | ||
{{- if $.Values.leaderElection.leaseDuration }} | ||
- name: CATTLE_ELECTION_LEASE_DURATION | ||
value: {{$.Values.leaderElection.leaseDuration}} | ||
{{- end }} | ||
{{- if $.Values.leaderElection.retryPeriod }} | ||
- name: CATTLE_ELECTION_RETRY_PERIOD | ||
value: {{$.Values.leaderElection.retryPeriod}} | ||
{{- end }} | ||
{{- if $.Values.leaderElection.renewDeadline }} | ||
- name: CATTLE_ELECTION_RENEW_DEADLINE | ||
value: {{$.Values.leaderElection.renewDeadline}} | ||
{{- end }} | ||
{{- if $.Values.proxy }} | ||
- name: HTTP_PROXY | ||
value: {{ $.Values.proxy }} | ||
- name: HTTPS_PROXY | ||
value: {{ $.Values.proxy }} | ||
- name: NO_PROXY | ||
value: {{ $.Values.noProxy }} | ||
{{- end }} | ||
{{- if $.Values.controller.reconciler.workers.gitrepo }} | ||
- name: HELMOPS_RECONCILER_WORKERS | ||
value: {{ quote $.Values.controller.reconciler.workers.gitrepo }} | ||
{{- end }} | ||
{{- if $.Values.extraEnv }} | ||
{{ toYaml $.Values.extraEnv | indent 12}} | ||
{{- end }} | ||
{{- if $.Values.debug }} | ||
- name: CATTLE_DEV_MODE | ||
value: "true" | ||
{{- end }} | ||
{{- if not $.Values.disableSecurityContext }} | ||
securityContext: | ||
allowPrivilegeEscalation: false | ||
readOnlyRootFilesystem: true | ||
privileged: false | ||
capabilities: | ||
drop: | ||
- ALL | ||
{{- end }} | ||
volumeMounts: | ||
- mountPath: /tmp | ||
name: tmp | ||
nodeSelector: {{ include "linux-node-selector" $shard.id | nindent 8 }} | ||
{{- if $.Values.nodeSelector }} | ||
{{ toYaml $.Values.nodeSelector | indent 8 }} | ||
{{- end }} | ||
{{- if $shard.nodeSelector -}} | ||
{{- range $key, $value := $shard.nodeSelector }} | ||
{{ $key | indent 8}}: {{ $value }} | ||
{{- end }} | ||
{{- end }} | ||
tolerations: {{ include "linux-node-tolerations" $shard.id | nindent 8 }} | ||
{{- if $.Values.tolerations }} | ||
{{ toYaml $.Values.tolerations | indent 8 }} | ||
{{- end }} | ||
{{- if $.Values.priorityClassName }} | ||
priorityClassName: "{{$.Values.priorityClassName}}" | ||
{{- end }} | ||
|
||
{{- if not $.Values.disableSecurityContext }} | ||
securityContext: | ||
runAsNonRoot: true | ||
runAsUser: 1000 | ||
runAsGroup: 1000 | ||
{{- end }} | ||
volumes: | ||
- name: tmp | ||
emptyDir: {} | ||
{{- end }} | ||
--- | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
{{- if .Values.gitops.enabled }} | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: ClusterRole | ||
metadata: | ||
name: helmops | ||
rules: | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- 'secrets' | ||
verbs: | ||
- "create" | ||
- "list" | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- 'configmaps' | ||
verbs: | ||
- '*' | ||
- apiGroups: | ||
- "fleet.cattle.io" | ||
resources: | ||
- "helmapps" | ||
- "helmapps/status" | ||
verbs: | ||
- "*" | ||
- apiGroups: | ||
- "fleet.cattle.io" | ||
resources: | ||
- "bundles" | ||
- "bundledeployments" | ||
verbs: | ||
- list | ||
- delete | ||
- get | ||
- watch | ||
- update | ||
- create | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- 'events' | ||
verbs: | ||
- '*' | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- serviceaccounts | ||
verbs: | ||
- "create" | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- namespaces | ||
verbs: | ||
- "create" | ||
- "delete" | ||
|
||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: ClusterRoleBinding | ||
metadata: | ||
name: helmops-binding | ||
roleRef: | ||
apiGroup: rbac.authorization.k8s.io | ||
kind: ClusterRole | ||
name: helmops | ||
subjects: | ||
- kind: ServiceAccount | ||
name: helmops | ||
namespace: {{ .Release.Namespace }} | ||
|
||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: Role | ||
metadata: | ||
name: helmops | ||
rules: | ||
- apiGroups: | ||
- "coordination.k8s.io" | ||
resources: | ||
- "leases" | ||
verbs: | ||
- "*" | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: RoleBinding | ||
metadata: | ||
name: helmops | ||
roleRef: | ||
apiGroup: rbac.authorization.k8s.io | ||
kind: Role | ||
name: helmops | ||
subjects: | ||
- kind: ServiceAccount | ||
name: helmops | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{{- if .Values.gitops.enabled }} | ||
apiVersion: v1 | ||
kind: ServiceAccount | ||
metadata: | ||
name: helmops | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
apiVersion: fleet.cattle.io/v1alpha1 | ||
kind: HelmApp | ||
metadata: | ||
name: {{.Name}} | ||
namespace: "fleet-local" | ||
spec: | ||
helm: | ||
releaseName: testhelm | ||
repo: {{.Repo}} | ||
chart: {{.Chart}} | ||
namespace: {{.Namespace}} | ||
helmSecretName: {{.HelmSecretName}} | ||
insecureSkipTLSVerify: true |
Oops, something went wrong.