Add aws-node-termination-handler (#945)

Co-authored-by: Andreas Sommer <[email protected]>

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add aws-node-termination-handler bundle
+
 ## [2.4.0] - 2024-11-12
 
 ### Added

helm/cluster-aws/README.md

@@ -24,6 +24,7 @@ Properties within the `.global.providerSpecific` object
 | `global.providerSpecific.ami` | **Amazon machine image (AMI)** - If specified, this image will be used to provision EC2 instances.|**Type:** `string`<br/>|
 | `global.providerSpecific.awsAccountId` | **AWS Account ID** - Only used when rendering the chart template locally, you shouldn't use this value. Used to calculate the IRSA service account issuer when using the China region.|**Type:** `string`<br/>|
 | `global.providerSpecific.awsClusterRoleIdentityName` | **Cluster role identity name** - Name of an AWSClusterRoleIdentity object. Learn more at https://docs.giantswarm.io/getting-started/cloud-provider-accounts/cluster-api/aws/#configure-the-awsclusterroleidentity .|**Type:** `string`<br/>**Value pattern:** `^[-a-zA-Z0-9_\.]{1,63}$`<br/>**Default:** `"default"`|
+| `global.providerSpecific.awsPartition` | **AWS Partition** - Only used when rendering the chart template locally, you shouldn't use this value.|**Type:** `string`<br/>|
 | `global.providerSpecific.flatcarAwsAccount` | **AWS account owning Flatcar image** - AWS account ID owning the Flatcar Container Linux AMI.|**Type:** `string`<br/>**Default:** `"706635527432"`|
 | `global.providerSpecific.instanceMetadataOptions` | **Instance metadata options** - Instance metadata options for the EC2 instances in the cluster.|**Type:** `object`<br/>|
 | `global.providerSpecific.instanceMetadataOptions.httpTokens` | **HTTP tokens** - The state of token usage for your instance metadata requests. If you set this parameter to `optional`, you can use either IMDSv1 or IMDSv2. If you set this parameter to `required`, you must use a IMDSv2 to access the instance metadata endpoint. Learn more at [What’s new in IMDSv2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html).|**Type:** `string`<br/>**Default:** `"required"`|
@@ -56,6 +57,13 @@ Configuration of apps that are part of the cluster.
 | `global.apps.awsEbsCsiDriverServiceMonitors.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
 | `global.apps.awsEbsCsiDriverServiceMonitors.extraConfigs[*].priority` | **Priority**|**Type:** `integer`<br/>**Default:** `25`|
 | `global.apps.awsEbsCsiDriverServiceMonitors.values` | **Config map** - Helm Values to be passed to the app as user config.|**Type:** `object`<br/>|
+| `global.apps.awsNodeTerminationHandler` | **App** - Configuration of a default app that is part of the cluster and is deployed as a HelmRelease resource.|**Type:** `object`<br/>|
+| `global.apps.awsNodeTerminationHandler.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
+| `global.apps.awsNodeTerminationHandler.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|
+| `global.apps.awsNodeTerminationHandler.extraConfigs[*].kind` | **Kind** - Specifies whether the resource is a config map or a secret.|**Type:** `string`<br/>|
+| `global.apps.awsNodeTerminationHandler.extraConfigs[*].name` | **Name** - Name of the config map or secret. The object must exist in the same namespace as the cluster App.|**Type:** `string`<br/>|
+| `global.apps.awsNodeTerminationHandler.extraConfigs[*].optional` | **Optional** - Optional marks this ValuesReference as optional. When set, a not found error for the values reference is ignored, but any ValuesKey, TargetPath or transient error will still result in a reconciliation failure.|**Type:** `boolean`<br/>|
+| `global.apps.awsNodeTerminationHandler.values` | **Values** - Values to be passed to the app. Values will have higher priority than values from configmaps.|**Type:** `object`<br/>|
 | `global.apps.awsPodIdentityWebhook` | **App resource** - Configuration of a default app that is part of the cluster and is deployed as an App resource.|**Type:** `object`<br/>|
 | `global.apps.awsPodIdentityWebhook.extraConfigs` | **Extra config maps or secrets** - Extra config maps or secrets that will be used to customize to the app. The desired values must be under configmap or secret key 'values'. The values are merged in the order given, with the later values overwriting earlier, and then inline values overwriting those. Resources must be in the same namespace as the cluster.|**Type:** `array`<br/>|
 | `global.apps.awsPodIdentityWebhook.extraConfigs[*]` | **Config map or secret**|**Type:** `object`<br/>|

helm/cluster-aws/ci/ci-values.yaml

@@ -16,6 +16,7 @@ global:
   providerSpecific:
     region: "eu-west-1"
     awsAccountId: "1234567890"
+    awsPartition: "aws"
   components:
     containerd:
       containerRegistries:

helm/cluster-aws/ci/test-local-registry-cache-values.yaml

@@ -16,6 +16,7 @@ global:
   providerSpecific:
     region: "eu-west-1"
     awsAccountId: "1234567890"
+    awsPartition: "aws"
   managementCluster: test
   components:
     containerd:

helm/cluster-aws/ci/test-multiple-authenticated-mirrors-values.yaml

@@ -16,6 +16,7 @@ global:
   providerSpecific:
     region: "eu-west-1"
     awsAccountId: "1234567890"
+    awsPartition: "aws"
   managementCluster: test
   components:
     containerd:

helm/cluster-aws/templates/_awspartition.tpl

@@ -0,0 +1,25 @@
+{{- /*
+Extracts the AWS partition from an ARN string.
+Example usage: {{ include "extractAWSPartition" "arn:aws:iam::1234567890:role/example-role" }}
+
+Input: An ARN string
+Output: The AWS partition (e.g., "aws", "aws-cn")
+*/ -}}
+{{- define "extractAWSPartition" -}}
+{{- $parts := (split ":" .) -}}
+{{- if ge (len $parts) 5 -}}{{- $parts._1 -}}{{- end -}}
+{{- end -}}
+
+{{- define "aws-partition" -}}
+{{- $roleName := .Values.global.providerSpecific.awsClusterRoleIdentityName -}}
+{{- $partition := .Values.global.providerSpecific.awsPartition -}}
+{{- $role := (lookup "infrastructure.cluster.x-k8s.io/v1beta2" "AWSClusterRoleIdentity" "" $roleName) -}}
+{{- if $role -}}
+{{- $partition = (include "extractAWSPartition" $role.spec.roleARN) -}}
+{{- end -}}
+{{- if eq $partition "" -}}
+{{- fail "failed to extract AWS Partition from AWSClusterRoleIdentity" -}}
+{{- else -}}
+{{- $partition -}}
+{{- end -}}
+{{- end -}}

helm/cluster-aws/templates/_machine_pools.tpl

@@ -86,6 +86,17 @@ spec:
     maxHealthyPercentage: {{ $value.maxHealthyPercentage }}
   ignition:
     version: "3.4"
+  lifecycleHooks:
+  - defaultResult: CONTINUE
+    # High enough heartbeat timeout because aws-node-termination-handler (shortened to "NTH" here)
+    # doesn't send heartbeats (https://github.com/aws/aws-node-termination-handler/issues/493),
+    # but low enough so that if the controller is down, instances can still terminate within
+    # a reasonable time.
+    heartbeatTimeout: 30m
+    lifecycleTransition: autoscaling:EC2_INSTANCE_TERMINATING
+    name: aws-node-termination-handler
+    notificationTargetARN: arn:{{ include "aws-partition" $}}:sqs:{{ include "aws-region" $ }}:{{ include "aws-account-id" $}}:{{ include "resource.default.name" $ }}-nth
+    roleARN: arn:{{ include "aws-partition" $}}:iam::{{ include "aws-account-id" $}}:role/{{ include "resource.default.name" $ }}-nth-notification
 ---
 {{ end }}
 {{- end -}}

helm/cluster-aws/templates/aws-nth-helmrelease.yaml

@@ -0,0 +1,61 @@
+{{/* Default Helm values for the app */}}
+{{/* See schema for the appropriate app version here https://github.com/giantswarm/aws-ebs-csi-driver-app/blob/master/helm/aws-ebs-csi-driver-app/values.schema.json */}}
+{{- define "defaultAwsNodeTerminationHandlerHelmValues" }}
+clusterID: {{ include "resource.default.name" $ }}
+{{- if (.Values.global.connectivity.proxy).enabled }}
+proxy:
+  noProxy: "{{ include "cluster.connectivity.proxy.noProxy" (dict "global" $.Values.global "providerIntegration" $.Values.cluster.providerIntegration) }}"
+  http: {{ .Values.global.connectivity.proxy.httpProxy | quote }}
+  https: {{ .Values.global.connectivity.proxy.httpsProxy | quote }}
+{{- end }}
+global:
+  image:
+    registry: {{ include "awsContainerImageRegistry" $ }}
+  podSecurityStandards:
+    enforced: {{ .Values.global.podSecurityStandards.enforced }}
+{{- end }}
+apiVersion: helm.toolkit.fluxcd.io/v2beta1
+kind: HelmRelease
+metadata:
+  name: {{ include "resource.default.name" $ }}-nth-bundle
+  namespace: {{ $.Release.Namespace }}
+  annotations:
+    cluster.giantswarm.io/description: "{{ .Values.global.metadata.description }}"
+  labels:
+    cluster-apps-operator.giantswarm.io/watching: ""
+    {{- include "labels.common" . | nindent 4 }}
+spec:
+  suspend: false # It can be unsuspended by the post-install/post-upgrade hook. Useful if we need to populate some fields later on.
+  releaseName: aws-nth-bundle
+  chart:
+    spec:
+      chart: aws-nth-bundle
+      {{- $_ := set $ "appName" "aws-nth-bundle" }}
+      version: {{ include "cluster.app.version" $ }}
+      sourceRef:
+        kind: HelmRepository
+        name: {{ include "resource.default.name" $ }}-{{ include "cluster.app.catalog" $ }}
+  kubeConfig:
+    secretRef:
+      name: {{ $.Values.global.managementCluster }}-kubeconfig
+  interval: 5m
+  install:
+    remediation:
+      retries: 30
+  {{- $AwsNodeTerminationHandlerHelmValues := (include "defaultAwsNodeTerminationHandlerHelmValues" .) | fromYaml -}}
+  {{- $customAwsNodeTerminationHandlerHelmValues := $.Values.global.apps.awsNodeTerminationHandler.values -}}
+  {{- if $customAwsNodeTerminationHandlerHelmValues }}
+  {{- $AwsNodeTerminationHandlerHelmValues = merge (deepCopy $customAwsNodeTerminationHandlerHelmValues) $AwsNodeTerminationHandlerHelmValues -}}
+  {{- end }}
+  {{- if $AwsNodeTerminationHandlerHelmValues }}
+  values: {{- $AwsNodeTerminationHandlerHelmValues | toYaml | nindent 4 }}
+  {{- end }}
+  {{- if $.Values.global.apps.awsNodeTerminationHandler.extraConfigs }}
+  valuesFrom:
+    {{- range $config := $.Values.global.apps.awsNodeTerminationHandler.extraConfigs }}
+    - kind: {{ $config.kind }}
+      name: {{ $config.name }}
+      valuesKey: values
+      optional: {{ $config.optional | default false  }}
+    {{- end }}
+  {{- end }}

helm/cluster-aws/values.schema.json

@@ -708,6 +708,12 @@
                             "title": "AWS EBS CSI driver service monitors",
                             "description": "Configuration of aws-ebs-csi-driver-servicemonitors. For all available values see https://github.com/giantswarm/aws-ebs-csi-driver-servicemonitors-app."
                         },
+                        "awsNodeTerminationHandler": {
+                            "$ref": "#/$defs/helmRelease",
+                            "type": "object",
+                            "title": "AWS EBS CSI driver",
+                            "description": "Configuration of aws-nth-bundle. For all available values see https://github.com/giantswarm/aws-nth-bundle."
+                        },
                         "awsPodIdentityWebhook": {
                             "$ref": "#/$defs/app",
                             "type": "object",
@@ -1750,6 +1756,11 @@
                             "minLength": 1,
                             "pattern": "^[-a-zA-Z0-9_\\.]{1,63}$"
                         },
+                        "awsPartition": {
+                            "type": "string",
+                            "title": "AWS Partition",
+                            "description": "Only used when rendering the chart template locally, you shouldn't use this value."
+                        },
                         "flatcarAwsAccount": {
                             "type": "string",
                             "title": "AWS account owning Flatcar image",

helm/cluster-aws/values.yaml

@@ -267,6 +267,7 @@ global:
     awsCloudControllerManager: {}
     awsEbsCsiDriver: {}
     awsEbsCsiDriverServiceMonitors: {}
+    awsNodeTerminationHandler: {}
     awsPodIdentityWebhook: {}
     capiNodeLabeler: {}
     certExporter: {}