issue with blockDeviceMappings and snapshotID

[uderik]

Description

Observed Behavior:
When specifying snapshotID for EBS in the blockDeviceMappings, we receive the following error: "You are not authorized to perform this operation". The error message from Amazon indicates that the AbsentTagComponent is not being set. If we remove the snapshotID, the instance launches without any issues. This happens because service control policy and tagging policy are enforced for all resources in our AWS environment.

Expected Behavior:
The instance should launch successfully with snapshotID provided, and the necessary tags should be applied to comply with the organization's tagging policy.

Reproduction Steps (Please include YAML):

1. Ensure that service control policy and tagging policy are enabled for your AWS account.
2. Include snapshotID in the blockDeviceMappings.
3. Launch the instance.
4. Observe the authorization error and missing tag components.

Versions:

• Chart Version: 0.35.9
• Kubernetes Version (Server Version: v1.29.7-eks-a18cd3a`):

[jmdeal]

If I understand correctly, you have an SCP that requires snapshotIDs to be tagged correctly before they can be used when launching an instance, and you want Karpenter to observe the authorization error and tag the snapshots so that they comply. Is that correct? In my opinion this falls outside the scope of Karpenter's responsibilities. Is there a reason you think this should be handled by Karpenter and not somewhere else in the process, e.g. whenever you're creating these snapshots?

[uderik]

@jmdeal When I try to launch an instance using a snapshot, Karpenter doesn't apply tags to the instance itself, which leads to the following error from AWS:

{
  "allowed": false,
  "explicitDeny": true,
  "matchedStatements": {
    "items": [
      {
        "statementId": "AbsentTagComponent",
        "effect": "DENY",
        "principals": {
          "items": [
..............

However, when I launch an instance without using a snapshot, it starts without any issues. The snapshot already has all the necessary tags; otherwise, I wouldn't have been able to create it.

It seems like the problem arises because Karpenter is not adding tags to the instance when using a snapshot.

[jmdeal]

Got it, is there any reason specifying the tags on your NodeClass directly isn't an option? https://karpenter.sh/docs/concepts/nodeclasses/#spectags

[uderik]

All the necessary tags are specified, and there are no issues when launching instances. However, when I specify snapshotId for EBS, I get the error that I mentioned above.

[jmdeal]

Any tags specified in the EC2NodeClass are included in the launch template for the instance (as well as the EBS volume and the launch template itself). Are there additional tags that are required on the instance by your SCP when launching with a snapshot ID that aren't specified on the NodeClass? Are you able to share the API request that's getting denied?