Not able to fetch secret from aws secret manager (cross-accounts)

[xval2307]

Hello maintainers,

I try to deploy helm secrets + vals on ArgoCD to fetch secrets from aws secrets manager in another account.

    helm:
      fileParameters:
        - name: data.exemple.Key1
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:/SECRET_NAME#/SECRET_KEY_1
        - name: data.exemple.Key2
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:/SECRET_NAME#/SECRET_KEY_2

The arn of mysecret on AWS Console is:

arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME

I haven’t “/” in mysecret name.

And if I remove the “/” in the secret path in helm parameter:

    helm:
      fileParameters:
        - name: data.exemple.Key1
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_1
        - name: data.exemple.Key2
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_2

I have the following error message in ArgoCD:

[helm-secrets] vals error: expand awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME?region=eu-west-3: parse "awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME?region=eu-west-3": invalid port ":SECRET_NAME" after host Error: failed parsing --set-file data: plugin "scripts/run.sh downloader" exited with error

Could you please confirm if this is really the case or it's a misconfiguration of my side?

[mumoshu]

@xval2307 Thanks for reporting! Ahh interesting... so our URL parser is messed up with the last part of your ARN.

Would you try adding a redundant and unused port number after your ARN? That would make the URL parser green and it vals would work as expected as it won't use the information provided via the port part for the AWS SecretsManager mode...

In other words, try changing this:

path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_1

to

path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME:12345#/SECRET_KEY_1

Notice that I've added :12345 after the ARN part to make the URL parser happy.

[fbordallo]

I'm seeing this problem too. Also, secret names can have forward slashes which could invalidate the workaround.

ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:secret/name/with/slashes#/SECRET_KEY_1

Thanks for you work!

[BabisK]

I'm facing this issue too, If I add the :12345 after the ARN I get another error when the variable is used:

in ./helmfile.yaml: error during helmfile.yaml.part.1 parsing: template: stringTemplate:22:42: executing "stringTemplate" at <fetchSecretValue>: error calling fetchSecretValue: expand awssecrets://arn:aws:secretsmanager:eu-central-1:xxxxxxx:secret:rds!vvvvvvvvvvvvvvvvvvv:12345: get parameter: ValidationException: Invalid name. Must be a valid name containing alphanumeric characters, or any of the following: -/_+=.@!
        status code: 400, request id: b4f1877d-1113-40f4-97c2-ad12e8e52514

[JBOClara]

Hi @mumoshu ,

We hit this bug and workaround isn't working.

Is there anything I can do to help fixing ?

[rikameajay1]

Worked for me.
Used following format ref+awssecrets://arn:aws:secretsmanager:AWS_REGION:ACCOUNT_ID:secret:/secret/name/with_slashes?region=AWS_REGION
Also created secret in AWS Secrets manager as specified in helmfile /secret/name/with_slashes
Create secret with "/" at the beginning

[anton-prymak]

It's the same problem for me. Unfortunately, I cannot use"/" at the beginning because of my project limitations.

[itssimon]

I'm also hitting this problem and workarounds aren't working

[okouyad]

Same issue. Given how critical it is, is there some plans to takle this anytime soon ? Else a recommended work around ? This is particularily problematic in situation where there is a cross account access and therefore we need the full arn

[hari-rad]

Worked fine for us. We used two AWS accounts in the same region. For cross-account access, a CMK (Customer-managed key) is required to encrypt the secret. Initially, the argocd-repo-server IRSA role was failing with "AccessDenied secretsmanager:GetSecretValue" and "Access to KMS is not allowed" while syncing the argocd application.
Turns out we were using the "kms:Decrypt" permission with the CMK Alias. Once we updated the IRSA role to use the CMK ARN (instead of the key alias), these errors got resolved.
For referencing the secret from another account, we had to use the secret ARN (while accessing from the same account, we used the secret name).

# use secret arn to access from another account
configtoken: ref+awssecrets://<secret-arn>?region=<region>#<secret-json-key>

# IRSA Role policy
Version: "2012-10-17"
Statement:
  - Action: secretsmanager:GetSecretValue
    Resource:
      - <CROSS-ACCOUNT SECRET ARN>
    Effect: Allow
  - Action: kms:Decrypt
    Resource:
      - <CROSS-ACCOUNT CMK ARN>
    Effect: Allow