kamal secrets extract JSON parse error on shell escaped secrets

[mblayman]

Hi!

I'm trying to use the new secret helpers from Kamal 2 to manage secrets for my project. I tried to follow the style listed in the secrets documentation under "Environment variables", but I'm running into a problem with shell escaping.

I think what is happening is that the raw secrets JSON data is getting to the Ruby parsing code with the shell escape characters still intact. Parsing fails because the JSON parser doesn't accept the backslashes.

I tried to do some puts debugging in the extract method of lib/kamal/cli/secrets.rb to confirm what was going into the JSON.parse call. You can see from my output below that the shell escape characters are still present.

$ SECRETS=$(kamal secrets fetch --adapter 1password --account *** --from some-vault/some-item KAMAL_REGISTRY_PASSWORD)

$ echo $SECRETS
\{\"some-vault/some-item/KAMAL_REGISTRY_PASSWORD\":\"just_alphanumeric_and_underscores\"\}

$ kamal secrets extract KAMAL_REGISTRY_PASSWORD $SECRETS
Hello modified <== my change of `puts "Hello modified"`
\{\"some-vault/some-item/KAMAL_REGISTRY_PASSWORD\":\"just_alphanumeric_and_underscores\"\} <== my change of `puts secrets`
  ERROR (JSON::ParserError): unexpected token at '\{\"some-vault/some-item/KAMAL_REGISTRY_PASSWORD\":\"just_alphanumeric_and_underscores\"\}'

I attempted this on bash and zsh to try to rule out differences in shells.

Is there supposed to be a different way to invoke kamal secrets fetch or kamal secrets extract?

Thanks for the help! I hope this bug report is useful. Please let me know if y'all need any other info.

[mblayman]

I'm not really a Ruby dev, so this might be an inappropriate solution, but Chat GPT helped me devise this alternative implementation of extract:

  desc "extract", "Extract a single secret from the results of a fetch call"
  option :inline, type: :boolean, required: false, hidden: true
  def extract(name, secrets)
    unescaped_secrets = Shellwords.shellsplit(secrets).first
    parsed_secrets = JSON.parse(unescaped_secrets)
    ...

If fetch consistently outputs a shell escaped JSON blob in a single line, I think this might work well.

[mblayman]

Even with my little hack above, something was not working right. I attempted to continue by switching KAMAL_REGISTRY_PASSWORD to use an env variable like KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD. Then I continued with running kamal setup. My setup failed for reasons that I think are app-related, but I when I checked on the web.env file on the server, I found some rather unexpected output. It looks like the command substitution did not happen because I found the following in web.env:

SECRET_KEY=$(kamal secrets extract SECRET_KEY \\{\\"some-vault/some-item/KAMAL_REGISTRY_PASSWORD\\":\\"the_password_was_here\\",\\"some-vault/some-item/SECRET_KEY\\":\\"the_key_was_here\\"\\})

I confirmed in my Django shell that the SECRET_KEY setting was set to that long raw value. My best guess is that not doing the command substitution was also why my KAMAL_REGISTRY_PASSWORD did not work until I changed it to use an env variable.

For completeness, here is my .kamal/secrets (with the sensitive bits removed):

# Secrets defined here are available for reference under registry/password, env/secret, builder/secrets,
# and accessories/*/env/secret in config/deploy.yml. All secrets should be pulled from either
# password manager, ENV, or a file. DO NOT ENTER RAW CREDENTIALS HERE! This file needs to be safe for git.

SECRETS=$(kamal secrets fetch --adapter 1password --account *** --from some-vault/some-item KAMAL_REGISTRY_PASSWORD SECRET_KEY)
#KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD $SECRETS)
KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
SECRET_KEY=$(kamal secrets extract SECRET_KEY $SECRETS)

[djmb]

Thanks for the report @mblayman!

This worked ok when you specified the kamal secrets commands inside .kamal/secrets, but that was because it "inlines" the calls to the secret commands and the inlining was unescaping the JSON for us. I've moved things around in #1008 so it should work ok now both on the command line and in .kamal/secrets

[djmb]

Ok that approach won't work, the problem is that the dotenv gem and the shell don't work in the same way.

#1009 adds a kamal secrets print command that can be used for debugging. I'll update the docs to explain this once it is released.

[blvrd]

Ran into the same issue. FWIW, this is my current workaround in .kamal/secrets:

KAMAL_REGISTRY_PASSWORD=$(op read op://vault/item/field)

[mblayman]

@blvrd, yeah, I ended up doing the same thing and used the 1password CLI directly just like you did.

[evdevdev]

@blvrd Thanks for posting your solution. I just burned about 45 knocking my head against this trying to figure out the escape error.