CustomManager seems to have a problem adding a github SHA hash to the end of a number-only Docker tag

[colinexl]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

GHE 38.80.0

Please tell us more about your question or problem

Hello all,

I have a file here "/path/to/var.json" with the following contents:

{
  "foo": {
    "bar": {
      "baz": {
        "MY_IMAGE_TAG": 123
      }
    }
  }
}

The MY_IMAGE_TAG is a hardcoded name and is not the name of the actual package. The actual package name is my.docker.repo/my-org/my_awesome_test_image from which the 123 tag is also generated.

I want to update the tag "MY_IMAGE_TAG": 123 whenever there is a new Docker image built for my.docker.repo/my-org/my_awesome_test_image.

Some more details that makes this trickier. Our docker images are tagged in two ways. The first way is just an incremental number tag which is what is currently used by vars.json:

123, 124, 125, ...

The second way is the incremental number tag with the Git commit SHA:

123-087d744d1d9e14c34c3bd8741b6b6ecb200bb434
124-027aba0471fc48727089d65282f61a4f3a388b09
...

We don't use the standard sha256 digest and I cannot change that.

I want to update the tag "MY_IMAGE_TAG": 123 whenever there is a new Docker image built for my.docker.repo/my-org/my_awesome_test_image using the "tag-SHA" format. Since Renovate doesn't have a manager out of the box for this, I tried creating my own customManagers:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "packageRules": [
    {
      "description": "Group all Docker updates",
      "groupName": "Renovate: all Docker updates",
      "groupSlug": "all-docker",
      "matchDatasources": ["docker"],
      "versioning": "loose",
      "allowedVersions": "/\\d+-\\w{40}$/", // force tag-SHA format
      "labels": ["all-docker"]
    }
  ],
  "customManagers": [
    {
      "description": "Custom manager for MY_IMAGE_TAG updates",
      "customType": "regex",
      "fileMatch": ["(^|/)vars\\.json$"],
      "matchStrings": [
        "\"MY_IMAGE_TAG\":\\s*(?<currentValue>\\d+)(?:-(?<currentDigest>\\w{40}))?"
      ],
      "autoReplaceStringTemplate": "\"{{{depName}}}\": {{{newValue}}}-{{{newDigest}}}",
      "depNameTemplate": "MY_IMAGE_TAG",
      "packageNameTemplate": "my.docker.repo/my-org/my_awesome_test_image",
      "datasourceTemplate": "docker",
      "versioningTemplate": "loose"
    }
  ]
}

I took some inspiration from prior art: #32085

I also tried "extractVersionTemplate": "(?<version>\\d+)(?:-\\w{40})?" but that caused Renovate to skip this file entirely.

The problem I'm seeing right now is that Renovate finds the newer tag 124-027aba0471fc48727089d65282f61a4f3a388b09, but is not able to update the tag 123 for some reason, possibly because it doesn't conform to the allowedVersions that I have defined in the packageRules?

The allowedVersions is important because we have other Dockerfiles that use this config to update the tag+SHA so I cannot remove it either.

Ideally, Renovate can update the vars.json with new tags using the "tag+SHA" format.

Update 1:

I played with the config some more and now I'm able to get PRs with the correct tag+sha format. However, the problem now is it is not being surrounded by ", which causes the JSON to be invalid.

"customManagers": [
    {
      "description": "Custom manager for MY_IMAGE_TAG updates",
      "customType": "regex",
      "fileMatch": ["(^|/)vars\\.json$"],
      "matchStrings": [
        "\"MY_IMAGE_TAG\":\\s*\"?(?<currentValue>\\d+)(?:-(?<currentDigest>\\w{40}))?\"?"
      ],
      "autoReplaceStringTemplate": "\"{{{depName}}}\": \"{{{newValue}}}{{#if newDigest}}-{{{newDigest}}}{{/if}}\"",
      "currentValueTemplate": "{{{ currentValue }}}{{#if currentDigest}}-{{{ currentDigest }}}{{else}}-1234567890123456789012345678901234567890{{/if}}",
      "depNameTemplate": "MY_IMAGE_TAG",
      "packageNameTemplate": "my.docker.repo/my-org/my_awesome_test_image",
      "datasourceTemplate": "docker",
      "versioningTemplate": "loose"
    }
  ]

So the diff looks something like:

-        "FIT_IMAGE_TAG": 99,
+        "FIT_IMAGE_TAG": 103-d2a41df8af8306082754ea89791b8a20505bfc96,

but I want it to be like:

-        "FIT_IMAGE_TAG": 99,
+        "FIT_IMAGE_TAG": "103-d2a41df8af8306082754ea89791b8a20505bfc96",

I thought updating the autoReplaceStringTemplate with \" would've done the trick but apparently not.

Logs (if relevant)

Logs

https://gist.github.com/colinexl/2b17cab809c6244bf4321de4238bd5a3

[github-actions[bot]]

Hi there,

This issue or discussion is missing some logs, making it difficult or impossible to help you. Depending on which situation applies follow one, some or all of these instructions.

No logs at all

If you haven't posted any log yet, we need you to find and copy/paste the log into the issue template.

Finding logs on hosted app

Select me to read instructions

If you use the Mend Renovate app (GitHub):

1. Log in to the Mend Developer Portal
2. Navigate to the correct organization and repository
3. Locate the appropriate log (it may not always be the latest one)
4. Copy/paste the log contents
5. Follow the steps in the formatting your logs section

Finding logs when self-hosting

Select me to read instructions

Read the Renovate docs, troubleshooting, self-hosted to learn how to find the logs.

Insufficient logs

Select me to read instructions

If you already gave us a log, and the Renovate team said it's not enough, then follow the instructions from the No logs at all section.

Formatting your logs

Select me to read instructions

Please put your logs in a <details> and <summary> element like this:

<details><summary>Select me to see logs</summary>

```
Copy/paste your logs here, between the starting and ending backticks
```

</details>

If you feel the logs are too large to paste here, please use a service like GitHub Gist and paste the link here.

Good luck,

The Renovate team

[colinexl]

I did provide logs in the section between the triple back ticks. Wonder why it's complaining about no logs?

[rarkins]

Keep reading that message

[colinexl]

Thanks for the reply. Apologies, I read over the instructions again but still don't understand what the ask is. If I have insufficient logs, I can go back and dig up more, but will have to sanitize them first.

[colinexl]

I did some more updates on the config and posted the full sanitized logs. Essentially the problem now is how do enclose 103-d2a41df8af8306082754ea89791b8a20505bfc96 in double-quotes.

[colinexl]

ok not sure what I did but it appears to be working now. This is my latest config:

"customManagers": [
    {
      "description": "Custom manager for MY_IMAGE_TAG updates",
      "customType": "regex",
      "fileMatch": ["(^|/)vars\\.json$"],
      "matchStrings": [
        "\"MY_IMAGE_TAG\":\\s*\"?(?<currentValue>\\d+)(?:-(?<currentDigest>\\w{40}))?\"?"
      ],
      "autoReplaceStringTemplate": "\"{{{depName}}}\": \"{{{newValue}}}{{#if newDigest}}-{{{newDigest}}}{{/if}}\"",
      "currentValueTemplate": "{{{ currentValue }}}{{#if currentDigest}}-{{{ currentDigest }}}{{else}}-1234567890123456789012345678901234567890{{/if}}",
      "depNameTemplate": "MY_IMAGE_TAG",
      "packageNameTemplate": "my.docker.repo/my-org/my_awesome_test_image",
      "datasourceTemplate": "docker",
      "versioningTemplate": "loose"
    }
  ]