Renovate CE fails to lookup private dependencies

[Wassim-AB]

Hello :)

Yesterday I migrated from Whitesource Renovate to Renovate-CE. After following the migration guide everything looked fine but then Renovate started raising errors about failures when looking up for private dependencies. Here is an error example :

{
    "files": [
      "package.json"
    ],
    "repository": "xxx/yyy/zzz",
    "warnings": [
      "Failed to look up npm package @bridge/database",
    ],
    "level": 40,
    "time": "2023-11-28T09:36:09.384Z",
    "msg": "Package lookup failures",
    "name": "renovate",
    "v": 0,
    "logContext": "b2b97b5e58",
}

It has always worked with Whitesource Renovate but ever since the migration I started having these errors.

Here is the .npmrc file inside my repository :

@bridge:registry=https://nexus.company.com/repository/company-npm-public/
always-auth = true

Here is my config.js file mounted at /usr/src/app :

module.exports = {
  timezone: 'Europe/Paris',
  onboardingCommitMessage: 'configure renovate bot',

  labels: [
    'dependencies',
  ],

  lockFileMaintenance: {
    enabled: true,
  },

  platformAutomerge: true,

  persistRepoData: true,

  repositoryCache: "enabled",

  hostRules: [
    {
      matchHost: 'europe-west1-docker.pkg.dev',
      username: "_json_key_base64",
      password: process.env.ARTIFACT_REGISTRY_KEY,
    },
    {
      hostType: 'npm',
      matchHost: 'https://nexus.company.com/repository/company-npm-public/',
      token: process.env.NEXUS_NPM_TOKEN,
      authType: "Basic"
    }
  ],

  onboardingConfig: {
    "extends": ["local>company/ci/renovate-config"]
  }
};

The environnement variables are well defined in the container. I also have the error for the Artifact Registry defined above. I set the Log Level to debug and I found these two logs at startup :

{
   "name":"renovate",
   "config":{
      "timezone":"Europe/Paris",
      "onboardingCommitMessage":"configure renovate bot",
      "labels":[
         "dependencies"
      ],
      "lockFileMaintenance":{
         "enabled":true
      },
      "platformAutomerge":true,
      "persistRepoData":true,
      "repositoryCache":"enabled",
      "hostRules":[
         {
            "matchHost":"europe-west1-docker.pkg.dev",
            "username":"_json_key_base64",
            "password":"***********"
         },
         {
            "hostType":"npm",
            "matchHost":"https://nexus.company.com/repository/company-npm-public/",
            "token":"***********"
         }
      ],
      "logContext":"a7d46a65e8",
      "onboardingConfig":{
         "$schema":"https://docs.renovatebot.com/renovate-schema.json",
         "extends":[
            "config:recommended"
         ]
      },
      "endpoint":"https://gitlab.company.com/api/v4/",
      "platform":"gitlab",
      "token":"***********",
      "username":"renovate-bot",
      "binarySource":"install",
      "autodiscover":false
   }
}

{
   "name":"renovate",
   "pid":25,
   "level":20,
   "logContext":"a7d46a65e8",
   "msg":"Adding token authentication for https://nexus.company.com/repository/company-npm-public/ (hostType=npm) to hostRules",
   "time":"2023-11-28T08:09:53.545Z",
}

I can provide extra logs and any additional information if needed, but I feel like I just missed an extra configuration somewhere with the migration.

Version : renovate-ce:6.7.0-full
Previous Whitesource version : 5.0.1

Thank you :)

[rarkins]

Something here isn't right:

[Wassim-AB]

Absolutely my mistake, I accidentally removed the line while cleaning the logs. The authType: "Basic" is present in the logs.

[PhilipAbed]

can we get debug logs?

if you want to trouble shoot this check the section right after
Dependency extraction complete ( in debug logs)
it would show you the lookup errors you are receiving

[Wassim-AB]

Here is something I found :

{
   "jsonPayload": {
      "msg": "GET https://nexus.company.com/repository/company-npm-public/@bridge%2Fi18n = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=401 retryCount=0, duration=231)",
      "v": 0,
      "pid": 46966,
      "level": 20,
      "name": "renovate",
      "repository": "xxx/yyy/zzz"
   }
}

So it reads the repositorie's .npmrc successfully but it didn't find the credentials.

Any other type of log that could be relevant in your opinion ?

[StinkyLord]

Authentication issue.

1. Could be your token has expired?
   next step is to try to Curl it with the token as the header, and see if your token actually has authorization or expired .
2. Could it be a wrong URL ?
   match your config: matchHost url to the log above, make sure the Host is the same.

[Wassim-AB]

1. I tried using Curl locally and it succeeded using the Header authorization: Bearer <my-token> so the token is ok. I tried removing the authType: "Basic" attribute inside my config but it didn't change anything. I actually changed it to Basic when I was trying to fix it before opening this discussion :)

2. I compared the URL in the .npmrc and in the hostRules and they are the same.

Like I said, it started failing when I switched image from Whitesource Renovate to using Renovate-CE and changed the configurations mentioned in the migration guide.

[Gabriel-Ladzaretti]

It might be related or it might not, but it seems that your onboarding configuration is being overwritten. This is likely due to it being hardcoded in the Dockerfile using an environment variable. Consider explicitly setting the onboarding config using its respective environment variable RENOVATE_ONBOARDING_CONFIG.

Therefore if this is a newly onboarded repository, the extends entry might not be as expected.

[Wassim-AB]

Indeed :) So should I use environnement variables for private registries as well ? I know it's possible to configure private registries using detectHostRulesFromEnv

Also I found that RENOVATE_CONFIG_FILE exists, should I configure It as well ? Maybe copy the file to a different location as well

[Gabriel-Ladzaretti]

The only environment variables we set are these:

• RENOVATE_CONFIG
• RENOVATE_AUTODISCOVER
• RENOVATE_BINARY_SOURCE
• RENOVATE_ONBOARDING_CONFIG

So, should I use environment variables for private registries as well?

I don't think so; using config.js is perfectly fine and encouraged. Just bear in mind that if you try to use any from the above list inside the config.js, it will be overwritten, similar to your experience with onboardingConfig & RENOVATE_ONBOARDING_CONFIG.

[Wassim-AB]

Ok thank you :) I still need to figure out why my private registries are not configured properly ...

[Wassim-AB]

I don't know why but fixing the onboardingConfiguration by setting it through RENOVATE_ONBOARDING_CONFIG rather than in the config.js file fixed my issue ! It now handles my dependencies properly :)

Thank you !