could not get access_tokens from GitHub API for installation ID

[XBeg9]

2021-07-13T08:31:28.346Z	DEBUG	controller-runtime.manager.events	Warning	{"object": {"kind":"Runner","namespace":"actions-runner-system","name":"runner-hrwpg-5th5d","uid":"44383ec5-84d9-47c1-80d4-345412a0d624","apiVersion":"actions.summerwind.dev/v1alpha1","resourceVersion":"65816"}, "reason": "FailedUpdateRegistrationToken", "message": "Updating registration token failed"}

2021-07-13T08:31:28.346Z	ERROR	controller-runtime.controller	Reconciler error	{"controller": "runner-controller", "request": "actions-runner-system/runner-hrwpg-5th5d", "error": "failed to create registration token: Post \"https://api.github.com/orgs/myorg/actions/runners/registration-token\": could not refresh installation id 18242346's token: could not get access_tokens from GitHub API for installation ID 18242346: EOF"}

This setup was working for a while and it did stop without any changes to app and configs, I thought while it's broken I will update everything to the latest version (0.12.7) and try again, but the issue hasn't disappeared and it's still throwing this error.

my helm config has only authSecret values, and I have validated them, everything looks good and passed using ENVs to the controller:

authSecret: {
        create: true,
        github_app_id: config.github.app.id,
        github_app_installation_id: config.github.app.installationId,
        github_app_private_key: config.github.app.pem
      }

my app permissions:

[XBeg9]

btw, I saw #427 , I did upgrade to CRDs using kubectl replace, it doesn't help. Btw, the suggestion here will be to add install_crd option inside helm chart, so you can disable it if there will be another IaC controlling the CRDs.

I am happy to share any debug logs here, @mumoshu this is a huge blocker for me, our CI did stop over the weekend without doing any changes. I will appreciate your help.

[XBeg9]

The issue has been resolved. FW was blocking requests to Github :(