-
Notifications
You must be signed in to change notification settings - Fork 445
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pre-upgrade migration hook terminates unsuccessfully #1890
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is this a request for help?:
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT
Version of Helm and Kubernetes:
Helm v3
Kubernetes v1.27.5
Which chart:
Jfrog Platform 1.18.0
Which product license (Enterprise/Pro/oss):
Enterprise Plus
JFrog support reference (if already raised with support team):
#301924
What happened:
JFrog platform currently deployed on our k8s clusters is 10.17.0 (version of the official helm chart).
We're trying to upgrade it to v10.18.0, but we're experiencing issues with the rabbitmq instances. There are 2 rabbitmq replicas running on staging, and 3 on prod cluster at the moment.
The new rabbitmq image included in the v10.18.0 jfrog-platform global values.yaml file is 3.12.10-debian-11-r1, while the one used in the older versions of the chart is 3.11.10-debian-11-r5. This upgrade is affected by some missing featureFlags, ex. classic_mirrored_queue_version, that is not enabled in the older version, but required for the new version of rabbitmq.
After some debugging, we found out that there is a migration-hook.yaml file that has to create a job that will enable all the flags before the actual rabbitmq upgrade when the migration flag is set to true.
In our case, the job finishes with the following message: "Rabbitmq pod is not in running state. Ignoring feature flag migration for rabbitmq".
This is happening for 2 reasons:
Namespace parameter is not added to the kubectl commands in the file, check this for example. I am not sure why -n {{ .Release.Namespace }} is not part of the commands because without it they won't give any results back and the migration job will terminate unsuccessfully.
The condition below works when there is ONLY one rabbitmq instance because it evaluates the status of only one pod. In our case, we have two/three rabbitmq replicas, in staging/prod accordingly, and the command returns back the status of all of them, which means that the condition will be skipped and the job will again finish unsuccessfully:
What you expected to happen:
This time we had to enable the flags manually in order to upgrade the platform, but we're expecting this to be done by the job.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:
The text was updated successfully, but these errors were encountered: