Self Hosted Air gapped production Pixie Cloud deployment: no job created

[Pk-sketch]

Describe the bug
Followed the self hosted Air gapped production instruction (https://docs.px.dev/installing-pixie/install-guides/self-hosted-pixie/production-readiness/) and switched to https://docs.px.dev/installing-pixie/install-guides/self-hosted-pixie/airgap-pixie/ per instruction on Production Readiness page. The login page is visible however, when I click on login, I receive 400 Error.

To Reproduce
Steps to reproduce the behavior:

1. follow the self hosted Air gapped production instruction
2. Verify all pods are up and running (there is no job create-admin-job or create-hydra-client-job created)
3. Click on login bottom on the custom domain page Authentication using Kratos / Hydra
4. Receiving HTTP Error 400 after clicking Login instead of login user/pass page

#kubectl get pod
NAME READY STATUS RESTARTS AGE
api-server-875984977-glwww 1/1 Running 0 4d13h
artifact-tracker-server-857d8f47c9-dj4g8 1/1 Running 0 4d13h
auth-server-7c78b47f94-jg5bz 1/1 Running 0 4d13h
cloud-proxy-5f5c59bfbf-d9vc2 2/2 Running 0 4d13h
config-manager-server-5dd98bbb76-wmlz9 1/1 Running 0 4d13h
cron-script-server-54f758b6fd-q6jvh 1/1 Running 0 4d13h
hydra-767f5cbd8-fn4z5 2/2 Running 0 4d13h
indexer-server-74c9f97d49-gzxms 1/1 Running 0 4d13h
kratos-795566bd8f-4rlxq 2/2 Running 0 4d13h
metrics-server-696f7f8446-58cjx 1/1 Running 0 4d13h
pl-elastic-es-data-0 1/1 Running 0 5d19h
pl-elastic-es-master-0 1/1 Running 0 5d19h
pl-elastic-es-master-1 1/1 Running 0 5d19h
pl-nats-0 1/1 Running 0 5d19h
pl-nats-1 1/1 Running 0 5d19h
pl-nats-2 1/1 Running 0 5d19h
plugin-server-7d76679674-l725t 1/1 Running 0 4d13h
postgres-6877f879-fz5wn 1/1 Running 0 5d19h
profile-server-5c974b5775-q5r6c 1/1 Running 0 4d13h
project-manager-server-799f8c5b6d-fzk46 1/1 Running 0 4d13h
scriptmgr-server-76f8c4b54c-dc5kq 1/1 Running 0 4d13h
vzconn-server-5f76f4dcdc-7bptf 1/1 Running 0 4d13h
vzmgr-server-6d4bccd56d-86blq 1/1 Running 0 4d13h

Expected behavior
Viewing user/pass page after click on Login button

Screenshots

Logs
log.tar.gz

** Yaml **
Uploading pixie_cloud.tar.gz…

App information (please complete the following information):

• Pixie version latest
• K8s cluster version v1.29.7
• Node Kernel version 4.18.0-553.8.1.el8_10.x86_64
• Browser version Chrome latest

Additional context
in addition, in https://docs.px.dev/installing-pixie/install-guides/airgap-pixie/#deploy-pixie-cloud-authentication-using-kratos-hydra is written to set the admin credentials should update k8s/cloud/base/ory_auth/kratos/kratos_deployment.yaml while no where in the document this yaml is called.

[Mr-Artemus]

Hi, I remember something similar happening.
Make sure to use the version 0.1.7 on all images instead of latest (latest is older than 0.1.7)