Sync seemingly unable to proceed on new empty opensearch install

[tobybellwood]

A brand new opensearch instance returns index patterns, but never progresses in the sync

2024-12-20T05:11:49.922Z DEBUG opensearch/indexpatterns.go:243 got all index patterns, returning result {"hits": 9} 2024-12-20T05:19:50.173Z DEBUG opensearch/indexpatterns.go:243 got all index patterns, returning result {"hits": 9}

all the ./lagoon-opensearch-sync dump-x commands seem to work properly and return expected data

~ $ ./lagoon-opensearch-sync dump-index-patterns
2024-12-20T06:05:43.149Z        DEBUG   opensearch/indexpatterns.go:243 got all index patterns, returning result        {"hits": 9}
{"-152937574_admintenant":{"*":["d94534f0-a5c0-11ef-9fd8-e9c9fe3ef9c9"],"application-logs-*":["application-logs-*"],"container-logs-*":["container-logs-*"],"lagoon-logs-*":["lagoon-logs-*"],"router-logs-*":["router-logs-*"]},"global_tenant":{"application-logs-*":["application-logs-*"],"container-logs-*":["container-logs-*"],"lagoon-logs-*":["lagoon-logs-*"],"router-logs-*":["router-logs-*"]}}

except dump-groups, but it appears that this is normal even on a running instance

~ $ ./lagoon-opensearch-sync dump-groups
[]

Will try a few options, but putting this here for reference.

[tobybellwood]

ooh - just thinking out loud - maybe it won't be able to create an index if there's no matching data yet - hence the hang - I'll see if we can seed some data to get it to continue to the roles creation etc - but it should probably handle the pre-release edge case better.

[smlx]

Ah I think it is a Keycloak permissions issue. The lagoon-opensearch-sync client doesn't have this fix applied, so it doesn't work on newer versions of Keycloak: uselagoon/lagoon#3797

You can test this by manually adding the view-users role in a test environment and then running dump-groups again.

Then we'll need a PR to update the base realm import over in uselagoon/lagoon.

Also noting here that lagoon-opensearch-sync currently ignores subgroups so if you would like subgroup support please open another issue :D

[tobybellwood]

I'll have a look. At least that should fix the dump-groups command

I'm only going on the basis that the sync is working in test 6 with the same keycloak version, but not an empty ch3.

[smlx]

Well it's not really working there either. Because it is getting an empty group list from Keycloak it has only "synced" the Global and admin tenants and deleted all the others 🙈