Generalized remote cluster targeting

[seaneagan]

The kustomize-controller has the ability to target remote clusters, such as those created by CAPI. It was mentioned that the kustomize-controller functionality was targeted at being able to install the GitOps Toolkit in the remote cluster. I think there are also use cases for running the GitOps Toolkit from a remote "management cluster" (to borrow the CAPI terminology), without having to have a permanent foothold within the targeted cluster. This can be thought of as a push model vs the currently supported pull model. This could be useful for managing:

• resource contrained clusters (or cost saving on compute resources)
• clusters with limited network access
• "tenant clusters" i.e. having an even stronger isolation than namespaces between admin and tenant, including separate LMA stacks etc.
• centralized fleet management of a large amount of clusters from a single or smaller number of management clusters

In cases where the Git repo (or other manifest source) is updated infrequently, a management cluster could even be created on demand (after git push to a given branch), and torn down as desired after the update is complete. A lightweight cluster manager like kind could of course be useful for this.

Enabling this would involve adding the above referenced kustomize-controller functionality to the helm-controller as well. The release metadata could be stored in the remote cluster in the same namespace as the HelmRelease exists in the local cluster, for the same reason that this is done already while targeting the local cluster (isolation of admin vs tenant namespace permissions). One would of course need to care for creating this namespace in the remote cluster ahead of time. If there were a desire to have the storageNamespace differ from the HelmRelease namespace, such a field could be added to HelmRelease to enable this.

It could make a lot of sense to run one's image/chart version update automation (#107) within such a "management cluster" as well.

[stefanprodan]

I agree with you, I think we should implement remote clusters reconciliation in helm-controller.

[stefanprodan]

Implemented in helm-controller, released in flux2 v0.3.0