Virtual Cluster as a Service - Phase 1

[waveywaves]

Tell us about your request
This feature request outlines a service which will provide virtual cluster as an environment for Uffizzi users.

This phase of development covers

• the creation of the virtual cluster
• and using a straightforward Github Actions / GitLab reusable workflow to create resources from a PR in the Virtual Cluster

Which service(s) is this request for?
Self Hosted Open Source

Describe the solution you'd like

• For creation of the Virtual Cluster there are two ways we can go about this.

1. Make the uffizzi CLI run vcluster commands as follows. This is really good for prototyping and initial demoing but not good for building on top of VCluster.

# Create a virtual cluster 
uffizzi cluster create my-uffizzi-cluster
# Corresponding vcluster command
vcluster create my-uffizzi-cluster

# Connect to a virtual cluster 
uffizzi cluster connect my-uffizzi-cluster
# Corresponding vcluster command
vcluster connect my-uffizzi-cluster

2. Use the Uffizzi Cluster Operator with which we can create a UffizziCluster resource. The operator will then deploys a VCluster Helm Chart with all necessary values already added in to the cluster. Right now the operator is still being developed but as of now, a UffizziCluster can be created with necessary helm charts installed and ingress enabled. The CLI commands will look like the following. And this is considering that the user has already logged in.

# Create a virtual cluster
uffizzi cluster create my-uffizzi-cluster

An API request will be called on the application side and the Uffizzi controller will create a UffizziCluster resource which will be reconciled by the Uffizzi Cluster Operator https://github.com/UffizziCloud/uffizzi-cluster-operator

# Connect to the virtual cluster
uffizzi cluster connect my-uffizzi-cluster

The address for the above vcluster can be found in the status of the UffizziCluster resource. We will be running the corresponding vcluster command for connecting to the cluster.

# Corresponding vcluster command
vcluster connect my-uffizzi-cluster --server=<address retrieved from the UffizziCluster resource>

This will ensure that the VCluster is created and the CLI can connect to it.

• For executing helm charts in the cluster
  Once the cluster is up and the CLI has access to it, we are going to create kubernetes resources in the cluster which the user can use. The vcluster CLI has the ability to download the kubeconfig. We can pass this kubeconfig in any Kubernetes CLI application and create resources in the VCluster.

[moklidia]

@waveywaves Some questions:

1. What problem does it solve?
2. Do we plan for the virtual clusters to be created in the Uffizzi cluster or self-hosted clusters?
3. Will the helm chart be provided by users?
4. Can this library be installed as a controller dependency?

[waveywaves]

Hey @moklidia

1. What problem does it solve?
   Currently we create previews from docker-compose and can create previews of applications containing more than one services. The scale of these applications is rather small when compared to applications which need to deployed with Helm or Kubernetes. With Virtual Clusters we can create Kubernetes clusters without having to deploy hardware resources to create a new cluster. Checkout how VClusters work over here https://www.youtube.com/watch?v=gQ-KG57ruvY

2. Do we plan for the virtual clusters to be created in the Uffizzi cluster or self-hosted clusters?
   Ideally both. Virtual clusters created in the Uffizzi cluster will be managed by us. The ones created in self-hosted clusters can probably get support from us for a fee.

3. Will the helm chart be provided by users?
   Yes and no. The Helm chart can be provided by the users but we will also have an artifact registry at some point to support pipelines. There, we can build an image from a git repo, push it to our image registry and then use that image in a Helm Chart. This helm chart will be pushed to our artifact registry (need to host one). This chart can now be used in a new VCluster deployment with the helm chart. This will work a lot like our reusable workflow. We should ideally have some links to some popular helm charts as well.

4. Can this library be installed as a controller dependency?
   Currently it is a standalone operator/controller. In theory it can run as a controller dependency. The uffizzi controller can create the UffizziCluster resource and relay it's status back to the user whereas the uffizzi cluster operator can do the reconcilation and heavy lifting of orchestrating all necessary resources for the UffizziCluster.

[gadkins]

User Story we want to support:

User story 1 (Platform maintainers): virtual cluster as a service (self-hosted)

1. As a platform engineering team leader, I want an off-the-shelf solution to provide ephemeral test environments to my application developers
2. I install Uffizzi (via helm chart?) onto a dedicated cluster
3. And add the Uffizzi reusable workflow for virtual clusters to my GitHub/GitLab pipeline
4. On merge requests, my pipeline builds images, pushes them to a private registry, after which a Charts/ directory / values.yaml file is updated
5. Uffizzi job executes:
6. Uffizzi CLI runs uffizzi cluster create
7. This saves a kubeconfig file (and also optionally dumps this file to stdout)
   Given the kubeconfig + ``Charts/directory / values.yaml` my pipeline creates a Helm Release
8. My pipeline then posts a comment to my merge request issue with the paths and/or relevant objects returned from the helm release
9. When a merge request is merged/closed, a uffizzi job deletes the virtual cluster via uffizzi cluster delete [CLUSTER_ID]

[zipofar]

Does this make sense?

[moklidia]

Can we support a directory of Helm charts?

[moklidia]

@waveywaves Please take a look at the chart above: #413 (comment)

[jpthurman]

@waveywaves Did you provide feedback to @moklidia

[waveywaves]

@moklidia This looks good. We can go ahead with this design. The below are some notes regarding some clarifications with respect to each section in the diagram

• Start creating vcluster
  ✔️

• Watching the creation of the cluster
  Here we can watch the UffizziCluster object status which will let the platform know if the UffizziCluster is ready or not

• User wants to see logs of container
  The pods created by the vcluster are created in the same namespace where the UffizziCluster object is created. The names of the pods will go by the formula -x--x- eg: pod1-x-ns1-x-uc-vcluster1