disposable-k8s-cluster

This repository contains a set of scripts to create a disposable kubernetes cluster on one of supported cloud providers.

General information

This project is intended to be used as a playground for testing and learning purposes. It is not intended to be used in production.

It was created in the way that should support extending it with new cloud providers and new applications.

Currently supported bootstrap options:

• DigitalOcean
• kind - not a cloud provider, but a tool to create a kubernetes cluster on a local machine

Additionally, to simplify access to the cluster, we are deploying external-dns and cert-manager to the cluster.

Those two controllers will be used to automatically create DNS records and TLS certificates for the ingress resources in the cluster.

Note

In case the certificates and dns records are not required, the project can be bootstrapped with SKIP_EXPOSE=true

Default applications

The following applications (controllers) are deployed to the cluster by default:

• ArgoCD - a declarative, GitOps continuous delivery tool for Kubernetes
• cert-manager - used to automatically create TLS certificates for the ingress resources
• external-dns - used to automatically create DNS records for the ingress resources
• reflector - used for copying objects between namespaces ¹
• traefik - used as an ingress controller

Prerequisites

Mandatory

• OpenTofu - used to create the cluster and deploy default applications
• tfswitch - used to ensure that the correct version of tofu is used
• GNU make - used to orchestrate the deployment
• kubectl - used to interact with the cluster

Optional

• direnv - used to automatically load environment variables from .env or .envrc files

Configuration

A list of environment variables that can be used to configure the deployment:

Variable	Description	Required	Notes
TF_VAR_do_token	DigitalOcean API token (required only when using DigitalOcean as a cloud provider)	No
TF_VAR_cloudflare_api_token	Cloudflare API token (required for generating TLS certificates and DNS records)	Yes	Unless SKIP_EXPOSE=true
KUBECONFIG	Path to the kubeconfig file (should be set to ./kubeconfig)	Yes
DISPOSABLE_DOMAIN	Domain that will be used to create DNS records and TLS certificates	Yes
SKIP_EXPOSE	Whether external-dns and cert-manager deployment should be skipped	No

Usage

To set up a temporary kubernetes cluster, including infrastructure and common services run:

make bootstrap

To destroy ephemeral kubernetes cluster run:

make destroy

To stop kind cluster run:

make stop

To start kind cluster run:

make start

Reflector

One of potential use cases would be to configure automatic copy of docker-registry secret across all namespaces:

kubectl create secret docker-registry regcred \
--docker-server=registry.example.com \
--docker-username=user \
--docker-password=verysecretsecret

And later on add the following annotation to the regcred secret:

reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-auto-enabled: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: ""

Contributing

Contributions are welcome! Feel free to submit a pull request. For major changes, please open an issue first to discuss.