enterprise-grade-k8s-local

Pseudo Enterprise Grade Kubernetes - On a Laptop

⚠️ Not Actually Enterprise Grade

The problem

Running a single Kubernetes cluster at home just isn't interesting enough. Sure, you may have moved from nodeport services up to MetalLB, and thanks to an extra couple hard drives and Longhorn you've got storage that doesn't require pinning Pods to a specific node. Heck, you've probably even decided enough is enough and you've gotta have something aside from your bash history keeping track of your Helm installs, prompting you to get most of your stack redefined within Helmfile.

But now, aside from giving ever more resources to your kids creative Minecraft server, there's not much to do except kick back and enjoy the clean browsing experience your PiHole deployment ensures while Plex streams some Futurama episodes to the TV.

The solution

It's time to take it to the next level. No more running a quick command to update a Helm release for us; we're implementing change management. No longer will your media applications blemish the same cluster as your infrastructure! We're splitting out your infra, prod, and test environments to their own clusters.

To keep this as approachable as possible, this lab is designed to run on a single machine. If your laptop happens to melt while chewing through this well... Don't blame me, blame the manufacturers chasing paper thin and feather light laptops instead of giving better battery life and usable keyboards.

The architecture

pre-requisites A half-way decent workstation.

Ensure you have the following installed FIRST:

Docker : We're running containers in containers so make sure you install Docker first!

k3d : A handy "little helper", k3d creates containerized k3s clusters - basically, use Docker containers as cluster nodes.

kubectl : You're going to need kubectl installed so you can run commands on your shiny new clusters.

---

We'll spin up 3 clusters, each with their own set of "nodes". Each will expose the Kubernetes API to a separate port on localhost. Most direct interaction will be with ArgoCD running on the infra cluster, allowing it to handle the configuration of the prod and test clusters.

Each cluster will run MetalLB in ARP mode (BGP is nicer, but requires support in your home network). Optionally, you can port-forward :80 and :443 to your prod cluster for external Ingress, or use a public hosted VPS along with a tunnel. Bonus points to skip either and take advantage of Argo Tunnels from Cloudflare (they're free!).

Finally, we will setup LinkerD & deploy the test application EmojiVoto to get some "live" traffic.

The result should be a questionably stable environment & a ton of fun!

The tools

k3s : For IoT, Edge computing, and just screwing around; k3s is a full-on certified Kubernetes distribution that will run anywhere.

k3d : A handy "little helper", k3d creates containerized k3s clusters - basically, use Docker containers as cluster nodes.

Terraform : From HashiCorp, Terraform is a command line tool designed to turn YAML into Cost (aka, Infrastructure as Code).

ArgoCD : It's got a cute Octopus mascot and the docs say "GitOps" a bunch -- ArgoCD keeps your running Kubernetes cluster in sync with the resource definitions stored in git (and yes, it's YAML all the way down).

Lets do this!

Terraform

Just build the things man!

cd terraform && terraform init && terraform apply

Setup initial pieces

kubectl --kubeconfig terraform/infra*.kubeconfig apply -k argocd/argocd  # terraform can do this
kubectl --kubeconfig terraform/infra*.kubeconfig apply -k app_manager/infra
# do ^ for each cluster to get the common bits sync'd

At this point we can

kubectl --kubeconfig ../terraform/infra-local-5a413c1635.kubeconfig \
  -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d

kubectl --kubeconfig ../../terraform/infra-local-5a413c1635.kubeconfig \
  -n argocd port-forward argocd-server-6f4fcdc5dc-czhnw 8080:8080

Login with admin and the password provided.

Custom DNS nameserver

If you want to use a different nameserver other than the default 8.8.8.8, you can do so by modifying the coredns configmap

# This gives you the current configmap in a yaml file
kubectl get configmaps coredns -n kube-system -o yaml > coredns_configmap.yaml

# Edit the file. Change 'forward . /etc/resolv.conf' to 'forward . <IP or FQDN of NS>'
# Then apply the modified file to update the configmap
kubectl apply -f coredns_configmap.yaml

## err, probably need to configure docker networks somewhere...