Kube Infra

Homelab based on Kubernetes

• Router: OPNsense
• Hypervisor: Proxmox
• Storage Server: Debian + Snapraid/MergerFS
• K8s Distribution: Talos
• Load Balancer: Metallb
• Ingress Controller: Traefik
• GitOps Toolkit: Flux CD
• Infra Tools: Packer, Terraform, Ansible
• Secrets Management: SOPS

Installation

Install the local tools yay -S flux-bin sops age terraform packer talosctl crane cilium-cli hubble-bin k9s ansible

Configure git hooks git config core.hooksPath scripts/hooks

Configuring Storage Server

To install and configure all the required packages for the storage server, go into kirito/ansible and run

ansible-galaxy collection install community.sops

ansible-playbook -i inventory ./install.yaml

Building Cloud Init Images

To generate the cloud-init images for proxmox, go in asuna/packer and run

packer init -upgrade .

packer build -var pm_password="" -force .

You can also restrict the build to a specific image (ex: -only=release.proxmox-iso.talos to skip nvidia image build)

Nvidia specific

Make sure that you have docker and crane installed.

You may need to edit your daemon configuration to add these features:

{
  "features": {
    "containerd-snapshotter": true
  },
  "experimental": true
}

To generate nvidia custom disk image (used by packer), run cd asuna/talos-nvidia && ./build.sh

Deploying Cluster

To create the VMs, install talos and bootstrap the cluster with flux, go in asuna/terraform and run:

terraform init

terraform apply

You can get the kubeconfig file with terraform output -raw kubeconfig > ~/.kube/config