Welcome to my HomeOps setup. Here I perform DevOps best practicies but at home. Checkout the hardware section where I describe what sort of hardware I am using. Thanks to terraform and ansible it's very easy for me to manage my home infrastructure and the cluster. Under the folder provision
I store all my ansible and terraform scripts my infrastructure. Some of them are used for k3s cluster some only for docker instances.
I deciced for my cluster that instead of high IO load using etcd, I am using the postgres database for storing critical data for my cluster and maintain it in Hight Availibity mode. That's just in case you would like to copy my cluster configuration, then keep this note in mind. Here is great guide from Devin's template which can help to spin up your own cluster at home πͺ.
- calico - CNI (container network interface)
- echo-server - REST Server Tests (Echo-Server) API (useful for debugging HTTP issues)
- longhorn - storage class provider for data persistence (yeah I'm giving longhorn second chance)
- k8s_gateway - DNS resolver for all types of external Kubernetes resources
- flux - GitOps tool for deploying manifests from the
cluster
directory - metallb - bare metal load balancer
- kube-vip - layer 2 load balancer for the Kubernetes control plane
- cert-manager - SSL certificates - with Cloudflare DNS challenge
- traefik: Provides ingress cluster services.
- botkube Bot that helps me to monitor the cluster with notifications.
- hajimari - start page with ingress discovery
- reflector - mirror configmaps or secrets to other Kubernetes namespaces
- reloader - restart pods when Kubernetes
configmap
orsecret
changes - system-upgrade-controller - upgrade k3s
- SOPS: Encrypts secrets which is safe to store - even to a public repository.
Following tools I use to setup infrastructure:
- Ubuntu - a pretty universal operating system that supports running all kinds of home related workloads in Kubernetes
- Ansible - tool I use for configuration ubuntu nodes and also to install k3s
- Terraform - in order to help with the DNS settings and setup VM where I run k3s I use terraform
My homelab runs on the following hardware (all k8s nodes are running on ubuntu 20.04):
Device | OS Disk Size | Data Disk Size | Ram | Purpose |
---|---|---|---|---|
Udoo Bolt V8 AMD Ryzen | 250GB NVMe | N/A | 32GB | k3s node |
Intel NUC | 250GB NVMe | 1TB HDD | 32GB | k3s node |
AMD GPU Server | 250GB NVMe | 1TB SSD | 32GB | k3s node with nvidia GPU |
TRUENAS | ZFS raidz 1 40TB | 4x10TB HDD | 32GB | Storage |
Unifi UDM Pro | SSD 14GB | HDD 1TB | 4GB | Router and security Gateway |
Unifi Switch 16 PoE | N/A | N/A | N/A | Switch with 802.3at PoE+ ports |
Offsite Machine | 60 GB | 8TB | 8GB | for backups and storage datababase,redis etc |
I've made a blog post about HW, what were my choices... which ones were good and which ones were bad. Click here.
I am proud to be memeber k8s-at-home community! A lot of help and inspiration for my k8s cluster came from this community. Without their help and inspiration I would not be where I am now. Thank you guys β€οΈ. Definately checkout their clusters - awesome-home-kubernetes and join the community.