Skip to content
/ hcloud-k8s-ctl Public

High available Kubernetes cluster on Hetzner Cloud with Autoscaling

License

Apache-2.0 license
6 stars 2 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

maksim-paskal/hcloud-k8s-ctl

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

114 Commits
.github/workflows
.github/workflows
 
 
cmd
cmd
 
 
e2e
e2e
 
 
examples/charts/test
examples/charts/test
 
 
internal
internal
 
 
pkg
pkg
 
 
scripts
scripts
 
 
utils
utils
 
 
.gitignore
.gitignore
 
 
.golangci.yml
.golangci.yml
 
 
.goreleaser.yml
.goreleaser.yml
 
 
LICENSE
LICENSE
 
 
Makefile
Makefile
 
 
README.md
README.md
 
 
config.yaml
config.yaml
 
 
go.mod
go.mod
 
 
go.sum
go.sum
 
 

Repository files navigation

High available Kubernetes cluster on Hetzner Cloud with Autoscaling

Motivation

AWS has eksctl tool for creating kubernetes cluster - Hetzner Cloud have no official tool for creating kubernetes cluster. This tool will create new production ready kubernetes clusters on Hetzner Cloud with minimum user interaction. New cluster will be available in High availability mode with automatic cluster autoscaling and automatic volume creation

Preparation

  • login to https://console.hetzner.cloud and create new project
  • select project, select in menu Security -> API Tokens and create new "Read & Write" token
  • save token to .hcloudauth file in current directory

Install binnary

MacOS

brew install maksim-paskal/tap/hcloud-k8s-ctl

for other OS download binnary from release pages

Create kubernetes cluster

This will create kubernetes cluster in Hetzner Cloud Europe region with 3 instances, 1 load balancer for the kubernetes control plane and 1 kubernetes worker node, after successful installation the cluster will have:

for HA needs odd number of master nodes (minimum 3) https://etcd.io/docs/v3.4/faq/#why-an-odd-number-of-cluster-members

Create a simple configuration file config.yaml full configuration example here

# Kubeconfig path
kubeConfigPath: ~/.kube/hcloud
# Hetzner Cloud internal network CIDR
ipRange: "10.0.0.0/16"
# Servers for Kubernetes master (recommended 3)
# For development purposes, the cluster can have 1 master node  
# In this case, the cluster will be created without a load balancer, and pods can be scheduled on the master
masterCount: 3

customize configuration file for your needs

# Kubeconfig path
kubeConfigPath: ~/.kube/hcloud
# Hetzner Cloud internal network CIDR
ipRange: "10.0.0.0/16"
# Servers for Kubernetes master (recommended 3)
# For development purposes, the cluster can have 1 master node  
# In this case, the cluster will be created without a load balancer, and pods can be scheduled on the master
masterCount: 3
# Server components for all nodes in the cluster
serverComponents:
  kubernetes:
    # Customize Kubernetes version
    version: 1.30.5-1.1
  docker:
    # Customize apt package version for Docker install
    # apt-cache madison docker-ce
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    # Customize apt package version for containerd install
    # apt-cache madison containerd.io
    version: 1.7.22-1
# Add autoscaler chart extra values
cluster-autoscaler:
  replicaCount: 3
  resources:
    requests:
      cpu: 200m
      memory: 300Mi
# Add a custom script for all nodes in the cluster
preStartScript: |
  # Add a custom cron job on the node
  crontab <<EOF
  0 0 * * * /usr/bin/docker system prune -af
  EOF

  # Add containerd config for some registries
  mkdir -p /etc/containerd/certs.d/some-registry.io
  cat > /etc/containerd/certs.d/some-registry.io/hosts.toml <<EOF
  server = "https://some-registry.io"

  [host."http://10.10.10.10:5000"]
  capabilities = ["pull", "resolve"]
  EOF
Kubernetes: v1.28, Arch: amd64, Ubuntu: 20.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: x86
  kubernetes:
    version: 1.28.2-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
Kubernetes: v1.28, Arch: amd64, Ubuntu: 22.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: x86
  kubernetes:
    version: 1.28.2-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
Kubernetes: v1.28, Arch: amd64, Ubuntu: 22.04, Region: us 
masterCount: 3
networkZone: us-east
location: ash
datacenter: ash-dc1
masterServers:
  servertype: cpx21
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: x86
  kubernetes:
    version: 1.28.2-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
cluster-autoscaler:
  autoscalingGroups:
  - name: cpx51-ash
    minSize: 1
    maxSize: 20
    instanceType: cpx51
    region: ash
Kubernetes: v1.28, Arch: arm64, Ubuntu: 20.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: arm
  kubernetes:
    version: 1.28.2-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1
Kubernetes: v1.28, Arch: arm64, Ubuntu: 22.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: arm
  kubernetes:
    version: 1.28.2-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1
Kubernetes: v1.29, Arch: amd64, Ubuntu: 20.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: x86
  kubernetes:
    version: 1.29.3-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
Kubernetes: v1.29, Arch: amd64, Ubuntu: 22.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: x86
  kubernetes:
    version: 1.29.3-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
Kubernetes: v1.29, Arch: amd64, Ubuntu: 22.04, Region: us 
masterCount: 3
networkZone: us-east
location: ash
datacenter: ash-dc1
masterServers:
  servertype: cpx21
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: x86
  kubernetes:
    version: 1.29.3-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
cluster-autoscaler:
  autoscalingGroups:
  - name: cpx51-ash
    minSize: 1
    maxSize: 20
    instanceType: cpx51
    region: ash
Kubernetes: v1.29, Arch: arm64, Ubuntu: 20.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: arm
  kubernetes:
    version: 1.29.3-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1
Kubernetes: v1.29, Arch: arm64, Ubuntu: 22.04, Region: eu 
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: arm
  kubernetes:
    version: 1.29.3-1.1
  docker:
    version: 5:24.0.6-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.6.24-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1
Kubernetes: v1.30, Arch: amd64, Ubuntu: 20.04, Region: eu 
ipRange: "10.0.0.0/16"
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: x86
  kubernetes:
    version: 1.30.5-1.1
  docker:
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.7.22-1
Kubernetes: v1.30, Arch: amd64, Ubuntu: 20.04, Region: us 
ipRange: "10.0.0.0/16"
masterCount: 3
networkZone: us-east
location: ash
datacenter: ash-dc1
masterServers:
  servertype: cpx21
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: x86
  kubernetes:
    version: 1.30.5-1.1
  docker:
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.7.22-1
cluster-autoscaler:
  autoscalingGroups:
  - name: cpx51-ash
    minSize: 1
    maxSize: 20
    instanceType: cpx51
    region: ash
Kubernetes: v1.30, Arch: amd64, Ubuntu: 22.04, Region: eu 
ipRange: "10.0.0.0/16"
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: x86
  kubernetes:
    version: 1.30.5-1.1
  docker:
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.7.22-1
Kubernetes: v1.30, Arch: arm64, Ubuntu: 20.04, Region: eu 
ipRange: "10.0.0.0/16"
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-20.04
    architecture: arm
  kubernetes:
    version: 1.30.5-1.1
  docker:
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.7.22-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1
Kubernetes: v1.30, Arch: arm64, Ubuntu: 22.04, Region: eu 
ipRange: "10.0.0.0/16"
masterCount: 3
serverComponents:
  ubuntu:
    version: ubuntu-22.04
    architecture: arm
  kubernetes:
    version: 1.30.5-1.1
  docker:
    version: 5:27.3.1-1~ubuntu.$(lsb_release -rs)~$(lsb_release -cs)
  containerd:
    version: 1.7.22-1
masterServers:
  servertype: cax11
cluster-autoscaler:
  autoscalingGroups:
  - name: cax-fsn1
    minSize: 1
    maxSize: 20
    instanceType: cax41
    region: fsn1

# create 3 instance with 1 load balancer
# kubernetes autoscaler will create 1 worker node
hcloud-k8s-ctl -action=create

all nodes in cluster initialized with official kubeadm - for all nodes use this script, for master initializing this script, for initial applications in cluster this script

Access to cluster

export KUBECONFIG=$HOME/.kube/hcloud

kubectl get no

Patch already created cluster

hcloud-k8s-ctl -action=patch-cluster

List available location/datacenter/servertype at Hezner

hcloud-k8s-ctl -action=list-configurations

Delete already created cluster

hcloud-k8s-ctl -action=delete

To install NFS provisioner

You can easy install NFS provisioner for your cluster adding to your config.yaml next lines

deployments:
  nfs:
    nfs-subdir-external-provisioner:
      enabled: true
    server:
      enabled: true

It will install NFS Provisioner for Kubernetes (optional) with NFS Server and Storage Class

you can easy create new NFS volumes to your pod with this PersistentVolumeClaim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nfs
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: nfs

About

High available Kubernetes cluster on Hetzner Cloud with Autoscaling

Topics

kubernetes kubernetes-cluster k8s autoscaling hetzner-cloud

Resources

Readme

License

Apache-2.0 license
Activity

Stars

6 stars

Watchers

3 watching

Forks

2 forks
Report repository

Releases 3

v0.3.7 Latest
Oct 3, 2024
+ 2 releases

Packages

No packages published

Contributors 2

Languages