Skip to content

Latest commit

 

History

History
226 lines (167 loc) · 8.09 KB

README.md

File metadata and controls

226 lines (167 loc) · 8.09 KB
Raw

terraform-eks-cluster

Deploy a full managed AWS EKS cluster with Terraform

What resources are created

  1. VPC
  2. Internet Gateway (IGW)
  3. Public and Private Subnets
  4. Security Groups, Route Tables and Route Table Associations
  5. IAM roles, instance profiles and policies
  6. An EKS Cluster
  7. Autoscaling group and Launch Configuration
  8. Worker Nodes in a private Subnet
  9. bastion host for ssh access to the VPC
  10. The ConfigMap required to register Nodes with EKS
  11. KUBECONFIG file to authenticate kubectl using the aws eks get-token command. needs awscli version 1.16.156 >

Configuration

You can configure you config with the following input variables:

Name Description Default
cluster-name The name of your EKS Cluster eks-cluster
aws-region The AWS Region to deploy EKS us-east-1
availability-zones AWS Availability Zones ["us-east-1a", "us-east-1b", "us-east-1c"]
k8s-version The desired K8s version to launch 1.13
node-instance-type Worker Node EC2 instance type m4.large
root-block-size Size of the root EBS block device 20
desired-capacity Autoscaling Desired node capacity 2
max-size Autoscaling Maximum node capacity 5
min-size Autoscaling Minimum node capacity 1
public-min-size Public Node groups ASG capacity 1
public-max-size Public Node groups ASG capacity 1
public-desired-capacity Public Node groups ASG capacity 1
vpc-subnet-cidr Subnet CIDR 10.1.2.3/16
private-subnet-cidr Private Subnet CIDR ["10.1.2.3/19", "10.1.32.0/19", "10.1.64.0/19"]
public-subnet-cidr Public Subnet CIDR ["10.1.128.0/20", "10.1.144.0/20", "10.1.160.0/20"]
db-subnet-cidr DB/Spare Subnet CIDR ["10.1.192.0/21", "10.1.200.0/21", "10.1.208.0/21"]
eks-cw-logging EKS Logging Components ["api", "audit", "authenticator", "controllerManager", "scheduler"]
ec2-key EC2 Key Pair for bastion and nodes my-key

You can create a file called terraform.tfvars or copy [variables.tf] into the project root, if you would like to over-ride the defaults.

How to use this example

git clone https://github.com/snigdhasambitak/terraform-k8s-eks-cluster
cd terraform-k8s-eks-cluster

Remote Module

NOTE use version = "2.0.0" with terraform 0.12.x > and version = 1.0.4 with terraform < 0.11x

You can use this module as a remote source:

module "eks" {
  source  = "/modules/eks"

  aws-region          = "us-east-1"
  availability-zones  = ["us-east-1a", "us-east-1b", "us-east-1c"]
  cluster-name        = "my-cluster"
  k8s-version         = "1.13"
  node-instance-type  = "t3.medium"
  root-block-size     = "40"
  desired-capacity    = "3"
  max-size            = "5"
  min-size            = "1"
  public-min-size     = "0" # setting to 0 will create the launch config etc, but no nodes will deploy"
  public-max-size     = "0"
  public-desired-capacity = "0"
  vpc-subnet-cidr     = "10.1.2.3/16"
  private-subnet-cidr = ["10.1.2.3/19", "10.1.32.0/19", "10.1.64.0/19"]
  public-subnet-cidr  = ["10.1.128.0/20", "10.1.144.0/20", "10.1.160.0/20"]
  db-subnet-cidr      = ["10.1.192.0/21", "10.1.200.0/21", "10.1.208.0/21"]
  eks-cw-logging      = ["api", "audit", "authenticator", "controllerManager", "scheduler"]
  ec2-key             = "my-key"
}

output "kubeconfig" {
  value = module.eks.kubeconfig
}

output "config-map" {
  value = module.eks.config-map
}

IAM

The AWS credentials must be associated with a user having at least the following AWS managed IAM policies

  • IAMFullAccess
  • AutoScalingFullAccess
  • AmazonEKSClusterPolicy
  • AmazonEKSWorkerNodePolicy
  • AmazonVPCFullAccess
  • AmazonEKSServicePolicy
  • AmazonEKS_CNI_Policy
  • AmazonEC2FullAccess

In addition, you will need to create the following managed policies

EKS

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "eks:*"
            ],
            "Resource": "*"
        }
    ]
}

Create a EKS cluster in AWS

Step 1: Create the modules for the creation of the eks cluster(eks-demo), master nodes, VPC and providers. Everything can be found in the root directory. the modules path is : "modules/eks/"

Step 2: Initialize the Terraform state:

$ terraform init

Step 3: Plan the Deployment

$ terraform plan -out EKS-cluster-tf

Step 4: We can apply the plan now

$ terraform apply "EKS-cluster-tf"

Step 5: Create the KubeConfig file and save it, that will be used to manage the cluster.

$ terraform output kubeconfig

$ terraform output kubeconfig > ${HOME}/.kube/config-EKS-cluster-tf

Step 6: Add this new config to the KubeCtl Config list.

$ export KUBECONFIG=${HOME}/.kube/config-EKS-cluster-tf:${HOME}/.kube/config $ echo "export KUBECONFIG=${KUBECONFIG}" >> ${HOME}/.bashrc

Authorize worker nodes

Get the config from terraform output, and save it to a yaml file:

terraform output config-map > config-map-aws-auth.yaml

Configure aws cli with a user account having appropriate access and apply the config map to EKS cluster:

kubectl apply -f config-map-aws-auth.yaml

You can verify the worker nodes are joining the cluster

kubectl get nodes --watch

Authorize users to access the cluster

Initially, only the system that deployed the cluster will be able to access the cluster. To authorize other users for accessing the cluster, aws-auth config needs to be modified by using the steps given below:

  • Open the aws-auth file in the edit mode on the machine that has been used to deploy EKS cluster:
sudo kubectl edit -n kube-system configmap/aws-auth
  • Add the following configuration in that file by changing the placeholders:
mapUsers: |
  - userarn: arn:aws:iam::111122223333:user/<username>
    username: <username>
    groups:
      - system:masters

So, the final configuration would look like this:

apiVersion: v1
data:
  mapRoles: |
    - rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-NodeInstanceRole-74RF4UBDUKL6
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
  mapUsers: |
    - userarn: arn:aws:iam::111122223333:user/<username>
      username: <username>
      groups:
        - system:masters
  • Once the user map is added in the configuration we need to create cluster role binding for that user:
kubectl create clusterrolebinding ops-user-cluster-admin-binding-<username> --clusterrole=cluster-admin --user=<username>

Replace the placeholder with proper values

Cleaning up

You can destroy this cluster entirely by running:

terraform plan -destroy
terraform destroy  --force