Skip to content

Commit

Permalink
Cert-manager - refactor (#326)
Browse files Browse the repository at this point in the history
  • Loading branch information
sekka1 authored Jun 17, 2022
1 parent d1416fe commit 357d278
Show file tree
Hide file tree
Showing 6 changed files with 211 additions and 54 deletions.
51 changes: 50 additions & 1 deletion terraform-environments/aws/terragrunt-dev/progress.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,8 +151,57 @@ PR for setting the 200-eks terragrunt to the release tag: https://github.com/Man
* It was on the branch for the module before so that I can test it out without having to merge and release the eks updated module

# 100-cert-manager
TestKube is dependent on cert-manager for it's internal usage

Cert manager was failing:

```
│Error: Failed to determine GroupVersionResource for manifest
│ with kubernetes_manifest.dns01_cluster_issuer[0],
│ on main.tf line 107, in resource "kubernetes_manifest" "dns01_cluster_issuer":
│ 107: resource "kubernetes_manifest" "dns01_cluster_issuer" {
│no matches for kind "ClusterIssuer" in group "cert-manager.io"
│Error: Failed to determine GroupVersionResource for manifest
│ with kubernetes_manifest.http01_cluster_issuer[0],
│ on main.tf line 131, in resource "kubernetes_manifest" "http01_cluster_issuer":
│ 131: resource "kubernetes_manifest" "http01_cluster_issuer" {
│no matches for kind "ClusterIssuer" in group "cert-manager.io"
time=2022-06-16T22:56:58Z level=error msg=1 error occurred:
* exit status 1
```

The cert-manager module does have a wait for the cert-manager helm chart to be installed first
then this error is trying to apply the cert-manager's CRDs for the ClusterIssuer which tells
cert-manager how you want to validate the Let's Encrypt certs like use the DNS and add a record there.


Reading the `kubernetes_manifest` doc:

https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest

Right at the top it does say that this will access to your kube API and try this out even
during the plan stage. This is why it is failing. This behavior did change b/c this module
was working before.

The idea now is to separate out the cert-manager helm chart install and then have another
module to apply the cert-manager's issuers.

The PR to separate this out to two modules:https://github.com/ManagedKube/kubernetes-ops/pull/326

This PR: https://github.com/ManagedKube/kubernetes-ops/pull/327
* Applies the cert-manager helm install
* no issuers

This PR: https://github.com/ManagedKube/kubernetes-ops/pull/328
* Applies the cert-manager-issuer items

* TestKube is dependent on cert-manager for it's internal usage



Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# doc: https://cert-manager.io/docs/configuration/acme/dns01/route53/
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod-dns01
spec:
acme:
server: ${letsEncryptServer}
email: ${emailAddress}
privateKeySecretRef:
name: dns01-issuer-account-key
solvers:

# example: cross-account zone management for example.com
# this solver uses ambient credentials (i.e. inferred from the environment or EC2 Metadata Service)
# to assume a role in a different account
- selector:
dnsZones:
- "${domainName}"
dns01:
route53:
region: ${awsRegion}
hostedZoneID: ${dnsZhostedZoneIDone} # optional, see policy above
# role: "arn:aws:iam::${awsAccountID}:role/${chartName}-${clusterName}"

# this solver handles example.org challenges
# and uses explicit credentials
# - selector:
# dnsZones:
# - "example.org"
# dns01:
# route53:
# region: eu-central-1
# accessKeyID: AKIAIOSFODNN7EXAMPLE
# secretAccessKeySecretRef:
# name: prod-route53-credentials-secret
# key: secret-access-key
# # you can also assume a role with these credentials
# role: arn:aws:iam::YYYYYYYYYYYY:role/dns-manager
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# doc: https://cert-manager.io/docs/configuration/acme/http01/
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod-http01
spec:
acme:
server: ${letsEncryptServer}
email: ${emailAddress}
privateKeySecretRef:
name: http01-issuer-account-key
solvers:
- http01:
ingress:
class: ${ingressClass}
50 changes: 50 additions & 0 deletions terraform-modules/aws/helm/cert-manager-issuers/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
locals {
official_chart_name = "cert-manager"
}

data "aws_caller_identity" "current" {}

#############################
# cert-manager DNS01 cluster-issuer
# https://cert-manager.io/docs/configuration/acme/dns01/route53/#creating-an-issuer-or-clusterissuer
#############################
# file templating
data "template_file" "dns01_cluster_issuer_yaml" {
template = file("${path.module}/dns01-cluster-issuer.tpl.yaml")

vars = {
awsRegion = var.aws_region
letsEncryptServer = var.lets_encrypt_server
emailAddress = var.lets_encrypt_email
dnsZhostedZoneIDone = var.route53_hosted_zones
domainName = var.domain_name
awsAccountID = data.aws_caller_identity.current.account_id
clusterName = var.cluster_name
chartName = local.official_chart_name
}
}

resource "kubernetes_manifest" "dns01_cluster_issuer" {
count = var.enable_dns01_cluster_issuer
manifest = yamldecode(data.template_file.dns01_cluster_issuer_yaml.rendered)
}

#############################
# cert-manager HTTP01 cluster-issuer
# https://cert-manager.io/docs/configuration/acme/http01/#configuring-the-http01-ingress-solver
#############################
# file templating
data "template_file" "http01_cluster_issuer_yaml" {
template = file("${path.module}/http01-cluster-issuer.tpl.yaml")

vars = {
emailAddress = var.lets_encrypt_email
letsEncryptServer = var.lets_encrypt_server
ingressClass = var.ingress_class
}
}

resource "kubernetes_manifest" "http01_cluster_issuer" {
count = var.enable_http01_cluster_issuer
manifest = yamldecode(data.template_file.http01_cluster_issuer_yaml.rendered)
}
55 changes: 55 additions & 0 deletions terraform-modules/aws/helm/cert-manager-issuers/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@


variable "aws_region" {
type = string
default = "us-east-1"
description = "AWS region"
}

variable "cluster_name" {
type = string
default = "cluster"
description = "EKS cluster name"
}

variable "route53_hosted_zones" {
type = string
default = "*"
description = "The hosted zone permissions granted to: arn:aws:route53:::hostedzone/<route53_hosted_zones ID"
}

variable "ingress_class" {
type = string
default = "nginx-external"
description = "The ingress class that will be used for the http01 resolver to put the inbound check onto"
}

variable "enable_http01_cluster_issuer" {
type = number
default = 1
description = "Enable an http01 cluster issuer"
}

variable "enable_dns01_cluster_issuer" {
type = number
default = 1
description = "Enable an dns01 cluster issuer"
}

variable "lets_encrypt_server" {
type = string
default = "https://acme-v02.api.letsencrypt.org/directory"
description = "The Lets Encrypt validation server to go to. The default is the live one."
}

variable "lets_encrypt_email" {
type = string
default = ""
description = "An email address for cert administration purposes"
}

variable "domain_name" {
type = string
default = "example.com"
description = "The domain name for DNS01 to resolve for"
}
53 changes: 0 additions & 53 deletions terraform-modules/aws/helm/cert-manager/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -83,56 +83,3 @@ module "cert-manager" {
module.iam_assumable_role_admin
]
}

#############################
# cert-manager DNS01 cluster-issuer
# https://cert-manager.io/docs/configuration/acme/dns01/route53/#creating-an-issuer-or-clusterissuer
#############################
# file templating
data "template_file" "dns01_cluster_issuer_yaml" {
template = file("${path.module}/dns01-cluster-issuer.tpl.yaml")

vars = {
awsRegion = var.aws_region
letsEncryptServer = var.lets_encrypt_server
emailAddress = var.lets_encrypt_email
dnsZhostedZoneIDone = var.route53_hosted_zones
domainName = var.domain_name
awsAccountID = data.aws_caller_identity.current.account_id
clusterName = var.cluster_name
chartName = local.official_chart_name
}
}

resource "kubernetes_manifest" "dns01_cluster_issuer" {
count = var.enable_dns01_cluster_issuer
manifest = yamldecode(data.template_file.dns01_cluster_issuer_yaml.rendered)

depends_on = [
module.cert-manager
]
}

#############################
# cert-manager HTTP01 cluster-issuer
# https://cert-manager.io/docs/configuration/acme/http01/#configuring-the-http01-ingress-solver
#############################
# file templating
data "template_file" "http01_cluster_issuer_yaml" {
template = file("${path.module}/http01-cluster-issuer.tpl.yaml")

vars = {
emailAddress = var.lets_encrypt_email
letsEncryptServer = var.lets_encrypt_server
ingressClass = var.ingress_class
}
}

resource "kubernetes_manifest" "http01_cluster_issuer" {
count = var.enable_http01_cluster_issuer
manifest = yamldecode(data.template_file.http01_cluster_issuer_yaml.rendered)

depends_on = [
module.cert-manager
]
}

0 comments on commit 357d278

Please sign in to comment.