Skip to content

Commit

Permalink
chore: Update examples to use terraform-aws-eks module where possib…
Browse files Browse the repository at this point in the history
…le (#1428)

Co-authored-by: Zvikan <[email protected]>
  • Loading branch information
bryantbiggs and Zvikan authored Feb 14, 2023
1 parent 78b796c commit 243ae23
Show file tree
Hide file tree
Showing 210 changed files with 985 additions and 216,964 deletions.
9 changes: 2 additions & 7 deletions .github/scripts/plan-examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,8 @@ def get_examples():
"""
exclude = {
'examples/appmesh-mtls', # excluded until Rout53 is setup
'examples/eks-cluster-with-external-dns', # excluded until Rout53 is setup
'examples/fully-private-eks-cluster/vpc', # skipping until issue #711 is addressed
'examples/fully-private-eks-cluster/eks',
'examples/fully-private-eks-cluster/add-ons',
'examples/ai-ml/ray', # excluded until #887 is fixed,
'examples/crossplane' #example removed
'examples/upgrade/blue-green-route53/core-infra'
'examples/blue-green-upgrade/core-infra',
'examples/blue-green-upgrade/modules/eks_cluster'
}

projects = {
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/e2e-parallel-destroy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,8 @@ jobs:
matrix:
include:
- example_path: examples/agones-game-controller
- example_path: examples/complete-kubernetes-addons
- example_path: examples/eks-cluster-with-new-vpc
- example_path: examples/fargate-serverless
- example_path: examples/gitops/argocd
- example_path: examples/grafana-loki
- example_path: examples/argocd
- example_path: examples/ipv4-prefix-delegation
- example_path: examples/ipv6-eks-cluster
- example_path: examples/karpenter
Expand Down Expand Up @@ -63,4 +60,5 @@ jobs:
terraform init -upgrade=true
terraform destroy -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform destroy -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform destroy -target=module.eks -no-color -input=false -auto-approve
terraform destroy -no-color -input=false -auto-approve
7 changes: 3 additions & 4 deletions .github/workflows/e2e-parallel-full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,8 @@ jobs:
matrix:
include:
- example_path: examples/agones-game-controller
- example_path: examples/complete-kubernetes-addons
- example_path: examples/eks-cluster-with-new-vpc
- example_path: examples/fargate-serverless
- example_path: examples/gitops/argocd
- example_path: examples/grafana-loki
- example_path: examples/argocd
- example_path: examples/ipv4-prefix-delegation
- example_path: examples/ipv6-eks-cluster
- example_path: examples/karpenter
Expand Down Expand Up @@ -105,6 +102,7 @@ jobs:
export AWS_CSM_HOST=127.0.0.1
terraform apply -target=module.vpc -no-color -input=false -auto-approve
terraform apply -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform apply -target=module.eks -no-color -input=false -auto-approve
terraform apply -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform apply -no-color -input=false -auto-approve
Expand All @@ -118,6 +116,7 @@ jobs:
export AWS_CSM_HOST=127.0.0.1
terraform destroy -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform destroy -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform destroy -target=module.eks -no-color -input=false -auto-approve
terraform destroy -no-color -input=false -auto-approve
- name: Fail if TF apply failed
Expand Down
10 changes: 3 additions & 7 deletions docs/add-ons/apache-airflow.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Apache Airflow add-on

This document describes the details of the best practices for building and deploying Self-managed **Highly Scalable Apache Airflow cluster on Kubernetes(Amazon EKS) Cluster**.
Alternatively, Amazon also provides a fully managed Apache Airflow service(MWAA). Please see this [example](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/aws-managed-airflow) if you are looking to build Amazon MWAA.
Alternatively, Amazon also provides a fully managed Apache Airflow service(MWAA).

Apache Airflow is used for the scheduling and orchestration of data pipelines or workflows.
Orchestration of data pipelines refers to the sequencing, coordination, scheduling, and managing complex data pipelines from diverse sources.
Expand Down Expand Up @@ -60,18 +60,15 @@ This deployment example uses internet facing Load Balancer to easily access the
You can modify the `values.yaml` to set the Load Balancer to `internal` and upload certificate to use HTTPS.
Ensure access to the WebUI using internal domain and network.


Checkout the [examples](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) of deploying and using Apache Airflow on Amazon EKS.

## Usage

The [Apache Airflow](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) can be deployed by enabling the add-on via the following.
The Apache Airflow can be deployed by enabling the add-on via the following.

```hcl
enable_airflow = true
```

For production workloads, you can use this [example](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) with custom Helm Config.
For production workloads, you can use the following custom Helm Config.

```hcl
enable_airflow = true
Expand Down Expand Up @@ -116,5 +113,4 @@ airflow-scheduler 2/2 2 2 77m
airflow-statsd 1/1 1 1 77m
airflow-triggerer 1/1 1 1 77m
airflow-webserver 2/2 2 2 77m

```
2 changes: 0 additions & 2 deletions docs/add-ons/argo-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@

For complete project documentation, please visit the [Argo Workflows documentation site](https://argoproj.github.io/argo-workflows/).

[DoEKS has an example demonstrates how to use Argo Workflows to assign jobs to Amazon EKS] (https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/argo-workflows-eks)

## Usage

Argo Workflows can be deployed by enabling the add-on via the following.
Expand Down
2 changes: 1 addition & 1 deletion docs/add-ons/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This addon deploys Strimizi Kafka Operator and it makes it really easy to spin u
For complete project documentation, please visit the [Strimzi Kafka](https://strimzi.io/).

## Usage
Strimzi Kafka Operator can be deployed by enabling the add-on via the following. Check out the full [example](https://github.com/awslabs/data-on-eks/tree/main/streaming/kafka) to deploy the EKS Cluster with Kafka.
Strimzi Kafka Operator can be deployed by enabling the add-on via the following.

```hcl
enable_strimzi_kafka_operator = true
Expand Down
Loading

0 comments on commit 243ae23

Please sign in to comment.