Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Update examples to use terraform-aws-eks module where possible #1428

Merged
merged 10 commits into from
Feb 14, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 2 additions & 7 deletions .github/scripts/plan-examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,8 @@ def get_examples():
"""
exclude = {
'examples/appmesh-mtls', # excluded until Rout53 is setup
'examples/eks-cluster-with-external-dns', # excluded until Rout53 is setup
'examples/fully-private-eks-cluster/vpc', # skipping until issue #711 is addressed
'examples/fully-private-eks-cluster/eks',
'examples/fully-private-eks-cluster/add-ons',
'examples/ai-ml/ray', # excluded until #887 is fixed,
'examples/crossplane' #example removed
'examples/upgrade/blue-green-route53/core-infra'
'examples/blue-green-upgrade/core-infra',
'examples/blue-green-upgrade/modules/eks_cluster'
}

projects = {
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/e2e-parallel-destroy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,8 @@ jobs:
matrix:
include:
- example_path: examples/agones-game-controller
- example_path: examples/complete-kubernetes-addons
- example_path: examples/eks-cluster-with-new-vpc
- example_path: examples/fargate-serverless
- example_path: examples/gitops/argocd
- example_path: examples/grafana-loki
- example_path: examples/argocd
- example_path: examples/ipv4-prefix-delegation
- example_path: examples/ipv6-eks-cluster
- example_path: examples/karpenter
Expand Down Expand Up @@ -63,4 +60,5 @@ jobs:
terraform init -upgrade=true
terraform destroy -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform destroy -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform destroy -target=module.eks -no-color -input=false -auto-approve
terraform destroy -no-color -input=false -auto-approve
7 changes: 3 additions & 4 deletions .github/workflows/e2e-parallel-full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,8 @@ jobs:
matrix:
include:
- example_path: examples/agones-game-controller
- example_path: examples/complete-kubernetes-addons
- example_path: examples/eks-cluster-with-new-vpc
- example_path: examples/fargate-serverless
- example_path: examples/gitops/argocd
- example_path: examples/grafana-loki
- example_path: examples/argocd
- example_path: examples/ipv4-prefix-delegation
- example_path: examples/ipv6-eks-cluster
- example_path: examples/karpenter
Expand Down Expand Up @@ -105,6 +102,7 @@ jobs:
export AWS_CSM_HOST=127.0.0.1
terraform apply -target=module.vpc -no-color -input=false -auto-approve
terraform apply -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform apply -target=module.eks -no-color -input=false -auto-approve
terraform apply -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform apply -no-color -input=false -auto-approve

Expand All @@ -118,6 +116,7 @@ jobs:
export AWS_CSM_HOST=127.0.0.1
terraform destroy -target=module.eks_blueprints_kubernetes_addons -no-color -input=false -auto-approve
terraform destroy -target=module.eks_blueprints -no-color -input=false -auto-approve
terraform destroy -target=module.eks -no-color -input=false -auto-approve
terraform destroy -no-color -input=false -auto-approve

- name: Fail if TF apply failed
Expand Down
10 changes: 3 additions & 7 deletions docs/add-ons/apache-airflow.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Apache Airflow add-on

This document describes the details of the best practices for building and deploying Self-managed **Highly Scalable Apache Airflow cluster on Kubernetes(Amazon EKS) Cluster**.
Alternatively, Amazon also provides a fully managed Apache Airflow service(MWAA). Please see this [example](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/aws-managed-airflow) if you are looking to build Amazon MWAA.
Alternatively, Amazon also provides a fully managed Apache Airflow service(MWAA).

Apache Airflow is used for the scheduling and orchestration of data pipelines or workflows.
Orchestration of data pipelines refers to the sequencing, coordination, scheduling, and managing complex data pipelines from diverse sources.
Expand Down Expand Up @@ -60,18 +60,15 @@ This deployment example uses internet facing Load Balancer to easily access the
You can modify the `values.yaml` to set the Load Balancer to `internal` and upload certificate to use HTTPS.
Ensure access to the WebUI using internal domain and network.


Checkout the [examples](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) of deploying and using Apache Airflow on Amazon EKS.

## Usage

The [Apache Airflow](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) can be deployed by enabling the add-on via the following.
The Apache Airflow can be deployed by enabling the add-on via the following.

```hcl
enable_airflow = true
```

For production workloads, you can use this [example](https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/self-managed-airflow) with custom Helm Config.
For production workloads, you can use the following custom Helm Config.

```hcl
enable_airflow = true
Expand Down Expand Up @@ -116,5 +113,4 @@ airflow-scheduler 2/2 2 2 77m
airflow-statsd 1/1 1 1 77m
airflow-triggerer 1/1 1 1 77m
airflow-webserver 2/2 2 2 77m

```
2 changes: 0 additions & 2 deletions docs/add-ons/argo-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@

For complete project documentation, please visit the [Argo Workflows documentation site](https://argoproj.github.io/argo-workflows/).

[DoEKS has an example demonstrates how to use Argo Workflows to assign jobs to Amazon EKS] (https://awslabs.github.io/data-on-eks/docs/job-schedulers-eks/argo-workflows-eks)

## Usage

Argo Workflows can be deployed by enabling the add-on via the following.
Expand Down
2 changes: 1 addition & 1 deletion docs/add-ons/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This addon deploys Strimizi Kafka Operator and it makes it really easy to spin u
For complete project documentation, please visit the [Strimzi Kafka](https://strimzi.io/).

## Usage
Strimzi Kafka Operator can be deployed by enabling the add-on via the following. Check out the full [example](https://github.com/awslabs/data-on-eks/tree/main/streaming/kafka) to deploy the EKS Cluster with Kafka.
Strimzi Kafka Operator can be deployed by enabling the add-on via the following.

```hcl
enable_strimzi_kafka_operator = true
Expand Down
Loading