Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Upgrade EKS to 1.27 #702

Closed
wants to merge 150 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
150 commits
Select commit Hold shift + click to select a range
b1424fc
Upgrade to EKS 1.27
niallthomson Oct 18, 2023
3851128
Fix VPC CNI version
niallthomson Oct 19, 2023
09b4708
Revert VPC CNI version change
niallthomson Oct 19, 2023
2e3d72d
Fix broken linkg in crossplane module
mostafafarzaneh Oct 20, 2023
95a2632
Make sure Inferentia lab tidies up after tests
niallthomson Oct 20, 2023
3d3dd7f
Fix to pick up errors when running multiple commands
niallthomson Oct 20, 2023
a5aec4e
Removed unused workflow
niallthomson Oct 21, 2023
78b7a9d
Upgrade eksctl
niallthomson Oct 21, 2023
0ae42c3
Migrated Terraform for several labs to new addons repository
niallthomson Oct 21, 2023
5e61371
Bump versions of kubectl and eksctl
niallthomson Oct 21, 2023
9baab17
Remove use of deprecated argument
niallthomson Oct 21, 2023
8b75d45
Migrate away from kustomize env substitution for 1.27
niallthomson Oct 21, 2023
5d715aa
Bump kubectl in website tools
niallthomson Oct 21, 2023
6d8e2ec
Migrate AIML section away from kustomize env substitution
niallthomson Oct 21, 2023
4042443
Revert change to IRSA dynamo configuration
niallthomson Oct 21, 2023
f591293
Increase timeout for AMI upgrade
niallthomson Oct 23, 2023
2ad246b
Migrate OSS metrics lab away from kustomize environment substitution
niallthomson Oct 23, 2023
cba43e6
Migrate CI metrics lab away from kustomize environment substitution
niallthomson Oct 24, 2023
a5b0c1a
Clean up load generator in OSS metrics lab
niallthomson Oct 24, 2023
f9f27e8
Fix apply for consolidation
niallthomson Oct 24, 2023
06ceabd
Fix deprecation warnings
niallthomson Oct 24, 2023
58ff11f
Removed use of standalone kustomize CLI in favor of kubectl kustomize
niallthomson Oct 24, 2023
3decfdb
Migrated SG for pods
niallthomson Oct 24, 2023
e722708
Workaround for slow kubecost deploy
niallthomson Oct 24, 2023
6ac5a6d
Migrate custom networking
niallthomson Oct 24, 2023
c16232c
Fix duplicate resource entries in kustomize files
niallthomson Oct 24, 2023
6b093fa
ArgoCD should use values.yaml
niallthomson Oct 24, 2023
a1ce1cd
Fix duplicate resources
niallthomson Oct 25, 2023
babf89c
Migrate custom networking
niallthomson Oct 25, 2023
8f277ab
Merge branch 'main' into eks-1.27
niallthomson Oct 25, 2023
70873d2
Fix multiple resources in storage
niallthomson Oct 25, 2023
160af94
Fix multiple resources in storage
niallthomson Oct 25, 2023
902eac5
Ensure EBS addon using correct IAM role
niallthomson Oct 25, 2023
7a69204
Downgrade eksctl due to bug
niallthomson Oct 25, 2023
ec03f23
Adding GuardDuty runtime findings
rodrigobersa Aug 26, 2023
582634e
Fix image link
rodrigobersa Aug 26, 2023
69a581a
Fix Runtime protection title, and cleanup menu position
rodrigobersa Aug 26, 2023
fa2f4ab
refactor: GuardDuty module, enable
rodrigobersa Aug 28, 2023
9a281b4
refactor: GuardDuty module, folder structure
rodrigobersa Aug 28, 2023
51fc90a
refactor: GuardDuty module, example finding
rodrigobersa Aug 29, 2023
4fb86c9
refactor: GuardDuty module, ommiting IDs
rodrigobersa Aug 29, 2023
fc2e0ea
refactor: GuardDuty module, log monitoring
rodrigobersa Aug 29, 2023
512c056
refactor: GuardDuty module, unifing privileged.
rodrigobersa Aug 29, 2023
b939f95
refactor: GuardDuty module, adding rshell.
rodrigobersa Aug 31, 2023
45e995a
refactor: GuardDuty module, updating rshell.
rodrigobersa Sep 6, 2023
ef7d5f4
refactor: GuardDuty module, fix rshell.
rodrigobersa Sep 6, 2023
3395d3c
refactor: GuardDuty module, scripting lab
rodrigobersa Sep 6, 2023
051a97a
refactor: GuardDuty module, fixing typos, cleanup
rodrigobersa Sep 6, 2023
0026ad8
Hide reverse-shell
rodrigobersa Oct 24, 2023
29d4588
Removing reverse-shell from Index
rodrigobersa Oct 24, 2023
64d9bc7
Updated governing members
svennam92 Oct 25, 2023
89817f9
Refactoring folder structure for Secrets Management
rodrigobersa Oct 6, 2023
44139ad
Adding Secrets manager index
rodrigobersa Oct 6, 2023
8288240
Changing module order
rodrigobersa Oct 6, 2023
2532098
Adding Deployment directory
rodrigobersa Oct 6, 2023
826ab34
Fixing directory structure
rodrigobersa Oct 7, 2023
a178f02
EKS Validation
rodrigobersa Oct 7, 2023
ac01951
Fixing Addon, and validation steps
rodrigobersa Oct 9, 2023
59952b9
Secrets module directory structure
rodrigobersa Oct 9, 2023
40b34e7
Adding secret store
rodrigobersa Oct 10, 2023
f0ff041
Fix typo
rodrigobersa Oct 10, 2023
2b04a87
Changing module strucutre
rodrigobersa Oct 11, 2023
c33d40f
Fixing descriptoon and adding describe command
rodrigobersa Oct 11, 2023
d11f67c
Running tests v1
rodrigobersa Oct 12, 2023
d79e80b
Running tests v1
rodrigobersa Oct 12, 2023
2611cf4
Adjusting validation step
rodrigobersa Oct 12, 2023
d692271
Updating mounting secrets section
rodrigobersa Oct 12, 2023
8e705c6
Fixing kustomization path
rodrigobersa Oct 12, 2023
78c8efe
Fixing linting
rodrigobersa Oct 12, 2023
8295df0
Adjusting code blocks
rodrigobersa Oct 12, 2023
2ae89ec
Fix typu
rodrigobersa Oct 12, 2023
72906bb
Adding external secrets
rodrigobersa Oct 12, 2023
e4ed747
Fix typo
rodrigobersa Oct 12, 2023
8f028c8
Review
rodrigobersa Oct 13, 2023
739b8c5
Fixing path
rodrigobersa Oct 13, 2023
89f176f
Code review
rodrigobersa Oct 19, 2023
2ed6309
Fix variable and rephrasing
rodrigobersa Oct 19, 2023
685c546
Fixing typo
rodrigobersa Oct 19, 2023
4c53106
Fixing typo
rodrigobersa Oct 19, 2023
6acebcb
Fixing typo
rodrigobersa Oct 19, 2023
7bdf2eb
Adding tag for Sealed Secrets
rodrigobersa Oct 19, 2023
130b1a8
Adding tag for Sealed Secrets
rodrigobersa Oct 19, 2023
983c54c
Adding tag for Sealed Secrets
rodrigobersa Oct 19, 2023
45476fc
Removing optinal tag from Sealed Secrets
rodrigobersa Oct 19, 2023
bc9bb94
Adding `cluster/terraform` deployment
rodrigobersa Aug 16, 2023
44a7150
Fixing Subnet CIDRs, and Public Subnet tags
rodrigobersa Aug 16, 2023
e64cf66
Adding subnet names sufix
rodrigobersa Aug 16, 2023
0a4d0eb
Adding subnet names sufix
rodrigobersa Aug 16, 2023
b8643da
Adjusting subnet names sufix
rodrigobersa Aug 16, 2023
d1ecc52
Updating `versions.tf`.
rodrigobersa Aug 16, 2023
080b6d9
Adding `docs/using-terraform`. Updating `scripts/installer.sh`.
rodrigobersa Aug 17, 2023
91d98c2
Adding file labels
rodrigobersa Aug 17, 2023
d4dd315
Fixing file labels
rodrigobersa Aug 17, 2023
f6294b2
Adjusting terraform commands, and .tf files
rodrigobersa Aug 17, 2023
2468cdf
Fix terraform files download
rodrigobersa Aug 17, 2023
fa9309f
Removing '(Comming soon)' comment
rodrigobersa Aug 17, 2023
6eb3e4b
Migrate kubecost to new helm addon for reliability
niallthomson Oct 19, 2023
08bd0c6
Split Terraform cluster creation to separate files and fixed to suppo…
niallthomson Oct 19, 2023
6bb3b6a
Reorder fundamentals section
niallthomson Aug 23, 2023
1cbdc06
Fixed some secrets manager issues with test repeatability, language c…
niallthomson Oct 26, 2023
89081e6
fix reset-environment script env PWD single quote
csantanapr Oct 26, 2023
737fb11
Make runtime monitoring testable, fix image sizes to be more readable
niallthomson Oct 26, 2023
c1c073c
Additional introduction to GuardDuty
niallthomson Oct 26, 2023
1111a5d
fix: change min node count to 4
yash555kumar Oct 20, 2023
bd5dde3
fix: remove word minimum size to match description with the command
yash555kumar Oct 26, 2023
232b327
Bump @babel/traverse from 7.19.1 to 7.23.2 in /website
dependabot[bot] Oct 18, 2023
e6f77fd
Pass TEST_OUTPUT to test hooks, added documentation on hooks
niallthomson Oct 26, 2023
e3c67da
Add new release workflow
niallthomson Oct 28, 2023
c0f82d0
Initial release doc
niallthomson Oct 28, 2023
1bee92c
Release show use content labels, added more documentation
niallthomson Oct 30, 2023
8fa2a27
add ci
Aug 16, 2023
ccf5830
initial setup
Aug 18, 2023
a97b81f
Add Flux ImageUpdater
Aug 29, 2023
d82fc6e
add mutli-arch ci
Sep 13, 2023
afa59d7
Add content to CI-Flix
Sep 26, 2023
6948a8e
Add content to CI-Flux
Sep 26, 2023
24c7e78
Add content to CI-Flux
Sep 26, 2023
ffa0dd4
Add content to CI-Flux
Sep 26, 2023
1eea0eb
move yaml to manifests
Sep 27, 2023
359f27b
move yaml to manifests
Sep 27, 2023
e0ca94e
rename file imagerepolicy
Oct 3, 2023
07d99e6
Update website/docs/automation/gitops/flux/ci.md
ybezsonov Oct 26, 2023
8f173be
Update website/docs/automation/gitops/flux/ci.md
ybezsonov Oct 26, 2023
7f3f060
Update website/docs/automation/gitops/flux/ci.md
ybezsonov Oct 26, 2023
9ec542f
improve setup and tests
Oct 26, 2023
e6831a6
Fix the terraform destroy command
Samuel-IH Oct 29, 2023
3172062
Upgrade remark to v15 and associated dependencies
niallthomson Oct 31, 2023
60929cb
Modified cleanup script for the workshop to better reflect what we're…
Oct 10, 2023
183bcb4
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 10, 2023
7658961
created Kubernetes manifests for creating DynamoDB tables, configmaps…
Oct 10, 2023
45d5091
modifying addon.tf to use ack TF and changes in workshop content
Oct 12, 2023
73577d6
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
1216cce
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
f151568
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
9880178
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
288d58f
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
8c1417e
Updated ACK DynamoDB Controller to install with EKS Blueprints v5 Ter…
Oct 22, 2023
666eeb0
Updated configmap creation with env, made changes to service account …
Oct 22, 2023
b2a135d
Updated steps to perform, screenshots and other manifest items
Oct 25, 2023
0ddaa6f
Made changes to language, titles and ENV vars for AWS region
Oct 25, 2023
8d243d9
Made changes as per PR comments
Oct 30, 2023
687a1b6
Made changes as per PR comments
Oct 30, 2023
eb61a60
broken website link
Oct 31, 2023
5925308
broken website link
Oct 31, 2023
f3f43a5
fixed bash prompt in md
Oct 31, 2023
9905023
Ensure ACK dynamo table and IAM role are named appropriately
niallthomson Oct 31, 2023
9ea5cdb
Fix how base is determined for kubectl 1.27
niallthomson Oct 31, 2023
304cf5a
Update ACK kustomization for EKS 1.27
niallthomson Oct 31, 2023
2009378
Use file view instead of kustomization for new resource
niallthomson Oct 31, 2023
fb5f250
Merge branch 'main' into eks-1.27
niallthomson Oct 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Migrate away from kustomize env substitution for 1.27
niallthomson committed Oct 21, 2023
commit 8b75d4565ff62d069d642110ddb43186184c2cce

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,24 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: provisioner-vars
namespace: other
env: config.properties
options:
disableNameSuffixHash: true
replacements:
- source:
kind: ConfigMap
name: provisioner-vars
version: v1
namespace: other
fieldPath: data.EKS_CLUSTER_NAME
targets:
- select:
kind: AWSNodeTemplate
name: default
fieldPaths:
- spec.subnetSelector.[karpenter.sh/discovery]
- spec.securityGroupSelector.[karpenter.sh/discovery]
resources:
- provisioner.yaml

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -5,5 +5,5 @@ metadata:
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap
fileSystemId: $(EFS_ID)
fileSystemId: ${EFS_ID}
directoryPerms: "700"

This file was deleted.

Original file line number Diff line number Diff line change
@@ -2,19 +2,3 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- efsstorageclass.yaml

configMapGenerator:
- name: assets-efsid
namespace: assets
envs: [config.properties]

vars:
- name: EFS_ID
objref:
kind: ConfigMap
name: assets-efsid
apiVersion: v1
fieldref:
fieldpath: data.EFS_ID
configurations:
- env-var-transformer.yaml
Original file line number Diff line number Diff line change
@@ -36,7 +36,7 @@ data "aws_route_table" "private" {
resource "aws_fsx_ontap_file_system" "fsxnassets" {
storage_capacity = 2048
subnet_ids = slice(data.aws_subnets.private_subnets_fsx.ids, 0, 2)
deployment_type = "MULTI_AZ_1"
deployment_type = "SINGLE_AZ_1"
throughput_capacity = 512
preferred_subnet_id = data.aws_subnets.private_subnets_fsx.ids[0]
security_group_ids = [aws_security_group.fsxn.id]

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ spec:
version: 1
backendName: backend-fsxn-ontap-
storageDriverName: ontap-nas
managementLIF: $(FSXN_IP)
managementLIF: ${FSXN_IP}
svm: fsxnsvm
autoExportPolicy: true
autoExportCIDRs: ["10.42.0.0/16","100.64.0.0/16"]
Original file line number Diff line number Diff line change
@@ -6,4 +6,4 @@ metadata:
type: Opaque
stringData:
username: fsxadmin
password: $(FSXN_ADMIN_PASSWORD)
password: ${FSXN_ADMIN_PASSWORD}
Original file line number Diff line number Diff line change
@@ -1,32 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- fsxn-secret.yaml
- fsxn-backend-nas.yaml

configMapGenerator:
- name: fsxnconfig
envs:
- config.env

generatorOptions:
disableNameSuffixHash: true

vars:
- name: FSXN_IP
objref:
kind: ConfigMap
name: fsxnconfig
apiVersion: v1
fieldref:
fieldpath: data.FSXN_IP
- name: FSXN_ADMIN_PASSWORD
objref:
kind: ConfigMap
name: fsxnconfig
apiVersion: v1
fieldref:
fieldpath: data.FSXN_ADMIN_PASSWORD
configurations:
- env-var-transformer.yaml
- fsxn-backend-nas.yaml
Original file line number Diff line number Diff line change
@@ -4,5 +4,5 @@ metadata:
name: catalog
namespace: catalog
data:
DB_ENDPOINT: $(CATALOG_RDS_ENDPOINT)
DB_READ_ENDPOINT: $(CATALOG_RDS_ENDPOINT)
DB_ENDPOINT: ${CATALOG_RDS_ENDPOINT}
DB_READ_ENDPOINT: ${CATALOG_RDS_ENDPOINT}

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -2,29 +2,8 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../../../base-application/catalog
configMapGenerator:
- name: catalog-env
namespace: catalog
env: config.properties
vars:
- name: CATALOG_RDS_ENDPOINT
objref:
kind: ConfigMap
name: catalog-env
apiVersion: v1
fieldref:
fieldpath: data.CATALOG_RDS_ENDPOINT
- name: CATALOG_RDS_PASSWORD
objref:
kind: ConfigMap
name: catalog-env
apiVersion: v1
fieldref:
fieldpath: data.CATALOG_RDS_PASSWORD
patches:
- path: catalog-configMap.yaml
- path: secrets.yaml
resources:
- nlb.yaml
configurations:
- configuration.yaml
Original file line number Diff line number Diff line change
@@ -3,4 +3,4 @@ kind: Secret
metadata:
name: catalog-db
data:
password: $(CATALOG_RDS_PASSWORD)
password: ${CATALOG_RDS_PASSWORD}
1 change: 0 additions & 1 deletion manifests/modules/security/irsa/dynamo/config.properties

This file was deleted.

Original file line number Diff line number Diff line change
@@ -4,4 +4,4 @@ metadata:
name: carts
namespace: carts
annotations:
eks.amazonaws.com/role-arn: $(CARTS_IAM_ROLE)
eks.amazonaws.com/role-arn: ${CARTS_IAM_ROLE}

This file was deleted.

12 changes: 0 additions & 12 deletions manifests/modules/security/irsa/service-account/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -2,17 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../dynamo
configMapGenerator:
- name: carts-env
namespace: carts
env: config.properties
vars:
- name: CARTS_IAM_ROLE
objref:
kind: ConfigMap
name: carts-env
apiVersion: v1
fieldref:
fieldpath: data.CARTS_IAM_ROLE
patches:
- path: carts-serviceAccount.yaml
Original file line number Diff line number Diff line change
@@ -25,7 +25,8 @@ The configuration for the provider is split into two parts. The first one define
* **Selectors**: This `AWSNodeTemplate` resource uses `securityGroupSelector` and `subnetSelector` to discover resources used to launch nodes. These tags were automatically set on the associated AWS infrastructure provided for the workshop.

```bash timeout=180
$ kubectl apply -k ~/environment/eks-workshop/modules/autoscaling/compute/karpenter/provisioner
$ kubectl kustomize ~/environment/eks-workshop/modules/autoscaling/compute/karpenter/provisioner \
| envsubst | kubectl apply -f-
```

Throughout the workshop you can inspect the Karpenter logs with the following command to understand its behavior:
4 changes: 2 additions & 2 deletions website/docs/fundamentals/storage/efs/efs-csi-driver.md
Original file line number Diff line number Diff line change
@@ -37,9 +37,9 @@ StorageClass/efs-sc
Let's apply this kustomization:

```bash
$ kubectl apply -k ~/environment/eks-workshop/modules/fundamentals/storage/efs/storageclass
$ kubectl kustomize ~/environment/eks-workshop/modules/fundamentals/storage/efs/storageclass \
| envsubst | kubectl apply -f-
storageclass.storage.k8s.io/efs-sc created
configmap/assets-efsid-48hg67g6fd created
```

Now we'll get and describe the StorageClass using the below commands. Notice that the provisioner used is the EFS CSI driver and the provisioning mode is EFS access point and ID of the file system as exported in the `EFS_ID` environment variable.
2 changes: 1 addition & 1 deletion website/docs/fundamentals/storage/efs/index.md
Original file line number Diff line number Diff line change
@@ -24,4 +24,4 @@ You can view the Terraform that applies these changes [here](https://github.com/
In this lab, we'll learn about the following concepts:
* Assets microservice deployment
* EFS CSI Driver
* Dynamic provisioning using EFS and Kuberneties deployment
* Dynamic provisioning using EFS and Kubernetes deployment
Original file line number Diff line number Diff line change
@@ -40,13 +40,14 @@ manifests/modules/fundamentals/storage/fsxn/backend/fsxn-backend-nas.yaml
Let's apply this kustomization:

```bash
$ kubectl apply -k ~/environment/eks-workshop/modules/fundamentals/storage/fsxn/backend
configmap/fsxnconfig created
$ kubectl kustomize ~/environment/eks-workshop/modules/fundamentals/storage/fsxn/backend \
| envsubst | kubectl apply -f-
secret/backend-fsxn-ontap-nas-secret created
tridentbackendconfig.trident.netapp.io/backend-fsxn-ontap-nas created
```

Now we'll get check that the TridentBackendConfig was create using the below command:

```bash
$ kubectl get tbc -n trident
NAME BACKEND NAME BACKEND UUID PHASE STATUS
2 changes: 0 additions & 2 deletions website/docs/networking/security-groups-for-pods/add-sg.md
Original file line number Diff line number Diff line change
@@ -77,8 +77,6 @@ $ kubectl apply -k ~/environment/eks-workshop/modules/networking/securitygroups-
namespace/catalog unchanged
serviceaccount/catalog unchanged
configmap/catalog unchanged
configmap/catalog-env-97g7bft95f unchanged
configmap/catalog-sg-env-54k244c6t7 created
secret/catalog-db unchanged
service/catalog unchanged
service/catalog-mysql unchanged
Original file line number Diff line number Diff line change
@@ -108,7 +108,8 @@ arn:aws:iam::1234567890:role/eks-workshop-carts-dynamo
Once we've verified the IAM Role to be used, we can run Kustomize to apply the change on the Service Account.

```bash
$ kubectl apply -k ~/environment/eks-workshop/modules/security/irsa/service-account
$ kubectl kustomize ~/environment/eks-workshop/modules/security/irsa/service-account \
| envsubst | kubectl apply -f-
```