Skip to content

Commit

Permalink
fix: Update to use cache builder module
Browse files Browse the repository at this point in the history
  • Loading branch information
bryantbiggs committed Oct 2, 2024
1 parent 4d29778 commit a532f4d
Show file tree
Hide file tree
Showing 10 changed files with 39 additions and 729 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ on:

env:
TERRAFORM_VERSION: 1.3.10
TERRAFORM_DOCS_VERSION: v0.16.0
TERRAFORM_DOCS_VERSION: v0.19.0
TFLINT_VERSION: v0.53.0
TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache
TFLINT_VERSION: v0.50.2

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
Expand Down
36 changes: 15 additions & 21 deletions patterns/ml-container-cache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@ When the PyTorch image is not present on the EBS volume, it takes roughly 6 minu

## Code

### Cache Builder

```terraform hl_lines="7-11 13-14"
{% include "../../patterns/ml-container-cache/cache_builder.tf" %}
```

### Cluster

```terraform hl_lines="5-9 48-60 62-74"
{% include "../../patterns/ml-container-cache/eks.tf" %}
```
Expand All @@ -45,12 +53,11 @@ When the PyTorch image is not present on the EBS volume, it takes roughly 6 minu

See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and steps to deploy this pattern.

1. Navigate to the `cache-builder/` directory in order to deploy the Step Function state machine that will create the EBS volume snapshots with the cached images.
1. First, deploy the Step Function state machine that will create the EBS volume snapshots with the cached images.

```sh
cd cache-builder
terraform init
terraform apply --auto-approve
terraform apply -target=module.ebs_snapshot_builder -target=module.vpc --auto-approve
```

2. Once the cache builder resources have been provisioned, execute the state machine by either navigating to the state machine within the AWS console and clicking `Start execution` (with the defaults or by passing in values to override the default values), or by using the provided output from the Terraform output value `start_execution_command` to start the state machine using the awscli. For example, the output looks similar to the following:
Expand All @@ -60,16 +67,14 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
aws stepfunctions start-execution \
--region us-west-2 \
--state-machine-arn arn:aws:states:us-west-2:111111111111:stateMachine:cache-builder \
--input "{\"InstanceType\":\"c6in.24xlarge\",\"Iops\":10000,\"SnapshotDescription\":\"ML container image cache\",\"SnapshotName\":\"ml-container-cache\",\"Throughput\":1000,\"VolumeSize\":128}"
--input "{\"SnapshotDescription\":\"ML container image cache\",\"SnapshotName\":\"ml-container-cache\"}"
EOT
```
3. Once the state machine execution has completed successfully and created an EBS snapshot volume, navigate back up to the root directory of the pattern to provision the cluster and node group that will utilize the cached images.
3. Once the state machine execution has completed successfully and created an EBS snapshot volume, provision the cluster and node group that will utilize the cached images.
```sh
cd ..
terraform init
terraform apply --auto-approve
```
Expand All @@ -95,17 +100,6 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
## Destroy
To remove the resources that were created, the destroy steps should be executed in the reverse order of the deployment steps:
1. Deprovision the cluster resources:
```sh
terraform destroy --auto-approve
```
2. Navigate to the `cache-builder/` directory to deprovision the Step Function state machine:
```sh
cd cache-builder
terraform destroy --auto-approve
```
```sh
terraform destroy --auto-approve
```
40 changes: 0 additions & 40 deletions patterns/ml-container-cache/cache-builder/README.md

This file was deleted.

62 changes: 0 additions & 62 deletions patterns/ml-container-cache/cache-builder/ec2.tf

This file was deleted.

65 changes: 0 additions & 65 deletions patterns/ml-container-cache/cache-builder/main.tf

This file was deleted.

Loading

0 comments on commit a532f4d

Please sign in to comment.