Skip to content

Latest commit

 

History

History
126 lines (85 loc) · 5.05 KB

deployment.md

File metadata and controls

126 lines (85 loc) · 5.05 KB
Raw

This document covers the instructions you need to follow before you can run you first calculation.

Requirements

Before you deploy, you must have the following in place:

Step 1: Clone The Repository

Note: To use this solution you need access to the integrals repository.

  1. Select a folder to hold the code in this repository, or create a new one.
  2. Open the terminal (or command prompt on Windows) and cd into the above folder.
  3. Clone this github repository by entering the following:
git clone https://github.com/UBC-CIC/electron-repulsion.git --recurse-submodules
  1. Navigate into the electron-repulsion folder by running the following command:
cd electron-repulsion
  1. Run this command if you want to update the submodule to the latest commit in the integrals repository. Note that the latest commit may or may not be compatible with the CDK in this repository:
git submodule update --remote

Step 2: Building The Image

Start Docker on your system if it is not already running. Run the following command to build and tag the integrals image locally:

./scripts/build_image.sh

Step 3: Deploying CDK Resources

We use AWS CDK to manage cloud resources in such a way that we can easily replicate our setup in other users' environments. To use our definitions, you will need to install the cdk utility, which you can do with:

npm install -g aws-cdk

You can verify that cdk is installed correctly on your path with:

cdk --version

CDK determines the AWS environment it manipulates through all the normal means. Typically this will be through environment variables like AWS_PROFILE, AWS_REGION, AWS_ACCESS_KEY_ID, etc. (This is exactly how aws-cli works, so we will not go into much more detail here, except to say that you should be able to confirm access in your current shell with aws sts get-caller-identity. It is enough to set AWS_PROFILE and AWS_REGION in the shell.)

CDK groups deployable things in named "stacks." We have the following stacks (in dependency order):

  1. IntegralsStack - All resources, including S3 buckets, ECS clusters, Lambda and Step Function definitions, etc.

There are options in the cdk utility to operate per-stack, but by default it operates on all stacks.

In the cdk directory, you can execute the following commands to initialize and deploy:

# Required: Install dependencies
npm install

# Optional: Print out a CloudFormation template with all managed resources
cdk synth

# Required: Bootstrap CDK in your environment
cdk bootstrap

# Deploy the current resources (potentially just updating changed resources, or doing nothing if up-to-date)
# You need to pass required parameters at deploy time.
cdk deploy --parameters IntegralsStack:bucketName=${YOUR_DESIRED_S3_BUCKET_TO_CREATE}

As you pull new versions of our CDK code, you can cdk synth and cdk deploy at your convenience. If you want to tear down your environment, you can run cdk destroy. Note that you may have to delete the ECR image manually (or use scripts/delete_image.sh). Be warned that destroying the stack will remove the S3 bucket with calculation results!

Step 4: Pushing The Image

Now that CDK has initialized all our resources, we need to push the integrals image to the ECR repo it has created. You can do that with:

./scripts/build_image.sh

Note that this script assumes the ability to read AWS configuration from environment variables.

There is a shortcut around building if you have access to another account with the image already in it:

AWS_PROFILE=NAME_OF_THE_SOURCE_ACCOUNT ./scripts/pull_image.sh
AWS_PROFILE=NAME_OF_THE_DESTINATION_ACCOUNT AWS_DEFAULT_REGION=ca-central-1 ./scripts/push_image.sh

Step 5: Setting up the CLI (Command-Line Interface)

There is a small Python library/application in cli that allows you to run and monitor jobs without having to know too much about the AWS resources themselves. Its dependencies are managed by pip, and you can initialize a virtual environment and run the CLI with:

# In the cli dir
# Install the dependencies needed to run the CLI
make venv

# Or equivalently
python3 -m venv .venv
.venv/bin/pip install -e .

# Run the CLI
./cli.sh --help

# Or equivalently
.venv/bin/python -m cli.main --help

See the Makefile for other development tasks (such as make test).

The interface is packaged as a Python library (with a setup.py) so you can depend on it in your own pip projects.

Now that the cloud resources have been deployed and the CLI has been set up, you can run your first calculation. Visit the User Guide for instructions on how to do that.