From 2b1da73fba595b5f6249477a367e75d0ed209b15 Mon Sep 17 00:00:00 2001 From: Marwan Zouinkhi Date: Wed, 5 Jun 2024 20:51:52 -0400 Subject: [PATCH] aws docs --- docs/source/aws.rst | 68 +++++++++++++++++++++++++++++++++++++++++++ docs/source/index.rst | 1 + 2 files changed, 69 insertions(+) create mode 100644 docs/source/aws.rst diff --git a/docs/source/aws.rst b/docs/source/aws.rst new file mode 100644 index 000000000..00391f299 --- /dev/null +++ b/docs/source/aws.rst @@ -0,0 +1,68 @@ +AWS EC2 Setup Guide +=================== + +This guide will help you to run your Docker image on an AWS EC2 instance and set up S3 access for storing data. + +Running Docker Image on AWS EC2 +------------------------------- + +To run your Docker image on an AWS EC2 instance, follow these steps: + +1. **Create Key Pair** (if you don't have one already): + ```bash + aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem + chmod 400 MyKeyPair.pem + ``` + +2. **Create a Security Group** (if you don't have one already): + ```bash + aws ec2 create-security-group --group-name my-security-group --description "My security group" + ``` + +3. **Authorize Inbound Traffic for the Security Group**: + ```bash + aws ec2 authorize-security-group-ingress --group-name my-security-group --protocol tcp --port 22 --cidr 0.0.0.0/0 + aws ec2 authorize-security-group-ingress --group-name my-security-group --protocol tcp --port 80 --cidr 0.0.0.0/0 + ``` + +4. **Run EC2 Instance with Docker Image**: + Use the following script to launch an EC2 instance, pull your Docker image from DockerHub, and run it with port forwarding: + + ```bash + aws ec2 run-instances \ + --image-id ami-0b8956f13d7bdfe7b \ + --count 1 \ + --instance-type p3.2xlarge \ + --key-name \ + --security-groups \ + --user-data '#!/bin/bash + yum update -y + amazon-linux-extras install docker -y + service docker start + docker pull mzouink/dacapo:0.3.0 + docker run -d -p 80:8000 mzouink/dacapo:0.3.0' + ``` + + Replace `` with the name of your key pair and `` with the name of your security group. + +S3 Access Configuration +----------------------- + +You can work locally using S3 data by setting the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. You can also set the `AWS_REGION` environment variable to specify the region to use. If you are using a profile, you can set the `AWS_PROFILE` environment variable to specify the profile to use. + +To configure AWS CLI, use the following command: + +```bash +aws configure +``` + +Storing Checkpoints and Experiments Data in S3 +---------------------------------------------- + +To store checkpoints and experiments data in S3, modify the `dacapo.yaml` file to include the following: + +```yaml +runs_base_dir: "s3://dacapotest" +``` + +This setup will help you run your Docker image on AWS EC2 and configure S3 access for storing checkpoints and experiment data. diff --git a/docs/source/index.rst b/docs/source/index.rst index a4390edd3..c35cb2dff 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -11,6 +11,7 @@ install tutorial docker + aws autoapi/index cli