From 40c6e4980b9f4ccf6e1d59ea6e9636d439d3178a Mon Sep 17 00:00:00 2001 From: Erin Becker Date: Fri, 29 Mar 2019 12:36:45 -0700 Subject: [PATCH 1/3] move instructions to main setup page Please review and merge in conjuction with https://github.com/datacarpentry/genomics-workshop/pull/67 --- _extras/LaunchingInstances.md | 328 ++++------------------------------ 1 file changed, 38 insertions(+), 290 deletions(-) diff --git a/_extras/LaunchingInstances.md b/_extras/LaunchingInstances.md index a15ab0ef..0ebb67e5 100644 --- a/_extras/LaunchingInstances.md +++ b/_extras/LaunchingInstances.md @@ -1,65 +1,38 @@ --- layout: page title: Launching your own instances -permalink: /LaunchingInstances/ -questions: -- How do I use cloud resources on my own? -objectives: -- Be able to launch an Amazon instance -- In beta- Be able to launch an Atmosphere/Cyverse instance -- Coming soon- Be able to connect to an HPC -keypoints: -- Always terminate your instance when your analysis is complete --- - - -## Launching and logging onto a cloud instance - -This document will cover common questions and advanced topics that we won't get to discuss in the lessons. - -## Launching cloud VMs on your own - -Objectives: - -* Choose a cloud platform and learn how to access -* Launch an instance -* Identify the address of your cloud instance -* Log in to your cloud instance - -**Important Caveat**: For many of the above steps, your instructor will have completed these tasks for you. If you are learning on your own, or trying this after the workshop, see the exercises and instructions below. +> ## Do I need to create my own instances? +> +> **If you are:** +> +> - teaching at or attending a centrally organized Data +Carpentry workshop, +> - a Maintainer for one of the Genomics lessons, or +> - contributing to the Genomics lessons, +> +> The Carpentries staff will create AMI instances for you. Please contact +> team@carpentries.org. +> +> **If you are:** +> +> - teaching at a self-organized workshop, +> - working through these lessons on your own outside of a workshop, +> - practicing your skills after a workshop, or +> - using these lessons for a teaching demonstration as part of your Instructor checkout for The Carpentries, +> +> you will need to create your own AMI instances using the [launching your own AMI instance instructions](https://datacarpentry.org/genomics-workshop/AMI-setup/). +> +> **If you are:** +> +> - considering what types of cloud computing resources are appropriate for you +> - interested in modifying the AMI used in this workshop, +> - want to create your own AMI from scratch, or +> - want to use these lessons on a different cloud resources, +> +> read the information and instructions below. +{: .callout} ### Choosing a cloud platform @@ -70,7 +43,6 @@ The most important thing about *The Cloud* is choice - instead of purchasing a p * Full administrative rights - install anything * Use pre-configured images (software already installed) - **Disadvantages of Cloud Computing** * Cloud computing costs money (you must keep track of your costs) * If you need help, you may not have a local system administrator @@ -89,242 +61,21 @@ There are several cloud providers to choose from. Some scientific clouds may eit * [Atmosphere](https://pods.iplantcollaborative.org/wiki/display/atmman/Getting+Started) * [JetStream](http://jetstream-cloud.org/) - -## Launching an Cloud Instance (Virtual Machine) - -We will provide instructions for working on the *Amazon* and beta instructions for the *Atmosphere* cloud. Follow the instructions for your platform of choice. - -**Please select the platform you wish to use for the exercises: ** - - -
- - ## AWS (Amazon EC2 instances) -In this tutorial, we'll show you how to launch your own instance. - -If you want to go through the the genomics lesson outside of a workshop, or if you just want to use -AWS to process your own data, you'll need to have your own account and launch your own instance. - -These instructions will help you launch an instance identical to the one we use in workshops, and -and this tutorial is not meant to be exhaustive, but for minor changes: - - - You can launch any publicly available image by changing the AMI that you load in step 4 below. - - You can choose not to load an AMI at all, and install your own software later by skipping step 4 below. +- See [the detailed instructions](https://datacarpentry.org/genomics-workshop/AMI-setup/) on launching an Amazon EC2 instance for this workshop. + - You can launch any publicly available image by changing the AMI that you load in step 4 on that page. + - You can choose not to load an AMI at all, and install your own software later by skipping step 4 on that page. - If you want to launch a different set of resources- more processors, more disk space, different operating system, etc; you can choose a different option at step 5. - - For more detailed instructions, and how-tos for other changes consult [EC2 Getting Started](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html) - -### Launching an instance on Amazon Web Services - -**Prerequisites** - -* Form of payment (credit card)* -* Understanding of Amazon's billing and payment (See: [Getting started with AWS Billing and Cost Management](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-getting-started.html)) - -\* You can use some of Amazon Web Services for free, or see if you qualify for an AWS Grant (See: [https://aws.amazon.com/grants/](https://aws.amazon.com/grants/) ) if you are using AWS for education. The free level of service *will not* be sufficient for working with the amount of data we are using for our lessons. - -#### Create an AWS account - -1. Go to Amazon Web Services [https://aws.amazon.com/](https://aws.amazon.com/) -2. Follow the button to sign up for an account - you will need to agree to Amazon's terms and conditions and provide credit card information. - - -#### Sign into AWS and Launch an Instance -1. Sign into AWS EC2 Dashboard: [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) - -2. Click the 'Launch Instance' button - -3. In 'Step 1' you will be asked to choose an Amazon Machine Image (AMI), on the lefthand side, look for 'Community AMIs' and then search for the AMI listed on this curriculum's [Setup page](https://datacarpentry.org/genomics-workshop/setup.html); select that image - -4. For 'Choose and Instance Type' select **t2.medium**; then click 'Review and Launch' - - - > **Tip:** You can select larger instances, but these will cost more to run. Also for the 'Review Instance Launch' you will get some warning about instance security. We will not go into detail here about security groups. However, you should know that when you launch an instance, you will want to take steps to make sure no one maliciously accesses your instance. For example, never give your security credentials (next step) to another user. -5. For the 'Review Instance Launch' step, click 'Launch' -6. You will be asked to Select an existing key pair or create a new key pair. Select 'Proceed without a key pair' and select the acknowledgement you are given. Then click 'Launch Instances' - - -You instance will now be launched. You should follow the links to 'Create billing alerts'. - -#### Connect to Amazon Instance - -1. Log into your AWC EC2 Dashboard [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) -2. You should see that you have one instance, to proceed the instance state must be 'running' (if you just launched the instance it will take <5 min for the instance to start running) - -3. At the bottom of the dashboard, you should see a **Public DNS** which will look something like *ec2.12.2.45.678.compute-1.amazonaws.com*. Copy that address (you may wish make a note of it as you will need this each time you connect.) - -4. Open the terminal application on your Mac and use 'ssh' to connect. Your command will be: - -```bash -$ ssh dcuser@your.amazon.dns -``` -5. Your computer will be unable to verify the authenticity of the host; type **yes** to continue connecting -6. Then enter the password for this computer: 'data4Carp' - -You should now be connected to your personal instance. You can confirm this with the following commands; ``whoami``,``pwd``,``ls``, which should yield the following results: - -```bash -Last login: Thu Jul 30 13:21:08 2015 from 8.sub-70-197-200.myvzw.com -$ whoami -dcuser -$ pwd -/home/dcuser -$ ls -dc_sample_data Downloads openrefine-2.6-beta.1 Trimmomatic-0.32 -Desktop FastQC R -$ -``` - -#**Very Important Warning - Avoid Unwanted Charges** -Please remember, for as long as this instance is running, you will be charged for your usage. You can see an estimate of the current charge from your AWS EC2 dashboard by clicking your name (Account name) on the upper right of the dashboard and selecting 'Billing & Cost Management'. **DO NOT FORGET TO TERMINATE YOUR INSTANCE WHEN YOU ARE DONE** - -### Terminating your instance - -When you are finished with your instance, you must terminate. Follow the following steps. - -1. Sign into AWS and go to the EC2 Dashboard: [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) -2. Under 'Resources' select 'Running Instances' -3. Select the instance you wish to terminate, then click 'Actions' - -4. Under 'Instance State' select terminate. - - > **Warning:** This will delete any data on this instance, so you must move any data you wish to save off the instance. - - > **Tip:** You can use iCommands to move data between your computer, a cloud instance, and the iPlant Data Store. iCommands is installed on the Data Carpentry Amazon AMI. You can download and see documentation for iCommands [here](https://pods.iplantcollaborative.org/wiki/display/DS/Using+iCommands) - there is also some documentation on setting up iCommands in the Atmosphere section below -5. Select 'Yes, Terminate' to terminate the instance. - -
- -
- -**Instructions for PC** - - -## AWS (Amazon EC2 instances) - -In this tutorial, we'll show you how to launch your own instance. - -If you want to go through the the genomics lesson outside of a workshop, or if you just want to use -AWS to process your own data, you'll need to have your own account and launch your own instance. - -These instructions will help you launch an instance identical to the one we use in workshops, and -and this tutorial is not meant to be exhaustive, but for minor changes: - - - You can launch any publicly available image by changing the AMI that you load in step 4 below. - - You can choose not to load an AMI at all, and install your own software later by skipping step 4 below. - - If you want to launch a different set of resources- more processors, more disk space, different operating system, etc; - you can choose a different option at step 5. - - For more detailed instructions, and how-tos for other changes consult [EC2 Getting Started](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html) - -### Launching an instance on Amazon Web Services - -**Prerequisites** - -* Form of payment (credit card)* -* Understanding of Amazon's billing and payment (See: [Getting started with AWS Billing and Cost Management](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-getting-started.html)) - -\* You can use some of Amazon Web Services for free, or see if you qualify for an AWS Grant (See: [https://aws.amazon.com/grants/](https://aws.amazon.com/grants/) ) if you are using AWS for education. The free level of service *will not* be sufficient for working with the amount of data we are using for our lessons. - -#### Create an AWS account - -1. Go to Amazon Web Services [https://aws.amazon.com/](https://aws.amazon.com/) -2. Follow the button to sign up for an account - you will need to agree to Amazon's terms and conditions and provide credit card information. - - -#### Sign into AWS and Launch an Instance - -1. Sign into AWS EC2 Dashboard: [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) - -2. Click the 'Launch Instance' button - - -3. In 'Step 1' you will be asked to choose an Amazon Machine Image (AMI), on the lefthand side, look for 'Community AMIs' and then search for the AMI listed on this curriculum's [Setup page](https://datacarpentry.org/genomics-workshop/setup.html); select that image - - -4. For 'Choose and Instance Type' select **t2.medium** - - > **Tip:** You can select larger instances, but these will cost more to run. - > Also for the 'Review Instance Launch' you will get some warning about - > instance security. We will not go into detail here about security groups. - > However, you should know that when you launch an instance, you will want to - > take steps to make sure no one maliciously accesses your instance. For - > example, never give your security credentials (next step) to another user. - -5. Click on "6. Configure Security Group" in the top navigation menu. Choose - "select an existing security group" under "assign a security group", and - choose "sg-a5dba9ef" which has the name "to use with DC genomics AMI". - -5. Click "Review and Launch" and the bottom of the screen, and then click - 'Launch' - -6. You will be asked to Select an existing key pair or create a new key pair. Select 'Proceed without a key pair' and select the acknowledgement you are given. Then click 'Launch Instances'. - - - -You instance will now be launched. You should follow the links to 'Create billing alerts'. - -#### Connecting to your Amazon Instance - -1. Download the PuTTY application at: [http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe](http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe) - -1. Log into your AWC EC2 Dashboard [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) - -2. You should see that you have one instance, make sure instance state is 'running' (if you just launched the instance it will take <5 min for the instance to start running) - - -3. At the bottom of the dashboard, you should see a **Public DNS** which will look something like *ec2.12.2.45.678.compute-1.amazonaws.com*. Copy that address (you may wish make a note of it as you will need this each time you connect.) - - -4. Start PuTTY. In the section 'Specify the destination you want to connect to' for 'Host Name (or IP address)' paste in the DNS address and click 'Open' - -5. When prompted to login as, enter 'dcuser'; you may be notified that the authenticity of the host cannot be verified - if so, ignore the warning an continue connecting - -6. When prompted for a password enter 'data4Carp' - -You should now be connected to your personal instance. You can confirm this with the following commands; ``whoami``,``pwd``,``ls``, which should yield the following results: - -```bash -Last login: Thu Jul 30 13:21:08 2015 from 8.sub-70-197-200.myvzw.com -$ whoami -dcuser -$ pwd -/home/dcuser -$ ls -dc_sample_data Downloads openrefine-2.6-beta.1 Trimmomatic-0.32 -Desktop FastQC R -$ -``` - - -#**Very Important Warning - Avoid Unwanted Charges** -Please remember, for as long as this instance is running, you will be charged for your usage. You can see an estimate of the current charge from your AWS EC2 dashboard by clicking your name (Account name) on the upper right of the dashboard and selecting 'Billing & Cost Management'. **DO NOT FORGET TO TERMINATE YOUR INSTANCE WHEN YOU ARE DONE** - -### Terminating your instance - -When you are finished with your instance, you must terminate. Follow the following steps. - -1. Sign into AWS and go to the EC2 Dashboard: [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/) -2. Under 'Resources' select 'Running Instances' -3. Select the instance you wish to terminate, then click 'Actions' - -4. Under 'Instance State' select terminate. - - > **Warning:** This will delete any data on this instance, so you must move any data you wish to save off the instance. - - > **Tip:** You can use iCommands to move data between your computer, a cloud instance, and the iPlant Data Store. iCommands is installed on the Data Carpentry Amazon AMI. You can download and see documentation for iCommands [here](https://pods.iplantcollaborative.org/wiki/display/DS/Using+iCommands) - there is also some documentation on setting up iCommands in the Atmosphere section below -5. Select 'Yes, Terminate' to terminate the instance. - -
- -
+ - For more detailed instructions, and how-tos for other changes consult [EC2 Getting Started](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html). ## Launching an instance on Atmosphere - These instructions are in beta and not fully tested! -**Prerequisites** - -* You must have an iPlant account (register at [https://user.cyverse.org/](https://user.cyverse.org/) ) - You must also request access to Atmosphere (see [documentation](https://wiki.cyverse.org/wiki/display/atmman/Atmosphere+Manual+Table+of+Contents) **CyVerse requires Atmosphere users to have a valid .edu or .org email address** ) +> ## Prerequisites +> You must have an iPlant account (register at [https://user.cyverse.org/](https://user.cyverse.org/) ) - You must also request access to +> Atmosphere (see [documentation](https://wiki.cyverse.org/wiki/display/atmman/Atmosphere+Manual+Table+of+Contents) **CyVerse requires Atmosphere users to have a valid .edu or .org email address** ) +{: .prereq} #### Sign into Atmosphere and launch an instance 1. Sign into Atmosphere at: [http://atmo.iplantcollaborative.org/](http://atmo.iplantcollaborative.org/) @@ -345,7 +96,6 @@ Click Launch. Your instance should be ready in 10-15 minutes. When your instance status is listed as 'active' you will be able to connect. - #### Connect to Atmosphere instance **Instructions for Mac\Linux** @@ -482,5 +232,3 @@ When you are finished with your instance, you must terminate. Follow the followi > **Warning:** This will delete any data on this instance, so you must move any data you wish to save off the instance (for example by using iCommands). 5. After reading the warning and confirming you are ready, click 'Terminate' to terminate the instance. - -
From 7ceb96ab6eedf41eeda3f5d4c5c0323aeafa77e0 Mon Sep 17 00:00:00 2001 From: Erin Becker Date: Mon, 8 Apr 2019 14:58:47 -0700 Subject: [PATCH 2/3] Add back in platform selection script --- _extras/LaunchingInstances.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/_extras/LaunchingInstances.md b/_extras/LaunchingInstances.md index 0ebb67e5..d2080e51 100644 --- a/_extras/LaunchingInstances.md +++ b/_extras/LaunchingInstances.md @@ -3,6 +3,27 @@ layout: page title: Launching your own instances --- + + + > ## Do I need to create my own instances? > > **If you are:** @@ -61,6 +82,8 @@ There are several cloud providers to choose from. Some scientific clouds may eit * [Atmosphere](https://pods.iplantcollaborative.org/wiki/display/atmman/Getting+Started) * [JetStream](http://jetstream-cloud.org/) +
+ ## AWS (Amazon EC2 instances) - See [the detailed instructions](https://datacarpentry.org/genomics-workshop/AMI-setup/) on launching an Amazon EC2 instance for this workshop. @@ -70,6 +93,10 @@ There are several cloud providers to choose from. Some scientific clouds may eit you can choose a different option at step 5. - For more detailed instructions, and how-tos for other changes consult [EC2 Getting Started](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html). +
+ +
+ ## Launching an instance on Atmosphere - These instructions are in beta and not fully tested! > ## Prerequisites @@ -232,3 +259,5 @@ When you are finished with your instance, you must terminate. Follow the followi > **Warning:** This will delete any data on this instance, so you must move any data you wish to save off the instance (for example by using iCommands). 5. After reading the warning and confirming you are ready, click 'Terminate' to terminate the instance. + +
From f1ae58b785a8c349a927bc64f4812000cca10fbc Mon Sep 17 00:00:00 2001 From: Amanda Charbonneau Date: Thu, 11 Apr 2019 16:45:26 -0400 Subject: [PATCH 3/3] fix dialogue code --- _extras/LaunchingInstances.md | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/_extras/LaunchingInstances.md b/_extras/LaunchingInstances.md index d2080e51..ec2eaaa7 100644 --- a/_extras/LaunchingInstances.md +++ b/_extras/LaunchingInstances.md @@ -25,33 +25,33 @@ window.onload = set_page_view_defaults; > ## Do I need to create my own instances? -> +> > **If you are:** -> -> - teaching at or attending a centrally organized Data -Carpentry workshop, +> +> - teaching at or attending a centrally organized Data +Carpentry workshop, > - a Maintainer for one of the Genomics lessons, or > - contributing to the Genomics lessons, -> +> > The Carpentries staff will create AMI instances for you. Please contact -> team@carpentries.org. -> +> team@carpentries.org. +> > **If you are:** -> -> - teaching at a self-organized workshop, -> - working through these lessons on your own outside of a workshop, +> +> - teaching at a self-organized workshop, +> - working through these lessons on your own outside of a workshop, > - practicing your skills after a workshop, or > - using these lessons for a teaching demonstration as part of your Instructor checkout for The Carpentries, -> -> you will need to create your own AMI instances using the [launching your own AMI instance instructions](https://datacarpentry.org/genomics-workshop/AMI-setup/). -> +> +> you will need to create your own AMI instances using the [launching your own AMI instance instructions](https://datacarpentry.org/genomics-workshop/AMI-setup/). +> > **If you are:** -> +> > - considering what types of cloud computing resources are appropriate for you > - interested in modifying the AMI used in this workshop, > - want to create your own AMI from scratch, or > - want to use these lessons on a different cloud resources, -> +> > read the information and instructions below. {: .callout} @@ -82,6 +82,8 @@ There are several cloud providers to choose from. Some scientific clouds may eit * [Atmosphere](https://pods.iplantcollaborative.org/wiki/display/atmman/Getting+Started) * [JetStream](http://jetstream-cloud.org/) +**Please select the platform you wish to use for the exercises: ** +
## AWS (Amazon EC2 instances) @@ -100,7 +102,7 @@ There are several cloud providers to choose from. Some scientific clouds may eit ## Launching an instance on Atmosphere - These instructions are in beta and not fully tested! > ## Prerequisites -> You must have an iPlant account (register at [https://user.cyverse.org/](https://user.cyverse.org/) ) - You must also request access to +> You must have an iPlant account (register at [https://user.cyverse.org/](https://user.cyverse.org/) ) - You must also request access to > Atmosphere (see [documentation](https://wiki.cyverse.org/wiki/display/atmman/Atmosphere+Manual+Table+of+Contents) **CyVerse requires Atmosphere users to have a valid .edu or .org email address** ) {: .prereq}