A lightweight wrapper around linting (e.g. yamllint) & infrastructure deployment tools (e.g. CloudFormation, Terraform, Serverless) to ease management of per-environment configs & deployment.
Very simple configuration to:
- Perform automatic linting/verification
- Ensure deployments are only performed when an environment config is present
- Define an IAM role to assume for each deployment
- Wrangle Terraform backend/workspace configs w/ per-environment tfvars
- Avoid long-term tool lock-in
- runway is a simple wrapper around standard tools. It simply helps to avoid convoluted Makefiles / CI jobs
Complete quickstart documentation, including Docker images, CloudFormation templates, and walkthrough can be found here
- Modules:
- A single-tool configuration of an application/component/infrastructure (e.g. a set of CloudFormation stacks to deploy a VPC, a Serverless app)
- Regions:
- AWS regions
- Environments:
- A Serverless stage, a Terraform workspace, etc.
- Environments are determined automatically from:
- Git branches. We recommend promoting changes through clear environment branches (prefixed with
ENV-). For example, when running a deployment in theENV-devbranchdevwill be the environment. Themasterbranch can also be used as a special 'shared' environment calledcommon(e.g. for modules not normally promoted through other environments). - The parent folder name of each module. For teams with a preference or technical requirement to not use git branches, each environment can be represented on disk as a folder. Instead of promoting changes via git merges, changes can be promoted by copying the files between the environment folders. See the
ignore_git_branchrunway.yml config option.- The folder name of the module itself (not its parent folder) if the
ignore_git_branchandcurrent_dirrunway.yml config config options are both used (see "Directories as Environments with a Single Module" in "Repo Structure").
- The folder name of the module itself (not its parent folder) if the
- The
DEPLOY_ENVIRONMENTenvironment variable.
- Git branches. We recommend promoting changes through clear environment branches (prefixed with
- Deployments:
- Mappings of modules to regions, optionally with AWS IAM roles to assume
- runway.yml:
- List of deployments
- When the
CIenvironment variable is set, all deployments are run in order; otherwise, the user is prompted for deployments to run.
Sample repo structure, showing 2 modules using environment git branches (these same files would be present in each environment branch, with changes to any environment promoted through branches):
.
├── myapp.cfn
│ ├── dev-us-west-2.env
│ ├── prod-us-west-2.env
│ ├── myapp.yaml
│ └── templates
│ └── foo.json
├── myapp.tf
│ ├── backend.tfvars
│ ├── dev-us-east-1.tfvars
│ ├── prod-us-east-1.tfvars
│ └── main.tf
└── runway.yml
Another sample repo structure, showing the same modules nested in environment folders:
.
├── dev
│ ├── myapp.cfn
│ │ ├── dev-us-west-2.env
| │ ├── prod-us-west-2.env
│ │ ├── myapp.yaml
│ │ └── templates
│ │ └── myapp_cf_template.json
│ ├── myapp.tf
│ │ ├── backend.tfvars
│ │ ├── dev-us-east-1.tfvars
| │ ├── prod-us-east-1.tfvars
│ │ └── main.tf
│ └── runway.yml
└── prod
├── myapp.cfn
│ ├── dev-us-west-2.env
│ ├── prod-us-west-2.env
│ ├── myapp.yaml
│ └── templates
│ └── myapp_cf_template.json
├── myapp.tf
│ ├── backend.tfvars
│ ├── dev-us-east-1.tfvars
│ ├── prod-us-east-1.tfvars
│ └── main.tf
└── runway.yml
Another sample repo structure, showing environment folders containing a single CloudFormation module at their root (combining the current_dir & ignore_git_branch "Runway Config File" options to merge the Environment & Module folders):
.
├── dev
│ ├── dev-us-west-2.env
│ ├── prod-us-west-2.env
│ ├── myapp.yaml
│ ├── runway.yml
│ └── templates
│ └── myapp_cf_template.json
└── prod
├── dev-us-west-2.env
├── prod-us-west-2.env
├── myapp.yaml
├── runway.yml
└── templates
└── myapp_cf_template.json
runway.yml example:
---
# Order that modules will be deployed. A module will be skipped if a
# corresponding env/config file is not present in its folder.
# (e.g., for cfn modules, if a dev-us-west-2.env file is not in the 'app.cfn'
# folder when running a dev deployment of 'app' to us-west-2 then it will be
# skipped.)
deployments:
- modules:
- myapp.cfn
regions:
- us-west-2
- modules:
- myapp.tf
regions:
- us-east-1
assume-role: # optional
# When running multiple deployments, post_deploy_env_revert can be used
# to revert the AWS credentials in the environment to their previous
# values
# post_deploy_env_revert: true
dev: arn:aws:iam::account-id1:role/role-name
prod: arn:aws:iam::account-id2:role/role-name
# A single ARN can be specified instead, to apply to all environments
# arn: arn:aws:iam::account-id:role/role-name
# Role duration can be set at the top level, or in a specific environment
# duration: 7200
# dev:
# arn: arn:aws:iam::account-id1:role/role-name
# duration: 7200
account-alias: # optional
# A mapping of environment -> alias mappings can be provided to have
# Runway verify the current assumed role / credentials match the
# necessary account
dev: my_dev_account
prod: my_dev_account
account-id: # optional
# A mapping of environment -> id mappings can be provided to have Runway
# verify the current assumed role / credentials match the necessary
# account
dev: 123456789012
prod: 345678901234
skip-npm-ci: false # optional, and should rarely be used. Omits npm ci
# execution during Serverless deployments
# (i.e. for use with pre-packaged node_modules)
# If using environment folders instead of git branches, git branch lookup can
# be disabled entirely (see "Repo Structure")
# ignore_git_branch: true
runway.yml can also be placed in a module folder (e.g. a repo/environment containing only one module doesn't need to nest the module in a subfolder):
---
# This will deploy the module in which runway.yml is located
deployments:
- current_dir: true
regions:
- us-west-2
assume-role:
arn: arn:aws:iam::account-id:role/role-name
# If using environment folders instead of git branches, git branch lookup can
# be disabled entirely (see "Repo Structure"). See "Directories as Environments
# with a Single Module" in "Repo Structure".
# ignore_git_branch: true
- Install Python
- On Linux (assuming default Bash shell; adjust for others appropriately):
- Setup your shell for user-installed (non-root) pip packages:
echo 'export PATH=$HOME/.local/bin:$PATH' >> ${HOME}/.bashrcsource ${HOME}/.bashrc
- Install Python/pip:
- Debian-family (e.g. Ubuntu):
sudo apt-get -y install python-pip python-minimal - Amazon Linux should should work out of the box
- RHEL-family:
- If easy_install is available:
easy_install --user pip - Otherwise, enable EPEL and
sudo yum install python-pip
- If easy_install is available:
- Debian-family (e.g. Ubuntu):
- Setup your shell for user-installed (non-root) pip packages:
- On macOS (assuming default Bash shell; adjust for others appropriately):
if ! which pip > /dev/null; then easy_install --user pip; fiecho 'export PATH="${HOME}/Library/Python/2.7/bin:${PATH}"' >> ${HOME}/.bash_profilesource ${HOME}/.bash_profile
- On Windows:
- This can be done via the Chocolately package manager (e.g.
choco install python2), or manually from their website- If installing via Chocolately, default options will be sufficient. Close/reopen terminals after installation to use the updated PATH
- If installing manually, use the default options with the exception of the "Add python to Path" (it should be enabled).
- Add
%USERPROFILE%\AppData\Roaming\Python\Scriptsto PATH environment variable
- This can be done via the Chocolately package manager (e.g.
- On Linux (assuming default Bash shell; adjust for others appropriately):
- Install runway (doesn't require sudo/admin permissions):
pip install --user runway- If this produces an error like
Unknown distribution option: 'python_requires', upgrade setuptools firstpip install --user --upgrade setuptools
- If this produces an error like
runway test(akarunway preflight) - execute this in your environment to catch errors; if it exits0, you're ready for...runway plan(akarunway taxi) - this optional step will show the diff/plan of what will be changed. With a satisfactory plan you can...runway deploy(akarunway takeoff) - if running interactively, you can choose which deployment to run; otherwise (i.e. on your CI system) each deployment will be run in sequence.
runway destroy(akarunway dismantle) - if running interactively, you can choose which deployment to remove; otherwise (i.e. on your CI system) every deployment will be run in reverse sequence (use with caution).
CloudFormation modules are managed by 2 files: a key/value environment file, and a yaml file defining the stacks/templates/params.
Environment - name these in the form of ENV-REGION.env (e.g. dev-us-east-1.env) or ENV.env (e.g. dev.env):
# Namespace is used as each stack's prefix
# We recommend an (org/customer)/environment delineation
namespace: contoso-dev
environment: dev
customer: contoso
region: us-west-2
# The stacker bucket is the S3 bucket (automatically created) where templates
# are uploaded for deployment (a CloudFormation requirement for large templates)
stacker_bucket_name: stacker-contoso-us-west-2
Stack config - these can have any name ending in .yaml (they will be evaluated in alphabetical order):
# Note namespace/stacker_bucket_name being substituted from the environment
namespace: ${namespace}
stacker_bucket: ${stacker_bucket_name}
stacks:
myvpcstack: # will be deployed as contoso-dev-myvpcstack
template_path: templates/vpc.yaml
# The enabled option is optional and defaults to true. You can use it to
# enable/disable stacks per-environment (i.e. like the namespace
# substitution above, but with the value of either true or false for the
# enabled option here)
enabled: true
myvpcendpoint:
template_path: templates/vpcendpoint.yaml
# variables map directly to CFN parameters; here used to supply the
# VpcId output from the myvpcstack to the VpcId parameter of this stack
variables:
VpcId: ${output myvpcstack::VpcId}
The config yaml supports many more features; see the full Stacker documentation for more detail (e.g. stack configuration options, additional lookups in addition to output (e.g. SSM, DynamoDB))
In addition or in place of the environment file(s), environment values can be provided via deployment and module options.
---
deployments:
- modules:
- path: mycfnstacks
environments:
dev:
namespace: contoso-dev
foo: bar
and/or
---
deployments:
- environments:
dev:
namespace: contoso-dev
foo: bar
modules:
- mycfnstacks
---
environments:
dev:
namespace: contoso-dev
foo: bar
(in runway.module.yaml)
Standard Serverless rules apply, with the following recommendations/caveats:
- Runway environments map directly to Serverless stages.
- A
package.jsonfile is required, specifying the serverless dependency, e.g.:
{
"name": "mymodulename",
"version": "1.0.0",
"description": "My serverless module",
"main": "handler.py",
"devDependencies": {
"serverless": "^1.25.0"
},
"author": "Serverless Devs",
"license": "ISC"
}
- We strongly recommend you commit the package-lock.json that is generated after running
npm install - Each stage requires either its own variables file (even if empty for a particular stage) in one of the following forms, or a configured environment in the module options (see
Specifying Environments Via Runway Module Optionsbelow):
env/STAGE-REGION.yml
config-STAGE-REGION.yml
env/STAGE.yml
config-STAGE.yml
env/STAGE-REGION.json
config-STAGE-REGION.json
env/STAGE.json
config-STAGE.json
Environments can be specified via deployment and module options in lieu of variable files.
---
deployments:
- modules:
- path: myslsmodule
environments:
dev: true
prod: true
and/or:
---
deployments:
- environments:
dev: true
prod: true
modules:
- myslsmodule
---
environments:
dev: true
prod: true
(in runway.module.yaml)
Standard Terraform rules apply, with the following recommendations/caveats:
- Each environment requires its own tfvars file, in the form of ENV-REGION.tfvars (e.g. dev-contoso.tfvars).
- We recommend (but do not require) having a backend configuration separate from the terraform module code:
main.tf:
terraform {
backend "s3" {
key = "some_unique_identifier_for_my_module" # e.g. contosovpc
}
}
# continue with code here...
backend-REGION.tfvars, or backend-ENV-REGION.tfvars, or backend-ENV.tfvars (e.g. backend-us-east-1.tfvars):
bucket = "SOMEBUCKNAME"
region = "SOMEREGION"
dynamodb_table = "SOMETABLENAME"
If a .terraform-version file is placed in the module, tfenv will be invoked to ensure the appropriate version is installed prior to module deployment.
In addition or in place of the variable file(s), variable values can be provided via deployment and module options.
---
deployments:
- modules:
- path: mytfmodule
environments:
dev:
foo: bar
and/or
---
deployments:
- environments:
dev:
foo: bar
modules:
- mytfmodule
---
environments:
dev:
namespace: contoso-dev
foo: bar
(in runway.module.yaml)
Execute runway whichenv to output the name of the currently detected environment (see Basic Concepts for an overview of how runway determines the environment name).
Runway comes pre-packaged with a module plugin for performing idempotent deployments of static websites. It combines CloudFormation stacks (for S3 buckets & CloudFront Distribution) with additional logic to build & sync the sites.
It can be used with a configuration like the following:
deployments:
- modules:
- path: web
class_path: runway.module.staticsite.StaticSite
environments:
dev:
namespace: contoso-dev
staticsite_cert_arn: arn:aws:acm:us-east-1:123456789012:certificate/...
options:
build_steps:
- npm ci
- npm run build
build_output: dist
regions:
- us-west-2
This will build the website in web via the specified build_steps and then upload the contents of web/dist to a S3 bucket created in the CloudFormation stack web-dev-conduit. On subsequent deploys, the website will be built and synced only if the (non-git-ignored) files in web change.
The site domain name is available via the CFDistributionDomainName output of the <namespace>-<path> stack (e.g. contoso-dev-web above) and will be displayed on stack creation/updates.
See additional options here, or a start-to-finish example walkthrough here.
Execute runway gen-sample followed by a module type to create a sample module directory, containing example files appropriate for the module type:
runway gen-sample cfn: Creates a sample CloudFormation module insampleapp.cfnrunway gen-sample sls: Creates a sample Serverless Framework module insampleapp.slsrunway gen-sample stacker: Creates a sample CloudFormation module (with Python templates using Troposphere and awacs) inrunway-sample-tfstate.cfnrunway gen-sample tf: Creates a sample Terraform module insampleapp.tf