Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Yaml include feature #43

Open
wants to merge 28 commits into
base: 094.branch
Choose a base branch
from
Open

Yaml include feature #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5dbcb28
Impliment yaml \!import and update factory defaults
Feb 3, 2016
6451ca7
Copy FACTORY_CONFIG verbatim to local config.json
Feb 3, 2016
dc63b48
Cleanup of resources.py w new tests
Feb 5, 2016
e05b37c
Some function name cleanup and added comments
Feb 5, 2016
3dde644
Implimented prompt flag for Res.generate_config() and integrate with …
Feb 6, 2016
85da65f
remove generated file instancetype_to_arch.json
Feb 6, 2016
02aef23
sort keys of generated config files
Feb 6, 2016
b2dd8b9
Bug fixes to reenable 'create'
Feb 6, 2016
8ad71ca
Fixed bug with get_template_from_s3, deleted old resources.py code
Feb 6, 2016
5a48e98
Removed dead code
Feb 6, 2016
80d41f1
Fixed test_config_yaml by reenabling named config files
Feb 9, 2016
103d5e4
Fixed config_override test, though its utility is reduced w/ new vers…
Feb 9, 2016
2c5375f
Fixed test test_config_validation
Feb 9, 2016
abbf515
Fixed test_extending_config by passing config_handlers to generate_co…
Feb 9, 2016
309938f
Fixed bug in test_template_file_flag by enabling Res.generate_config(…
Feb 9, 2016
e51b667
Fixed test_factory_default
Feb 10, 2016
f22b097
Cleanup of region_arch_2_ami.py and refresh of amazon linux AMIs.
Feb 10, 2016
d720d32
Added support for suggested_instance_types to HaNAT pattern
Feb 15, 2016
6b867ea
Supress output during tests by adding is_silent to generate_config() …
Feb 15, 2016
60a857d
Forgot to remove instance_type config param from data/config.json
Feb 16, 2016
b34ba3d
pylint changes to HaNAT
Feb 16, 2016
5a8a5f8
Added suggested_instance_types to add_asg(), Bastion and HaCluster. …
Feb 16, 2016
1e2cf2a
Removing legacy config param ami_map_file
Feb 17, 2016
16a2734
Complete removal of network config from data/config.json
Feb 17, 2016
1bdaa01
Forgot to include nat.enable_ntp
Feb 17, 2016
34a6258
Added region_map.json to contian region config info in separate file.
Feb 17, 2016
47a4425
Fixed tests
Feb 18, 2016
593dbe1
Removed dead test files
Feb 19, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
src/cfn_environment_base.egg-info/
ami_cache.*
config.*
instancetype_to_arch.*
.DS_Store
.idea/
.eggs
93 changes: 42 additions & 51 deletions src/environmentbase/data/ami_cache.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,53 +1,44 @@
{
"us-east-1": {
"natAmiId": "ami-4f9fee26",
"ubuntu1404LtsAmiId": "ami-018c9568",
"amazonLinuxAmiId": "ami-0d4cfd66",
"centOS65AmiId": "ami-8997afe0"
},
"us-west-1": {
"natAmiId": "ami-7850793d",
"ubuntu1404LtsAmiId": "ami-ee4f77ab",
"amazonLinuxAmiId": "ami-87ea13c3",
"centOS65AmiId": "ami-1a013c5f"
},
"us-west-2": {
"natAmiId": "ami-69ae8259",
"ubuntu1404LtsAmiId": "ami-6ac2a85a",
"amazonLinuxAmiId": "ami-d5c5d1e5",
"centOS65AmiId": "ami-b6bdde86"
},
"eu-central-1": {
"amazonLinuxAmiId": "ami-a6b0b7bb"
},
"eu-west-1": {
"natAmiId": "ami-ed352799",
"ubuntu1404LtsAmiId": "ami-896c96fe",
"amazonLinuxAmiId": "ami-e4d18e93",
"centOS65AmiId": "ami-42718735"
},
"ap-southeast-1": {
"natAmiId": "ami-780a432a",
"ubuntu1404LtsAmiId": "ami-9a7724c8",
"amazonLinuxAmiId": "ami-68d8e93a",
"centOS65AmiId": "ami-a08fd9f2"
},
"ap-southeast-2": {
"natAmiId": "ami-0154c73b",
"ubuntu1404LtsAmiId": "ami-43128a79",
"amazonLinuxAmiId": "ami-db7b39e1",
"centOS65AmiId": "ami-e7138ddd"
},
"ap-northeast-1": {
"natAmiId": "ami-5f840e5e",
"ubuntu1404LtsAmiId": "ami-bddaa2bc",
"amazonLinuxAmiId": "ami-1c1b9f1c",
"centOS65AmiId": "ami-81294380"
},
"sa-east-1": {
"natAmiId": "ami-7660c56b",
"ubuntu1404LtsAmiId": "ami-7315b76e",
"amazonLinuxAmiId": "ami-55098148",
"centOS65AmiId": "ami-7d02a260"
"amazonLinuxAmiId": {
"ap-northeast-1": { "HVM64": "ami-393f1a57", "PV64": "ami-393f1a57" },
"ap-northeast-2": { "HVM64": "ami-249b554a" },
"ap-southeast-1": { "HVM64": "ami-34bd7a57", "PV64": "ami-34bd7a57" },
"ap-southeast-2": { "HVM64": "ami-97d58af4", "PV64": "ami-97d58af4" },
"eu-central-1": { "HVM64": "ami-794a5915", "PV64": "ami-794a5915" },
"eu-west-1": { "HVM64": "ami-10e43b63", "PV64": "ami-10e43b63" },
"sa-east-1": { "HVM64": "ami-d412aab8", "PV64": "ami-d412aab8" },
"us-east-1": { "HVM64": "ami-60b6c60a", "PV64": "ami-60b6c60a" },
"us-west-1": { "HVM64": "ami-f7e48897", "PV64": "ami-f7e48897" },
"us-west-2": { "HVM64": "ami-b03420d1", "PV64": "ami-b03420d1" }
},
"natAmiId": {
"ap-northeast-1": {"HVM64": "ami-5f840e5e"},
"ap-southeast-1": {"HVM64": "ami-780a432a"},
"ap-southeast-2": {"HVM64": "ami-0154c73b"},
"eu-west-1": {"HVM64": "ami-ed352799"},
"sa-east-1": {"HVM64": "ami-7660c56b"},
"us-east-1": {"HVM64": "ami-4f9fee26"},
"us-west-1": {"HVM64": "ami-7850793d"},
"us-west-2": {"HVM64": "ami-69ae8259"}
},
"centOS65AmiId": {
"ap-northeast-1": {"HVM64": "ami-81294380"},
"ap-southeast-1": {"HVM64": "ami-a08fd9f2"},
"ap-southeast-2": {"HVM64": "ami-e7138ddd"},
"eu-west-1": {"HVM64": "ami-42718735"},
"sa-east-1": {"HVM64": "ami-7d02a260"},
"us-east-1": {"HVM64": "ami-8997afe0"},
"us-west-1": {"HVM64": "ami-1a013c5f"},
"us-west-2": {"HVM64": "ami-b6bdde86"}
},
"ubuntu1404LtsAmiId": {
"ap-northeast-1": {"HVM64": "ami-bddaa2bc"},
"ap-southeast-1": {"HVM64": "ami-9a7724c8"},
"ap-southeast-2": {"HVM64": "ami-43128a79"},
"eu-west-1": {"HVM64": "ami-896c96fe"},
"sa-east-1": {"HVM64": "ami-7315b76e"},
"us-east-1": {"HVM64": "ami-018c9568"},
"us-west-1": {"HVM64": "ami-ee4f77ab"},
"us-west-2": {"HVM64": "ami-6ac2a85a"}
}
}
}
82 changes: 4 additions & 78 deletions src/environmentbase/data/config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,9 @@
"environment_name": "environmentbase",
"monitor_stack": false,
"write_stack_outputs": false,
"stack_outputs_directory": "stack_outputs",
"valid_regions": [
"ap-northeast-1",
"ap-northeast-2",
"ap-southeast-1",
"ap-southeast-2",
"eu-central-1",
"eu-west-1",
"sa-east-1",
"us-east-1",
"us-west-1",
"us-west-2"
]
"stack_outputs_directory": "stack_outputs"
},
"template": {
"ami_map_file": "ami_cache.json",
"description": "Environment Generator Base deployment",
"s3_bucket": "dualspark",
"s3_prefix": "templates",
Expand All @@ -39,68 +26,7 @@
"aws_access_key_id": null,
"aws_secret_access_key": null
},
"network": {
"network_cidr_base": "10.0.0.0",
"network_cidr_size": "16",
"az_count": 3,
"subnet_types": [
"public",
"private"
],
"subnet_config": [
{
"type": "public",
"size": "18",
"name": "public"
},
{
"type": "private",
"size": "22",
"name": "private"
},
],
},
"nat": {
"instance_type": "t2.micro",
"enable_ntp": false
},
"instancetype_to_arch": {
"c1.medium": "PV64", "c1.xlarge": "PV64", "c3.2xlarge": "HVM64", "c3.4xlarge": "HVM64",
"c3.8xlarge": "HVM64", "c3.large": "HVM64", "c3.xlarge": "HVM64", "c4.2xlarge": "HVM64",
"c4.4xlarge": "HVM64", "c4.8xlarge": "HVM64", "c4.large": "HVM64", "c4.xlarge": "HVM64",
"cc2.8xlarge": "PV64", "cg1.4xlarge": "PV64", "cr1.8xlarge": "PV64", "d2.2xlarge": "HVM64",
"d2.4xlarge": "HVM64", "d2.8xlarge": "HVM64", "d2.xlarge": "HVM64", "g2.2xlarge": "HVMG2",
"g2.8xlarge": "HVM64", "hi1.4xlarge": "PV64", "hs1.8xlarge": "PV64", "i2.2xlarge": "HVM64",
"i2.4xlarge": "HVM64", "i2.8xlarge": "HVM64", "i2.xlarge": "HVM64", "m1.large": "PV64",
"m1.medium": "PV64", "m1.small": "PV64", "m1.xlarge": "PV64", "m2.2xlarge": "PV64",
"m2.4xlarge": "PV64", "m2.xlarge": "PV64", "m3.2xlarge": "HVM64", "m3.large": "HVM64",
"m3.medium": "HVM64", "m3.xlarge": "HVM64", "m4.10xlarge": "HVM64", "m4.2xlarge": "HVM64",
"m4.4xlarge": "HVM64", "m4.large": "HVM64", "m4.xlarge": "HVM64", "r3.2xlarge": "HVM64",
"r3.4xlarge": "HVM64", "r3.8xlarge": "HVM64", "r3.large": "HVM64", "r3.xlarge": "HVM64",
"t1.micro": "PV64", "t2.large": "HVM64", "t2.medium": "HVM64", "t2.micro": "HVM64",
"t2.nano": "HVM64", "t2.small": "HVM64"
},
"image_map": {
"amazonLinuxAmiId": {
"ap-northeast-1": { "HVM64": "ami-cbf90ecb", "PV64": "ami-27f90e27" },
"ap-southeast-1": { "HVM64": "ami-68d8e93a", "PV64": "ami-acd9e8fe" },
"ap-southeast-2": { "HVM64": "ami-fd9cecc7", "PV64": "ami-ff9cecc5" },
"eu-central-1": { "HVM64": "ami-a8221fb5", "PV64": "ami-ac221fb1" },
"eu-west-1": { "HVM64": "ami-a10897d6", "PV64": "ami-bf0897c8" },
"sa-east-1": { "HVM64": "ami-b52890a8", "PV64": "ami-bb2890a6" },
"us-east-1": { "HVM64": "ami-1ecae776", "PV64": "ami-1ccae774" },
"us-west-1": { "HVM64": "ami-d114f295", "PV64": "ami-d514f291" },
"us-west-2": { "HVM64": "ami-e7527ed7", "PV64": "ami-ff527ecf" }
},
"natAmiId": {
"ap-northeast-1": {"HVM64": "ami-5f840e5e"},
"ap-southeast-1": {"HVM64": "ami-780a432a"},
"ap-southeast-2": {"HVM64": "ami-0154c73b"},
"eu-west-1": {"HVM64": "ami-ed352799"},
"sa-east-1": {"HVM64": "ami-7660c56b"},
"us-east-1": {"HVM64": "ami-4f9fee26"},
"us-west-1": {"HVM64": "ami-7850793d"},
"us-west-2": {"HVM64": "ami-69ae8259"}
}
}
"valid_regions": !include region_map.json,
"instancetype_to_arch": !include instancetype_to_arch.json,
"image_map": !include ami_cache.json
}
8 changes: 3 additions & 5 deletions src/environmentbase/data/config_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@
# Enable monitoring of stack event stream
"monitor_stack": "bool",
# Directory for writing stack outputs
"stack_outputs_directory": "basestring",
"valid_regions": "list"
"stack_outputs_directory": "basestring"
},
"template": {
# Name of json file containing mapping labels to AMI ids
"ami_map_file": "basestring",
# Canned ACL permissions for the template file upload
# Legal values: private, public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control
"s3_upload_acl": "basestring",
Expand All @@ -38,5 +35,6 @@
},
"instancetype_to_arch": {
"*": "basestring"
}
},
"valid_regions": "list"
}
23 changes: 23 additions & 0 deletions src/environmentbase/data/instancetype_to_arch.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
# Previous gen instance types (paravirtualized)
"c1.medium": "PV64", "c1.xlarge": "PV64",
"cc2.8xlarge": "PV64",
"cg1.4xlarge": "PV64",
"cr1.8xlarge": "PV64",
"hi1.4xlarge": "PV64",
"hs1.8xlarge": "PV64",
"m1.large": "PV64", "m1.medium": "PV64", "m1.small": "PV64", "m1.xlarge": "PV64",
"m2.2xlarge": "PV64", "m2.4xlarge": "PV64", "m2.xlarge": "PV64",
"t1.micro": "PV64",

# Current gen instance types (hardware virtualized)
"c3.2xlarge": "HVM64", "c3.4xlarge": "HVM64", "c3.8xlarge": "HVM64", "c3.large": "HVM64", "c3.xlarge": "HVM64", "c4.2xlarge": "HVM64",
"c4.4xlarge": "HVM64", "c4.8xlarge": "HVM64", "c4.large": "HVM64", "c4.xlarge": "HVM64",
"d2.2xlarge": "HVM64", "d2.4xlarge": "HVM64", "d2.8xlarge": "HVM64", "d2.xlarge": "HVM64",
"g2.2xlarge": "HVMG2", "g2.8xlarge": "HVM64",
"i2.2xlarge": "HVM64", "i2.4xlarge": "HVM64", "i2.8xlarge": "HVM64", "i2.xlarge": "HVM64",
"m3.2xlarge": "HVM64", "m3.large": "HVM64", "m3.medium": "HVM64", "m3.xlarge": "HVM64",
"m4.10xlarge": "HVM64", "m4.2xlarge": "HVM64", "m4.4xlarge": "HVM64", "m4.large": "HVM64", "m4.xlarge": "HVM64",
"r3.2xlarge": "HVM64", "r3.4xlarge": "HVM64", "r3.8xlarge": "HVM64", "r3.large": "HVM64", "r3.xlarge": "HVM64",
"t2.large": "HVM64", "t2.medium": "HVM64", "t2.micro": "HVM64", "t2.nano": "HVM64", "t2.small": "HVM64"
}
12 changes: 12 additions & 0 deletions src/environmentbase/data/region_map.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
"ap-northeast-1",
"ap-northeast-2",
"ap-southeast-1",
"ap-southeast-2",
"eu-central-1",
"eu-west-1",
"sa-east-1",
"us-east-1",
"us-west-1",
"us-west-2"
]
Loading