Terraform module to create an Elasticsearch resource on AWS.
We eat, drink, sleep and most importantly love DevOps. We are working towards strategies for standardizing architecture while ensuring security for the infrastructure. We are strong believer of the philosophy Bigger problems are always solved by breaking them into smaller manageable problems. Resonating with microservices architecture, it is considered best-practice to run database, cluster, storage in smaller connected yet manageable pieces within the infrastructure.
This module is basically combination of Terraform open source and includes automatation tests and examples. It also helps to create and improve your infrastructure with minimalistic code instead of maintaining the whole infrastructure code yourself.
We have fifty plus terraform modules. A few of them are comepleted and are available for open source usage while a few others are in progress.
This module has a few dependencies:
IMPORTANT: Since the master
branch used in source
varies based on new modifications, we suggest that you use the release versions here.
Here are examples of how you can use this module in your inventory structure:
module "elasticsearch" {
source = "clouddrove/elasticsearch/aws"
name = "es"
environment = "test"
label_order = ["name", "environment"]
domain_name = "clouddrove"
#IAM
enable_iam_service_linked_role = false
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
#Networking
vpc_enabled = false
availability_zone_count = 2
zone_awareness_enabled = true
allowed_cidr_blocks = ["51.79.69.69"]
#ES
elasticsearch_version = "7.8"
instance_type = "c5.large.elasticsearch"
instance_count = 2
# Volumes
volume_size = 30
volume_type = "gp2"
#DNS
dns_enabled = false
es_hostname = "es"
kibana_hostname = "kibana"
dns_zone_id = false
advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
#Cognito
cognito_enabled = false
user_pool_id = ""
identity_pool_id = ""
#logs
log_publishing_index_enabled = true
log_publishing_search_enabled = true
log_publishing_application_enabled = true
log_publishing_audit_enabled = false
}
}
module "elasticsearch" {
source = "clouddrove/elasticsearch/aws"
name = "es"
environment = "test"
label_order = ["name", "environment"]
domain_name = "clouddrove"
#IAM
enable_iam_service_linked_role = false
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
#Networking
vpc_enabled = true
security_group_ids = [module.security_group.security_group_ids]
subnet_ids = tolist(module.public_subnets.public_subnet_id)
availability_zone_count = length(module.public_subnets.public_subnet_id)
zone_awareness_enabled = true
#ES
elasticsearch_version = "7.8"
instance_type = "c5.large.elasticsearch"
instance_count = 2
# Volumes
volume_size = 30
volume_type = "gp2"
#DNS
dns_enabled = false
es_hostname = "es"
kibana_hostname = "kibana"
dns_zone_id = false
advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
#Cognito
cognito_enabled = false
user_pool_id = ""
identity_pool_id = ""
#logs
log_publishing_index_enabled = true
log_publishing_search_enabled = true
log_publishing_application_enabled = true
log_publishing_audit_enabled = false
}
module "elasticsearch" {
source = "clouddrove/elasticsearch/aws"
name = "es"
environment = "test"
label_order = ["name", "environment"]
#IAM
enable_iam_service_linked_role = false
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
#Networking
vpc_enabled = false
allowed_cidr_blocks = ["51.79.69.69"]
#Es
elasticsearch_version = "7.8"
instance_type = "c5.large.elasticsearch"
instance_count = 1
#Volume
volume_size = 30
volume_type = "gp2"
#Logs
log_publishing_application_enabled = true
log_publishing_search_cloudwatch_log_group_arn = true
log_publishing_index_cloudwatch_log_group_arn = true
#Cognito
cognito_enabled = false
user_pool_id = ""
identity_pool_id = ""
#DNS
kibana_hostname = "kibana"
dns_zone_id = "Z1XJD7SSBKXLC1"
dns_enabled = false
es_hostname = "es"
advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
enforce_https = true
tls_security_policy = "Policy-Min-TLS-1-0-2019-07"
public_enabled = false
}
module "elasticsearch" {
source = "clouddrove/elasticsearch/aws"
name = "es"
environment = "test"
label_order = ["name", "environment"]
#IAM
enable_iam_service_linked_role = false
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
#Networking
vpc_enabled = true
security_group_ids = [module.security_group.security_group_ids]
subnet_ids = tolist(module.public_subnets.public_subnet_id)
#Es
elasticsearch_version = "7.8"
instance_type = "c5.large.elasticsearch"
instance_count = 1
#Volume
volume_size = 30
volume_type = "gp2"
#Logs
log_publishing_application_enabled = true
log_publishing_search_cloudwatch_log_group_arn = true
log_publishing_index_cloudwatch_log_group_arn = true
#Cognito
cognito_enabled = false
user_pool_id = ""
identity_pool_id = ""
#DNS
kibana_hostname = "kibana"
dns_zone_id = "Z1XJD7SSBKXLC1"
dns_enabled = false
es_hostname = "es"
advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
enforce_https = true
tls_security_policy = "Policy-Min-TLS-1-0-2019-07"
public_enabled = false
}
Note: There are some type of instances which not support encryption and EBS option, Please read about this here. Also, there are some limitation for instance type, Please read here
Name | Description | Type | Default | Required |
---|---|---|---|---|
advanced_options | Key-value string pairs to specify advanced configuration options. | map(string) |
{} |
no |
advanced_security_options_enabled | AWS Elasticsearch Kibana enchanced security plugin enabling (forces new resource) | bool |
false |
no |
advanced_security_options_internal_user_database_enabled | Whether to enable or not internal Kibana user database for ELK OpenDistro security plugin | bool |
false |
no |
advanced_security_options_master_user_arn | ARN of IAM user who is to be mapped to be Kibana master user (applicable if advanced_security_options_internal_user_database_enabled set to false) | string |
"" |
no |
advanced_security_options_master_user_name | Master user username (applicable if advanced_security_options_internal_user_database_enabled set to true) | string |
"" |
no |
advanced_security_options_master_user_password | Master user password (applicable if advanced_security_options_internal_user_database_enabled set to true) | string |
"" |
no |
allowed_cidr_blocks | List of CIDR blocks to be allowed to connect to the cluster | list(string) |
[] |
no |
attributes | Additional attributes (e.g. 1 ). |
list(any) |
[] |
no |
auto_tune_desired_state | Desired state of Auto-Tune for the domain. Valid values are ENABLED, DISABLED. | string |
"DISABLED" |
no |
automated_snapshot_start_hour | Hour at which automated snapshots are taken, in UTC. | number |
0 |
no |
availability_zone_count | Number of Availability Zones for the domain to use. | number |
2 |
no |
cloudwatch_kms_key_id | The KMS key ID to encrypt the Cloudwatch logs. | string |
"" |
no |
cognito_enabled | Set to false to prevent enable cognito. | bool |
true |
no |
custom_endpoint | Fully qualified domain for custom endpoint. | string |
"" |
no |
custom_endpoint_certificate_arn | ACM certificate ARN for custom endpoint. | string |
"" |
no |
custom_endpoint_enabled | Whether to enable custom endpoint for the Elasticsearch domain. | bool |
false |
no |
dedicated_master_count | Number of dedicated master nodes in the cluster. | number |
0 |
no |
dedicated_master_enabled | Indicates whether dedicated master nodes are enabled for the cluster. | bool |
false |
no |
dedicated_master_type | Instance type of the dedicated master nodes in the cluster. | string |
"t2.small.elasticsearch" |
no |
dns_enabled | Flag to control the dns_enable. | bool |
false |
no |
dns_zone_id | Route53 DNS Zone ID to add hostname records for Elasticsearch domain and Kibana. | string |
"" |
no |
domain_name | Domain name. | string |
"" |
no |
elasticsearch_version | Version of Elasticsearch to deploy. | string |
"6.5" |
no |
enable_iam_service_linked_role | Whether to enabled service linked with role. | bool |
false |
no |
enable_logs | enable logs | bool |
true |
no |
enabled | Set to false to prevent the module from creating any resources. | bool |
true |
no |
encrypt_at_rest_enabled | Whether to enable encryption at rest. | bool |
true |
no |
encryption_enabled | Whether to enable node-to-node encryption. | bool |
true |
no |
enforce_https | Whether or not to require HTTPS. | bool |
true |
no |
environment | Environment (e.g. prod , dev , staging ). |
string |
"" |
no |
es_hostname | The Host name of elasticserch. | string |
"" |
no |
iam_actions | List of actions to allow for the IAM roles, e.g. es:ESHttpGet , es:ESHttpPut , es:ESHttpPost . |
list(string) |
[] |
no |
identity_pool_id | ID of the Cognito Identity Pool to use. | string |
"" |
no |
instance_count | Number of data nodes in the cluster. | number |
4 |
no |
instance_type | Elasticsearch instance type for data nodes in the cluster. | string |
"t2.small.elasticsearch" |
no |
iops | The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type. | number |
0 |
no |
kibana_hostname | The Host name of kibana. | string |
"" |
no |
kms_key_id | The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key. | string |
"" |
no |
label_order | Label order, e.g. name ,application . |
list(any) |
[] |
no |
log_publishing_application_enabled | Specifies whether log publishing option for ES_APPLICATION_LOGS is enabled or not. | bool |
false |
no |
log_publishing_audit_enabled | Specifies whether log publishing option for AUDIT_LOGS is enabled or not. | bool |
true |
no |
log_publishing_index_enabled | Specifies whether log publishing option for INDEX_SLOW_LOGS is enabled or not. | bool |
false |
no |
log_publishing_search_enabled | Specifies whether log publishing option for SEARCH_SLOW_LOGS is enabled or not. | bool |
false |
no |
managedby | ManagedBy, eg 'CloudDrove'. | string |
"[email protected]" |
no |
name | Name (e.g. app or cluster ). |
string |
"" |
no |
repository | Terraform current module repo | string |
"https://github.com/clouddrove/terraform-aws-elasticsearch" |
no |
retention_in_days | Days of retention of cloudwatch. | number |
90 |
no |
rollback_on_disable | Whether to roll back to default Auto-Tune settings when disabling Auto-Tune. Valid values: DEFAULT_ROLLBACK or NO_ROLLBACK. | string |
"DEFAULT_ROLLBACK" |
no |
security_group_ids | Security Group IDs. | list(string) |
[] |
no |
subnet_ids | Subnet IDs. | list(string) |
[] |
no |
tls_security_policy | The name of the TLS security policy that needs to be applied to the HTTPS endpoint. | string |
"Policy-Min-TLS-1-0-2019-07" |
no |
ttl | The TTL of the record to add to the DNS zone to complete certificate validation. | string |
"300" |
no |
type | Type of DNS records to create. | string |
"CNAME" |
no |
user_pool_id | ID of the Cognito User Pool to use. | string |
"" |
no |
volume_size | EBS volumes for data storage in GB. | number |
0 |
no |
volume_type | Storage type of EBS volumes. | string |
"gp2" |
no |
vpc_enabled | Set to false if ES should be deployed outside of VPC. | bool |
true |
no |
warm_count | Number of UltraWarm nodes | number |
2 |
no |
warm_enabled | Whether AWS UltraWarm is enabled | bool |
false |
no |
warm_type | Type of UltraWarm nodes | string |
"ultrawarm1.medium.elasticsearch" |
no |
zone_awareness_enabled | Enable zone awareness for Elasticsearch cluster. | bool |
false |
no |
Name | Description |
---|---|
domain_arn | ARN of the Elasticsearch domain. |
tags | A mapping of tags to assign to the resource. |
In this module testing is performed with terratest and it creates a small piece of infrastructure, matches the output like ARN, ID and Tags name etc and destroy infrastructure in your AWS account. This testing is written in GO, so you need a GO environment in your system.
You need to run the following command in the testing folder:
go test -run Test
If you come accross a bug or have any feedback, please log it in our issue tracker, or feel free to drop us an email at [email protected].
If you have found it worth your time, go ahead and give us a ★ on our GitHub!
At CloudDrove, we offer expert guidance, implementation support and services to help organisations accelerate their journey to the cloud. Our services include docker and container orchestration, cloud migration and adoption, infrastructure automation, application modernisation and remediation, and performance engineering.
We are The Cloud Experts!
We ❤️ Open Source and you can check out our other modules to get help with your new Cloud ideas.