Skip to content

This module creates an S3 bucket suitable for receiving logs from other AWS services such as S3, CloudFront, and CloudTrail

License

Notifications You must be signed in to change notification settings

appzen-oss/terraform-aws-s3-log-storage

 
 

Repository files navigation

This module creates an S3 bucket suitable for receiving logs from other AWS services such as S3, CloudFront, and CloudTrails.

It implements a configurable log retention policy, which allows you to efficiently manage logs across different storage classes (e.g. Glacier) and ultimately expire the data altogether.

It enables server-side default encryption.

https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html

It blocks public access to the bucket by default.

https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html


Usage

IMPORTANT: We do not pin modules to versions in our examples because of the difficulty of keeping the versions in the documentation in sync with the latest released versions. We highly recommend that in your code you pin the version to the exact version you are using so that your infrastructure remains stable, and update versions in a systematic way so that they do not catch you by surprise.

Also, because of a bug in the Terraform registry (hashicorp/terraform#21417), the registry shows many of our inputs as required when in fact they are optional. The table below correctly indicates which inputs are required.

module "log_storage" {
  source = "cloudposse/s3-log-storage/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"
  name                     = "logs"
  stage                    = "test"
  namespace                = "eg"
  acl                      = "log-delivery-write"
  standard_transition_days = 30
  glacier_transition_days  = 60
  expiration_days          = 90
}

Makefile Targets

Available targets:

  help                                Help screen
  help/all                            Display help for all targets
  help/short                          This help short screen
  lint                                Lint terraform code

Requirements

Name Version
terraform >= 0.13.0
aws >= 2.0

Providers

Name Version
aws >= 2.0

Inputs

Name Description Type Default Required
abort_incomplete_multipart_upload_days Maximum time (in days) that you want to allow multipart uploads to remain in progress number 5 no
access_log_bucket_name Name of the S3 bucket where s3 access log will be sent to string "" no
acl The canned ACL to apply. We recommend log-delivery-write for compatibility with AWS services string "log-delivery-write" no
additional_tag_map Additional tags for appending to tags_as_list_of_maps. Not added to tags. map(string) {} no
analytics_bucket_name Name of the S3 bucket where s3 analytics report will be sent to string "" no
attributes Additional attributes (e.g. 1) list(string) [] no
block_public_acls Set to false to disable the blocking of new public access lists on the bucket bool true no
block_public_policy Set to false to disable the blocking of new public policies on the bucket bool true no
context Single object for setting entire context at once.
See description of individual variables for details.
Leave string and numeric variables as null to use default value.
Individual variable settings (non-null) override settings in context object,
except for attributes, tags, and additional_tag_map, which are merged.
any
{
"additional_tag_map": {},
"attributes": [],
"delimiter": null,
"enabled": true,
"environment": null,
"id_length_limit": null,
"label_key_case": null,
"label_order": [],
"label_value_case": null,
"name": null,
"namespace": null,
"regex_replace_chars": null,
"stage": null,
"tags": {}
}
no
delimiter Delimiter to be used between namespace, environment, stage, name and attributes.
Defaults to - (hyphen). Set to "" to use no delimiter at all.
string null no
enable_glacier_transition Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files bool true no
enabled Set to false to prevent the module from creating any resources bool null no
enabled_analytics Set to false to prevent the module from creating s3 analytics bucket bool false no
enabled_bucket_notification Set to false to prevent the module from creating bucket notification bool false no
environment Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' string null no
expiration_days Number of days after which to expunge the objects number 90 no
event_notifications bucket event multiple-notification, only required if enabled_bucket_notification is true list [] no
force_destroy (Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable bool false no
glacier_transition_days Number of days after which to move the data to the glacier storage tier number 60 no
id_length_limit Limit id to this many characters (minimum 6).
Set to 0 for unlimited length.
Set to null for default, which is 0.
Does not affect id_full.
number null no
ignore_public_acls Set to false to disable the ignoring of public access lists on the bucket bool true no
kms_master_key_arn The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms string "" no
label_key_case The letter case of label keys (tag names) (i.e. name, namespace, environment, stage, attributes) to use in tags.
Possible values: lower, title, upper.
Default value: title.
string null no
label_order The naming order of the id output and Name tag.
Defaults to ["namespace", "environment", "stage", "name", "attributes"].
You can omit any of the 5 elements, but at least one must be present.
list(string) null no
label_value_case The letter case of output label values (also used in tags and id).
Possible values: lower, title, upper and none (no transformation).
Default value: lower.
string null no
lifecycle_rule_enabled Enable lifecycle events on this bucket bool true no
lifecycle_rules lifecycle values for multiple lifecycle rules list [] no
lifecycle_tags Tags filter. Used to manage object lifecycle events map(string) {} no
name Solution name, e.g. 'app' or 'jenkins' string null no
namespace Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' string null no
noncurrent_version_expiration_days Specifies when noncurrent object versions expire number 90 no
noncurrent_version_transition_days Specifies when noncurrent object versions transitions number 30 no
object_ownership Object ownership. Valid values: BucketOwnerPreferred or ObjectWriter string BucketOwnerPreferred no
policy A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy string "" no
queue_arn SQS Queue ARN to send S3 object creation events, only required if enabled_bucket_notification is true string null no
regex_replace_chars Regex to replace chars with empty string in namespace, environment, stage and name.
If not set, "/[^a-zA-Z0-9-]/" is used to remove all characters other than hyphens, letters and digits.
string null no
restrict_public_buckets Set to false to disable the restricting of making the bucket public bool true no
sse_algorithm The server-side encryption algorithm to use. Valid values are AES256 and aws:kms string "AES256" no
stage Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' string null no
standard_transition_days Number of days to persist in the standard storage tier before moving to the infrequent access tier number 30 no
tags Additional tags (e.g. map('BusinessUnit','XYZ') map(string) {} no
versioning_enabled A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket bool true no

Outputs

Name Description
bucket_arn Bucket ARN
bucket_domain_name FQDN of bucket
bucket_id Bucket Name (aka ID)
enabled Is module enabled
prefix Prefix configured for lifecycle rules

About

This module creates an S3 bucket suitable for receiving logs from other AWS services such as S3, CloudFront, and CloudTrail

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • HCL 83.0%
  • Makefile 11.1%
  • Go 5.9%