Skip to content

Commit

Permalink
Airflow variables configuration (#436)
Browse files Browse the repository at this point in the history
  • Loading branch information
bcarranza authored May 24, 2023
1 parent 85bbc1e commit 3b3d5d6
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 2 deletions.
3 changes: 2 additions & 1 deletion terraform-modules/aws/airflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ No requirements.

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_airflow_configuration_options"></a> [airflow\_configuration\_options](#input\_airflow\_configuration\_options) | (Optional) The airflow\_configuration\_options parameter specifies airflow override options. Check the Official documentation for all possible configuration options. | `map(string)` | `null` | no |
| <a name="input_airflow_name"></a> [airflow\_name](#input\_airflow\_name) | Airflow name | `string` | `"airflow"` | no |
| <a name="input_airflow_version"></a> [airflow\_version](#input\_airflow\_version) | (Optional) Airflow version of your environment, will be set by default to the latest version that MWAA supports. | `string` | `null` | no |
| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | The AWS region | `string` | `"us-east-1"` | no |
Expand All @@ -42,6 +43,7 @@ No requirements.
| <a name="input_iam_extra_policies"></a> [iam\_extra\_policies](#input\_iam\_extra\_policies) | List of additional policies to create and attach to the IAM role | <pre>list(object({<br> name_prefix = string<br> policy_json = string<br> }))</pre> | `[]` | no |
| <a name="input_max_workers"></a> [max\_workers](#input\_max\_workers) | (Optional) The maximum number of workers that can be automatically scaled up. Value need to be between 1 and 25. Will be 10 by default. | `number` | `10` | no |
| <a name="input_min_workers"></a> [min\_workers](#input\_min\_workers) | (Optional) The minimum number of workers that you want to run in your environment. Will be 1 by default. | `number` | `1` | no |
| <a name="input_requirements_s3_path"></a> [requirements\_s3\_path](#input\_requirements\_s3\_path) | The S3 path for the MWAA requirements file. | `string` | `""` | no |
| <a name="input_scheduler_log_level"></a> [scheduler\_log\_level](#input\_scheduler\_log\_level) | The log level: INFO \| WARNING \| ERROR \| CRITICAL | `string` | `"INFO"` | no |
| <a name="input_sg_extra_ids"></a> [sg\_extra\_ids](#input\_sg\_extra\_ids) | List of additional sg to create and attach to Airflow | `list(string)` | `[]` | no |
| <a name="input_source_bucket_arn"></a> [source\_bucket\_arn](#input\_source\_bucket\_arn) | The Dag's S3 bucket arn: arn:aws:s3:::bucketname | `string` | `"s3://foo"` | no |
Expand All @@ -61,7 +63,6 @@ No requirements.
| <a name="output_arn"></a> [arn](#output\_arn) | n/a |
| <a name="output_webserver_url"></a> [webserver\_url](#output\_webserver\_url) | n/a |


## Rough edges
In default section we have a statement policy as the following
```
Expand Down
3 changes: 2 additions & 1 deletion terraform-modules/aws/airflow/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ resource "aws_mwaa_environment" "this" {
execution_role_arn = module.iam_assumable_role_admin.iam_role_arn
webserver_access_mode = var.webserver_access_mode
requirements_s3_path = var.requirements_s3_path

airflow_configuration_options = var.airflow_configuration_options

logging_configuration {
dag_processing_logs {
enabled = true
Expand Down
8 changes: 8 additions & 0 deletions terraform-modules/aws/airflow/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,11 @@ variable "requirements_s3_path" {
type = string
default = ""
}

#You can looking for variables in the following link:
#https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-env-variables.html
variable "airflow_configuration_options" {
description = "The Airflow override options"
type = any
default = null
}

0 comments on commit 3b3d5d6

Please sign in to comment.