Skip to content

Commit

Permalink
feat: update variables
Browse files Browse the repository at this point in the history
  • Loading branch information
@JianLi-Expedia
JianLi-Expedia committed Apr 18, 2024
1 parent d651cdb commit 5f60ef2
Show file tree
Hide file tree
Showing 4 changed files with 27 additions and 9 deletions.
4 changes: 2 additions & 2 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [4.4.0] - 2024-04-17
## [4.4.0] - 2024-04-18
### Added
- Add new variable `pod_resource_guaranteed` to support set pod resource guaranteed.
- Add new variable `cpu_limit` and `memory_limit` to support set pod resource guaranteed.

## [4.3.2] - 2024-03-26
### Updated
Expand Down
3 changes: 2 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
| aws_region | AWS region to use for resources. | string | - | yes |
| bastion_ssh_key_secret_name | Secret name in AWS Secrets Manager which stores the private key used to log in to bastions. The secret's key should be `private_key` and the value should be stored as a base64 encoded string. Max character limit for a secret's value is 4096. | string | `` | no |
| cpu | The number of CPU units to reserve for the Waggle Dance container. Valid values can be 256, 512, 1024, 2048 and 4096. Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `1024` | no |
| cpu_limit | The number of CPU limit units to reserve for the Waggle Dance container. Valid values can be 256, 512, 1024, 2048 and 4096. It will use `cpu` * 1.25 if not specified. Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `1024` | no |
| cpu_scale_in_cooldown | Cool down time(seconds) of scale in task by cpu usage | number | 300 | no |
| cpu_scale_out_cooldown | Cool down time(seconds) of scale out task by cpu usage | number | 120 | no |
| default_latency | Latency used for other (not primary) metastores that don't override it in their own configurations. See `latency` parameter in https://github.com/ExpediaGroup/waggle-dance/blob/main/README.md. | number | `0` | no |
Expand All @@ -32,6 +33,7 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
| k8s_max_replica_count | Max Number of k8s pod replicas to create. | number | `10` | no |
| local_metastores | List of federated Metastore endpoints directly accessible on the local network. See section [`local_metastores`](#local_metastores) for more info. | list | `<list>` | no |
| memory | The amount of memory (in MiB) used to allocate for the Waggle Dance container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `4096` | no |
| memory_limit | The amount of memory limit (in MiB) used to allocate for the Waggle Dance container, it will use `memory` * 1.25 if the limit is not specified. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `null` | no |
| primary_metastore_access_type | Primary Hive Metastore access control type. | string | `READ_AND_WRITE_ON_DATABASE_WHITELIST` | no |
| primary_metastore_host | Primary Hive Metastore hostname configured in Waggle Dance. | string | `localhost` | no |
| primary_metastore_port | Primary Hive Metastore port | string | `9083` | no |
Expand All @@ -41,7 +43,6 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
| primary_metastore_mapped_databases | List of Hive databases mapped from primary Metastore. | list | `<list>` | no |
| primary_metastore_read_only_host | Primary Hive Metastore READ ONLY hostname configured in Waggle Dance. Optional. | string | `` | no |
| primary_metastore_read_only_port | Primary Hive Metastore READ ONLY port configured in Waggle Dance. Optional. | string | `9083` | no |
| pod_resource_guaranteed | Guarantee Pod required resources. Optional. | bool | `false` | no |
| remote_metastores | List of VPC endpoint services to federate Metastores in other accounts. See section [`remote_metastores`](#remote_metastores) for more info. | list | `<list>` | no |
| remote_region_metastores | List of VPC endpoint services to federate Metastores in other region,other accounts. The actual data from tables in these metastores can be accessed using Alluxio caching instead of reading the data from S3 directly. See section [`remote_region_metastores`](#remote_region_metastores) for more info. | list | `<list>` | no |
| secondary_vpcs | List of VPCs to associate with Service Discovery namespace. | list | `<list>` | no |
Expand Down
4 changes: 2 additions & 2 deletions k8s.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@

locals {
heapsize = ceil((var.memory * 85) / 100)
memory_limit = var.pod_resource_guaranteed ? var.memory : ceil((var.memory * 120) / 100)
memory_limit = length(var.memory_limit) != 0 ? var.memory_limit : ceil((var.memory * 120) / 100)
actuator_port = 18000
wd_port = 48869
k8s_cpu = var.cpu / 1024
k8s_cpu_limit = var.pod_resource_guaranteed ? var.cpu / 1024 : (var.cpu / 1024) * 1.25
k8s_cpu_limit = length(var.cpu_limit) != 0 ? var.cpu_limit / 1024 : (var.cpu / 1024) * 1.25
}

resource "kubernetes_service_account" "waggle_dance" {
Expand Down
Loading

0 comments on commit 5f60ef2

Please sign in to comment.