Skip to content

Commit

Permalink
Added keepalive config for EKS
Browse files Browse the repository at this point in the history
  • Loading branch information
@patduin
patduin committed Jan 18, 2024
1 parent 8696cd5 commit ce48dbc
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 3 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [4.2.3] - 2024-01-19
### Fixed
- Enable optional sysctl settings for EKS. To overwrite TCP keepalive settings.

## [4.2.2] - 2024-01-05
### Fixed
- Conditional reading of Secrets from SecretManager fix.
Expand Down
7 changes: 4 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,10 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
| root_vol_type | Waggle Dance EC2 root volume type. | string | `gp2` | no |
| root_vol_size | Waggle Dance EC2 root volume size. | string | `10` | no |
| enable_query_functions_across_all_metastores | This controls the thrift call for `get_all_functions`. It is generally used to initialize a client and get built-in functions and registered UDF's from a metastore. Setting this to `false` is more performant as WD then only gets the functions from the `primary` metastore. However, setting this to `true` will collate results by calling `get_all_functions` from all configured metastores. This could be potentially slow if some of the metastores are slow to respond. If all the metastores configured are of the same version and no additional UDF's are installed, then WD gets the same functions back so it's not very useful to call this across metastores. For backwards compatibility, this property can be set to `true`. Further read: https://github.com/ExpediaGroup/waggle-dance#server | bool | false | no |
| tcp_keepalive_time | Sets net.ipv4.tcp_keepalive_time (seconds), currently only supported in ECS. | number | `200` | no |
| tcp_keepalive_intvl | Sets net.ipv4.tcp_keepalive_intvl (seconds), currently only supported in ECS. | number | `30` | no |
| tcp_keepalive_probes | Sets net.ipv4.tcp_keepalive_probes (seconds), currently only supported in ECS. | number | `2` | no |
| enable_sysctl_config_in_eks | Enable sysctl configuration for Hive Metastore. For EKS you need to allow this on your cluster (https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/ check EKS version for details). Also see tcp_keepalive_* variables. | bool | false | no |
| tcp_keepalive_time | Sets net.ipv4.tcp_keepalive_time (seconds), currently only supported in ECS. | number | `200` | no |
| tcp_keepalive_intvl | Sets net.ipv4.tcp_keepalive_intvl (seconds), currently only supported in ECS. | number | `30` | no |
| tcp_keepalive_probes | Sets net.ipv4.tcp_keepalive_probes (seconds), currently only supported in ECS. | number | `2` | no |
| datadog_key_secret_name | Name of the secret containing the DataDog API key. This needs to be created manually in AWS secrets manager. | string | | no |
| datadog_agent_version | Version of the Datadog Agent running in the ECS cluster. | string | `7.46.0-jmx` | no |
| include_datadog_agent | Whether to include the datadog-agent container alongside Waggledance. | string | bool | no |
Expand Down
15 changes: 15 additions & 0 deletions k8s.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,21 @@ resource "kubernetes_deployment_v1" "waggle_dance" {
spec {
service_account_name = kubernetes_service_account.waggle_dance[0].metadata.0.name
automount_service_account_token = true
dynamic "security_context" {
for_each = var.enable_sysctl_config_in_eks ? ["enabled"] : []
content {
sysctl = [{
name="net.ipv4.tcp_keepalive_time"
value="${var.tcp_keepalive_time}"
},{
name="net.ipv4.tcp_keepalive_intvl"
value="${var.tcp_keepalive_intvl}"
},{
name="net.ipv4.tcp_keepalive_probes"
value="${var.tcp_keepalive_probes}"
}]
}
}
container {
image = "${var.docker_image}:${var.docker_version}"
name = local.instance_alias
Expand Down
6 changes: 6 additions & 0 deletions variables.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,12 @@ variable "datadog_metrics_enabled" {
default = false
}

variable "enable_sysctl_config_in_eks" {
description = "Enable sysctl configuration for Hive Metastore. For EKS you need to allow this on your cluster (https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/ check EKS version for details). Also see tcp_keepalive_* variables."
type = bool
default = false
}

variable "tcp_keepalive_time" {
description = "Sets net.ipv4.tcp_keepalive_time (seconds), currently only supported in ECS."
type = number
Expand Down

0 comments on commit ce48dbc

Please sign in to comment.