Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keda cron scaler #509

Merged
merged 13 commits into from
Jan 3, 2024
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Running various samples requires access to the Kyma runtime. There are also othe
## Advanced scenarios

| Name | Description | References |
| ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|-------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Redis and Kyma Functions](./redis-function/README.md) | This sample provides a Redis deployment and two serverless functions that interact with it | [Tutorial](https://developers.sap.com/tutorials/cp-kyma-redis-function.html) |
| [Cloud Integration Multi-cloud](./pi-scenario/README.md) | This sample details how to bi-directionally connect SAP Process Integration and Kyma | - |
| [Cloud Integration Neo](./pi-scenario/README-neo.md) | This sample details how to bi-directionally connect SAP Process Integration and Kyma in NEO | - |
Expand All @@ -123,10 +123,12 @@ Running various samples requires access to the Kyma runtime. There are also othe
| [Next.js app with Kyma eventing & Go backend connected to SAP HANA Cloud database](./nextjs-app-with-kyma-eventing/README.md) | This sample provides a tutorial for a conference registration app using Next.js | [Post](https://blogs.sap.com/2022/02/24/going-jamstack-with-kyma-runtime-building-a-high-performance-web-app/) |
| [Data Backup and Restore](./data-backup-and-restore/README.md) | This sample demonstrates performing a backup and restore for a stateful application where data is stored using Persistence Volume Claim. | - |
| [HandsOn DSAG Technology Days 2022](./dsagtt22/) | This sample gives a walk-through setting up a scenario combining on prem systems with Kyma Functions and the Event Mesh | - |
| [Query LDAP Users on on-premise](./sample-ldap/README.md) | This sample queries the LDAP users from an on premise LDAP Server via SAP Connectivity proxy | - |
| [Deploy Highly Available Workloads](./multi-zone-ha-deployment/README.md) | This sample demonstrates deploying highly available workloads in Kyma runtime | - |
| [Power of serverless with SAP BTP, Kyma runtime.](./kyma-serverless/README.md) | This sample demonstrates how to leverage latest features of kyma functions with SAP HANA Cloud and SAP libraries | [Post](https://blogs.sap.com/2023/02/06/power-of-serverless-with-sap-btp-kyma-runtime.-secrets-mounted-as-volumes./) |
| [Using the on-premise Docker registry with Kyma runtime](./on-premise-docker-registry/README.md) | This sample demonstrates how to pull images from the on-premise Docker registry for applications deployed on Kyma runtime | - |
| [Query LDAP Users on on-premise](./sample-ldap/README.md) | This sample queries the LDAP users from an on premise LDAP Server via SAP Connectivity proxy | - |
| [Deploy Highly Available Workloads](./multi-zone-ha-deployment/README.md) | This sample demonstrates deploying highly available workloads in Kyma runtime | - |
| [Power of serverless with SAP BTP, Kyma runtime.](./kyma-serverless/README.md) | This sample demonstrates how to leverage latest features of kyma functions with SAP HANA Cloud and SAP libraries | [Post](https://blogs.sap.com/2023/02/06/power-of-serverless-with-sap-btp-kyma-runtime.-secrets-mounted-as-volumes./) |
| [Using the on-premise Docker registry with Kyma runtime](./on-premise-docker-registry/README.md) | This sample demonstrates how to pull images from the on-premise Docker registry for applications deployed on Kyma runtime | - |
| [KEDA Cron based scaler](./keda-cron-scaler/README.md) | This sample demonstrates how to leverage KEDA Cron scaler for efficient scaling strategies. | - |


## Multitenancy and SaaS

Expand Down
149 changes: 149 additions & 0 deletions keda-cron-scaler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# SAP BTP Kyma Runtime: Leveraging KEDA module capabilities for efficient and cost-effective scaling
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

SAP Business Technology Platform (BTP) Kyma runtime is currently in the process of undergoing a pivotal change as it transitions to a modular architecture. This transformation will provide customers with the advantage of a la carte selection of components or capabilities, thereby reducing unnecessary overhead and complexity. One of the first modules to emerge within this novel framework is KEDA (Kubernetes Event-driven Autoscaling).
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

## Introduction to KEDA

[KEDA](https://keda.sh/), an open-source initiative that facilitates event-driven autoscaling for Kubernetes workloads, was originally developed by Microsoft and Red Hat. It has since become a sandbox project under the Cloud Native Computing Foundation (CNCF). KEDA focuses on autoscaling applications in response to events sourced from a variety of platforms, including Kafka, RabbitMQ, and cloud-specific services such as Azure Service Bus and Google Pub/Sub.

## The Benefits of KEDA

KEDA ushers in a new era of flexibility and efficiency in autoscaling. It enhances Kubernetes' capacity to support fine-grained autoscaling for event-driven workloads. Leveraging KEDA, you can dynamically scale your deployments from zero to any arbitrary number, contingent on the volume of events they are designed to process.

## Activating the KEDA Module in SAP BTP Kyma Runtime

You can activate KEDA like any other module by adhering to the official guidelines on how to [enable and disable a module](https://help.sap.com/docs/btp/sap-business-technology-platform/enable-and-disable-kyma-module).

## KEDA's Cron-Based Scaler

KEDA offers a broad range of scaling strategies, one of which is the **cron-based scaler**. This scaler allows you to schedule scaling actions according to the time of day, an invaluable feature for managing predictable fluctuations in workload.

As an illustration, the cron-based scaler enables you to:

- **Manage High Traffic and Request Volume Peaks**: With the cron-based scaler, you can program your applications to upscale during peak hours or during high-traffic events, such as Black Friday or New Year sales. The same functionality can be used to schedule your applications to upscale during off-peak hours for batch processing tasks.

![bf](assets/keda-scale-bf.png)

- **Optimize Resource Utilization and Reduce Expenses**: The cron-based scaler offers a solution to optimize resource utilization and reduce costs by allowing you to schedule your applications to downscale during non-working hours. This feature is useful for your **dev/stage/QA** clusters, which are not required during off-working hours.

> **Note:** This only benefits when your workloads require more resources than the base setup. The [current base setup](https://kyma-project.github.io/price-calculator/) consists of 3 VMs, each with 4 CPU and 16 GB of RAM. Therefore, *if you your workloads need 4 or more VMs to be provisioned, this feature can provide benefits to control costs during off-work hours.*
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

![off-work](assets/keda-scale-off-work.png)

## Scenario

Lets put cron based scaler to action.

Assume we have a **development cluster** where we want to run workloads only during work hours.

Lets say **Monday - Friday, 8 AM to 6 PM**

### Prerequisites

- [SAP BTP, Kyma runtime instance](../prerequisites/#kyma)
- [Kubernetes tooling](../prerequisites/#kubernetes)
- [KEDA and Serverless Modules enabled in Kyma]((https://help.sap.com/docs/btp/sap-business-technology-platform/enable-and-disable-kyma-module))
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

### Steps

- Export environment variables
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```shell
export NS={your-namespace}
```

- Create namespace and enable istio injection if not already done

```shell
kubectl create ns ${NS}
# only required once to enable istio sidecar. Ignore if done already
kubectl label namespaces ${NS} istio-injection=enabled
```
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

- Create sample workloads. One deployment and one function
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl -n ${NS} apply -f k8s/deployment.yaml
kubectl -n ${NS} apply -f k8s/function.yaml
```

- Apply the KEDA cron based scaling to these workloads
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl -n ${NS} apply -f k8s/keda-cron-scaler.yaml
```

### How it works

KEDA `scaledobject` resource can be configured with a trigger of type cron.

In the cron scaler, we can then specify to have the workloads running only during the working hours.
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```yaml
triggers:
- type: cron
metadata:
# The acceptable values would be a value from the IANA Time Zone Database.
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved
timezone: Europe/Berlin
# At 08:00 AM, Monday through Friday
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved
start: 0 8 * * 1-5
# At 06:00 PM, Monday through Friday
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved
end: 0 18 * * 1-5
# ie. Your MINIMUM replica count for this workload
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved
desiredReplicas: "1"
```

For each type of workload, the **scaleTargetRef** need to be specified
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```yaml
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: test-keda-cron-nginx
```

```yaml
spec:
scaleTargetRef:
apiVersion: serverless.kyma-project.io/v1alpha2
kind: Function
name: test-keda-cron-function
```

### View the events

Check the events during the trigger start or end time. Here you can see KEDA scaling down the replicas
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl -n ${NS} get events
```

```shell
LAST SEEN TYPE REASON OBJECT MESSAGE
7m34s Normal Killing pod/test-keda-cron-nginx-86b78b79df-r42zd Stopping container istio-proxy
7m34s Normal Killing pod/test-keda-cron-nginx-86b78b79df-r42zd Stopping container nginx
7m31s Warning Unhealthy pod/test-keda-cron-nginx-86b78b79df-r42zd Readiness probe failed: HTTP probe failed with statuscode: 503
7m34s Normal SuccessfulDelete replicaset/test-keda-cron-nginx-86b78b79df Deleted pod: test-keda-cron-nginx-86b78b79df-r42zd
7m34s Normal KEDAScaleTargetDeactivated scaledobject/test-keda-cron-nginx Deactivated apps/v1.Deployment demos/test-keda-cron-nginx from 1 to 0
7m34s Normal ScalingReplicaSet deployment/test-keda-cron-nginx Scaled down replica set test-keda-cron-nginx-86b78b79df to 0 from 1
```

> Note: Events are only available until after 1 hour of trigger.
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

### First hand experience
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

I applied the KEDA cron scaler to all my custom workloads in my Kyma cluster.
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

All my deployment replicas scaled down to zero
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

![dep-off](assets/keda-off-hours.png)

and number of nodes (VMs) reduced from 4 to 3
abbi-gaurav marked this conversation as resolved.
Show resolved Hide resolved

![nodes-off](assets/nodes-off-hours.png)

## References

- <https://medium.com/@CloudifyOps/optimizing-kubernetes-workloads-with-keda-custom-metric-driven-pod-autoscaling-7332e674fdc6>
- <https://doc.kaas.thalesdigital.io/docs/Features/keda>
Binary file added keda-cron-scaler/assets/keda-off-hours.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added keda-cron-scaler/assets/keda-scale-bf.png
Copy link
Contributor

@nataliasitko nataliasitko Dec 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd export the diagrams with a white background (not transparent). Otherwise, the captions are not visible in the dark mode.
The officially approved name is SAP BTP, Kyma runtime, with a comma after BTP. I suggest adding it also in the diagrams

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions keda-cron-scaler/assets/keda-scale-bf.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added keda-cron-scaler/assets/keda-scale-off-work.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions keda-cron-scaler/assets/keda-scale-off-work.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added keda-cron-scaler/assets/keda-work-hours.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added keda-cron-scaler/assets/nodes-off-hours.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 24 additions & 0 deletions keda-cron-scaler/k8s/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: test-keda-cron-nginx
name: test-keda-cron-nginx
spec:
replicas: 1
selector:
matchLabels:
app: test-keda-cron-nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: test-keda-cron-nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
19 changes: 19 additions & 0 deletions keda-cron-scaler/k8s/function.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
apiVersion: serverless.kyma-project.io/v1alpha2
kind: Function
metadata:
name: test-keda-cron-function
labels:
app: test-keda-cron-function
spec:
runtime: nodejs18
source:
inline:
source: |
module.exports = {
main: function(event, context) {
console.log('I only run during work hours!!');
}
}
scaleConfig:
minReplicas: 1
maxReplicas: 1
51 changes: 51 additions & 0 deletions keda-cron-scaler/k8s/keda-cron-scaler.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: test-keda-cron-nginx
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: test-keda-cron-nginx
pollingInterval: 10
# immediately scale down the workload
cooldownPeriod: 0
# The workload is ASLEEP by DEFAULT, otherwise, it's Awake.
minReplicaCount: 0
triggers:
- type: cron
metadata:
# The acceptable values would be a value from the IANA Time Zone Database.
timezone: Europe/Berlin
# At 08:00 AM, Monday through Friday
start: 0 8 * * 1-5
# At 06:00 PM, Monday through Friday
end: 0 18 * * 1-5
# ie. Your MINIMUM replica count for this workload
desiredReplicas: "1"
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: test-keda-cron-function
spec:
scaleTargetRef:
apiVersion: serverless.kyma-project.io/v1alpha2
kind: Function
name: test-keda-cron-function
pollingInterval: 10
# immediately scale down the workload
cooldownPeriod: 0
# The workload is ASLEEP by DEFAULT, otherwise, it's Awake.
minReplicaCount: 0
triggers:
- type: cron
metadata:
# The acceptable values would be a value from the IANA Time Zone Database.
timezone: Europe/Berlin
# At 08:00 AM, Monday through Friday
start: 0 8 * * 1-5
# At 06:00 PM, Monday through Friday
end: 0 18 * * 1-5
# ie. Your MINIMUM replica count for this workload
desiredReplicas: "1"
Binary file added on-premise-docker-registry/assets/using-tp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions on-premise-docker-registry/assets/using-tp.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading