Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt benchmarks configuration to be in line with updated charts. #213

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b0844de
fix: replace release tag
rodrigo-lourenco-lopes Nov 20, 2024
0daea08
fix: fix tests to accommodate the changes in charts refactorings.
rodrigo-lourenco-lopes Nov 21, 2024
60122c1
fix: fix maintainer name.
rodrigo-lourenco-lopes Nov 21, 2024
3f7531d
refactor: remove unused, and rename appropriately golden files.
rodrigo-lourenco-lopes Nov 21, 2024
15bcf72
feat: switch to camunda "core" component, instead of individual zeebe…
remcowesterhoud Dec 2, 2024
0a609f8
test: update golden files
remcowesterhoud Dec 3, 2024
c70e9ba
fix: only print operate in notes if it's enabled
remcowesterhoud Dec 3, 2024
d93097d
feat: increase resources of single core app
remcowesterhoud Dec 3, 2024
6053086
fix: run operate and tasklist in realistic benchmark
remcowesterhoud Dec 4, 2024
1b170c1
fix: add operate resources to core app
remcowesterhoud Dec 4, 2024
b904949
fix: add operate stackdriver env variables to core app
remcowesterhoud Dec 4, 2024
2e73baa
fix: add env variables to realistic benchmark
remcowesterhoud Dec 4, 2024
578e380
feat: print active profiles
remcowesterhoud Dec 4, 2024
ad4ce5b
test: regenerate golden files
remcowesterhoud Dec 4, 2024
7b2e1db
fix: fix the way we display active profiles.
rodrigo-lourenco-lopes Dec 18, 2024
30d10c0
feat: append dependency to "0.0.0-8.7.0-alpha2"
rodrigo-lourenco-lopes Dec 18, 2024
68be354
feat: add the option to fetch image form a different repo in starters…
rodrigo-lourenco-lopes Dec 18, 2024
e81f0b3
feat: fetch a different image for starters and workers.
rodrigo-lourenco-lopes Dec 18, 2024
56f5d76
feat: remap the values.yaml to fit the new chart.
rodrigo-lourenco-lopes Dec 18, 2024
5c33f51
fix: remove global values in values-realistic-benchmark.yaml
rodrigo-lourenco-lopes Dec 18, 2024
faffd19
fix: fix golden tests.
rodrigo-lourenco-lopes Dec 18, 2024
a7514c6
feat: re-enable zeebe config, set global image.
rodrigo-lourenco-lopes Jan 3, 2025
1364d7e
feat: fetch images from global.
rodrigo-lourenco-lopes Jan 3, 2025
54ee9ee
feat: fix golden files.
rodrigo-lourenco-lopes Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ sources:
dependencies:
- name: camunda-platform
repository: "oci://ghcr.io/camunda/helm"
version: "0.0.0-12.0.0-alpha1"
version: "0.0.0-8.7.0-alpha1"
condition: "camunda.enabled"
- name: prometheus-elasticsearch-exporter
repository: "https://prometheus-community.github.io/helm-charts"
Expand Down
4 changes: 2 additions & 2 deletions charts/zeebe-benchmark/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@ sources:
dependencies:
- name: camunda-platform
repository: "oci://ghcr.io/camunda/helm"
version: "0.0.0-12.0.0-alpha1"
version: "0.0.0-8.7.0-alpha2"
condition: "camunda-platform.enabled"
- name: prometheus-elasticsearch-exporter
repository: "https://prometheus-community.github.io/helm-charts"
version: 6.4.0
condition: "prometheus-elasticsearch-exporter.enabled"
maintainers:
- name: Zelldon
rodrigo-lourenco-lopes marked this conversation as resolved.
Show resolved Hide resolved
- name: ChrisKujawa
email: [email protected]
annotations:
artifacthub.io/links: |
Expand Down
8 changes: 3 additions & 5 deletions charts/zeebe-benchmark/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,9 @@

Installed Zeebe cluster with:

* {{ index .Values "camunda-platform" "zeebe" "clusterSize" }} Brokers ({{ index .Values "camunda-platform" "zeebe" "image" "repository" }}:{{ index .Values "camunda-platform" "zeebe" "image" "tag" }})
* {{ index .Values "camunda-platform" "zeebeGateway" "replicas" }} Gateways ({{ index .Values "camunda-platform" "zeebeGateway" "image" "repository" }}:{{ index .Values "camunda-platform" "zeebeGateway" "image" "tag" }})
{{- if index .Values "camunda-platform" "operate" "enabled" }}
* Operate ({{ index .Values "camunda-platform" "operate" "image" "repository" }}:{{ index .Values "camunda-platform" "operate" "image" "tag" }})
{{ end }}
* {{ index .Values "camunda-platform" "core" "clusterSize" }} Brokers ({{ index .Values "camunda-platform" "core" "image" "repository" }}:{{ index .Values "camunda-platform" "core" "image" "tag" }})
* Active profiles: {{- range index .Values "camunda-platform" "core" "env" }}{{- if eq .name "SPRING_PROFILES_ACTIVE" }}{{ .value }}{{- end }}{{- end }}


The benchmark is running with:

Expand Down
2 changes: 1 addition & 1 deletion charts/zeebe-benchmark/templates/publisher.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ spec:
- name: JDK_JAVA_OPTIONS
value: >-
-Dconfig.override_with_env_vars=true
-Dapp.brokerUrl={{ .Release.Name }}-zeebe-gateway:26500
-Dapp.brokerUrl={{ .Release.Name }}-core:26500
-Dapp.starter.rate={{ .Values.publisher.rate }}
-Dzeebe.client.requestTimeout=62000
-XX:+HeapDumpOnOutOfMemoryError
Expand Down
2 changes: 1 addition & 1 deletion charts/zeebe-benchmark/templates/starter.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ spec:
{{- if $.Values.saas.enabled }}
-Dapp.tls=true
{{- else }}
-Dapp.brokerUrl={{ $.Release.Name }}-zeebe-gateway:26500
-Dapp.brokerUrl={{ $.Release.Name }}-core:26500
{{- end }}
-Dapp.starter.rate={{ .Values.starter.rate }}
-Dapp.starter.durationLimit=0
Expand Down
2 changes: 1 addition & 1 deletion charts/zeebe-benchmark/templates/timer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ spec:
- name: JDK_JAVA_OPTIONS
value: >-
-Dconfig.override_with_env_vars=true
-Dapp.brokerUrl={{ .Release.Name }}-zeebe-gateway:26500
-Dapp.brokerUrl={{ .Release.Name }}-core:26500
-Dapp.starter.rate={{ .Values.timer.rate }}
-Dzeebe.client.requestTimeout=62000
-XX:+HeapDumpOnOutOfMemoryError
Expand Down
2 changes: 1 addition & 1 deletion charts/zeebe-benchmark/templates/workers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ spec:
{{- if $.Values.saas.enabled }}
-Dapp.tls=true
{{- else }}
-Dapp.brokerUrl={{ $.Release.Name }}-zeebe-gateway:26500
-Dapp.brokerUrl={{ $.Release.Name }}-core:26500
{{- end }}
-Dzeebe.client.requestTimeout=62000
{{- if $worker.capacity }}
Expand Down
212 changes: 212 additions & 0 deletions charts/zeebe-benchmark/test/golden/c8-core-configmap.golden.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
---
# Source: zeebe-benchmark/charts/camunda-platform/templates/core/configmap.yaml
kind: ConfigMap
metadata:
name: benchmark-test-core-configuration
labels:
app: camunda-platform
app.kubernetes.io/name: camunda-platform
app.kubernetes.io/instance: benchmark-test
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/part-of: camunda-platform
app.kubernetes.io/component: core
app.kubernetes.io/version: "8.7.0-alpha2"
apiVersion: v1
data:
startup.sh: |
# The Node ID depends on the Pod name so it cannot be templated in the StatefulSet level.
export ZEEBE_BROKER_CLUSTER_NODEID="${ZEEBE_BROKER_CLUSTER_NODEID:-$[${K8S_NAME##*-} * 1 + 0]}"
echo "export ZEEBE_BROKER_CLUSTER_NODEID=${ZEEBE_BROKER_CLUSTER_NODEID}"

exec /usr/local/camunda/bin/camunda
application.yaml: |

spring:
profiles:
active: "operate,tasklist,broker,auth"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Ah ok so it seems Operate is enabled.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be 🤔 I modified the SPRING_PROFILES_ACTIVE to disable it.


management:
server:
port: 9600

server:
forward-headers-strategy: none
address: 0.0.0.0
port: 8080
servlet:

zeebe:
host: 0.0.0.0
log:
level: "info"

broker:
# zeebe.broker.gateway
gateway:
enable: true
network:
host: 0.0.0.0
port: 26500

# zeebe.broker.network
network:
advertisedHost: "${K8S_NAME}.${K8S_SERVICE_NAME}"
host: 0.0.0.0
commandApi:
port: 26501
internalApi:
port: 26502

# zeebe.broker.cluster
cluster:
# The value of "nodeId" is set via the "ZEEBE_BROKER_CLUSTER_NODEID" env var.
# As it depends on the Pod name, which cannot be templated at the installation time.
# nodeId:
initialContactPoints:
- benchmark-test-zeebe-0.${K8S_SERVICE_NAME}:26502
- benchmark-test-zeebe-1.${K8S_SERVICE_NAME}:26502
- benchmark-test-zeebe-2.${K8S_SERVICE_NAME}:26502
clusterSize: "3"
replicationFactor: "3"
partitionsCount: "3"
clusterName: benchmark-test-zeebe

# zeebe.broker.data
data:
snapshotPeriod: 5m
freeSpace:
processing: 2GB
replication: 3GB

# zeebe.broker.threads
threads:
cpuThreadCount: "3"
ioThreadCount: "3"

# zeebe.broker.exporters
exporters:
elasticsearch:
className: "io.camunda.zeebe.exporter.ElasticsearchExporter"
args:
url: "http://benchmark-test-elasticsearch:9200"
index:
prefix: "zeebe-record"
retention:
enabled: true
minimumAge: "10m"
policyName: "core-record-retention-policy"
CamundaExporter:
className: "io.camunda.exporter.CamundaExporter"
args:
connect:
type: elasticsearch
url: "http://benchmark-test-elasticsearch:9200"
retention:
enabled: true
minimumAge: "10m"
policyName: "core-record-retention-policy"
createSchema: true

camunda:
persistent:
sessions:
enabled: true

rest:
query:
enabled: true

security:
authorizations:
enabled: false

#
# Camunda Database Configuration.
#
database:
type: elasticsearch
# Cluster name
clusterName: elasticsearch
# Elasticsearch full url
url: "http://benchmark-test-elasticsearch:9200"

#
# Camunda Operate Configuration.
#
operate:
persistentSessionsEnabled: true

# ELS instance to store Operate data
elasticsearch:
# Operate index prefix.
# Cluster name
clusterName: elasticsearch
# Host
host: benchmark-test-elasticsearch
# Transport port
port: 9200
# Elasticsearch full url
url: "http://benchmark-test-elasticsearch:9200"
# ELS instance to export Zeebe data to
zeebeElasticsearch:
# Cluster name
clusterName: elasticsearch
# Host
host: benchmark-test-elasticsearch
# Transport port
port: 9200
# Index prefix, configured in Zeebe Elasticsearch exporter
prefix: "zeebe-record"
# Elasticsearch full url
url: "http://benchmark-test-elasticsearch:9200"
# Zeebe instance
zeebe:
# Gateway address
gatewayAddress: "benchmark-test-core:26500"
archiver:
ilmEnabled: true
ilmMinAgeForDeleteArchivedIndices: 10m

#
# Camunda Tasklist Configuration.
#
tasklist:

# Set Tasklist username and password.
# If user with <username> does not exists it will be created.
# Default: demo/demo
#username:
#password:
# ELS instance to store Tasklist data
elasticsearch:
# Tasklist index prefix.
# Cluster name
clusterName: elasticsearch
# Host
host: benchmark-test-elasticsearch
# Transport port
port: 9200
# Elasticsearch full url
url: "http://benchmark-test-elasticsearch:9200"
# ELS instance to export Zeebe data to
zeebeElasticsearch:
# Cluster name
clusterName: elasticsearch
# Host
host: benchmark-test-elasticsearch
# Transport port
port: 9200
# Index prefix, configured in Zeebe Elasticsearch exporter
prefix: "zeebe-record"
# Elasticsearch full url
url: "http://benchmark-test-elasticsearch:9200"
# Zeebe instance
zeebe:
# Gateway address
gatewayAddress: benchmark-test-core:26500
restAddress: "http://benchmark-test-core:8080"
archiver:
ilmEnabled: true
ilmMinAgeForDeleteArchivedIndices: 10m

log4j2.xml: |
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
# Source: zeebe-benchmark/charts/camunda-platform/templates/service-monitor/zeebe-service-monitor.yaml
# Source: zeebe-benchmark/charts/camunda-platform/templates/service-monitor/core-service-monitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: benchmark-test-camunda-platform-zeebe
name: benchmark-test-camunda-platform-core
labels:
app: camunda-platform
app.kubernetes.io/name: camunda-platform
Expand All @@ -15,9 +15,9 @@ spec:
selector:
matchLabels:
app: camunda-platform
app.kubernetes.io/component: zeebe-broker
app.kubernetes.io/component: core
endpoints:
- honorLabels: true
path: /actuator/prometheus
port: http
port: server
interval: 30s
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
---
# Source: zeebe-benchmark/charts/camunda-platform/templates/zeebe/service.yaml
# Source: zeebe-benchmark/charts/camunda-platform/templates/core/service.yaml
apiVersion: v1
kind: Service
metadata:
name: "benchmark-test-zeebe"
name: benchmark-test-core
labels:
app: camunda-platform
app.kubernetes.io/name: camunda-platform
app.kubernetes.io/instance: benchmark-test
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/part-of: camunda-platform
app.kubernetes.io/component: zeebe-broker
app.kubernetes.io/version: "SNAPSHOT"
app.kubernetes.io/component: core
app.kubernetes.io/version: "8.7.0-alpha2"
annotations:
spec:
clusterIP: None
publishNotReadyAddresses: true
type: ClusterIP
ports:
- port: 9600
- port: 8080
protocol: TCP
name: http
- port: 26502
Expand All @@ -27,10 +27,16 @@ spec:
- port: 26501
protocol: TCP
name: command
- port: 9600
protocol: TCP
name: server
- port: 26500
protocol: TCP
name: gateway
selector:
app: camunda-platform
app.kubernetes.io/name: camunda-platform
app.kubernetes.io/instance: benchmark-test
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/part-of: camunda-platform
app.kubernetes.io/component: zeebe-broker
app.kubernetes.io/component: core
Loading
Loading