From b56beca3410c789f8b3f592f835de3dabd95ec88 Mon Sep 17 00:00:00 2001 From: Adam Wegrzynek Date: Wed, 7 Feb 2024 09:14:02 +0100 Subject: [PATCH 1/3] [OMON-753] Update docs for reading metrics from Kafka --- README.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index f263e160..2adb8a5e 100644 --- a/README.md +++ b/README.md @@ -206,13 +206,15 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr ## Receiving metrics from Monitoring system (development instructions) ### Requirements - - CS8 or CC7 with `devtoolset-9` - - Boost >= 1.70 + - RHEL8, RHE9, CS8 or CC7 with `devtoolset-9` + - Boost >= 1.70, CMake ### Compile Monitoring library with Kafka backend + +#### Manually - Compile `librdkafka` ```bash - git clone -b v1.8.2 https://github.com/edenhill/librdkafka && cd librdkafka + git clone -b v2.3.0 https://github.com/edenhill/librdkafka && cd librdkafka cmake -H. -B./_cmake_build -DENABLE_LZ4_EXT=OFF -DCMAKE_INSTALL_LIBDIR=lib -DRDKAFKA_BUILD_TESTS=OFF -DRDKAFKA_BUILD_EXAMPLES=OFF -DCMAKE_INSTALL_PREFIX=~/librdkafka_install cmake --build ./_cmake_build --target install -j ``` @@ -223,6 +225,11 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr cmake --build ./_cmake_build --target install -j ``` +#### aliBuild +- Modify `monitoring.sh`: add `- librdkafka` to "requires" +- Compile Monitoring: `aliBuild build Monitoring --defaults o2-dataflow --always-prefer-system` +- Add `Monitoring` as dependency of your project + ### Look for Monitoring library in your CMake As `librdkafka` is optional dependency of Monitoring it is not handled by CMakeConfig, therefore you need: ```cmake @@ -238,7 +245,7 @@ And then, link against `AliceO2::Monitoring` target. ... std::vector topics = {"topic-to-subscribe"}; -auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics); +auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics, ""); for (;;) { auto metrics = client->pull(); if (!metrics.empty()) { @@ -248,5 +255,7 @@ for (;;) { } ``` +`` should be unique, self-explainable string describing the client, eg. `dcs-link-status` or `its-link-status`. + ### Data format Native data format is [Influx Line Protocol](https://docs.influxdata.com/influxdb/latest/reference/syntax/line-protocol/) but metrics can be converted into any format listed in here: https://docs.influxdata.com/telegraf/latest/data_formats/output/ From a4a07b892562952fa527e22ae9b207507f7e3b13 Mon Sep 17 00:00:00 2001 From: Adam Wegrzynek Date: Wed, 7 Feb 2024 09:18:50 +0100 Subject: [PATCH 2/3] Run time params --- README.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2adb8a5e..03e8811a 100644 --- a/README.md +++ b/README.md @@ -206,8 +206,8 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr ## Receiving metrics from Monitoring system (development instructions) ### Requirements - - RHEL8, RHE9, CS8 or CC7 with `devtoolset-9` - - Boost >= 1.70, CMake + - RHEL8, CS8 or CC7 with `devtoolset-9` + - Boost >= 1.83, CMake ### Compile Monitoring library with Kafka backend @@ -244,8 +244,8 @@ And then, link against `AliceO2::Monitoring` target. #include "Monitoring/MonitoringFactory.h" ... -std::vector topics = {"topic-to-subscribe"}; -auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics, ""); +std::vector topics = {""}; +auto client = MonitoringFactory::GetPullClient("", topics, ""); for (;;) { auto metrics = client->pull(); if (!metrics.empty()) { @@ -255,7 +255,10 @@ for (;;) { } ``` -`` should be unique, self-explainable string describing the client, eg. `dcs-link-status` or `its-link-status`. +Run-time parameters: +- `` - List of topics to subscribe +- `` - Kafka broker (staging or production) +- `` - unique, self-explainable string describing the client, eg. `dcs-link-status` or `its-link-status`. ### Data format Native data format is [Influx Line Protocol](https://docs.influxdata.com/influxdb/latest/reference/syntax/line-protocol/) but metrics can be converted into any format listed in here: https://docs.influxdata.com/telegraf/latest/data_formats/output/ From 88c230f242752fe781f7cc27e7e4bfaed9120afd Mon Sep 17 00:00:00 2001 From: Adam Wegrzynek Date: Wed, 7 Feb 2024 09:20:03 +0100 Subject: [PATCH 3/3] Update oses --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 03e8811a..b5cfc6a3 100644 --- a/README.md +++ b/README.md @@ -206,7 +206,7 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr ## Receiving metrics from Monitoring system (development instructions) ### Requirements - - RHEL8, CS8 or CC7 with `devtoolset-9` + - Ubuntu, RHEL9, RHEL8, CS8, macOS, or CC7 with `devtoolset-9` - Boost >= 1.83, CMake ### Compile Monitoring library with Kafka backend