Skip to content

Commit

Permalink
[OMON-753] Update docs for reading metrics from Kafka (#340)
Browse files Browse the repository at this point in the history
  • Loading branch information
awegrzyn authored Feb 7, 2024
1 parent 9c4de85 commit dc349d0
Showing 1 changed file with 17 additions and 5 deletions.
22 changes: 17 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,13 +206,15 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr
## Receiving metrics from Monitoring system (development instructions)

### Requirements
- CS8 or CC7 with `devtoolset-9`
- Boost >= 1.70
- Ubuntu, RHEL9, RHEL8, CS8, macOS, or CC7 with `devtoolset-9`
- Boost >= 1.83, CMake

### Compile Monitoring library with Kafka backend

#### Manually
- Compile `librdkafka`
```bash
git clone -b v1.8.2 https://github.com/edenhill/librdkafka && cd librdkafka
git clone -b v2.3.0 https://github.com/edenhill/librdkafka && cd librdkafka
cmake -H. -B./_cmake_build -DENABLE_LZ4_EXT=OFF -DCMAKE_INSTALL_LIBDIR=lib -DRDKAFKA_BUILD_TESTS=OFF -DRDKAFKA_BUILD_EXAMPLES=OFF -DCMAKE_INSTALL_PREFIX=~/librdkafka_install
cmake --build ./_cmake_build --target install -j
```
Expand All @@ -223,6 +225,11 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr
cmake --build ./_cmake_build --target install -j
```

#### aliBuild
- Modify `monitoring.sh`: add `- librdkafka` to "requires"
- Compile Monitoring: `aliBuild build Monitoring --defaults o2-dataflow --always-prefer-system`
- Add `Monitoring` as dependency of your project

### Look for Monitoring library in your CMake
As `librdkafka` is optional dependency of Monitoring it is not handled by CMakeConfig, therefore you need:
```cmake
Expand All @@ -237,8 +244,8 @@ And then, link against `AliceO2::Monitoring` target.
#include "Monitoring/MonitoringFactory.h"
...

std::vector<std::string> topics = {"topic-to-subscribe"};
auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics);
std::vector<std::string> topics = {"<topic-to-subscribe>"};
auto client = MonitoringFactory::GetPullClient("<kafka-server:9092>", topics, "<client-id>");
for (;;) {
auto metrics = client->pull();
if (!metrics.empty()) {
Expand All @@ -248,5 +255,10 @@ for (;;) {
}
```

Run-time parameters:
- `<topic-to-subscribe>` - List of topics to subscribe
- `<kafka-server:9092>` - Kafka broker (staging or production)
- `<client_id>` - unique, self-explainable string describing the client, eg. `dcs-link-status` or `its-link-status`.

### Data format
Native data format is [Influx Line Protocol](https://docs.influxdata.com/influxdb/latest/reference/syntax/line-protocol/) but metrics can be converted into any format listed in here: https://docs.influxdata.com/telegraf/latest/data_formats/output/

0 comments on commit dc349d0

Please sign in to comment.