Cerebro: open alerting for DevOps teams

Table of Contents

• Cerebro
• Features
• Getting started
• User documentation
• Installation / Configuration
• Code structure
• License
• Frequently Asked Questions
• Contact

(open screenshots)

Cerebro

Cerebro is an open alerting system over Graphite, the opensource timeseries database. The service layer currently integrates with Seyren to store alarms, schedule checks and send alerts. (More backends in the future.)

Cerebro provides a service layer (REST API) and a dashboard (Web UI) to make alerting over various types of metrics more accessible to Graphite non-experts. It is dedicated to DevOps teams monitoring their product/services.

Whereas still a beta, we already use Cerebro over various types of production metrics: system, applicative, business. We have teams and users, from ops to developers as well as product owners, which find it useful as a general-purpose open alerting service. That is why we chose to share Cerebro with the community, even though there is (much) place for improvements: scalability, resilience, API, integrations…​

Features

• Supports alarms over Graphite timeseries and email alerts for now. (More backends and alert types in the future.)

• Use wizard-like forms to create basic, as well as advanced alarms:

  • Define basic fixed-thresholds alarms, as well as history-leveraging targets (combining Graphite functions, such as summarize, timeShift or Holt-Winters forecasts)

  • Follow natural-language instructions and illustrations to set alarm modes and options

  • Preview data and thresholds results at each step (over 1h, 1d, 1w)

• Browse, inspect, susbscribe to existing alarms:

  • Find / browse alarms and recent alerts

  • Subscribe/Unsubscribe to any alarm (setting active status/weekdays/hours of notifications)

  • Inspect alarm details, previews, subscribers

• Edit alarms and subscriptions in a non-blocking manner:

  • Enable/Disable, edit or duplicate existing alarms (no owner - subscribers will be notified instead)

  • Enable/Disable, edit and delete subscriptions (when no subscription remains, the alarm is deleted to)

• Leverage the REST API for programmatic usage (deploy alarms automatically with applications?)

• Browse basic stats over alarms, alerts and subscribers

• Get online help, links, info:

  • Contextual help from every page, aka Prof. Xavier (using Intro.js)

  • Customizable links to your wiki, chatroom, issues, mailing list…​

  • Technical details about the version (commit, author, build job, etc.)

  • API online documentation (with Swagger UI)

Note	Questions about existing/missing features and supports? auth? alarm ownership?…​ Please check the FAQ wiki.

Getting started

The following instructions show how to create an alarm with Cerebro, as well as several other features, running everything locally as Docker containers.

• Install Maven and Docker Compose.

• Build Cerebro:

  $ mvn clean verify

• Run Cerebro over fresh Graphite+Seyren backends as containers:

  $ cd docker
  $ docker-compose up

• Open the Cerebro dashboard on http://localhost (IP may differ for Docker Toolbox and VM users).

• Once the dashboard is loaded, click on the Prof. Xavier button to get a short description of what you see. Click on Learn more…​ to get more info and a bunch of external links.

• Create your first alarm:

  • Open Create an alarm…​,

  • Enter some title and description,

  • Set the Graphite target, for instance carbon.agents.*.cpuUsage, then check the Preview…​ to verify there actually are values in the backend,

  • In Next step, select a Time range (summarize) function, such as the Average value over the last hour,

  • In Next step, set some Fixed thresholds values for warnings/errors, then check the Preview…​ to see the results of smoothing and thresholds,

  • In Next steps, configure email notifications (who, what, when), check the summary then Finish saving the alarm.

• From the alarm page, scroll to Subscribers then Subscribe…​ another email with different notification params, try disabling or editing a subscription (the buttons on the cards) or the alarm itself.

• When back to the All alarms page, verify your first alarm is visible. Click it to go back to details.

• Cerebro Docker containers won’t actually send emails (no SMTP configured), but congratulations: you’ve just created your first alarm and subscriptions with Cerebro.

User documentation

In addition to comments and labels in the user interface, the dashboard Prof. Xavier button provides contextual information from for every page.

Cerebro has been used to perform an introduction to Graphite-based anomaly detection at Devoxx France. The talk is available online (in French).

More user documentation will come soon.

Installation / Configuration

See the INSTALL.adoc file for installation and configuration instructions, as well as general information about dependencies, integrations, etc.

Code structure

• dashboard : graphical user interface, running Angular (TypeScript, Bootstrap, etc.)

• docker : Docker / Compose implementation to run a Graphite+Mongo+Seyren+Cerebro stack of containers

• selenium : Selenium integration tests over dashboard features

• services : REST API over backends, running Spring Boot / Java

License

See the LICENSE file.

NOTE

Cerebro sources include several links to external images from the X-Men licence/movies. Those images are not part of Cerebro sources, only their URL is embedded, merely illustrating a possible look’n’feel for the application. Please see the configuration instructions to replace them with other resources.

Frequently Asked Questions

Why Seyren? Angular 2? Why not <insert_your_language>?

Get auth? Why no ownership over alarms?

Does it scale? Can I use the beta in production?

Where is the commit history? Who are you? Where do you want to go with Cerebro?

Please check the project wiki for FAQ.

Contact

For most developers in the team, this is the first project we share to the community.

We would be glad to have your feedback.

mailto:cerebro-team(at)framalistes.org.