A containerized Prometheus exporter/data scraper for the CircuitSetup Split Single Phase Real Time Whole House Energy Meter written in Node.js by Ryan Jacobs. It subscribes to the MQTT topics published by the Energy Meter and provides a /metrics HTTP endpoint with for Prometheus to scrape.
Note: This software should be compatible with any meter running the OpenEnergyMonitor project's EmonESP software and publishing data to an MQTT broker, although it only subscribes to specific MQTT topics. Since its purpose is to scrape data into a time-series database (in this case, Prometheus), it can replace EmonCMS, which is the solution that EmonESP is really designed to communicate with.
You will need to define at least the environment variable MQTT_SERVER and MQTT_TOPIC (MQTT_USERNAME, MQTT_PASSWORD, and MQTT_PORT may also be required, depending on your MQTT configuration) in order for the container to be useful. You will also need to ensure that port 3000 is available to the container. Further configuration information is available below.
docker run --env MQTT_HOST=<your-mqtt-hostname-or-ip> --env MQTT_TOPIC=<your-mqtt-base-topic> -p 3000:3000 djryanj/energy-scraper:latest
Configuration is done via environment variables only.
The following environment variables directly affect the operation of the scraper. Two are required to be set (provided that the defaults aren't sufficient); the rest are optional depending on what features you want to do (and the configuration required for them).
| Variable | Required | Description | Default |
|---|---|---|---|
MQTT_HOST |
Yes | IP or hostname of the MQTT server that the energy monitor is publishing to. NOTE: At this time, providing a self-signed certificate file for TLS communications is not supported. If your MQTT server uses PKI from a well-known certificate authority such as letsencrypt (be sure that the WHOLE chain is sent), it should work. However, at this time, cert behaviour is untested. | 192.168.1.1 |
MQTT_TOPIC |
Yes | Base topic to subscribe to for energy moitor MQTT data. See MQTT section for more information. | prometheus/# |
MQTT_PORT |
No | TCP port that your MQTT broker is listening on. | 1883 |
PORT |
No | The port that the container listens on (internally). Be sure to change the port Docker forwards if you change this. | 3000 |
MQTT_USERNAME |
No | If you need to supply a username for your MQTT server, set it here. | null/unset (works for anonymous servers) |
MQTT_PASSWORD |
No | If you need to supply a password for your MQTT server, set it here. IMPORTANT! This will be visible in the logs and if you get container information from the Docker host. You could provide this via Docker secrets. | null/unset (works for anonymous servers) |
MONITOR_SOLAR |
No | If your Energy Meter is configured to monitor solar, set this to true. | False |
MAINS_GAUGES |
No | Publish Prometheus gauges for the following MQTT topics sent by the meter: CT1, CT2, W, totI. See the metrics section for more information. | True |
MAINS_COUNTERS |
No | Publish Prometheus counters for the following MQTT topics sent by the meter: CT1, CT2, W, totI. See the metrics section for more information. | False |
MONITOR_EXTENDED |
No | If your meter is publishing the extended metrics, enable this to capture them. | False |
The following environment variables provide information into the scraper but do not affect operation beyond the cosmetic (they are exported with metrics and are visible on the browseable home page). None of these are required.
| Variable | Description | Default |
|---|---|---|
BUILDID |
The BUILDID is intended to be passed in from an automation pipeline such as Azure DevOps and should be equivalent to the git hash (this was mostly used during development; a future release may hide this for production builds). The BUILDID can be read from the local .git directory, but that is not present in the Docker container. | missingBuildId (when nothing is passed to Docker or the .git directory is not present locally); otherwise the 7-character git hash representing the commit used to build the image. |
BUILDNUMBER |
The BUILDNUMBER is intended to be passed in from an automation pipeline such as Azure DevOps and should be equivalent to the automated build number, e.g. 20190910.1. | local |
SOURCEBRANCHNAME |
The SOURCEBRANCHNAME is intended to be passed in from an automation pipeline such as Azure DevOps and should be equivalent to the git branch that the container was built from, e.g. master. | local |
Exposes port tcp/3000 by default. Change the PORT environment variable if you want to change it. Leverage something like jwilder/nginx-proxy to automatically forward a hostname from the Docker host's port 80 or 443 to this container.
There is a default cosmetic endpoint at / (e.g. http://docker-host:3000/) that will respond if the scraper is up. No data is available on that page except for build information.
Prometheus expects an endpoint at /metrics, which is where it is. In addition to the custom metrics provided by this scraper, there are node.js-specific metrics exported as well that are not presented here. You can browse the endpoint if you wish.
All metrics are prefixed by home_ (except one). The list of metrics is below (including the prefix for clarity).
| Metric | Type | Description | MQTT Topic from Energy Meter * |
|---|---|---|---|
home_voltage_1 |
Counter | Current voltage on leg 1 of the home power feed in Volts (V). | V1 |
home_voltage_2 |
Counter | Current voltage on leg 2 of the home power feed in Volts (V). | V2 |
home_current_power |
Gauge | Current total home power usage in Watts (W). Only published if the MAINS_GAUGES environment variable is true. |
W |
home_current_1 |
Gauge | Current amperage (current) on leg 1 of the home power feed in Amps (A). Only published if the MAINS_GAUGES environment variable is true. |
CT1 |
home_current_2 |
Gauge | Current amperage (current) on leg 2 of the home power feed in Amps (A). Only published if the MAINS_GAUGES environment variable is true. |
CT2 |
home_total_current |
Gauge | Total amperage (current) usage for the home in Amps (A). Only published if the MAINS_GAUGES environment variable is true. |
totI |
home_current_power_counter |
Counter | Absolute value of the current total home power usage in Watts (W). Only published if the MAINS_COUNTERS environment variable is true. |
W |
home_current_1_counter |
Counter | Absolute value of the current amperage (current) on leg 1 of the home power feed in Amps (A). Only published if the MAINS_COUNTERS environment variable is true. |
CT1 |
home_current_2_counter |
Counter | Absolute value of the current amperage (current) on leg 2 of the home power feed in Amps (A). Only published if the MAINS_COUNTERS environment variable is true. |
CT2 |
home_total_current_counter |
Counter | Absolute value of the total amperage (current) usage for the home in Amps (A). Only published if the MAINS_COUNTERS environment variable is true. |
totI |
solar_current_1 |
Counter | Current amperage (current) on leg 1 of the solar power feed in Amps (A). Only published if the MONITOR_SOLAR environment variable is true. |
SCT1 |
solar_current_2 |
Counter | Current amperage (current) on leg 2 of the solar power feed in Amps (A). Only published if the MONITOR_SOLAR environment variable is true. |
SCT2 |
solar_current_power |
Counter | Current total solar power output in Watts (W). Only published if the MONITOR_SOLAR environment variable is true. |
SolarW |
solar_voltage |
Counter | SolarV | |
solar_total_current |
Counter | Total solar feed amperage (current) output in Amps (A). Only published if the MONITOR_SOLAR environment variable is true. |
totSolarI |
home_power_factor |
Gauge | Power factor of the home power feed. | PF |
home_fundamental_power |
Counter | Current fundamental power in Watts (W). Only published if the MONITOR_EXTENDED environment variable is true. |
FundPow |
home_harmonic_power |
Counter | Current harmonic power Watts (W). Only published if the MONITOR_EXTENDED environment variable is true. |
HarPow |
home_reactive_power |
Counter | Current reactive power in volt-amperes reactive (VAR). Only published if the MONITOR_EXTENDED environment variable is true. |
ReactPow |
home_apparent_power |
Counter | Current apparent power in volt-amperes (VA). Only published if the MONITOR_EXTENDED environment variable is true. |
AppPow |
home_phase_1 |
Counter | Current phase angle on leg 1 of the home power feed in degrees (°). Only published if the MONITOR_EXTENDED environment variable is true. |
PhaseA |
home_phase_2 |
Counter | Current phase angle on leg 1 of the home power feed in degrees (°). Only published if the MONITOR_EXTENDED environment variable is true. |
PhaseC |
home_power_monitor_temp |
Gauge | Current temperature of the home power feed in degrees C | temp |
home_grid_freq |
Counter | Current grid frequency for home power feed in Hertz (Hz) | freq |
home_power_monitor_free_ram |
Counter | Power monitor free RAM in bytes (b) | freeram |
energy_scraper_info{version="<version>",hostname="<hostname>",buildId="<buildId>"} |
Gauge | Version and other information. |
*Note: The MQTT topics represented here as published by the Energy Meter are prefaced by the MQTT Base Topic entered into the EmonESP interface, e.g. emon/emonesp
*Note on Prometheus counters and gauges: As a general statement, gauges are more intuitive to look at for humans because they provide the exact last value, which is why they are published by default. On the other hand, counters increment continuously and getting the current value requires using a rate() or irate() function. However, once accounted for properly, counters tend to provide more functionality and less subsequent processing with downstream tools to give real data.
When configured to do so (and I hope that it is otherwise why are you here), the Energy Monitor will publish its data to a base topic and a series of other topics. This is configurable within either the EmonESP's configuration web page or by hard coding it into the code and compiling your own firmware. This scraper will only read the topics identified in the metrics section above.
This scraper does not publish any data of its own.
Here is a sample config block for Prometheus:
- job_name: energy_monitor
scrape_interval: 5s
static_configs:
- targets:
- <hostname>