eGeoffrey
Replies: 3 comments
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-09-19 11:18:35] Lovely How To, thanks for sharing it! Very comprehensive and detailed. Question, are you also aiming to publish the weather station data to some online weather service? I had in the past something in eGeoffrey for publishing to Weatherunderground before being acquired by IBM, just trying to understand if there is the need of something similar for other services (e.g. Windy, etc.). Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-09-19 11:25:30] I do have yet another wish where you could do me a tremendous fever. [https://pypi.org/project/buienradar/](link url) |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-09-19 12:41:44] @eporocrail said in Create a weather station: Sure let me have a look and I'll let you know. |
Beta Was this translation helpful? Give feedback.
-
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-09-16 08:21:33]
MY WEATHER STATION
AIM
The aim of this project is to create a weather-station from scratch.
Several sensors are integrated. The data of some will be compared to see the differences between the different types of sensor measuring the same quantity.
Sensor integration, data transmission, data processing and data presentation are addressed.
Only the eGeoffrey part of the system is described in detail.
The mechanical part is not described.
Sensors
The following sensors are applied:
BME280 : pressure, humidity and temperature
TSL2561 : light level
ML8511 : UV level
ML6070 : UV level
DS18B20 : temperature
HMC5883: compass for the wind direction
Anemometer: wind speed sensor
Windsensor Rev. P : a hot wire anemometer with temperature sensor
Tipping bucket rain volume sensor
They are combined in one single mechanical construction.
They are hooked-up to one LOLIN D1 mini. This module contains a microcontroller able to transmit via incorporated WiFi capabilities. Several GPIO and communication protocols are available.
The following sensors use the I2C protocol :
BME280, TSL2561, HMC5883, ML6070
The following sensor uses the "one wire" protocol:
DS18B20
The following sensors provide an analog output:
ML8511
The Windsensor Rev. P has two analog outputs, one for the temperature and one for the windspeed.
The anemometer has an analogue output.
The tipping bucket works with two simple on/off signals.
One sensor has a 5V output. The rest have 3.3V outputs. One requires 15 V, one 12V and the rest need 3.3V power supply.
Data-collection
Per measured quantity it is adjustable which interval between measurements is applied.
It might happen that values change suddenly. To be able to catch those rapidly changing values also, per quantity an adjustable difference between sensed values is applied to collect and transmit data. This results in a regular update of data per sensor. On a sudden change in value a sensor transmits extra data.
Data-transmission
For data-transmission the MQTT protocol is used. Using the "ESP8266MQTTMesh" library the MQTT protocol is used in an automatically self-establishing radio network. All nodes together create a network using up-to four radio-links each simultaneously. As long as a node has at least one radio-link with another node, it has full transmission capability. The node with the best signal of the WiFi network establishes one single WiFi link used by all nodes. This WiFi link is used by all nodes for their individual connection with the MQTT broker. The network uses normal WiFi protection mechanisms.
Data processing and presentation
eGeoffrey is used for data processing, data presentation and sensor and module control.
DESIGN
People have a place. At that place are buildings. At that place is a garden. For the software a building and the garden both are spaces. A space has rooms. A room has modules. A module has sensors.
Messages are going out of the sensor module or are coming into the sensor module.
The data presentation is grouped into functional areas e.g.:
Naming convention in- and out going messages
The sensor module uses a root "in-topic" and a root "out-topic". eGeoffrey uses a topic sequence to send and a separate topic sequence to receive MQTT messages.
The sensor module is leading. It's "topic-out" is "out". It's "topic-in" is "in".
For eGeoffrey it is automatically the other way around. eGeoffrey receives messages via "out" and sends messages via "in".
Data display
To gain a better overview of the data presented several pages are used.
One page shows the actual weather data out of the internet for the location. Another page shows the actual values of the sensors of the weather-station. Per quantity a separate page is created. Data and widgets displaying data of the same quantity are visible on one page. There is e.g. a page bringing together actual values and an historic overview of temperature data.
To control sensors and modules a control page for each purpose is available.
To have an overview of several modules being on-line a "heartbeat" page is made.
An extra page shows data of eGeoffrey (the raspi where eGeoffrey lives).
Sensor module
The weather station has several sensors. Each sensor transmits data with an operational interval. For testing purposes each sensor can transmit data with a test interval. When the quantity measured changes rapidly more than a "delta" value, extra data is transmitted to catch this kind of rapid strong variations.
Furthermore the measured value of each sensor can be modified with a positive or negative float value. The aim is to move the curve representing the historic data of the sensor up and down on the screen.
With eGeoffrey the following aspects of each sensor can be modified:
When an aspect of a sensor is changed, the feedback of each aspect is displayed together with the name of the sensor. Between changes the content of these widgets is blanked.
Each sensor can be interrogated to transmit the value of it's aspects.
The weather station can be switched:
It can be rebooted.
During testing of the hardware it is necessary to trace why a widget is not behaving as expected. There can be several causes. Therefore individual parts of a data transfer path need to be checked.
It is possible to:
Controlling sensors
To send control parameters from within eGeoffrey a separate sensor control page is made.
On this page:
The module receives the MQTT messages transmitted by eGeoffrey. As soon as the “acknowledgement” is received the data are processed and stored into EEPROM to be used from now on.
Because eGeoffrey has to address sensors of an individual module, the module ID needs to be part of the topic eGeoffrey uses to send data to that module. Receiving data the wildcard "+" is used instead of the module ID.
To be sure that the data are received correctly the sensor module sends the data back to eGeoffrey.
Six widgets are used to display the data received from the module.
On this page the widgets are available to query the sensors.
Controlling the weather station
On a separate page it is possible to switch all sensors together on/off. They can also be switched from ops to test mode together. After finishing a test series it is handy to reset the module to the default values with only two actions. To have the module reboot a separate topic is used.
A dedicated widget provides feedback from the weather station.
MENU STRUCTURE
toplevel:
below weather:
below control:
below system:
TOPIC STRUCTURE
The "ESP8266MQTTMESH" library does not allow for topics having more than 65 characters. Therefore some topic elements have been condensed:
module-in -> in
module-out -> out
weatherstation -> wthr
sensor -> sr
temperature -> tmp
windspeed -> wspd
winddirection -> wdrtn
pressure -> prs
humidity -> hm
light -> lght
rain -> rn
anemometer -> anemo
opsinterval -> opsntrvl
testinterval -> tstntrvl
Topic structure weather station data
area-ID/space-ID/module-ID → module topics
area-ID/space-ID/module-ID/sensor → sensor general topics
area-ID/space-ID/module-ID/sensor/sensor-quantity/sensor-Id → sensor specific topics
Topic structure weather station sensor data
myplace/garden/wthr/sr/tmp/ds18
myplace/garden/wthr/sr/tmp/bme
myplace/garden/wthr/sr/tmp/xp
myplace/garden/wthr/sr/wspd/anemo
myplace/garden/wthr/sr/wspd/xp
myplace/garden/wthr/sr/wdrtn/compass
myplace/garden/wthr/sr/wdrtn/txt
myplace/garden/wthr/sr/prs/bme
myplace/garden/wthr/sr/hm/bme
myplace/garden/wthr/sr/lght/tsl2561
myplace/garden/wthr/sr/uv/ml8511
myplace/garden/wthr/sr/uv/ml6070
myplace/garden/wthr/sr/rn/volume
Topic structure control data
control sensor:
myplace/garden/wthr
myplace/garden/wthr/sr
myplace/garden/wthr/sr/switch
myplace/garden/wthr/sr/opstest
myplace/garden/wthr/sr/opsntrvl
myplace/garden/wthr/sr/tstntrvl
myplace/garden/wthr/sr/adjust
myplace/garden/wthr/sr/ack
control module:
myplace/garden/wthr
myplace/garden/wthr/switch
myplace/garden/wthr/opstest
myplace/garden/wthr/default
myplace/garden/wthr/reboot
myplace/garden/wthr/ack
myplace/garden/wthr/info
retrieve settings:
myplace/garden/wthr/sr
myplace/garden/wthr/sr/retrieve/select
myplace/garden/wthr/sr/retrieve/ack
myplace/garden/wthr/sr/retrieve/info
Topic structure system data
myplace/garden/wthr/heartbeat
DATA EXCHANGE
Sending data
Sensor module publishes sensor data with topic:
out/myplace/garden/wthr/sr/tmp/ds18
out/myplace/garden/wthr/sr/tmp/bme
out/myplace/garden/wthr/sr/tmp/xp
data type: float
out/myplace/garden/wthr/sr/wspd/anemo
out/myplace/garden/wthr/sr/wspd/xp
data type: integer
out/myplace/garden/wthr/sr/wdrtn/compass
data type: integer
out/myplace/garden/wthr/sr/wdrtn/txt
data type: string
out/myplace/garden/wthr/sr/prs/bme
data type: float
out/myplace/garden/wthr/sr/hm/bme
data type: integer
out/myplace/garden/wthr/sr/lght/tsl2561
out/myplace/garden/wthr/sr/uv/ml8511
out/myplace/garden/wthr/sr/uv/ml6070
data type: float
Sensor module publishes control feedback data with topic:
out/myplace/garden/wthr/sr
data type: string
out/myplace/garden/wthr/sr/switch
out/myplace/garden/wthr/sr/opstest
out/myplace/garden/wthr/sr/opsntrvl
out/myplace/garden/wthr/sr/tstntrvl
data type: integer
out/myplace/garden/wthr/sr/adjust
out/myplace/garden/wthr/sr/delta
data type: float
out/myplace/garden/wthr/sr/retrieve/info
data type: string
out/myplace/garden/wthr
data type: string
eGeoffrey
In eGeoffrey data are published by "sensors". For publishing data the "processing" aspects are not important.
eGeoffrey publishes sensor control data with topic:
in/module-ID/myplace/garden/wthr/sr
data: integer - -1.. number of sensors (12, "-1" used to blank the widget)
in/module-ID/myplace/garden/wthr/sr/switch
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/sr/opstest
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/sr/opsntrvl
data: integer - 1 .. 60 (minutes)
in/module-ID/myplace/garden/wthr/sr/tstntrvl
data: integer - 1 .. 15 (seconds)
in/module-ID/myplace/garden/wthr/sr/adjust
data: float - -5 .. 5
in/module-ID/myplace/garden/wthr/sr/delta
data: float - 0.1 .. 150
in/module-ID/myplace/garden/wthr/sr/ack
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/sr/retrieve/select
data: integer - 0 .. 4
in/module-ID/myplace/garden/wthr/sr/retrieve/ack
data: integer 1 / 0
eGeoffrey publishes module control data with topic:
in/module-ID/myplace/garden/wthr
data: integer - -1 .. 1 number of modules (1, "-1" used to blank the widget)
in/module-ID/myplace/garden/wthr/switch
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/opstest
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/default
data: integer 1 / 0
in/module-ID/myplace/garden/wthr/reboot
data: integer - -1 .. number of modules (1, "-1" used to blank the widget)
in/module-ID/myplace/garden/wthr/ack
data: integer 1 / 0
Receiving data
In eGeoffrey data are received by "sensors". For receiving data also the "processing" aspects are of importance.
eGeoffrey receives sensor data with topic:
Processing:
Automatic aggregation – avg – min – max
Retention Policies – history
Beware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr/sr/tmp/ds18
data: float
out/+/myplace/garden/wthr/sr/tmp/bme
data: float
out/+/myplace/garden/wthr/sr/tmp/xp
data: float
out/+/myplace/garden/wthr/sr/wspd/anemo
data: integer
out/+/myplace/garden/wthr/sr/wspd/xp
data: integer
out/+/myplace/garden/wthr/sr/wdrtn/compass
data: integer
out/+/myplace/garden/wthr/sr/wdrtn/txt
data: string
(Processing:
Automatic Aggregation - None
Retention Policies - Single Value)
out/+/myplace/garden/wthr/sr/prs/bme
data: float
out/+/myplace/garden/wthr/sr/hm/bme
data: integer
out/+/myplace/garden/wthr/sr/lght/tsl2561
data: float
out/+/myplace/garden/wthr/sr/uv/ml8511
data: float
out/+/myplace/garden/wthr/sr/uv/ml6070
data: float
out/+/myplace/garden/wthr/sr/rn/volume
data: integer
(Processing:
Automatic aggregation – sum
Retention Policies – history)
eGeoffrey receives sensor control data with topic:
Processing:
Automatic aggregation - none
Retention Policies – single value
Beware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr/sr
data: string
out/+/myplace/garden/wthr/sr/switch
data: string
out/+/myplace/garden/wthr/sr/opstest
data: string
out/+/myplace/garden/wthr/sr/opsntrvl
data: integer
out/+/myplace/garden/wthr/sr/tstntrvl
data: integer
out/+/myplace/garden/wthr/sr/adjust
data: float
out/+/myplace/garden/wthr/sr/delta
data: float
out/+/myplace/garden/wthr/sr/retrieve/info
data: string
eGeoffrey receives module control data with topic:
Processing:
Automatic aggregation - none
Retention Policies – single value
Beware: the eGeoffrey sensor name can not contain "+". The topic can.
out/+/myplace/garden/wthr
data: string
DATA INPUT and PRESENTATION
Sensor data widgets:
Widget Type: value
For overviews more Widget Type's are available.
out/myplace/garden/wthr/sr/tmp/ds18
out/myplace/garden/wthr/sr/tmp/bme
out/myplace/garden/wthr/sr/tmp/xp
out/myplace/garden/wthr/sr/wspd/anemo
out/myplace/garden/wthr/sr/wspd/xp
out/myplace/garden/wthr/sr/wdrtn/compass
out/myplace/garden/wthr/sr/wdrtn/txt
out/myplace/garden/wthr/sr/prs/bme
out/myplace/garden/wthr/sr/hm/bme
out/myplace/garden/wthr/sr/lght/tsl2561
out/myplace/garden/wthr/sr/uv/ml8511
out/myplace/garden/wthr/sr/uv/ml6070
out/myplace/garden/wthr/sr/rn/volume
Sensor control widgets:
in/myplace/garden/wthr/sr
type: input box
in/myplace/garden/wthr/sr/switch
type: On/Off switch
in/myplace/garden/wthr/sr/opstest
type: On/Off switch
in/myplace/garden/wthr/sr/opsntrvl
type: input box
in/myplace/garden/wthr/sr/tstntrvl
type: input box
in/myplace/garden/wthr/sr/adjust
type: input box
in/myplace/garden/wthr/sr/delta
type: input box
in/myplace/garden/wthr/sr/ack
type: button
action: SET in/1D8445/myplace/garden/wthr/sr/ack 1
Sensor Feedback widgets:
Widget Type: value
out/myplace/garden/wthr/sr
out/myplace/garden/wthr/sr/switch
out/myplace/garden/wthr/sr/opstest
out/myplace/garden/wthr/sr/opsntrvl
out/myplace/garden/wthr/sr/tstntrvl
out/myplace/garden/wthr/sr/adjust
out/myplace/garden/wthr/sr/delta
out/myplace/garden/wthr/sr/retrieve/info
Module control widgets:
in/myplace/garden/wthr/
type: input box
in/myplace/garden/wthr/switch
type: On/Off switch
in/myplace/garden/wthr/opstest
type: On/Off switch
in/myplace/garden/wthr/default
type: button
action: SET in/1D8445/myplace/garden/wthr/default 1
in/myplace/garden/wthr/reboot
type: button
action: SET in/1D8445/myplace/garden/wthr/reboot 1
in/myplace/garden/wthr/ack
type: button
action: SET in/1D8445/myplace/garden/wthr/ack 1
Module feedback widgets:
Widget Type: value
out/myplace/garden/wthr
out/myplace/garden/wthr/sr/retrieve/info
TESTING
Test configuration
Laptop1
Webbrowser with aGeoffrey website to:
Laptop 2
Each controlled sensor is tested. Each switch is put in both positions. Each quantity is tested with the minimum, middle and maximum value.
Quantity position
switch on/off on off
switch ops/test on off
values
ops interval: 1 - 29 - 60
test interval: 1 - 8 - 5
adjustment: -4.99 - 0 - 4.99
delta: 0.1 - 9 - 149.99
Every time before a test series the info widgets are cleaned.
Unexpected results are noted down.
After conclusion of testing the results are analysed and corrective actions are performed.
The tests are rehearsed until all control data are transmitted correctly, received correctly, evoked the correct action, the feedback is sent correctly and the feedback is displayed correctly.
Than the system is ready for operation.
Rules
Rules are a means to evaluate the data.
This rule supervises the heartbeat signal of a module.
The simplest way is to have a recurrent rule running every minute with:
value = sensor_id
value_elapsed = ELAPSED sensor_id
value_off = 0
elapsed_threshold = 1
value == value_off
value_elapsed > elapsed_threshold
After the weather station has been on-line for several weeks some adjustments to the software are done. Some of the sensors produce spikes in the data. Some sensors are delivering nothing but spikes.
To do away with the spikes, the ability to produce extra data on strong fluctuations can be switched off. When the sensor is in overload, transmission of data is suppressed.
This results in a far more intelligible presentation of the data.
All this preparation and labour resulted in a working weather station.
To give you an impression of how my weather station looks like here are a few screenshots:
Local weather data
All temperature sensors
Changing settings per sensor
Feedback from the weather station
Adjusting all sensors together
Enjoy!
Beta Was this translation helpful? Give feedback.
All reactions