eGeoffrey
My general sensor system #133
Replies: 103 comments
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-06 10:46:47] In the following posts I will explain the procedure I used to come to a working system. Installing and adapting eGeoffrey Starting point is a Raspi 3B+ with the OS on USB-stick instead of on SD card. Nowadays not only Raspberry Pi OS (old: Raspbian) is available but also some others of which "Ubuntu MATE" is one. Because I do not like some features of Raspberry Pi OS, I decided to give Ubuntu MATE a try. I am using Linux Mint on my laptop. That is an Ubuntu flavour. On the laptop I got acquainted with the network manager. Especially this software tool is quit different from the one of the Raspi. Installing this OS on Raspi is straight forward. Burn the image onto the USB stick instead of the Raspberry Pi OS and the rest is the same as with the latter. In my house I have a network with several WiFi access points connected to the wired LAN. During installation of the OS on the Raspi, my network server allocated an IP-address to the Raspi via DNS. I do not provide fixed IP-addresses. The validity (lease-time) of the addresses the server provides is very long. On that server an MQTT broker, mosquitto, is also always running. Once the Raspi is up and running only every now and than the peripherals are needed. Configuration of eGeoffrey is done via webbrowser. Because interaction with the Raspi is required only for installation of the core of egeoffrey and some extra modules, I think it is not worthwhile to spend time on some remote access possibilities like VNC for this. Just use the monitor, keyboard and mouse hooked-up to the Raspi directly. OK lets get started with egeoffrey. Open a terminal window on the Raspi. As far as I experienced the commands require "root" rights to be performed. So use "sudo" to issue the commands. Issue the command: Now lean back and wait until several software packages have been downloaded and installed. To see if everything is OK in the terminal window goto the "egeoffrey directory"; issue the command: Now issue the command: It takes some time but than you have an overview of the status of the different software modules of egeoffrey. The status of each component should read "Up". To see if the website of egeoffrey is also up and running, on a separate machine log-in in the network of which the Raspi is a user. Point a browser to the IP-address of the Raspi by inserting the IP-address in the URL field of the browser. It might take a little bit longer than you are used to but the welcome page of the egeoffrey website should be presented. Standard you are logged-in as "guest". If you see "bells and whistles" egeoffrey is OK. Now we have to proceed to extend the standard eGeoffrey with the "mysensors" module. Go back again to the Raspi and in the terminal window in the "egeoffrey directory" issue the command: than issue the command: Now the actual download and install procedure starts. Lean back again until everything is ready. To see if everything is OK just request the status of eGeoffrey again. Issue the command: Again everything should be "Up". This concludes the installation of eGeoffrey. If no surprises pop-up it is a piece of cake! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-06 11:48:32] Now we are going a step further. Now on a second machine, point your webbrowser to the IP address of the Raspi on which you installed eGeoffrey. You are logged in as "Guest". To get to the Admin part of the website you have to log out and log in as Administrator. Log out by clicking on the "arrow" before "Guest" in the left panel. In the panel in the middle under "User" replace "Guest"with "admin". Insert "admin" in the field "Password". Check what is there: House: House leads to "House configuration" window. Name of your house: insert what you like. Timezone offset: google on "timezone offset". Via the "Wiki" article you can find the offset you need. Insert the number into the field. Language: only "en" is possible. Units: select "Metric" or "Imperial". Latitude: For the location of your house, click the link. If everything went OK, your location is given in two ways: DD (decimal degrees) and DMD (degrees, minutes, seconds) Take the DD values with two decimals. Insert the Latitude value into the field Latitude and the Longitude value into the field Latitude. Confirm the data by clicking the "Save" at the bottom right side of the window. The next tab on the "Settings" page is "Web interface". Leave it for the time being. The next tab on the "Settings" page is "Users". Leave the "admin" data untouched for the time being. The same pertains to the data of "Guest" The possibility to add users is activated by clicking on "guest". Now on the bottom right side of the panel the possibility to add, delete and save a user becomes available. Click on "New User". A small window pops up. Insert the username of the user. Than the field "Icon*" needs to be filled (with what?). I inserted the username. After inserting a password into the "Password" field, save the data with the "Save" button. The next tab is "Groups" Three groups are available: At the bottom right side four buttons are available: The use of "New User" is not clear. To add a user to a specific group select that group in the most left column. Clicking "New Group" provides a pop-up window to insert the name of the new group. The use of creating a new group is not clear. For the issues which are not clear I will post a separate question. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-07 10:10:25] Thanks again for keeping track of your progress this helps me in understanding typical user's process and I will for sure take inspiration from this for enhancing the current documentation! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-07 10:56:48] Some weird things happening having nothing to do with Geoffrey but with Raspi OS. I gave Raspi OS in the most recent version another try. After update/upgrade the marketplace is not accessible anymore????????? Going to try some things to look if I can find a workaround. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-07 18:52:38] @eporocrail wonder if the update includes some new curl/wget version which requires different switches than the one I am using. Would you mind detailing the steps so I can fully reproduce? (e.g. which raspbian should I start with, which upgrades, etc.) Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-08 08:07:40] I am using RaspioOs Install without automatic update. Manual update some info from terminal: Unpacking rpi-eeprom (7.2-1) over (7.0-1) ... After this update I have no access any more to the marketplace |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-08 08:57:49] @eporocrail ok so if I understand correctly you start from 2020-05-27-raspios-buster.armhf.img then apt-get update & apt-get upgrade and when trying to install a new eGeoffrey package you get the behaviour you described on https://forum.egeoffrey.com/topic/15/installing-new-service/3, correct? If so I'll try to reproduce the problem. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-08 09:44:05] @user2684 |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-08 12:17:45] @eporocrail ok so the problem is when accessing with the web browser, not with the CLI, my bad in understanding. Are you using the browser on the raspi or on a different device to access? If you experience the problem with both my guess is that the thing has nothing to do with the upgrade but when you refresh multiple times the marketplace website, since it is building the page on the fly by pulling information from github, when reloaded multiple times, github is blocking too many requests (try visiting https://api.github.com/repos/egeoffrey/egeoffrey-marketplace/contents/marketplace for confirmation). If so I need to add an explicit warning or something for this. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-08 13:45:01] @user2684 I think you are right. I was able to install the mysensors module. I expect to proceed with the installation not earlier than in two days. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-08 14:01:54] @eporocrail looks like the case, I had a similar issue a while ago but with many many times the page realoaded, but I see this happening faster now, could easily be github lowering down the limit. I've opened up an issue to fix this (both for the marketplace and the CLI - egeoffrey/marketplace#1), it could take a while since I need to figure out the best way to keep a local cache, hope it is not a blocking issue for now. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-12 09:05:36] I did a complete new install starting with Raspi OS. Inserted the details of the house. Started inserting a new sensor. I inserted the data again and clicked "Save". The system did not only "save" but also "close". After a "save" one leaves the sensor configuration. Where do I find information about the values for the field "Sensor Icon"? On the tab "data" it is not clear what "format" means. I wanted to insert a simple on/off button. What "format" do I have to select? Where I am using MQTT I would expect to insert data for the broker somewhere. I found it under "eGeoffrey - modules" Later I will continue configuration and testing |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-12 11:11:19] @user2684 Register a new sensor. Weird behaviour: as soon as the the data on tab "General" is saved the input window is closed instead of data only being saved. After inserting data on tab "general data" go to "Advanced Editor". All empty. Go "Back". Unclear what data is inserted into filed "Sensor Icon". Do NOT save. Tab "Data" Tab "Processing" Tab "Service" Tab "Configuration" Configuring the "mysensors" module. Under "eGeoffrey" goto "Modules". At the bottom three versions of the "service/mysensors" are visible and running. Now only the module "service/mysensors_mqtt" is running. For the "mqtt" version click on the "arrow-down" in the "Actions" field. Module Configuration. The MQTT hostname to connect to. Insert the "hostname" or the IP-Address. "192.168.2.1" in my case. The port of the MQTT Broker. Is set to "1883". Leave as is. The username for authenticating against the MQTT broker. In my case leave empty. The password for authenticating against the MQTT broker. In my case leave empty. Topic prefix to subscribe to. "kamer/uit". Topic prefix to publish to. "kamer/in". Under "eGeoffrey" goto "Icons". Here they are. It is a list with general purpose icons. Not many I think useful for eGeoffrey. Looking into the logs it appeared that errors where caused by the mysensor modules which are not used. Stopped both modules. No effect. Both modules keep being interrogated. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-13 17:31:13] The marketplace issues has been solved, both for https://marketplace.egeoffrey.com and when using the CLI (please upgrade to v1.0-26 with |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-13 17:52:40] Thanks once again for these precious feedbacks. Let me reply point by point:
This is expected, the button save and close the window. Do you think a different label on the button would work better?
Still expected (or better a different behaviour would be really difficult to implement). I'd recommend avoiding using the Advanced Editor at the beginning since the configuration schema is not documented yet and there are no options by the wizard. I could take the link away if this is creating confusion.
This has to be documented better for sure. This field (any icon-related field) is expecting the name of an icon taken from the "eGeoffrey" / "Icons" menu under Administration
I've added a bunch of new pages on the docs website, have a look at https://docs.egeoffrey.com/configure/sensors/ if not done yet, wonder if clear enough. Format is the format of the data supposed to be stored in this sensor (being a sensor a logical container of one or more values). For a on/off sensor you can use "integer" and store 0 or 1. If the remote sensor is instead sending or expecting the string on/off, then would be string.
This entirely depends on the logic of the attached sensor. If you set integer and store a 0 or 1 and the remote relay expect 0 or 1 to trigger this is not required. When you set "1_to_ON" means every time a new value comes in (from the remote sensor or from the web interface, the 1 is converted to the string "ON" which is not necessarily what you want to use. The web interface has a on/off widget which expects a 0/1 stored in the sensor.
Yes, this is usually to be done before configuring the sensor (I tried to describe the workflow in this new page: https://docs.egeoffrey.com/configure/workflow/). Configuring the module set up the integration, the sensor leverages then the integration
An alternative (more advanced) would be to edit the docker-compose.yml file in the eGeoffrey directory, identify the entry for the MySensors package and under EGEOFFREY_MODULES, removing those you would not need.
Ok so you're saying even after disabling those modules they are still running after a reboot. I'll try to reproduce it. What about the mqtt one, is everything working there? |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-27 10:08:30] Would it be possible to limit the amount of email messages when rules for sensing them triggers it. Maybe it is possible to incorporate on the "schedule" page of the rule a value "maximum messages". |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-27 17:53:32] @eporocrail said in My general sensor system:
Ok, just some background first in case in the future somebody else could be interested. Something very handy of eGeoffrey architecture is that the different modules can reside everywhere and as far as they are connected to the same message bus, everything just works. This means they can stay all together like you have at home or something can can stay in a publicly accessible server and something else at home; as far as they are connected, it just works. This opens open two scenarios which are NOT yet available but in case this project will get any traction I can think of fully implementing:
Both the two methods leverage the same principle which is connecting the local mosquitto broker with one in the cloud. We are basically just leveraging the bridge capability of mosquitto to make the local broker connecting to the cloud one: every new message published locally will be mirrored to the cloud and viceversa and since the entire eGeoffrey inter-communication is through mqtt, when accessing the cloud gateway it is like you are at home. Of course there are strict ACLs in place to prevent one house to see the other. Having said that, all of this is of not user-friendly yet, mostly manual and no assurance it is working perfectly. Steps are the following:
In this way the local mosquitto gateway in addition to provide the same services it was providing before will also establish a secure connection to gateway.egeoffrey.com. From now you can connect via mobile app (or even using the same local web interface for testing) to gateway.egeoffrey.com, port 443, SSL enabled, using your house_name and password for authenticating. Your admin and guest users instead will be the same as before. As you can see it is a bit manual process right now. Let me know first of all if interested in this capability and if you are willing to go through this manual process. If very interested but prefer some more guided way, let me know and I'll see if I can automate all of this through the egeoffrey-cli (could take some time though) |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-27 17:55:14] @eporocrail said in My general sensor system:
Right now this capability is available for each notification module (maximum number of notifications per hour). Do you believe would make more sense if per rule? (or both) |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-27 18:12:26] Because we are talking about email notifications I think it would be enough when each rule sends an alert only a small number of times (let the user decide on the exact number). I myself would go for some odd three times for each separate rule. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-27 18:57:57] @eporocrail understand makes sense, just help me in understanding which is from your point of view the best way to have this applied at the rule level. Maximum message for which timeframe? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-27 19:00:59] I do not know if it is feasible but maybe via "edit rule" a button to reset the counter. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-27 19:19:56] @user2684 said in My general sensor system:
I have no idea where to find it. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-27 19:27:42] Today I have been playing around with a light level and UV level sensor. I managed to finish the software for my decoder. That means that the messages are entering eGeoffrey. It is a charm to implement it in eGeoffrey. The more experience I gain the more I get to like eGeoffrey!!!! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-28 08:11:00] Is this the "chat function" ?? |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-28 15:08:57]
I guess this is something feasible. So basically something like X maximum notifications from this rule per hour with the possibility to reset the counter, right?
Glad to hear that. Probably the getting started is a bit steep but happy to hear you like the flexibility there. I will try to publish some more HOW TOs, hope this can be helpful as well. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-28 17:55:43] @user2684 said in My general sensor system:
Sounds good to me. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-06-28 18:01:43] Ok thanks, tracked with egeoffrey/egeoffrey-controller#18 |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-07-12 09:58:27] @user2684 said in My general sensor system:
@eporocrail we left a while ago with this advanced configuration you preferred not to do (correctly). This is now possible through the updated |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-04 09:10:55] We addressed the issue that with me the topic had to begin with "/". This turned out to be a misunderstanding from my side. When looking into another problem I discovered that I had introduced the leading "/" myself, while there is no real need to do it. So just forget this issue. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-08-05 11:02:25] @eporocrail said in My general sensor system:
Well, there was an issue indeed with the MySensors module actually and even if not completely related this has helped me in identifying it. So was useful anyhow |
Beta Was this translation helpful? Give feedback.
-
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-05 15:36:50]
Overview sensor system
The sensors are controlled by ESP8266 modules.
The ESP8266 modules are interconnected with the ESP8266MQTTmesh network.
The transmission protocol is MQTT.
The MQTT broker is Mosquitto.
The data transfer protocol is the "mysensors" protocol.
Data are processed and displayed by "eGeoffrey"
Sensor controller
The software is developed and built in a modular way.
Parts of the software are derived from my Wireless Track Control System. This system is made to control tracks and sensors of my model railroad. WTCS is operational. It consists of decoders interconnected by the ESP mesh network.
Per type of sensor the data specific for that sensor are processed in the ESP module to be transmitted using the mysensors protocol over MQTT.
The mesh only transmits "char" type of data.
This means that data from a sensor have to be transformed into "char" data type.
Also the format of the mysensors protocol has to be constructed.
Data transfer
Simulated data from a simple on/off sensor and simulated negative temperature data with one decimal are processed and transferred to Mosquitto.
Mosquitto is running on my LAN-server.
A separate PC is receiving the test data correctly in the right format.
Data processing and presentation
The next step is to install eGeoffrey with the "mysensors" interface on a Raspi 3B+.
When eGeoffrey is up and running knowledge has to be gathered about the way eGeoffrey has to be tuned to have the data processed and displayed as envisioned.
Have fun
Ellard
Beta Was this translation helpful? Give feedback.
All reactions