-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Weather Condition generating locally #17
Comments
This is high on my list as well, and thinking through how I might do the same locally. I have already expanded my use of current conditions/icon selection in HA with a template sensor and pulling information from multiple sources, just not to the extent as you contemplate above... yet. I suppose the HA weather component is necessarily limited to a LCD of conditions. To me, partly cloudy, mostly cloudy, and cloudy are three very distinct and recognizable conditions. And I was hoping to be able to distinguish between them with logic as you mention. I am also not a huge fan of how WeatherFlow does its current condition. They will use something like "rain possible" even though the sky is clear or partly cloudy. To me, current condition is what it is NOW, the possibility of rain is a forecast. Dark sky does a better job of this, so for now I am using their current conditions to drive much of my NOW, sensor, at least until apple shuts it down next year. I am certainly happy to contribute to this however I can. I think it could be a significant value-add. But I dont think the calculations will be trivial to make it work for everyone. Local latitude, current season, and sun angle/time of day will all likely need to be factored in. So, it may be more feasible as a suggested template sensor with perhaps some guidance on tweaking for local use. In any case, count me in. |
What spurred it for me was exactly what you said above, I had a condition of raining as reported by api but there was no rain. I would like it to be actual not predicted just at you stated. On a slightly separate note you could look up purpleair on the HA forums to see some of the gauges I've done. If you look up pollen, I have an updated gauge with a color coordinated background to describe conditions such as cool for temperature or dry for dew point. I will see what I can knock out this weekend. I am open to ideas either now or after I post my first attempt to see if anything can be smoothed over. |
You know, before your post, I was blissfully unaware of the Purlple Air sensor. Now I NEED one! My wife is not going to be happy with you ;) |
Hi both, I guess the best way to start is by what Glenn has already done, trying to make some algorithms for each condition, and then tune these a as we go a long. I will be happy to make a new sensor, we could call it If everything fails, what would be the default condition: Cloudy? I think it is good that you posted on the forum Glenn, there might be others already doing some of this. As soon as I have time, I will start building it, based on what is already here. |
@chasut enjoy: I am still tweeking the backgounds on Dewpoint and Temperature that is why there are no words there yet. The UV background started it all, I saw someones background for their UV index and I modified it for other things. I was using an integration that pulled the UV data before, but I removed that integration since I have the Tempest now. So from this the UV is the only thing coming from the Tempest, I still am using the PurpleAir for Temperature/Dewpoint/Humidity and of course Air Quality. The Tempest and PurpleAir track fairly well except for humidity; the Tempest seems to run high sometimes like 99 and 100% too often. Here is my other dash (AQI is duplicated here) The PurpleAir has a local web server on the device that allows you to look at the device directly, or you can look at the web map, or you can pull the data from the Rest API (local and/or web) (which is what I do). |
I might do this, but it will not be prioritized right now. But please keep posting how you would calculate the different Conditions, then I will start adding when I have time. |
I'm not sure what happened, but I had updated a lot of the conditions in the above post...I probably closed the tab without saving the edit. |
Establishing order of checks (priority): ‘lightning’ 3 hour lightning count 10 < (sensor.wf_lightning_count_3_hours < 450 (attempt to eliminate false reports from noise on detector) & sensor.wf_precipitation_type > 0 ‘hail’ sensor.wf_precipitation_type = 2 ‘snowy-rainy’ sensor.wf_precipitation_type > 0, sensor.wf_temperature = 32F / 0C ‘snowy’ --- snowy is a guess since there is no 'sensor' that detects snow ‘pouring’ sensor.wf_rain_rate > 0.31 in/hr (7.8 mm/hr) or sensor.wf_precipitation_type = 3 ‘rainy’ sensor.wf_precipitation_type = 1 ‘windy-variant’ Wind Speed > 25mph (11.17 m/s) & cloudy ‘windy’ Wind Speed > 25mph (11.17 m/s) ‘fog’ Fog is likely when: ‘cloudy’ reduced luminescence with a steady tread ‘partlycloudy’ trend of alternating lower and higher lux ‘clear-night’ sun angle < 0 and no other events ‘sunny’ no other events and sun angle > 0 (minimum of 10,000 lux) ‘exceptional --?? (looked at some weather cards and does not seem to be used) Need to look at more data and adjust for latitude and time of day. |
Still gathering information. Discussed with an individual on the WeatherFlow forums that calculates cloudy based on UV level comparison with historical data for each month for a given latitude; I could go this route but this complicates it much more than I anticipated. I am still looking for a 'simple' formula for UV or even solar radiation for latitude. I might end up breaking it up to latitude segments of 5-10 degrees for an approximation and then see how to figure the bell curves for daylight for a given time of year for a given latitude. I will test what I have as a sensor in Home Assistant and fine tune as needed. Once I have it fairly reasonably working for me, then I'll put it on the forum for HA and hopefully get others at different latitudes to test. |
Working on 'normal' UV levels. Need the following: Solar Declination Angle = 23.45 * sin(360/365(d-81)) Max Elevation at Solar Noon = 90 + h * (Latitude - Declination Angle) Elevation (already available) Assumption that weather station UV sensor sees direct sunlight most of the day. More calculations to come. |
I will start looking at this, in a few weeks when my vacation is coming up, and see what I can do. |
I think I have to reevaluate how I am determining cloudy. Today was completely overcast but UV levels were up significantly. I know that clouds can reflect some UV back down but I saw UV index of 6 without clouds (two days ago) and 11.76 with clouds (today). Solar Radiation and Light Levels were both down though (yes, I know that UV is a different sensor). I might go with comparing current light levels to calculated light levels (based on time of day, latitude, and time of year); and also a comparison of current UV to current LUX. I think I can get away with only calculating for the Northern Hemisphere (NH) and then make a 180 day adjustment for the Southern Hemisphere (SH). The one thing I can not get locally is Ozone levels which affect UV levels, but they are fairly constant during the SH summer/ NH winter Jan/Feb/Mar. This has become a much deeper rabbit hole than I expected and I'm still digging down. It is very easy to look at either the UV or Lux or Sol Rad graphs and see when there are clouds. This could be used if the day started out sunny and cloud(s) moved in causing a more rapid than drop than sunset; but if full clouds for the whole day then there would not be any drops. Just documenting my thoughts. |
@briis is there a easy way to determine if the forecast connection is down? I know there is a similar issue here but my need is to provide the local conditions (mainly the current conditions icon) only when there is no internet. Yesterday I was without grid power and internet at home for 7 hours and I had all my WF sensors but I found that I really missed my big animated icon of the current weather conditions. So I was thinking of supplying my estimations of current conditions when either there is no connection to the weather flow server or someone has opted to not get forecast data. |
@GlennGoddard There are couple of ways to test if the connection fails and/or the data we have for forecast is empty, and then based on that switch to the local format. I guess the most important part is that the dataset is formatted in the same way as the data we retrieve from WeatherFlow. I have attached a sample json file - this is what gets returned when calling the forecast. I can help with generating that, if you make the dataset and then implement the switch-over function. |
I have not abandoned this, I have been very very very busy lately; hopefully I will have some time over the Christmas break to work on it. |
No worries. |
The Zambretti Forecast is the same that my Davis Vantage station delivers. Honestly I have never checked how accurate that is, but I assume it is better than nothing, if without Internet connection. |
It would be great to the weather condition locally! I also reaslied a couple of days ago that Tempest Weather Sensor showed I can't really contribute on this feature of full local forecast, but for now, I created a sensor which merges the weather condition in the following priority:
Since we can't calculate anything which involves a "mixture" where we only have one of the two conditions (rain and wind), or if it is exceptional (in particular: snowy-rainy, windy-variant and exceptional), we need to check the internet forecast first. For example, we need first to check if its You may adapt the parameters to your needs (lightning/lightning-rainy: strike <=1km away within the last 10mins, windy: 32-48km/h).
You can then use this sensor to create a weather entity which has, at the time of writing, the most possible local weather condition :) Here is the code for your
|
@badewanne1234 this may not be a bad approach to compare local station data to WeatherFlow's current condition when there is internet and the forecast option is setup. I was working off the idea of providing the current conditions completely locally, to allow for no internet; but I like your idea of comparing local and web data for the current condition. |
@GlennGoddard So for now, I'm sticking with the above approach, and for me it works very well, the current weather condition is much more accurate now. I'm also updating the above post, I'm still finetuning the weather state entity. It will be very interesting to see how the lightning calculation works, I probably need some tweaks there, but for that I'll need data from a lightning event... |
Narrowing in on Cloudy (will need a more data points at solar noon on a sunny day). The higher the solar elevation the higher solar radiation (the assumption is made that the weather station's sensor is parallel with ground). 90 degress is max solar elevation. Solar Elevation: α = sin^(-1)((sin(ϕ) * sin(δ)) + (cos(ϕ) * cos(δ) * cos(h))) δ = (-23.44) * cos(360/365 * (d + 10)) h = 15 * (LST -12) LST = LT + TC/60 TC = 4(λ - LSTM) + EOT LSTM = 15 * ΔTutc EOT = 9.87 * sin(2*β) - 7.53 * cos(β) - 1.5 * sin(β) I think the way I will go with this is to convert this to a percentage of max solar radiation for a given time and day and compare that with the measured solar radiation. If in a certain (yet to be determined) range then it is sunny. Specifically within a range since higher than 'max' solar radiation is an indication of fluffy white clouds reflecting UV light back down to the ground (as long as not directly between sun and monitor i.e. blocking direct sun light). If outside that range and no other conditions such as rain then cloudy. This also makes the assumption that station is never in the shade (not always practical but the easiest approach). This is also only valid during 'normal' daylight hours. Once I have a good working approximation for the Northern hemisphere, I will try to simulate for the Southern. I 'think' I have that already figured into the above calculations but I'm not entirely sure. As much as I would love to continue to use the Greek alphabet in the code, I will use more appropriate Latin characters for variable names. @badewanne1234 I can always use more data points to see how accurate this is; admittedly this is not ready for anyone else to try since I don't have the approximation done yet but any input is welcome. I have considered the consideration of solar eclipses but I don't know if I want to go there. It could be done with some packages but I don't think it is worth the effort (maybe). |
wow, looks like you're an expert! Let me know if I can assist with testing etc |
@badewanne1234 Things that would help, but not required if you don't feel comfortable: Don't post here yet. Probably IM in either the Home Assistant or WeatherFlow community boards. |
Partly Cloudy and Cloudy, I still have to figure out. I had no idea that figuring out local conditions would be this complicated when I started this. |
Sure not a problem. |
@badewanne1234 |
Update: I decided to test these in Node-Red since it is easy to modify and it was very easy to pull in lat/long. I still had to establish station elevation since that is not sent to MQTT (nor does it need to be). I am currently testing 'cloudy' based on estimated Solar Radiation vs measured Solar Radiation, my unfortunate issue is that I have lots of data for cloudy days but none with any sun. I am exporting my results from Node-Red to MQTT and then using MQTT explorer to look at the output in graphical form. I am testing a few algorithms for Solar Radiation estimation and I can see that sometimes they appear too high at solar noon but I'm not sure since I have not had a sunny and non-cloudy day in weeks. If anyone else utilizes Node-Red I would be willing to share my node for weather. I pull in the data from the Tempest via Home Home Assistant and not MQTT nor UDP directly. My intention is to get this working as much as possible before incorporating into this repo. So far we would need Latitude and Longitude added as a variable (just like elevation is already provided). I am making the assumption that the time for this container is in accurate local time (stating this in the case anyone is using an isolated machine that does not have a good time source); I think currently as long as time is within 16 mins then it will work ok. Just to note that time is important since the earth rotates 1 degree every 4 minutes, which in turn effects the received (and estimated) sunlight on the top of the weather station and therefore the Solar Radiation. |
my temptest is exposed to the sun directly. I have some data from the DB, but I need to compare the timestamps from the different sensors (working on this). I might do a live logging instead (write to file every 2mins or so), it is supposed to be sunny mid next week here. |
Thanks Glenn, I've sent you a PM on the HA forum. |
Could you compare solar elevation that this outputs to what it is in your area? I think I have this right, but just want to verify. It looks like I need to do some modifications for the southern hemisphere in the calculations for the actual estimation of Solar Radiation. I will see if I can resurrect my node-red node for this testing. I had it where every little sub calculation had it's own block, that way I could easily find and make changes. I can let you fine tune it for your location and then with other input from people in other locations I can try to make a generalization for anywhere in the world. |
@GlennGoddard for my understanding, |
Yes Zambretti is it's own sensor seperate from local current conditions. I have the current conditions effectively done, but life has gotten in the way of me finalizing and putting in the PR. So all your statements are correct. As soon as life will let me get it done, I'll get it in. There is not much left for me to do, other than make sure I correctly convert everything I have in JavaScript to Python. |
Ok, send it through when you are ready. |
Submitted PR for Fog Probability, if that works then Snow Probability will be next and then that should cover all remaining perquisites for local current conditions. I still need to fine tune Solar Insolation for other Latitudes but I need more data from other latitudes |
Fog and Snow Probability are in the Dev build (I'm still working some issues, but sensors are getting to HA). Fog has issues still, won't be able to verify Snow for a few months (myself anyways). Note to self: try to convert ALL syntax from java-script to python. |
Hey @GlennGoddard great work as always, now with the Snow and Fog Probability which just arrived in 3.1.4. |
Zambretti is a forecast based off the Zambretti Forecaster wheel. Current Conditions is an attempt to mimic the results of the 'current conditions' received from WeatherFlow but locally. I have actually had everything needed for some time now, and honestly I forgot about finishing moving it over to this integration since I have been running my own test state. I will see if I can port everything over this weekend. |
That would be amazing!! I would love to have full local conditions, right now this is only partially the case for me.. |
@badewanne1234 I'm sure there will be tweaking of the calculations once more people in different locations start using this and providing feedback. |
I can totally live with these caveats. Do you have any experience yet with the snow probability sensor? |
I will probably end up with two different sensors for Current Conditions: I 'should' be able to get (1) done this weekend. (2) won't be too hard but not 'essential' for Current Conditions, just really helpful. |
@badewanne1234 I do have some experience with the Snow Probability, and as a result the calculation was modified. It was not 'wrong' per se; I just neglected to to cap the maximum probability of snow. The way it was that if all conditions were met for snow and the temperature went into the teens (Fahrenheit) this would send a probability over 100%. The snow probability calculations came from someone else, I just put them to use with a little modification. It is not perfect, but it has proven to be more accurate then the local news weather forecasters. The more input we get from people in various locations, hopefully, the better the calculations can be. |
PR #204 submitted for local current conditions but just the Home Assistant accepted inputs. My brain is fried from too many other things, so I'm sure it is not perfect. The translations for Human Readable Current Conditions will come later once this is ironed out. |
I am busy all week, so I will not have time to look at the PR before the weekend. |
@briis any joy with looking into the PR yet? |
It is now merged and published to the dev tag. However, at my end it does not work, as there are some errors in the formula itself. I have asked @GlennGoddard to look at that. |
wind_speed was not called |
Just pushing a new dev build, which now returns current_condition. Please note it will take a little while before it is pushed as there are many values that need to have data before it works. |
That is not a bad idea. For me, Cloudy is the norm in the winter. We could pass 'exceptional', as a tell-tale; not sure how the animated weather card I use would handle it since it doesn't have any icons for exceptional. I think sunny or clear-night would be better for most. Clear Night would be the most likely at night and would indicate a reboot/issue during the day. |
OK. I will change from None to |
I need to look at fog and cloudy; they seem to be swapped with each other. When my node-red says fog, this says cloudy; when node-red says cloudy, this says fog. Everything else seems to track. I have my Node-Red output in 'readable' text, the capital is intentional in this case. I'm not sure why I am inverting the fog/cloudy....??? |
@briis @badewanne1234 current conditions is in the dev build, I have to find one issue before it reaches stable. Fog and Cloudy are swapped....it is bugging me that don't see why yet. |
Fog and Cloudy is not always reversed; it might be a metric/imperial thing...I set it up for all metric...so something might be translated to imperial. Wind Speed is probably the likely culprit but not sure yet. It might also be that the calculation are done with slightly different data since I was only performing every 3 mins vice every 3 seconds here...should not make a difference...but. I think if I make a tweak to when Fog is called to a higher number from the current of 50% then that would change it. I finished all the different translations, might need some review there. I don't have the function called yet but the work in the translation directory is done. |
Well...that is what I get for 2nd guessing myself....no fog visible at my house today...but I'm watching the news and massive fog everywhere around me.... I'm torn.... I guess I will keep this for now |
Hey @GlennGoddard have you made further progress on this? Last update was that you found an issue, need to fix before you push it to stable. I would love to have fully local conditions 💪🏻💪🏻 |
@jazzyisj |
The below are the only inputs allowed for HA weather condition. Currently this is pulled from the web API. I was toying with the idea of expanding my weather condition template to pull this locally. I understand I can't get the forecast local only; but I like as much self-hosted as possible. I think the only things I can't verify is clear-night, due to light conditions at night. Thoughts below:
‘clear-night’ can't check if fog at night since no light level to compare to (open to ideas)
‘cloudy’ can get a base line lux amount and compare and if not any other precipitation
‘fog’ same as cloudy but with window below a determined temperature
‘hail’ sensor for this
‘lightning’ sensor for this (maybe any number in last 3 hours)
‘lightning-rainy’ combine sensors for this
‘partlycloudy’ same as cloudy, with maybe a trend/derivative sensor to determine intermittent light levels
‘pouring’ above a certain rain rate
‘rainy’ sensor for this
‘snowy’ a little more complicated but maybe temperature combined with blocked UV sensor and rain sensor
‘snowy-rainy’ same as snowy but easier to assume around 0C / 32F
‘sunny’ easiest just base on UV/lux level
‘windy’ wind above a certain speed or gust above a certain speed
‘windy-variant’ not sure what this is checking
‘exceptional’ dewpoint between 50-60F, temperature between 68-77F, UV below 2.5 index, not any of the above (except sunny).
So...the actual point of this... should I make my condition template and then PR to the readme example or should we incorporate this to give another calculated sensor output? I'm leaning toward a sensor, but this would add a lot more code and wanted to run it past you before a PR without a big change with any other info. I would keep the current web api returned status as you have it and create some other named sensor. Thoughts?
The text was updated successfully, but these errors were encountered: