You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We'll need to create and run a back-filling script for two related issues. It would make sense to do these at the same time.
We need to standardize the database and ingest to consistently utilize metric units throughout.
We also need to make sure the wind fields store scalar (speed/direction) & vector (u/v) components in obs & model documents.
Potentially, we could update the Wind field to something like below. That way we could get the Wind object and be able to consistently ask for scalar & vector variables. Whether this is a good idea heavily depends on if it's a good idea in Couchbase.
I re-ran the backfill of METAR obs over the weekend to re-calculate the RH using DegF instead of DegC. The obs should all be good now, to my estimation. The backfill is single threaded and it took about 6 hours. That means that if we want to backfill the model data that we have - like to change the units or to calculate WindU and WindV this single threaded technique would take 2(models) * 48 (fcstLens) * 6(hours) => 576 hours which I think is probably a no go. So the model backfill will need to be either multi-threaded or broken up into a bunch of different containers that each process a subset of the data. I lean to the later myself. That would be easy enough. We just need to figure out where to run it. Probably on rancher.
After thinking about this over the break, I would really prefer not to store the wind in a different structure than the rest of the variables, as it will make our templating way more difficult.
We'll need to create and run a back-filling script for two related issues. It would make sense to do these at the same time.
Potentially, we could update the
Wind
field to something like below. That way we could get theWind
object and be able to consistently ask for scalar & vector variables. Whether this is a good idea heavily depends on if it's a good idea in Couchbase.The text was updated successfully, but these errors were encountered: