Remote Data Feeds - When to sync?

[rburghol]

When syncing a remote timeseries data feed, it is best to avoid redundant updates.

• Is the date populated with data? (fully populated? i.e. 24 values, or more?
  • script to get a count of weather data for a give date and station ID
  • if count < 23, or if max date has hour less than 23, refresh
• Speed... hobo rest for multiple days is glacial.
  • but hobo export format is convoluted with huge column names that are specific to the sensor name and ID
• Get latest date of data
  • this could be a sql query for the batch of stations, or, just select few
  • when a node has been offline for XX time period, notify, and decrease the frequency of checking back on it,
    • store as a property om_setprop.php query_type entity_type featureid varkey propname propvalue propcode [extras as urlenc key1=val1&key2=val2...]
    • retrieve with get_propval.php will work for this (btw: om/src/get_propval.php can get propvalue or propcode via om_getval.php entity_type featureid propname [target=propvalue]