Metric latest value is always 0

[royjs]

I'm using the check-azurerm-monitor-metric.rb check with Azure Postgres metrics (cpu_percent, memory_percent, storage_percent)
The issue I'm having is that metrics are always a few minutes behind so the latest value is almost always 0.0.

My first thought was to select a different timespan to use a start time from 2 or 3 minutes ago but it doesn't seem to be possible currently with this check. I did test it by changing the code directly and it worked.

Any ideas on how I could get around this issue?
Should we add a new option for the timespan start time?

[majormoses]

@tombuildsstuff any ideas on either how to select all but the most recent data point or why its always returning 0 in the first place? It's been a while since I touched azure.

[majormoses]

After taking a closer look at the code it seems that any data point that matches the threshold will trigger it: https://github.com/sensu-plugins/sensu-plugins-azurerm/blob/3.0.0/bin/check-azurerm-monitor-metric.rb#L187-L197 perhaps we should use https://github.com/sensu-plugins/sensu-plugins-azurerm/blob/3.0.0/bin/check-azurerm-monitor-metric.rb#L130-L134 as more than which metric data points we want to pull from azure and use it for also what the alert conditions are. I think this makes sense because otherwise as its currently written (even ignoring the last metric data point of 0) it would mean that once an alert is triggered it will meet the failure criteria for config[:lookback_period] + interval.

[majormoses]

To help further clarify based on a slack conversation

perhaps we should use https://github.com/sensu-plugins/sensu-plugins-azurerm/blob/3.0.0/bin/check-azurerm-monitor-metric.rb#L130-L134 as more than which metric data points we want to pull from azure and use it for also what the alert conditions are.

Currently we grab a list of list of metric points from azure, we can fetch various aggregations of those values. These are pre-aggragted values. For example if say we grab a 2 minute window and internally azure keeps track of the data points in say 10 second intervals but only reports them in say 1 minute intervals. This is still an array of metric points passed back in an array. Rather than looping through all the results and inspecting each one perhaps what makes more sense is to use another form of aggregation which may or may not align with what we pull from azure, one option is to re-use the same config argument to set them the same but I can think of cases where you might want to report on say max usage from an azure perspective but then alert based on an average in a different time frame.