Correct time_summary settings for madis2nc config file #1232
-
Hello, After some trial and error, it's still unclear to me how to set the time_summary settings properly in the madis2nc config file, Madis2NcConfig_default. I would like to compute the max temp over a 24-h window from 0600-0600 UTC each day. I keep getting multiple time windows apparently related to the value of 'step', even while doing simple tests with a much shorter beg..end period that doesn't doesn't overlap two days. I have the flag, grib_code and type parameters set properly. Any help would be much appreciated. This is the file's contents from the user manual: time_summary = { Thanks. |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 10 replies
-
Hi John, And thank you for your question. It sounds like there may be an improper setting in your Looking at the MET user's guide for the dictionary in question, we find:
Additionally, the guide provides the following example (which is the example you copied):
Without seeing your configuration file, your description sounds like the step parameter may be set small enough that it does something similar to the example; centering at the top of the step and so summarizing both the previous day's and day of focus' values. So if I took a first crack at the step and width settings for what you're looking for (max temp over 24 hour period), I'd say you need to adjust the width to be a same 24 hour period, or 86400 (both width = {beg = 0; end = 86400} which says to center the width at the 12 hour mark with 12 hours on each side, eliminating any issue of the width being centered at the step (which would grab the previous day's last 12 hours and your focus day's first 12 hours). Hopefully this helps you find a successful path through your problem! If it has, please feel free to mark the answer and I'll close the discussion. If you're still running into issues, please reply and upload your configuration file, along with the command you're using to run madis2nc. Thanks, -John O. |
Beta Was this translation helpful? Give feedback.
-
Hi John, Thanks for your thoughts. Previously, I had set width to 0..86400, plus other shorter windows for testing efficiency, in order to scope out the possible responses. I think a main concern is what value of step should be used when there should not be any stepping happening? That is, I only want one max/min window - the entire width. I've tried 0, 1, 86400 and a very large number for step. These result in either multiple windows (as expected), a hang (really, I think it was churning away still) and an empty output file. |
Beta Was this translation helpful? Give feedback.
-
Hi again John, I should also state that I'd like to have the time window be the 24-hour window after the valid/file time, so, e.g., 0600 UTC 1 September to 0600 UTC 2 September 2021 would be a file with the date of 20210901_0600. |
Beta Was this translation helpful? Give feedback.
-
John, Would you be able to upload your config file for MADIS2NC? It would probably help illustrate the issues you're running into. From the previous discussion, it sounds like you could accomplish the 0600 to 0600 summation by using
I think by setting your end time to 1 minute prior to the begin time, it avoids an issue where MET thinks your begin and end time is the same, and hence no summation data. The step and width are actually the same (24hours), but you need to tell the width parameter where the center of your summary is (or from a different perspective, where the edges are). The above write up informs MET to start at 0600 (when it should start reading in data) and proceed 24 hours to get to the outer limit (the next 0600) for the summary. The step parameter will then move the next summary to that next 0600 where the previous summary ended, and repeat the process. Hopefully that's not something that's been tried yet and yields some success. If not, I'll see if some guidance can come from having your configuration file in hand. Thanks, John O. |
Beta Was this translation helpful? Give feedback.
-
Hi @JohnMHenderson, I wanted to follow up on this issue. I see you have a question about how the time_summary settings translate into the valid times that appear in the output. And I do see how these settings can be confusing! Let's work through another example to see if that helps clarify. Let's compute 1-hour time summaries every 3 hours during the day. So we want time summary output for hours 00, 03, 06, 09, 12, 15, 18, and 21. That's specified in the time_summary config section like this:
We should see time_summary output spaced 3 hours apart in the output. The next step is defining time windows relative to those output times listed above. And here's the confusing part. You have 2 choices:
By convention, this gives you a "symmetric" time summary, i.e. +/- 30 minutes. So the time_summary value for 12Z will contain all observations from 11:30 to 12:30. And 18Z contains all observations from 17:30 to 18:30.
This gives you data for the hour before the timestamp. It defines a retention time window as [ output time + beg, output_time + end ]. So the value reported for 12Z summarizes obs from [12Z - 1 hour, 12Z + 0 hours], which is 11 to 12Z.
This gives you data for the hour after the timestamp. So the value at 12Z summaries obs from 12 to 13Z. So beg/end/step define the valid times that are written to the output... and width defines the time windows to be used relative to those valid times. While you technically could run with neither beg or end set to 0, I doubt that's what you really want. If I'm reading the last message right, I think you want the min/max value to be computed for the previous 24 hours. If that's correct, I'd try using:
|
Beta Was this translation helpful? Give feedback.
-
@JohnMHenderson I did some additional testing and have some useful info for you. The short answer is this. Please try updating your Point-Stat config file like this:
Let me explain what's happening here. Within Point-Stat and Ensemble-Stat, when verifying against point observations, one important step is making sure that the point observations exist at or within the expected vertical level. Typically that's a pressure level or height level. Just depends on what forecast data you're trying to verify. However, there is an exception to this rule. If verifying against observations at the surface, we do not need to check the vertical level. The code skips the vertical level validation when verifying against observations with a "surface" message type. In earlier versions of MET, those surface message types were hard-coded to something like "ADPSFC" or "SFCSHP". However in more recent versions, the surface message type are defined by the "SURFACE" entry of the message_type_group_map (as seen above). By putting those derived time summary message type strings in that list, it should enable Point-Stat to find matched pairs. Please give that a try and confirm it produces the desired result. If so, I'd recommend that we enhance the processing of the message_type_group_map entries to support wildcards or regular expressions. That way we could use "ADPSFC.*" without needing to explicitly list all time summary types. |
Beta Was this translation helpful? Give feedback.
Hi @JohnMHenderson, I wanted to follow up on this issue. I see you have a question about how the time_summary settings translate into the valid times that appear in the output. And I do see how these settings can be confusing!
Let's work through another example to see if that helps clarify. Let's compute 1-hour time summaries every 3 hours during the day.
So we want time summary output for hours 00, 03, 06, 09, 12, 15, 18, and 21. That's specified in the time_summary config section like this:
We should see time_summary output spaced 3 hours apar…