Row removal different in scorecard vs. batch plots

[bmatilla]

Hi there,

I am currently working to generate verification scorecards and make corresponding plots for the various metrics to be able to quantify the differences from the scorecard using METviewer. However, when event equalization is enabled for both, I've noticed that the number of rows omitted between metrics for the scorecard and metrics for the batch plots can be different, substantially in some cases, despite sourcing the data from the same database. This will lead to discrepancies in the batch plotting results where the scorecard would suggest one model is performing better than another, but the opposite shows up in the series plots. Has this been an issue that has raised in the past?

[jprestop]

Hi @bmatilla. Thank you for your question. I am tagging @TatianaBurek, our METviewer expert, who should be able to help you with your question.

[TatianaBurek]

@bmatilla we did not see any differences between the results that were produced by batch engine and the scorecard.
Could you please send us your config files for both modules so we could provide you more info about this case.

[bmatilla]

Hi @TatianaBurek, I've uploaded the two .xml config files I am using through the anonymous FTP server. I can also look to upload data as well, but there are numerous output files that span the data time frame that is encompassed in both of those files and wanted to notify you in advance prior to just uploading a substantial amount of files.

[TatianaBurek]

@bmatilla I had a wrong assumption that you are using NCAR instance of METviewer.
Which METviewer do you use? Do you use Python or Rscript implementation?
I don't know what do you mean by 'the anonymous FTP server' and where I could find XML files.

[bmatilla]

My apologies as I should have been clearer from the beginning. I am currently using METviewer v5.0.1 (along with METcalcpy 2.0.2, METplotpy 2.0.1, and METdataio 2.0.2), and am using the Python implementation. As for the example XML config files, I've uploaded them through the FTP as per the instructions here; they should be located in incoming/irap/met_help/matilla_data.

[TatianaBurek]

@bmatilla I compared the configuration in your XML files.
To simplify the comparison I focused on fcst_var 'REFC_ENS_NMEP_geSFP99_NBRHD25_GAUSSIAN1' and stat 'NBR_CSI' in batch and scorecard configs. Both XMLs define the same models but I see the difference in the configuration of other fields:
batch

• fcst_init_beg: 2018-04-29 19:00:00 - 2018-06-02 03:00:00

scorecard:

• fcst_valid_beg: 2018-04-29 19:00:00 - 2018-06-02 03:00:00
• obs_thresh: >=38.97
• interp_pnts: 1

Each process gets different set of data from the database and of course the event equalization ends up with different number of rows. To really compare batch and scorecard results you need to create a new XML for batch that would contain exactly the same configuration as the scorecard.

[bmatilla]

Thanks for looking into this @TatianaBurek. I see the difference more clearly now. Let me give this a try on my end with the necessary changes to the XML and I will report back if I still see differences.