bufr2ioda: ATMS DB and ERAS data

[xyzemc]

Found the bufr2ioda doesn't work correctly when converting ATMS DBnet and ERAS bufr data to ioda format. The ObsValue in converted ioda file are all missing value. Both RAP and GDAS dumped bufr have the same issue. I will work closely with Emily Liu to fix it.

RAP Bufr to IODA

[xyzemc]

Emily has found the bug and fixed it. This PR NOAA-EMC/bufr-query#24 described the bug in both GSI and bufr2ioda.
Although this bug fix can convert the atms DBNet data from gdas bufr to ioda, but it still has problem for rap dumped bufr data. Here is the example of atms_n20 DBNet bufr data converted to ioda with the bufr2ioda bug fix.

Only half of the scan line converted. This is still under debugging.

[xyzemc]

The further improvement for converting RAP ATMS BUFR data has been applied. And the bufr2ioda code has been under review for merging this bug fix. As it described in PR NOAA-EMC/bufr-query#24

• FFT calculation requires the calculation of scanline from observation time. To get accurate scanline numbers, the observation time precision needs to be increased from seconds to milliseconds
• Also, for scanline calculation, the observation time used for the scanline calculation should be the observation time at fov number 1 (fovn = 1)
• The code needs to ensure that no field of view footprints (for) are missing in one scan. If numbers are missing in one scan, this needs to be identified, and the observation time at fov=1 needs to be calculated.

RAP hourly dumped bufr data with short data window has more incomplete scanline. Compare the following two plots, the data count of ATMS DBNet is 65991, but the converted ramapped IODA data count is 52461. This means the remapping code (FFT) will skip the incomplete scanline and put it as missing value.

Original ATMS DBNet BUFR data :

IODA DBNet data:

[xyzemc]

In addition of individual test for each ATMS bufr data, I also tested combine the normal feed, DBNet, and ERAs bufr data together by an python code before doing remapping as it does in GSI, and have proved the bug fix in bufr_query works correctly for the combined bufr too. Here is one example:

The combined BUFR ATMS_N20 data:

After remapping , the IODA data is here:

[xyzemc]

This issue has been resolved.