Ingesting Stage IV Precipitation Observations

[scatt11]

Hello MET team,

I am working to ingest Stage IV precipitation data and running into a few roadblocks (file attached below). I am simply using a raw file for 1-hr precip accumulation acquired from: https://data.eol.ucar.edu/cgi-bin/codiac/fgr_form/id=21.093

st4_conus.2021060700.01h.grb2.txt

First step always, plot with plot_data_plane to ensure the the new dataset is read as expected (a not helpful error occurred). In troubleshooting I quickly converted it to a netCDF with wgrib2 and attempted with the new .nc file. The error hinted at the projection being the issue because MET can only process Latitude/Longitude files where the latitudes are evenly spaced. This gave the idea to try regrid, and regrid gets the same unhelpful error when trying to read the grb2 file. Unhelpful error looks like the following:

DEBUG 1: Reading data file: /met_data/surface_data/precip_gridded/1hr/st4_conus.2021060700.01h.grb2 DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Gb2". *** stack smashing detected ***: <unknown> terminated
One example of what prompts this error
../../src/tools/other/plot_data_plane/plot_data_plane \ /met_data/surface_data/precip_gridded/1hr/st4_conus.2021060700.01h.grb2 \ ./plot_precip.ps \ 'name="APCP";'\ -title "GRIB2 Stage IV 1-hr precip" \ -v 10

I have attempted multiple variations of the name and level arguments with the same error result which leads me to believe this is an issue reading the file.

Using Met V10.0.0

Any assistance/ guidance with reading in Stage IV precip data would be greatly appreciated.

[jprestop]

Hi @scatt11.

I see that you are having trouble running Plot-Data-Plane on some Stage IV precipitation data. Thanks for passing along your input file.

Based on this part of your post:

One example of what prompts this error

../../src/tools/other/plot_data_plane/plot_data_plane \ /met_data/surface_data/precip_gridded/1hr/st4_conus.2021060700.01h.grb2 \ ./plot_precip.ps \ 'name="APCP";'\ -title "GRIB2 Stage IV 1-hr precip" \ -v 10

I ran the following using METv10.0.0:

 /usr/local/met-10.0.0/bin/plot_data_plane st4_conus.2021060700.01h.grb2 st4_conus.2021060700.01h.ps 'name="APCP";' -title "GRIB2 Stage IV 1-hr precip"

which produced the following:

DEBUG 1: Opening data file: st4_conus.2021060700.01h.grb2
ERROR  : 
ERROR  : VarInfo::set_level_info_grib() - either level or GRIB_lvl_typ, GRIB_lvl_val1 and GRIB_lvl_val2 (if necessary) must be specified in field information
ERROR  : 

indicating that the level must be set.

I modified my command to run the following to include the level information:

/usr/local/met-10.0.0/bin/plot_data_plane st4_conus.2021060700.01h.grb2 st4_conus.2021060700.01h.ps 'name="APCP"; level="A01";' -title "GRIB2 Stage IV 1-hr precip"

which produced a successful plot:

DEBUG 1: Opening data file: st4_conus.2021060700.01h.grb2
DEBUG 1: Creating postscript file: st4_conus.2021060700.01h.ps

Please give that a try and let us know how it goes. I hope it helps!

[scatt11]

Thanks @jprestop for confirming this is not an issue with the data. I have made the advised changes and still no success which leads me to believe this has to do with my own configuration of MET. Looking back at my install file it seems I did not compile with GRIB support. You can see I commented out this section in my install config file.

export TEST_BASE=/usr/meteo/
export COMPILER=gnu_7.5.0
export MET_SUBDIR=${TEST_BASE}
export MET_TARBALL=met-10.0.0.20210510.tar.gz
export USE_MODULES=FALSE
export PYTHON_LOC=/usr/bin
export MET_PYTHON=${PYTHON_LOC}/python3
export MET_PYTHON_LD=-L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu\ -L/usr/lib\ -lpython3.6m\ -lpthread\ -ldl\ -lutil\ -lm\ -Xlinker\ -export-dynamic\ -Wl,-O1\ -Wl,-Bsymbolic-functions\

export MET_PYTHON_CC=-I/usr/include/python3.6m\

export SET_D64BIT=TRUE
export CFLAGS="-D_64BIT_"
export CXXFLAGS="-Wall -g"

export COMPILE_GSL=0
COMPILE_HDF=0

#If you've already compiled these and don't need to compile them again, set the following;
#Otherwise, if you do need to install these libraries, comment out the variables below
export EXTERNAL_LIBS=/usr/meteo/MET_install/external_lib
#export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib
#export MET_GRIB2CINC=${EXTERNAL_LIBS}/include
#export GRIB2CLIB_NAME=-BUFRLIB_v11-3-0
#export MET_BUFRLIB=${EXTERNAL_LIBS}/lib
#export BUFRLIB_NAME=-lbufr
#export MET_NETCDF=${EXTERNAL_LIBS}

Is this likely the root cause? I did a quick attempt to uncomment these lines and recompile MET. This was unsuccessful, but regrettably I have not done my homework on correctly linking these environment variables. If this is likely the root cause I will dig deeper on recompiling.

[jprestop]

Hi @scatt11. Ah! Good debugging!

Is this likely the root cause?

Yes. Without GRIB enabled, this functionality won't work.

Since the installation and linking issues are a new topic, I'd like to mark this discussion as being answered and "lock it" to prevent future posts. Then, please feel free to create a new discussions for dealing with the installation and linking issues. But I wanted to give you an opportunity to comment on it before doing so.

[scatt11]

Thank you, I will make a new discussion if needed for installation issues. My apologies for any confusion here. As I have been using GRIB abbreviations to ingest madis2nc output, it totally escaped me that I had not enabled GRIB.

[jprestop]

Great! Thanks. No apology is necessary. I'm glad you realized GRIB was not enabled.