Errors in the RAP_OPS ingest

[ian-noaa]

Describe the bug

The ingest is failing to ingest RAP_OPS properly. It's having issues with the Vegetation Type field in the GRIB file. However, the job is marking itself as having completed successfully.

To Reproduce

Steps to reproduce the behavior:

1. Run a RAP_OPS ingest job
2. Observe an error message like the one included in the "additional context" section below
3. Note that the job completes "Successfully"

Expected behavior

The ingest should appropriately handle the Vegetation Type field. If not, it should error instead of succeeding.

Additional context

Sample error message:

2023-12-19T20:21:24+0000 [ERROR] <VxIngestManager-14> (vxingest.grib2_to_cb.grib_builder_parent): GribModelBuilderV01: Exception with builder build_document: file_name: /public/data/grids/rap/iso_130/grib2/2335319000021, exception None
Traceback (most recent call last):
  File "/app/vxingest/grib2_to_cb/grib_builder_parent.py", line 711, in build_document
    "Vegetation Type": ds_surface_vegetation_type.variables[
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/xarray/core/utils.py", line 455, in __getitem__
    return self.mapping[key]
           ~~~~~~~~~~~~^^^^^
KeyError: None
2023-12-19T20:21:24+0000 [INFO] <VxIngestManager-14> (vxingest.builder_common.ingest_manager): VxIngestManager-1: write_document_to_files output /opt/data/outdir/grib2_to_cb/output/20231219201012:
2023-12-19T20:21:24+0000 [INFO] <VxIngestManager-14> (vxingest.builder_common.ingest_manager): VxIngestManager-1: write_document_to_files: would write documents but DOCUMENT_MAP IS EMPTY
2023-12-19T20:21:24+0000 [INFO] <VxIngestManager-14> (vxingest.grib2_to_cb.vx_ingest_manager): IngestManager.process_element: elapsed time: 2
2023-12-19T20:21:24+0000 [INFO] <VxIngestManager-14> (vxingest.builder_common.ingest_manager): VxIngestManager-1: IngestManager - finished processing /public/data/grids/rap/iso_130/grib2/2335319000021

Two tar archives on ascend-test2 that can provide additional context:

• old ingest - /data-ingest/data/xfer/archive/job_v01_metar_grib2_model_rap__ops__130_a781650d0cff_1703009854.tar.gz
• ingest running from main - /data-ingest/data/xfer/archive/job_v01_metar_grib2_model_rap__ops__130_ab53158cbaad_1703016612.tar.gz

[ian-noaa]

@JeffHamiltonNOAA - do you know if the RAP_OPS GRIB files are expected to have the Vegetation Type table?

[randytpierce]

For this issue I checked for the existence of the dataset in the cfgrib ds and also the variable aAND if the variable had any data. For now if there is no data in the variable the program will fill the vegetation_type (or any other such variable) with None and the data document will get a null value.

In the future we may want to change this to make the data variable simply "MISSING"

[randytpierce]

This is fixed in branch "origin/278-errors-in-the-rap_ops-ingest"