-
Notifications
You must be signed in to change notification settings - Fork 6
RTC‐S1 Interface Acceptance Testing Instructions
This page contains instructions for performing Acceptance Testing for the RTC-S1 interface delivery from the OPERA-ADT team. These instructions assume the user has access to the JPL FN-Artifactory, and has Docker installed on their local machine.
The image is currently hosted on JPL FN-Artifactory, which requires JPL VPN access and JPL credentials. You may also need to be added to the gov.nasa.jpl.opera.adt organization.
Once you have access, the container tarball delivery is available under general/gov/nasa/jpl/opera/adt/rtc_s1/r1/interface/dockerimg_rtc_interface_0.1.tar. Sample inputs and outputs are also available under general/gov/nasa/jpl/opera/adt/rtc_s1/r1/interface/delivery_1_interface_0.1.zip.
Download both images to a location on your local machine. This location will be referred to throughout this instructions as <RTC_DIR>
Note that the sample data is quite large, so the download from AF can take some time.
The first step in running the RTC-S1 image is to load it into Docker via the following command:
docker load -i <RTC_DIR>/dockerimg_rtc_interface_0.1.tar
This should add the Docker image to your local repository with the name opera/rtc and the tag interface_0.1.
Once the delivery_1_interface_0.1.zip file is downloaded to your local machine, unpack it to <RTC_DIR>:
cd <RTC_DIR>; delivery_1_interface_0.1.zip
This will create a delivery_1_interface_0.1 directory within <RTC_DIR> containing the following files/directories:
- peru
|_ S1B_IW_SLC__1SDV_20180504T104507_20180504T104535_010770_013AEE_919F.zip
|_ S1B_OPER_AUX_POEORB_OPOD_20180524T110543_V20180503T225942_20180505T005942.EOF
|_ dem.tif
|_ expected_output_dir/
|_ output_dir/
|_ rtc_s1.yaml
|_ scratch_dir/
In order to execute the SAS, the input file directory, runconfig, scratch and output locations will be mounted into the container instance as Docker Volumes. To help streamline this process, we recommend making the following changes within the delivery_1_interface_0.1 directory:
-
Create a directory named
input_dir, and copy the expected input files into it:mkdir -p <RTC_DIR>/delivery_1_interface_0.1/peru/input_dirmv <RTC_DIR>/delivery_1_interface_0.1/peru/S1B_IW_SLC__1SDV_20180504T104507_20180504T104535_010770_013AEE_919F.zip <RTC_DIR>/delivery_1_interface_0.1/peru/input_dir/mv <RTC_DIR>/delivery_1_interface_0.1/peru/S1B_OPER_AUX_POEORB_OPOD_20180524T110543_V20180503T225942_20180505T005942.EOF <RTC_DIR>/delivery_1_interface_0.1/peru/input_dir/mv <RTC_DIR>/delivery_1_interface_0.1/peru/dem.tif <RTC_DIR>/delivery_1_interface_0.1/peru/input_dir/ -
Create a directory named
runconfig, and copy the existing runconfig YAML file into it:mkdir -p <RTC_DIR>/delivery_1_interface_0.1/peru/runconfigcp <RTC_DIR>/delivery_1_interface_0.1/peru/rtc_s1.yaml <RTC_DIR>/delivery_1_interface_0.1/peru/runconfig/rtc_s1.yaml -
Modify the copied runconfig to point all inputs to the
input_dircreated earlier. Note that this is done relative to the container's filesystem. Theinput_file_group,dynamic_ancillary_file_groupandproduct_pathsections of the Runconfig should be modified to match the following:
input_file_group:
# Required. List of SAFE files (min=1)
safe_file_path: [input_dir/S1B_IW_SLC__1SDV_20180504T104507_20180504T104535_010770_013AEE_919F.zip] # <- update to point to input_dir
# Required. List of orbit (EOF) files (min=1)
orbit_file_path: [input_dir/S1B_OPER_AUX_POEORB_OPOD_20180524T110543_V20180503T225942_20180505T005942.EOF] # <- update to point to input_dir
# Burst to process. Empty to process all bursts
burst_id:
dynamic_ancillary_file_group:
# Digital elevation model
dem_file: input_dir/dem.tif # <- update to point to input_dir
product_path_group:
# Directory where PGE will place results
product_path: output_dir # <- update to point at output_dir
# Directory where SAS writes temporary data
scratch_path: scratch_dir
# Intermediate file name. SAS writes the output to this file.
# PGE may rename the product according to file naming convention
output_dir: output_dir
product_id: rtc_product
mosaic_bursts: False
output_format: NETCDF
-
Make sure the output and scratch directories have write permissions set
chmod ga+w output_dir/ scratch_dir/
We're now ready to execute the RTC-S1 Interface. Run the following the command to kick off execution with the test assets:
docker run --rm -u $UID:$(id -g) \
-w /home/rtc_user \
-v <RTC_DIR>/delivery_1_interface_0.1/peru/runconfig:/home/rtc_user/runconfig:ro \
-v <RTC_DIR>/delivery_1_interface_0.1/peru/input_dir:/home/rtc_user/input_dir:ro \
-v <RTC_DIR>/delivery_1_interface_0.1/peru/output_dir:/home/rtc_user/output_dir \
-v <RTC_DIR>/delivery_1_interface_0.1/peru/scratch_dir:/home/rtc_user/scratch_dir \
-i --tty opera/rtc:interface_0.1 -- runconfig/rtc_s1.yaml
Note that the interface delivery does not produce any console output until processing has completed. Execution should take roughly 75 minutes. Once execution is complete, you should see a number of directories within the <RTC_DIR>/delivery_1_interface_0.1/peru/output_dir/ directory, one for each burst ID processed. There should be 28 sub-directories, each containing an rtc_product.nc file.
Now that we've successfully executed the SAS container and generated outputs, the last step is to perform a QA check against the expected outputs.
For the current interface delivery, the comparison routine is only available as a stand-alone script to be run outside of the RTC container. The script used with this AT can be obtained here: https://github.com/opera-adt/RTC/blob/432c37cf3fde2d9258a107bcd5c3422ee506ef35/app/rtc_compare.py
The rtc_compare.py script compares a single pair of expected/output products at a time, so the following shell script can be used to automated the comparisons across all burst-based products:
#!/bin/bash
declare -a burst_ids=( "t069_147170_iw1"
"t069_147170_iw3"
"t069_147171_iw1"
"t069_147171_iw2"
"t069_147171_iw3"
"t069_147172_iw1"
"t069_147172_iw2"
"t069_147172_iw3"
"t069_147173_iw1"
"t069_147173_iw2"
"t069_147173_iw3"
"t069_147174_iw1"
"t069_147174_iw2"
"t069_147174_iw3"
"t069_147175_iw1"
"t069_147175_iw2"
"t069_147175_iw3"
"t069_147176_iw1"
"t069_147176_iw2"
"t069_147176_iw3"
"t069_147177_iw1"
"t069_147177_iw2"
"t069_147177_iw3"
"t069_147178_iw1"
"t069_147178_iw2"
"t069_147178_iw3"
"t069_147179_iw2"
"t069_147179_iw3")
expected_dir=$1
output_dir=$2
for burst_id in "${burst_ids[@]}"; do
echo "Comparing results for $burst_id"
python3 rtc_compare.py $expected_dir/$burst_id/rtc_product.nc $output_dir/$burst_id/rtc_product.nc
done
After giving the script a name, such as rtc_compare_products.sh and making it executable, you can compare all expected/actual products with the following command:
rtc_compare_products.sh <RTC_DIR>/delivery_1_interface_0.1/peru/expected_output_dir <RTC_DIR>/delivery_1_interface_0.1/peru/output_dir
The rtc_compare.py script performs a number of checks on both the image layers and metadata within each RTC NetCDF product. All tests should be marked as PASSED, or WARNING for some instances of metadata where we expect a mismatch between input file paths, time stamps etc...
An example report for a single burst product is provided below.
Comparing results for t069_147170_iw1
Checking the dataset.
PASSED. Dataset 1 of 46: //science/CSAR/RTC/grids/frequencyA/rangeBandwidth
PASSED. Dataset 2 of 46: //science/CSAR/RTC/metadata/orbit/interpMethod
PASSED. Dataset 3 of 46: //science/CSAR/RTC/metadata/processingInformation/algorithms/radiometricTerrainCorrection
PASSED. Dataset 4 of 46: //science/CSAR/RTC/grids/frequencyA/VV
PASSED. Dataset 5 of 46: //science/CSAR/RTC/grids/frequencyA/localIncidenceAngle
PASSED. Dataset 6 of 46: //science/CSAR/identification/zeroDopplerStartTime
PASSED. Dataset 7 of 46: //science/CSAR/RTC/grids/frequencyA/polarizationOrientationFlag
PASSED. Dataset 8 of 46: //science/CSAR/RTC/grids/frequencyA/xCoordinateSpacing
PASSED. Dataset 9 of 46: //science/CSAR/RTC/metadata/orbit/time
PASSED. Dataset 10 of 46: //science/CSAR/RTC/grids/frequencyA/faradayRotationFlag
PASSED. Dataset 11 of 46: //science/CSAR/RTC/grids/frequencyA/radiometricTerrainCorrectionFlag
WARNING. Dataset 12 of 46: //science/CSAR/RTC/metadata/processingInformation/inputs/orbitFiles
Ignoring the comparison result based on the exclusion list provided.
PASSED. Dataset 13 of 46: //science/CSAR/RTC/grids/frequencyA/VH
PASSED. Dataset 14 of 46: //science/CSAR/identification/processingType
PASSED. Dataset 15 of 46: //science/CSAR/RTC/grids/frequencyA/xCoordinates
PASSED. Dataset 16 of 46: //science/CSAR/identification/missionId
PASSED. Dataset 17 of 46: //science/CSAR/identification/productType
PASSED. Dataset 18 of 46: //science/CSAR/RTC/metadata/processingInformation/inputs/l1SlcGranules
PASSED. Dataset 19 of 46: //science/CSAR/RTC/metadata/orbit/orbitType
WARNING. Dataset 20 of 46: //science/CSAR/RTC/metadata/processingInformation/algorithms/ISCEVersion
Ignoring the comparison result based on the exclusion list provided.
PASSED. Dataset 21 of 46: //science/CSAR/RTC/grids/frequencyA/zeroDopplerTimeSpacing
PASSED. Dataset 22 of 46: //science/CSAR/identification/isGeocoded
PASSED. Dataset 23 of 46: //science/CSAR/identification/lookDirection
PASSED. Dataset 24 of 46: //science/CSAR/identification/zeroDopplerEndTime
WARNING. Dataset 25 of 46: //science/CSAR/RTC/metadata/processingInformation/inputs/auxcalFiles
Ignoring the comparison result based on the exclusion list provided.
PASSED. Dataset 26 of 46: //science/CSAR/RTC/grids/frequencyA/projection
PASSED. Dataset 27 of 46: //science/CSAR/identification/trackNumber
WARNING. Dataset 28 of 46: //science/CSAR/RTC/metadata/processingInformation/inputs/configFiles
Ignoring the comparison result based on the exclusion list provided.
PASSED. Dataset 29 of 46: //science/CSAR/RTC/metadata/processingInformation/algorithms/geocoding
PASSED. Dataset 30 of 46: //science/CSAR/identification/absoluteOrbitNumber
PASSED. Dataset 31 of 46: //science/CSAR/identification/listOfFrequencies
PASSED. Dataset 32 of 46: //science/CSAR/RTC/metadata/orbit/velocity
PASSED. Dataset 33 of 46: //science/CSAR/identification/orbitPassDirection
PASSED. Dataset 34 of 46: //science/CSAR/RTC/grids/frequencyA/slantRangeSpacing
PASSED. Dataset 35 of 46: //science/CSAR/identification/diagnosticModeFlag
PASSED. Dataset 36 of 46: //science/CSAR/RTC/metadata/orbit/position
PASSED. Dataset 37 of 46: //science/CSAR/identification/isUrgentObservation
PASSED. Dataset 38 of 46: //science/CSAR/RTC/grids/frequencyA/numberOfLooks
WARNING. Dataset 39 of 46: //science/CSAR/RTC/metadata/processingInformation/inputs/demFiles
Ignoring the comparison result based on the exclusion list provided.
PASSED. Dataset 40 of 46: //science/CSAR/RTC/grids/frequencyA/listOfPolarizations
PASSED. Dataset 41 of 46: //science/CSAR/RTC/grids/frequencyA/incidenceAngle
PASSED. Dataset 42 of 46: //science/CSAR/RTC/grids/frequencyA/yCoordinateSpacing
PASSED. Dataset 43 of 46: //science/CSAR/RTC/grids/frequencyA/areaNormalizationFactor
PASSED. Dataset 44 of 46: //science/CSAR/RTC/grids/frequencyA/yCoordinates
PASSED. Dataset 45 of 46: //science/CSAR/RTC/metadata/processingInformation/algorithms/demInterpolation
PASSED. Dataset 46 of 46: //science/CSAR/RTC/grids/frequencyA/centerFrequency
Checking the attributes.
Test summary:
PASSED: Same dataset structure confirmed.
PASSED: Same attributes structure confirmed.
PASSED: The datasets of the two HDF files are the same within the tolerance.
Relative tolerance = 0.0001, Absolute tolerance = 1e-05
PASSED: The attributes of the two HDF files are the same within the tolerance
Relative tolerance = 0.0001, Absolute tolerance = 1e-05
For the Acceptance Test to pass, the test summary across all burst products should display PASSED for each catagory.