Add params to logs, update readme (#80)

* Add params to logs, update readme

* Automatically update integration test validation results

* Apply suggestions from code review

Co-authored-by: Vanessa Newey <[email protected]>

---------

Co-authored-by: robbibt <[email protected]>
Co-authored-by: Vanessa Newey <[email protected]>

README.md

@@ -9,7 +9,7 @@
 
 **License:** The code in this repository is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). Digital Earth Australia data is licensed under the [Creative Commons by Attribution 4.0 license](https://creativecommons.org/licenses/by/4.0/).
 
-**Contact:** For assistance with any of the Python code or Jupyter Notebooks in this repository, please post a [Github issue](https://github.com/GeoscienceAustralia/dea-intertidal/issues). For questions or more information about this workflow, email Robbi.BishopTaylor@ga.gov.au.
+**Contact:** For assistance with any of the Python code or Jupyter Notebooks in this repository, please post a [Github issue](https://github.com/GeoscienceAustralia/dea-intertidal/issues). For questions or more information about DEA Intertidal, email earth.observation@ga.gov.au.
 
 **To cite:** 
 > Bishop-Taylor, R., Sagar, S., Lymburner, L., Beaman, R.L., 2019. Between the tides: modelling the elevation of Australia's exposed intertidal zone at continental scale. Estuarine, Coastal and Shelf Science. https://doi.org/10.1016/j.ecss.2019.03.006
@@ -19,3 +19,20 @@
 > Sagar, S., Roberts, D., Bala, B., Lymburner, L., 2017. Extracting the intertidal extent and topography of the Australian coastline from a 28 year time series of Landsat observations. Remote Sensing of Environment 195, 153-169. https://doi.org/10.1016/j.rse.2017.04.009
 
 ---
+
+The DEA Intertidal product suite maps the changing extent, elevation and topography of Australia's exposed intertidal zone. It is the next generation of DEA's intertidal products that have been used across government and industry to help better characterise and understand this complex zone that defines the interface between land and sea.
+
+Incorporating both Sentinel-2 and Landsat data, the product suite adds a temporal component to the elevation product for the intertidal zone, enabling users to better monitor and understand some of the most dynamic regions of Australia's coastlines. With an improved tidal modelling capability, the product suite has been expanded to include a continental scale mapping of intertidal exposure over time, enabling scientists and managers to integrate the data into ecological and migratory species applications and modelling.  
+
+## Repository structure
+
+The DEA Intertidal Github repository contains the following important sections:
+* `intertidal`: The DEA Intertidal Python package, containing modules required for loading data, tide modelling, intertidal elevation, and exposure calculations
+* `notebooks`: Jupyter Notebooks providing workflows for generating key DEA Intertidal outputs. Importantly:
+  * `notebooks/Intertidal_CLI.ipynb`: For running the entire DEA Intertidal workflow via the Command Line Interface
+  * `notebooks/Intertidal_workflow.ipynb`: For running the entire DEA Intertidal workflow via interactive notebook cells
+  * `notebooks/Intertidal_elevation.ipynb`: For customising and running the DEA Intertidal Elevation portion of the workflow
+  * `notebooks/experimental/Intertidal_elevation_pc.ipynb`: For running DEA Intertidal Elevation on global satellite data loaded from Microsoft Planetary Computer using STAC metadata
+* `data`: Contains required `raw` input data files and output `interim` and `processed` outputs
+* `metadata`: Open Data Cube (ODC) metadata required for indexing DEA Intertidal into an ODC datacube
+* `tests`: Unit and integration tests, including automatically generated validation test results

intertidal/elevation.py

@@ -1168,8 +1168,12 @@ def intertidal_cli(
 ):
     # Create a unique run ID for analysis based on input params and use
     # for logs
+    input_params = locals()
     run_id = f"[{output_version}] [{label_date}] [{study_area}]"
     log = configure_logging(run_id)
+
+    # Record params in logs
+    log.info(f"{run_id}: Using parameters {input_params}")
 
     # Configure S3
     configure_s3_access(cloud_defaults=True, aws_unsigned=aws_unsigned)

tests/README.md

@@ -10,7 +10,7 @@ Integration tests
 
 This directory contains tests that are run to verify that DEA Intertidal code runs correctly. The ``test_intertidal.py`` file runs a small-scale full workflow analysis over an intertidal flat in the Gulf of Carpentaria using the DEA Intertidal [Command Line Interface (CLI) tools](../notebooks/Intertidal_CLI.ipynb), and compares these results against a LiDAR validation DEM to produce some simple accuracy metrics.
 
-The latest integration test completed at **2024-03-25 17:38**. Compared to the previous run, it had an:
+The latest integration test completed at **2024-03-26 11:54**. Compared to the previous run, it had an:
 - RMSE accuracy of **0.14 m ( :heavy_minus_sign: no change)**
 - MAE accuracy of **0.12 m ( :heavy_minus_sign: no change)**
 - Bias of **0.12 m ( :heavy_minus_sign: no change)**

tests/validation.csv

@@ -53,3 +53,4 @@ time,Correlation,RMSE,MAE,R-squared,Bias,Regression slope
 2024-03-25 05:50:44.245009+00:00,0.975,0.141,0.121,0.95,0.116,1.11
 2024-03-25 06:08:57.564906+00:00,0.975,0.141,0.121,0.95,0.116,1.11
 2024-03-25 06:38:58.505117+00:00,0.975,0.141,0.121,0.95,0.116,1.11
+2024-03-26 00:54:18.363134+00:00,0.975,0.141,0.121,0.95,0.116,1.11