Skip to content

Commit

Permalink
Merge branch 'support/lisf-557ww-7.5' into support/lisf-557ww-7.6
Browse files Browse the repository at this point in the history
  • Loading branch information
jvgeiger committed Nov 8, 2024
2 parents 3d4b0c6 + fab34ed commit 5bc25f8
Show file tree
Hide file tree
Showing 35 changed files with 1,870 additions and 254 deletions.
4 changes: 2 additions & 2 deletions docs/LDT_users_guide/LDT_users_guide.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Land Data Toolkit (LDT): LDT {lisfrevision} Users Guide
:revnumber: 3.0
:revdate: 10 Jan 2024
:revnumber: 2.6
:revdate: 1 Nov 2024
:doctype: book
:sectnums:
:toc:
Expand Down
1 change: 1 addition & 0 deletions docs/LDT_users_guide/revision_table.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
|====
| Revision | Summary of Changes | Date

| 2.6 | LISF 557WW 7.5.19 release | Nov 1, 2024
| 2.5 | LISF 557WW 7.5.18 release | Aug 27, 2024
| 3.0 | LISF Public 7.5.0 release | Jan 10, 2024
| 2.4 | LISF Public 7.4.3 release | Oct 03, 2023
Expand Down
4 changes: 2 additions & 2 deletions ldt/configs/ldt.config.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3463,8 +3463,8 @@ for ensemble restart generation. Options are:
|Value |Description

|none | Do not apply random sampling strategy
|random sampling | Use random sampling strategy
|unperturbed sampling | Only use the non-perturbed member
|"`random sampling`" | Use random sampling strategy
|"`unperturbed sampling`" | Only use the non-perturbed member
|===

.Example _ldt.config_ entry
Expand Down
89 changes: 49 additions & 40 deletions lis/utils/usaf/s2s/README_GHI-S2S_LIS7.5
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Subseasonal-to-Seasonal (S2S) End-to-End System (E2ES) Master Doc

NASA/GSFC GHI-S2S Team
Last Modified 24 Aug 2023
Last Modified 24 Oct 2024

TABLE OF CONTENTS
_________________
Expand Down Expand Up @@ -56,26 +56,27 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms
grib configuration settings), and netCDF4 with compression.

* For running on Discover, you will load the following module:
LISF/env/discover/lisf_7.5_intel_2021.4.0_s2s
LISF/env/discover/lisf_7.5_intel_2023.2.1_s2s

* Python 3.9, with the following extra libraries installed:
* Python 3.11, with the following extra libraries installed:
NCCS HPC
- xarray 2022.10.0 2022.11.0
- dask 2022.10.0 2022.10.2
- xesmf 0.6.3 0.6.3
- cfgrib 0.9.10.2 0.9.10.2
- dateutil 2.8.2 2.8.2
- pandas 1.5.1 1.5.1
- numpy 1.23.4 1.23.4
- PyYAML 6.0 6.0
- GDAL 3.5.2 3.5.2
- OSGEO 3.5.2 3.5.2
- NetCDF4 1.6.1 1.6.1
- Cartopy 0.21.0 0.21.0
- Matplotlib 3.6.1 3.6.2

** Note: Future tests and any code updates will be
made when we upgrade to Python 3.10.
- xarray 2024.6.0 2024.6.0
- dask 2024.7.0 2024.6.2
- xesmf 0.8.6 0.8.5
- cfgrib 0.9.13.0 0.9.14.0
- dateutil 2.9.0 2.9.0
- pandas 2.2.2 2.2.2
- numpy 1.26.4 1.26.4
- PyYAML 6.0.1 6.0.01
- GDAL 3.8.4 3.8.1
- OSGEO 3.8.4 3.8.1
- NetCDF4 1.7.1 1.6.5
- Cartopy 0.23.0 0.23.0
- Matplotlib 3.9.1 3.8.4

** Note: Updates to the above libraries and modules
could get updated as frequent as every 3- to
6-months, depending on the system.


2.2 SETTING UP THE EXPERIMENT DIRECTORY
Expand Down Expand Up @@ -110,7 +111,7 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms
LISFDIR: [DIRECTORY_PATH_OF_YOUR_LISF_TOP-LEVEL-CODE_DIRECTORY]
E2ESDIR: [YOUR_LOCAL_WORKING-RUN_DIRECTORY_PATH]
METFORC: [MAIN_USAF_FORCING_AND_SATELLITE_DATA_DIRECTORY_PATH]
LISFMOD: lisf_7.5_intel_2021.4.0_s2s [OR LOCAL MACHINE LISF MODULE LIBRARY LIST]
LISFMOD: lisf_7.5_intel_2023.2.1_s2s [OR LOCAL MACHINE LISF MODULE LIBRARY LIST]
SPCODE: s1189 [YOUR LOCAL MACHINE GROUPID CODE]
DATATYPE: forecast [OPTIONS INCLUDE: "forecast" or "hindcast"]
supplementarydir: [YOUR GHI_S2S/supplementary_files DIRECTORY PATH WITH RUNTIME INPUT FILES]
Expand Down Expand Up @@ -193,7 +194,7 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms
To run a single step, you will select the following options, for example
the BCSD step:

sh s2s_app/s2s_run.sh -y 2022 -m 9 -c s2s_config_global_fcst -s BCSD -o Y
sh s2s_app/s2s_run.sh -y 2024 -m 9 -c s2s_config_global_fcst -s BCSD -o Y

would run just the BCSD step and scripts.

Expand All @@ -206,10 +207,10 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms
The report option allows the user to check the order, processing SLURM
job time, and final computational resource usage and time information.

An example of how to monitor an E2ES set of SLURM jobs, for an 09-2022
An example of how to monitor an E2ES set of SLURM jobs, for an 09-2024
forecast:

sh s2s_app/s2s_run.sh -y 2022 -m 9 -c s2s_config_global_fcst -r Y
sh s2s_app/s2s_run.sh -y 2024 -m 9 -c s2s_config_global_fcst -r Y

The "-r" flag option with "Y" set allows the user to see the
following information, e.g.:
Expand All @@ -220,24 +221,28 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms

JOB FILE WALLTIME (HH:MM:SS)

1/105 lisda_run.j 1h 12m 0s
2/105 ldtics_run.j 0h 11m 4s
3/105 bcsd01_run.j 5h 13m 34s
4/105 bcsd03_CCM4_run.j 0h 1m 2s
5/105 bcsd03_CCSM4_run.j 0h 1m 3s
6/105 bcsd03_CFSv2_run.j 0h 1m 5s
7/105 bcsd03_GEOSv2_run.j 0h 0m 54s
8/105 bcsd03_GFDL_run.j 0h 1m 6s
9/105 bcsd03_GNEMO5_run.j 0h 1m 0s
1/36 lisda_run.j 3h 40m 57s
2/36 ldtics_run.j 0h 7m 6s
3/36 bcsd01_01_run.j 0h 50m 53s
4/36 bcsd01_02_run.j 0h 50m 52s
5/36 bcsd01_03_run.j 0h 50m 43s
6/36 bcsd01_04_run.j 0h 51m 11s
7/36 bcsd03_run.j 0h 1m 9s
8/36 bcsd04_01_run.j 3h 11m 46s
9/36 bcsd04_02_run.j 3h 11m 48s
10/36 bcsd05_01_run.j 1h 35m 56s
11/36 bcsd05_02_run.j 2h 9m 44s
...
...
101/105 s2spost_GNEMO5_202301_run.j 0h 29m 32s
102/105 s2spost_GNEMO5_202302_run.j 0h 27m 14s
103/105 s2spost_GNEMO5_202303_run.j 0h 29m 11s
104/105 s2spost_GNEMO5_202304_run.j 0h 29m 23s
105/105 s2spost_GNEMO5_202305_run.j 0h 29m 58s
30/36 lis_fcst_GFDL_07_run.j 2h 45m 44s
31/36 lis_fcst_GFDL_08_run.j 2h 51m 39s
32/36 s2spost_01_run.j 1h 38m 44s
33/36 s2spost_02_run.j 2h 15m 31s
34/36 s2smetric_run.j 1h 16m 58s
35/36 s2smetric_tiff_run.j 0h 49m 56s
36/36 s2splots_run.j 1h 39m 15s

ELAPSED TIME : 1d 08h 14m 52s
ELAPSED TIME : 2d 7h 39m 13s
-------------------------------

The above example report output will show all the E2ES
Expand Down Expand Up @@ -367,7 +372,7 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms
files are available yet, a message will be given be along with the
following option, for example:

Precipitation forecasts are available for only 2 NMME models (GEOS5v2 CCM4).
Precipitation forecasts are available for only 2 NMME models (GEOS5v2 CESM1).
Do you want to continue (Y/N)?

We recommend the user to wait until all files are available on the IRI/Columbia
Expand Down Expand Up @@ -466,7 +471,11 @@ Postprocessing of LIS-based NoahMP4.0.1 and HYMAP2 is supported in terms

- Entries for the NMME and CFSv2 models are found in the main S2S
configuration script (e.g., s2s_config_global_fcst):
CCM4, CCSM4, CFSv2, GEOSv2, GFDL, GNEMO
== As of 01-Aug-2024 ==
CCM4, CCSM4, CFSv2, GEOSv2, GFDL, GNEMO5

== After 01-Aug-2024 ==
CanESM5, CESM1, CFSv2, GEOSv2, GFDL, GNEMO52

4.7 S2S POST-PROCESSING
s2spost: S2S forecast output post-process step
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@

LDT running mode: "Ensemble restart processing"

Processed LSM parameter filename: ../LDT_Params/lis_input.s2s_global.noahmp401_hymap2.25km.nc
LIS number of nests: 1
Number of surface model types: 2
Surface model types: "LSM" "Openwater"
Land surface model: "Noah-MP.4.0.1"
Routing model: none
Lake model: none
Water fraction cutoff value: 0.5
Incorporate crop information: .false.
Number of met forcing sources: 0
Met forcing sources: none
Undefined value: -9999.0

# -- New LDT restart generation sampling option

LIS restart source: "LSM"
Ensemble restart generation mode: downscale
Ensemble restart generation sampling strategy: "random sampling"

Input restart filename: ./output/SURFACEMODEL/202208/LIS_RST_NOAHMP401_202208312345.d01.nc
Output restart filename: ./CanESM5/LIS_RST_NOAHMP401_202208312345.ICS_Sep2022.ens10.nc
Number of ensembles per tile (input restart): 12
#Number of ensembles per tile (input restart): 1
Number of ensembles per tile (output restart): 10

Number of ensembles per tile: 12
#Number of ensembles per tile: 1

Maximum number of surface type tiles per grid: 1
Minimum cutoff percentage (surface type tiles): 0.05
Maximum number of soil texture tiles per grid: 1
Minimum cutoff percentage (soil texture tiles): 0.05
Maximum number of soil fraction tiles per grid: 1
Minimum cutoff percentage (soil fraction tiles): 0.05
Maximum number of elevation bands per grid: 1
Minimum cutoff percentage (elevation bands): 0.05
Maximum number of slope bands per grid: 1
Minimum cutoff percentage (slope bands): 0.05
Maximum number of aspect bands per grid: 1
Minimum cutoff percentage (aspect bands): 0.05

LDT diagnostic file: ./CanESM5/ldtlog_noahmp401_Sep2022
Mask-parameter fill diagnostic file: ./CanESM5/MaskParamFill.log
LDT output directory: ./CanESM5/

#--

#AFRICOM 25KM domain:
Map projection of the LIS domain: latlon
Run domain lower left lat: -89.875
Run domain upper right lat: 89.875
Run domain lower left lon: -179.875
Run domain upper right lon: 179.875
Run domain resolution (dx): 0.25
Run domain resolution (dy): 0.25

#Landcover parameter inputs
Landcover data source: "MODIS_Native"
Landcover classification: "IGBPNCEP"
Landcover file: ./input/LS_PARAMETERS/noah_2dparms/igbp.bin
Landcover spatial transform: tile
Landcover map projection: latlon
Landcover fill option: neighbor
Landcover fill radius: 5
Landcover fill value: 10

#Landmask parameter inputs
Create or readin landmask: "readin"
Landmask data source: "UKMO_CAP_Netcdf"
Landmask file: ./input/data/cap2ldt_ps41.nc
Landmask spatial transform: mode # Go from ~10 KM to 25KM domain ...
Landmask map projection: latlon
Landmask lower left lat: -89.9531250
Landmask lower left lon: -179.9296875
Landmask upper right lat: 89.9531250
Landmask upper right lon: 179.9296875
Landmask resolution (dx): 0.1406250
Landmask resolution (dy): 0.0937500

#Soil parameter inputs
Soil fraction data source: none
Soils spatial transform: none
Soils map projection: latlon
Soils fill option: none
Porosity data source: none
Porosity map: none

#Soil texture map:
Soil texture data source: none
#Soil texture data source: "STATSGOFAO_Native"
Soil texture map: ./input/LS_PARAMETERS/noah_2dparms/topsoil30snew
Soil texture spatial transform: mode
Soil texture map projection: latlon
Soil texture fill option: neighbor
Soil texture fill radius: 5
Soil texture fill value: 6
Soil texture fill value for Antarctica: 16
Soil texture force exclusion of water points during fill: true

#Topography parameter inputs
Elevation data source: "none"
Elevation number of bands: 1
Elevation map: none
Slope data source: "none"
Slope number of bands: 1
Slope map: none
Aspect data source: "none"
Aspect number of bands: 1
Aspect map: none
Topography spatial transform: average
Topography map projection: latlon

#Albedo inputs
Albedo data source: "none"
Albedo map: ./input/LS_PARAMETERS/noah_2dparms/albedo
Albedo climatology interval: monthly
Albedo spatial transform: "budget-bilinear"
Albedo map projection: latlon
Albedo fill option: neighbor
Albedo fill radius: 5
Albedo fill value: 0.15

#Maximum snow albedo inputs
Max snow albedo data source: "none"
Max snow albedo map: ./input/LS_PARAMETERS/noah_2dparms/maximum_snow_albedo.hdf
Max snow albedo spatial transform: average
Max snow albedo map projection: latlon
Max snow albedo fill option: neighbor
Max snow albedo fill radius: 5
Max snow albedo fill value: 0.3

#Greenness inputs
Greenness data source: "none"
Greenness fraction map: ./input/LS_PARAMETERS/noah_2dparms/gfrac
Greenness climatology interval: monthly
Calculate min-max greenness fraction: .false.
Greenness maximum map: ./input/LS_PARAMETERS/noah_2dparms/gfrac_max.asc
Greenness minimum map: ./input/LS_PARAMETERS/noah_2dparms/gfrac_min.asc
Greenness spatial transform: "budget-bilinear"
Greenness map projection: latlon
Greenness fill option: neighbor
Greenness fill radius: 5
Greenness fill value: 0.3
Greenness maximum fill value: 1.0
Greenness minimum fill value: 0.0

#Slope type inputs
Slope type data source: "none"

#Bottom temperature inputs
Bottom temperature data source: "none"
Bottom temperature map: ./input/LS_PARAMETERS/noah_2dparms/SOILTEMP.60
Bottom temperature spatial transform: "budget-bilinear"
Bottom temperature map projection: latlon
Bottom temperature fill option: average
Bottom temperature fill radius: 5
Bottom temperature fill value: 287.0
Bottom temperature topographic downscaling: "lapse-rate"

#Noah-MP LSM inputs
Noah-MP PBL Height Value: 900.

# -----

Loading

0 comments on commit 5bc25f8

Please sign in to comment.