-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.txt
265 lines (218 loc) · 12.8 KB
/
README.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
The below scripts were used to create the ESMAP soil evaporation product
Screen_SMAP_QC_Flag.m - Turns all SMAP retrievals that have an uncertain quality flag (flags other than 0 or 8) and turns the retrievals to the fill value (-9999). Writes new netcdf file, identical to originals.
Get_SMAP_QC.r - Extracts SMAP observations at each grid point and creates a directory structure for gridded ESMAPSMAP file columns. These SMAP obs have been screened by the quality flag (Screen_SMAP_QC_Flag.m).
1 - Year
2 - Month
3 - Day
4 - SMAP Observation
Get_EVI.r - Extracts EVI observations at each SMAP grid point
SMAP file columns
1 - Year
2 - Month
3 - Day
4 - EVI Observation
** Note that EVI needs to be divided by 1000 to get an LAI value
Get_Forcing.r - Extracts forcing file information at each SMAP grid pont
Forcing file columns
1 - Year
2 - Month
3 - Day
4 - Hour
5 - Pressure, Pa
6 - Temperature, K
7 - Specific Humidity, kg/kg
8 - Precipitation, kg/m^2
** Note that NLDAS time is in UTC, while SMAP is at 6am local time
** Initial file took a while to process. Suggest changing the dates to be for new time period wanted and deleting the file.remove statement
Get_NOAH.r - Extracts NOAH LSM information at each SMAP grid pont
Forcing file columns
1 - Year
2 - Month
3 - Day
4 - Hour
5 - Net Shortwave, W/m^2
6 - Net Longwave, W/m^2
** Note that NLDAS time is in UTC, while SMAP is at 6am local time
** Initial file took a while to process. Suggest changing the dates to be for new time period wanted and deleting the file.remove statement
Get_Vegetation.r - Extracts UMD vegetation classification at each SMAP grid point
No columns, just one number
0 Water
1 Evergreen Needleleaf Forest
2 Evergreen Broadleaf Forest
3 Deciduous Needleleaf Forest
4 Deciduous Broadleaf Forest
5 Mixed Forest
6 Woodland
7 Wooded Grassland
8 Closed Shrubland
9 Open Shrubland
10 Grassland
11 Cropland
12 Bare Ground
13 Urban and Built
** Note that Water and Urban SHOULD NOT APPEAR IN FILES GENERATED
** Source: http://glcf.umd.edu/data/landcover/
Get_Soil.r - Extracts predominant soil classification type at each SMAP grid point
Reads NLDAS_Soil_Class.txt
Column 1 X Coordinate Index
Column 2 Y Coordinate Index
Column 3 Longitude (center of 1/8th-degree grid boxes)
Column 4 Latitude (center of 1/8th-degree grid boxes)
Columns 5-21 Number of Occurrences of Soil Classes 1-16 in Each 1/8th-Grid Box
Soil Types:
1 Sand
2 Loamy sand
3 Sandy loam
4 Silt loam
5 Silt
6 Loam
7 Sandy clay loam
8 Silty clay loam
9 Clay loam
10 Sandy clay
11 Silty clay
12 Clay
13 Organic materials
14 Water
15 Bedrock
16 Other
Sources: https://ldas.gsfc.nasa.gov/nldas/NLDASsoils.php
https://ldas.gsfc.nasa.gov/nldas/asc/soils/STEX_TAB.01
https://ral.ucar.edu/sites/default/files/public/product-tool/noah-multiparameterization-land-surface-model-noah-mp-lsm/SOILPARM.TBL_.txt
** Note that we are excluding water as a valid option
** Commented out option in code to produce a file will all soil classes present in gridbox
Calc_Roots.r - R Function that calculates root distribution for transpiration purposes, called by Calc_Mu_Transpiration.r
Uses parameters from CLM4 Technical Note
Cites Zeng (2001)
http://www.cesm.ucar.edu/models/ccsm4.0/clm/CLM4_Tech_Note.pdf
Zeng, X. 2001. Global vegetation root distribution for land modeling. J. Hydrometeor. 2:525-530
Calc_Mu_Transpiration.r - Calculates the Mu transpiration quantities at each of the SMAP points
Uses the output from:
Get_SMAP.r
Get_EVI.r
Get_Forcing.r
Get_NOAH.r
Get_Vegetation.r
Get_Soil.r
Calls Calc_Roots.r to get root density of surface layer
Due to NLDAS and SMAP on different timings (NLDAS is UTC and SMAP is 6am local time), use an offset to get NLDAS on SMAP time
Longitude UTC Conversion
Long_1 Long_2 Offset
-67.5 -82.5 5
-82.5 -97.5 6
-97.5 -112.5 7
-112.5 -127.5 8
Outputs a file of transpiration estimates, columns are:
1 - Year
2 - Month
3 - Day
4 - PM (Mu eqn. 1), mm
5 - PM (Mu eqn. 22), mm
6 - Root restriction (applied to Mu eqn. 22, column 5), mm
7 - SM restriction (applied to the root restriction ET, column 6), mm
Soil_Constants_Hydrus.txt
Source for Van Genuchten parameters:
https://www.ars.usda.gov/ARSUserFiles/80420525/EnvironmentalTransport/CalcPTFFiles/PTF_Manual.version_3.0.pdf
Source for soil texture:
soil texture triangle - https://upload.wikimedia.org/wikipedia/commons/6/65/SoilTextureTriangle.jpg
Ks comes from (in units of m/s, converted to cm/hr):
Chen, F., & Dudhia, J. (2001). Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Monthly Weather Review, 129(4), 569-585.
Organic Materials was given same properties as loam - they have same Ks
Bedrock was given same properties as clay - they had most similar Ks
Other was given same properties as Silty Clay - they have same Ks
Por connectivity (l) is set to 0.5 for all soils - HYDRUS documentation says it is 0.5 for many soils and I found no source to dispute or verify this
Make_Hydrus_Spinup_Files.r
Creates hydrus spinup file for the 9km grid
Create_Final_Run_Files.r
Views output from Hydrus spinup runs, determines whether the Hydrus simulation ran to completion at this point (good_point) or not (bad_point). Then this script creates either new final run files or new spinup files (larger maxIT) depending on if the point was identified as having a completed/fully-converged Hydrus simulation or not.
Get_Soil_Bad_points.r
Creates a new soil file (analogous to that created in Get_Soil.r) with an updated soil type for the 10,161 points that did not converge in the original attempt. The first attempt to alter the soil parameters for points that did not converge looks at the second primary soil type at the grid cell (Secondary_Altered_Points). If the grid cell only has 1 soil type defined, i.e. all other soil types are noted as 0% of the grid cell, the next closest soil type with a different soil parameter set is chosen, other is the default, but in some cases other's parameter set is the same as the original parameter set. In these cases, Silt is the chosen soil classification because it has the most similar soil parameters to 'other'.
1 Sand --> other
2 Loamy sand --> other
3 Sandy loam --> other
4 Silt loam --> other
5 Silt --> other -->other
6 Loam --> other --other
7 Sandy clay loam --> other
8 Silty clay loam --> other
9 Clay loam --> other
10 Sandy clay --> Silt
11 Silty clay --> Silt
12 Clay --> Silt
13 Organic materials --> other
14 Water
15 Bedrock --> Silt
16 Other --> Silt
Create_Selector_Spinup_SoilMod_BadPoints.r
Create new Hydrus Selector file for the points that did not originally converge, but now have update soil parameters.
Run_Hydrus_SpinUp_Loop_9_template.sh
Template for scripts that run Hydrus spinup simulations. The template is iteratively altered by Create_Summit_Run_Hydrus_SpinUp_Loop_9.m, to create scripts that can run in parallel for different sub-regions in the ESMAP domain.
Run_Hydrus_Final_Loop_9_GoodPoints_Template.sh
Template for scripts that run Hydrus final simulations. The template is iteratively altered by Create_Summit_Final_Run_Loop_9.m, to create scripts that can run in parallel for different sub-regions in the ESMAP domain.
create_lat_lon_list_9km_final_summit_parallel_portions.m
Creates sub-regions from a large lat-lon coordinate list to run multiple smaller jobs in parallel on the supercomputer (Summit)
Get_NLDAS_Evap_SMAP_Time_Grid.r
Creates gridded NLDAS Esoil on SMAP intervals at each of the 9km grid points, units are cm
Calculate_ESoil.r
Calculates water balance terms and then ESOIL
Outputs a file with the following information:
1 - Time, Truncated day of mid-point between observations
2 - Length of SMAP overpass interval [days]
3 - EVI, mean of EVI used in Hydrus forcing for SMAP overpass interval
4 - Precipitation [cm] summed for SMAP overpass interval, want less than 0.2 cm
5 - Delta SMAP[cm], change in SMAP for SMAP overpass interval
6 - Transpiration [cm] or SMAP overpass interval, calculate in Calc_Mu_Transpiration.r with the SM and root restriction
7 - Qbot [cm] bottom flux calculated in Hydrus
8 - Esoil [cm], calculated Esoil from water balance calculation
Define_Soil_Param_Flag.m
Outputs the coordinate list of the points that converged using the 'Secondary_params' and those that used the next closest parameter set to the original parameter set 'no_Secondary_Params'. Because the 'Silt' parameter set is most similar to the 'Other' parameter set, in these cases where the original parameter set was identical to other's the 'Silt' texture was chosen.
Grid_QC_ESMAP_revised.r
Creates gridded netcdf files for ESMAP outputs after screening for the SMAP qual_flag
convert_hdf_to_netcdf.m
Converts SMAP native hdf files to netCDF files.
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________
The below scripts use CDO and NCO to interpolate and aggregate data in netCDF format.
NLDAS2.Mosaic.remapbil.pl
Uses CDO bilinear interpolation to bring NLDAS2 Mosaic Data to the ESMAP 9km EASE Grid
NLDAS2.Noah.remapbil.pl
Uses CDO bilinear interpolation to bring NLDAS2 Noah Data to the ESMAP 9km EASE Grid
GLEAM.Esoil.remapbil.v3.1a.pl
Uses CDO bilinear interpolation to bring GLEAM Esoil Data to the ESMAP 9km EASE Grid
GLEAM.ET.remapbil.v3.1a.pl
Uses CDO bilinear interpolation to bring GLEAM ET Data to the ESMAP 9km EASE Grid
command.line.loop.mosaic.[5-8]hr.offset.pl
Uses NCO aggregate hourly NLDAS-2 Mosaic data to daily data accounting for a 5-8 hour UTC to local time difference.
command.line.loop.noah.[5-8]hr.offset.pl
Uses NCO aggregate hourly NLDAS-2 Noah data to daily data accounting for a 5-8 hour UTC to local time difference.
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________
The below scripts were used as a part of the analysis and validation of the ESMAP data product
Screen_Mosaic_Esoil_Daily.m
Screens NLDAS-2 Mosaic's temporally continuous Esoil outputs to match the ESMAP Esoil gridded product's 'valid' intervals
Screen_Noah_Esoil_Daily.m
Screens NLDAS-2 Noah's temporally continuous Esoil outputs to match the ESMAP Esoil gridded product's 'valid' intervals
Screen_GLEAM_Esoil.m
Screens GLEAM's temporally continuous Esoil estimates to match the ESMAP Esoil gridded product's 'valid' intervals
Convert_Lv_of_T.m
Converts Esoil, Ec and Trans NLDAS outputs from W/m2 to mm/day using a latent heat of vaporization formula from equation 4 from Henderson-Sellers, 1986 where Lv is a function of temp
PaperPlot_Screening.m
Creates Figure 2a in the ESMAP data product's data descriptor manuscript
PaperPlot_ESMAP_Slength.m
Creates Figure 2b in ESMAP data product's data descriptor manuscript
PaperPlot_ESMAP_Timing.m
Creates Figure 2c in ESMAP data product's data descriptor manuscript
PaperPlot_KDE_of_ESMAP_Components.m
Plots all panels in Figure 3 in the ESMAP data product's data descriptor manuscript
Calculate_GLEAM_Esoil_ET_ratio.m
Calculates the ratio of the mean Esoil/ET for screened/unscreened time series from GLEAM. Where the screened time series temporally matches the ESMAP time series and the unscreened is the continuous GLEAM time series. PaperPlot_GLEAM_Esoil_ET_ratio.m uses these calculations to creates Figure 4a in the ESMAP data product's data descriptor manuscript
Calculate_Mosaic_Esoil_ET_ratio.m
Calculates the ratio of the mean Esoil/ET for screened/unscreened time series from NLDAS-2 Mosaic. Where the screened time series temporally matches the ESMAP time series and the unscreened is the continuous Mosaic time series. PaperPlot_Mosaic_Esoil_ET_ratio.m uses these calculations to creates Figure 4b in the ESMAP data product's data descriptor manuscript
Calculate_Noah_Esoil_ET_ratio.m
Calculates the ratio of the mean Esoil/ET for screened/unscreened time series from NLDAS-2 Noah. Where the screened time series temporally matches the ESMAP time series and the unscreened is the continuous Noah time series. PaperPlot_Noah_Esoil_ET_ratio.m uses these calculations to creates Figure 4c in the ESMAP data product's data descriptor manuscript
PaperPlot_Boxplot_Esoil_ET_ratio.m
Creates Figure 4d in the ESMAP data product's data descriptor manuscript
Fig4_Sginificance_Testing.R
Employs a Paired Wilcoxon Rank Sum test, to test the significance of the difference in medians between Screened and unscreened evaluation data sets. Uses wilcox.exact package in R from exactRankTests
PaperPlot_ESMAP_Difference_Maps.m
Creates all panels in Figure 5 in the ESMAP data product's data descriptor manuscript
Create_SCALE_FACTOR_files.m
Creates the corresponding scale factor file in the data repository, based on Fig. 4 of the data descriptor (Abolafia-Rosenzweig et al., 2020)