From be44954e8228d5bbc9a257eb4c4b68b37372a444 Mon Sep 17 00:00:00 2001 From: Michael Kavulich Date: Wed, 3 Apr 2024 15:11:35 -0600 Subject: [PATCH] Moving documentation to ReadTheDocs (#438) This PR removes the old, LaTeX-based users guide in favor of RST files that are buildable with Sphinx. These new docs have been configured to automatically build with each commit to main on ReadTheDocs, and can be viewed here: https://ccpp-scm.readthedocs.io/en/latest/ --- .readthedocs.yaml | 33 + LICENSE | 4 +- README.md | 2 +- scm/doc/README.md | 17 + scm/doc/TechGuide/Makefile | 13 +- scm/doc/TechGuide/acknow.rst | 12 + scm/doc/TechGuide/acknow.tex | 19 - scm/doc/TechGuide/chap_cases.rst | 611 +++++++++++++++++++ scm/doc/TechGuide/chap_cases.tex | 364 ----------- scm/doc/TechGuide/chap_ccpp.rst | 383 ++++++++++++ scm/doc/TechGuide/chap_ccpp.tex | 160 ----- scm/doc/TechGuide/chap_function.rst | 157 +++++ scm/doc/TechGuide/chap_function.tex | 66 -- scm/doc/TechGuide/chap_hsd.rst | 351 +++++++++++ scm/doc/TechGuide/chap_hsd.tex | 213 ------- scm/doc/TechGuide/chap_intro.rst | 116 ++++ scm/doc/TechGuide/chap_intro.tex | 46 -- scm/doc/TechGuide/chap_quick.rst | 902 ++++++++++++++++++++++++++++ scm/doc/TechGuide/chap_quick.tex | 481 --------------- scm/doc/TechGuide/chap_repo.rst | 62 ++ scm/doc/TechGuide/chap_repo.tex | 46 -- scm/doc/TechGuide/conf.py | 211 +++++++ scm/doc/TechGuide/index.rst | 18 + scm/doc/TechGuide/main.pdf | Bin 813248 -> 0 bytes scm/doc/TechGuide/main.tex | 36 -- scm/doc/TechGuide/preface.rst | 34 ++ scm/doc/TechGuide/preface.tex | 28 - scm/doc/TechGuide/requirements.txt | 1 + scm/doc/TechGuide/title.tex | 35 -- 29 files changed, 2919 insertions(+), 1502 deletions(-) create mode 100644 .readthedocs.yaml create mode 100644 scm/doc/README.md create mode 100644 scm/doc/TechGuide/acknow.rst delete mode 100644 scm/doc/TechGuide/acknow.tex create mode 100644 scm/doc/TechGuide/chap_cases.rst delete mode 100644 scm/doc/TechGuide/chap_cases.tex create mode 100644 scm/doc/TechGuide/chap_ccpp.rst delete mode 100644 scm/doc/TechGuide/chap_ccpp.tex create mode 100644 scm/doc/TechGuide/chap_function.rst delete mode 100644 scm/doc/TechGuide/chap_function.tex create mode 100644 scm/doc/TechGuide/chap_hsd.rst delete mode 100644 scm/doc/TechGuide/chap_hsd.tex create mode 100644 scm/doc/TechGuide/chap_intro.rst delete mode 100644 scm/doc/TechGuide/chap_intro.tex create mode 100644 scm/doc/TechGuide/chap_quick.rst delete mode 100644 scm/doc/TechGuide/chap_quick.tex create mode 100644 scm/doc/TechGuide/chap_repo.rst delete mode 100644 scm/doc/TechGuide/chap_repo.tex create mode 100644 scm/doc/TechGuide/conf.py create mode 100644 scm/doc/TechGuide/index.rst delete mode 100644 scm/doc/TechGuide/main.pdf delete mode 100644 scm/doc/TechGuide/main.tex create mode 100644 scm/doc/TechGuide/preface.rst delete mode 100644 scm/doc/TechGuide/preface.tex create mode 100644 scm/doc/TechGuide/requirements.txt delete mode 100644 scm/doc/TechGuide/title.tex diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 000000000..95693767e --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,33 @@ +# .readthedocs.yaml +# Read the Docs configuration file +# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details + +# Required +version: 2 + +# Set the OS, Python version and other tools you might need +build: + os: ubuntu-22.04 + tools: + python: "3.12" + # You can also specify other tool versions: + # nodejs: "19" + # rust: "1.64" + # golang: "1.19" + +# Build documentation in the "docs/" directory with Sphinx +sphinx: + configuration: scm/doc/TechGuide/conf.py + +# Optionally build your docs in additional formats such as PDF and ePub +formats: + - htmlzip + - pdf + - epub + +# Optional but recommended, declare the Python requirements required +# to build your documentation +# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html +python: + install: + - requirements: scm/doc/TechGuide/requirements.txt diff --git a/LICENSE b/LICENSE index 5f497a483..8e61668f4 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -Copyright 2022, NOAA, UCAR/NCAR, CU/CIRES, CSU/CIRA +Copyright 2024, NOAA, UCAR/NCAR, CU/CIRES, CSU/CIRA Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -10,4 +10,4 @@ Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. \ No newline at end of file +limitations under the License. diff --git a/README.md b/README.md index 1649d16b4..0db822ec2 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,3 @@ # User's Guide -Please find the up-to-date User's Guide located in this repository in [`scm/doc/TechGuide/main.pdf`](https://github.com/NCAR/ccpp-scm/blob/main/scm/doc/TechGuide/main.pdf) \ No newline at end of file +Please find the up-to-date User's Guide located on ReadTheDocs: https://ccpp-scm.readthedocs.io/en/latest/ diff --git a/scm/doc/README.md b/scm/doc/README.md new file mode 100644 index 000000000..fc82b5461 --- /dev/null +++ b/scm/doc/README.md @@ -0,0 +1,17 @@ +# SCM documentation + +The SCM technical documentation/users' guide is written in [reStructured Text markup language (RST)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html). It is designed to be built with the [Sphinx documentation tool](https://www.sphinx-doc.org/en/master/index.html), and is documentation from the `main` branch is configured to build automatically on ReadTheDocs: https://ccpp-scm.readthedocs.io/en/latest/ + +## Building docs locally + +To build the HTML documentation locally (for example, when testing modifications to the documentation), you must have Sphinx [installed on your machine](https://www.sphinx-doc.org/en/master/usage/installation.html), as well as the required python packages listed in the `requirements.txt` file. Once the prerequisite software is installed, you can build the HTML documentation using the following commands: + + python -m sphinx -T -b html -d _build/doctrees -D language=en . html + +To build a PDF locally, in addition to Sphinx you must have Perl as well as the Perl-based [`latexmk` utility](https://mg.readthedocs.io/latexmk.html#installation). You can then build a PDF using the following commands: + + python -m sphinx -T -b latex -d _build/doctrees -D language=en . latex + latexmk -f -pdf -pdflatex="pdflatex" -use-make index.tex + +In lieu of running the above commands manually, you can also use `make`. The command `make all` will build both HTML and PDF documentation, while `make html` and `make latex` will build HTML and PDF respectively. + diff --git a/scm/doc/TechGuide/Makefile b/scm/doc/TechGuide/Makefile index 501e745a6..6d48b2a61 100644 --- a/scm/doc/TechGuide/Makefile +++ b/scm/doc/TechGuide/Makefile @@ -2,13 +2,16 @@ # Makefile for the CCPP (SCM) Users Guide # -.PHONY: main.pdf all clean +.PHONY: html latex all clean -all: main.pdf +all: html latex -main.pdf: main.tex - latexmk -f -pdf -pdflatex="pdflatex" -use-make main.tex - open main.pdf +latex: + python -m sphinx -T -b latex -d _build/doctrees -D language=en . latex + cd latex && latexmk -f -pdf -pdflatex="pdflatex" -use-make index.tex + +html: + python -m sphinx -T -b html -d _build/doctrees -D language=en . html clean: latexmk -CA diff --git a/scm/doc/TechGuide/acknow.rst b/scm/doc/TechGuide/acknow.rst new file mode 100644 index 000000000..c8014aba5 --- /dev/null +++ b/scm/doc/TechGuide/acknow.rst @@ -0,0 +1,12 @@ +Acknowledgement +=============== +If significant help was provided via the helpdesk, email, or +support forum for work resulting in a publication, please +acknowledge the Developmental Testbed Center team. + +For referencing this document please use: + + Firl, G., D. Swales, L. Carson, L. Bernardet, D. Heinzeller, M. Harrold, T. Hertneky, and + M. Kavulich, 2024. Common Community Physics Package Single Column Model v7.0.0 User and + Technical Guide. Available at https://ccpp-scm.readthedocs.io/en/latest/. + diff --git a/scm/doc/TechGuide/acknow.tex b/scm/doc/TechGuide/acknow.tex deleted file mode 100644 index f089cb91d..000000000 --- a/scm/doc/TechGuide/acknow.tex +++ /dev/null @@ -1,19 +0,0 @@ -\begin{titlepage} -%\BgThispage -%\newgeometry{left=1cm,right=4cm} -\vspace*{0.5cm} -\noindent - -\begin{flushleft} -\textcolor{darkgray}{\LARGE Acknowledgement} -\vspace*{1cm}\par - -If significant help was provided via the helpdesk, email, or support forum for work resulting in a publication, please acknowledge the Developmental Testbed Center team.\\ -\vspace*{1cm}\par -For referencing this document please use:\\ -\vspace*{1cm}\par -Firl, G., D. Swales, L. Carson, L. Bernardet, D. Heinzeller, and M. Harrold, 2022. Common Community Physics Package Single Column Model v6.0.0 User and Technical Guide. 40pp. Available at https://dtcenter.org/sites/default/files/paragraph/scm-ccpp-guide-v6-0-0.pdf. - -\end{flushleft} -\end{titlepage} -\pagebreak{} diff --git a/scm/doc/TechGuide/chap_cases.rst b/scm/doc/TechGuide/chap_cases.rst new file mode 100644 index 000000000..b6bd183dd --- /dev/null +++ b/scm/doc/TechGuide/chap_cases.rst @@ -0,0 +1,611 @@ +Cases +===== + +How to run cases +---------------- + +Only two files are needed to set up and run a case with the SCM. The +first is a configuration namelist file found in ``ccpp-scm/scm/etc/case_config`` that contains parameters +for the SCM infrastructure. The second necessary file is a NetCDF file +containing data to initialize the column state and time-dependent data +to force the column state. The two files are described below. + +.. _`case config`: + +Case configuration namelist parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``case_config`` namelist expects the following parameters: + +- ``case_name`` + + - Identifier for which dataset (initialization and forcing) to load. + This string must correspond to a dataset included in the directory + ``ccpp-scm/scm/data/processed_case_input/`` (without the file extension). + +- ``runtime`` + + - Specify the model runtime in seconds (integer). This should + correspond with the forcing dataset used. If a runtime is + specified that is longer than the supplied forcing, the forcing is + held constant at the last specified values. + +- ``thermo_forcing_type`` + + - An integer representing how forcing for temperature and moisture + state variables is applied (1 :math:`=` total advective + tendencies, 2 :math:`=` horizontal advective tendencies with + prescribed vertical motion, 3 :math:`=` relaxation to observed + profiles with vertical motion prescribed) + +- ``mom_forcing_type`` + + - An integer representing how forcing for horizontal momentum state + variables is applied (1 :math:`=` total advective tendencies; not + implemented yet, 2 :math:`=` horizontal advective tendencies with + prescribed vertical motion, 3 :math:`=` relaxation to observed + profiles with vertical motion prescribed) + +- ``relax_time`` + + - A floating point number representing the timescale in seconds for + the relaxation forcing (only used if ``thermo_forcing_type = 3`` or ``mom_forcing_type = 3``) + +- ``sfc_flux_spec`` + + - A boolean set to ``.true.`` if surface flux are specified from the forcing + data (there is no need to have surface schemes in a suite + definition file if so) + +- ``sfc_roughness_length_cm`` + + - Surface roughness length in cm for calculating surface-related + fields from specified surface fluxes (only used if ``sfc_flux_spec`` is True). + +- ``sfc_type`` + + - An integer representing the character of the surface (0 :math:`=` + sea surface, 1 :math:`=` land surface, 2 :math:`=` sea-ice + surface) + +- ``reference_profile_choice`` + + - An integer representing the choice of reference profile to use + above the supplied initialization and forcing data (1 :math:`=` + “McClatchey” profile, 2 :math:`=` mid-latitude summer standard + atmosphere) + +- ``year`` + + - An integer representing the year of the initialization time + +- ``month`` + + - An integer representing the month of the initialization time + +- ``day`` + + - An integer representing the day of the initialization time + +- ``hour`` + + - An integer representing the hour of the initialization time + +- ``column_area`` + + - A list of floating point values representing the characteristic + horizontal domain area of each atmospheric column in square meters + (this could be analogous to a 3D model’s horizontal grid size or + the characteristic horizontal scale of an observation array; these + values are used in scale-aware schemes; if using multiple columns, + you may specify an equal number of column areas) + +- ``model_ics`` + + - A boolean set to ``.true.`` if UFS atmosphere initial conditions are used + rather than field campaign-based initial conditions + +- ``C_RES`` + + - An integer representing the grid size of the UFS atmosphere + initial conditions; the integer represents the number of grid + points in each horizontal direction of each cube tile + +- ``input_type`` + + - 0 => original DTC format, 1 => DEPHY-SCM format. + +Optional variables (that may be overridden via run script command line +arguments) are: + +- ``vert_coord_file`` + + - File containing FV3 vertical grid coefficients. + +- ``n_levels`` + + - Specify the integer number of vertical levels. + +.. _`case input`: + +Case input data file (CCPP-SCM format) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The initialization and forcing data for each case is stored in a NetCDF +(version 4) file within the ``ccpp-scm/scm/data/processed_case_input`` directory. Each file has at least two +dimensions (``time`` and ``levels``, potentially with additions for vertical snow and soil +levels) and is organized into 3 groups: scalars, initial, and forcing. +Not all fields are required for all cases. For example the fields ``sh_flux_sfc`` and ``lh_flux_sfc`` +are only needed if the variable ``sfc_flx_spec = .true.`` in the case configuration file +and state nudging variables are only required if ``thermo_forcing_type = 3`` or ``mom_forcing_type = 3`` +. Using an active LSM (Noah, NoahMP, RUC) requires many more variables +than are listed here. Example files for using with Noah and NoahMP LSMs +are included in ``ccpp-scm/scm/data/processed_case_input/fv3_model_point_noah[mp].nc``. + +.. _`case input arm`: +.. literalinclude:: arm_case_header.txt + :name: lst_case_input_netcdf_header_arm + :caption: example NetCDF file (CCPP-SCM format) header for case initialization and forcing data + +Case input data file (DEPHY format) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Development and Evaluation of Physics in atmospheric models (DEPHY) +format is an internationally-adopted data format intended for use by SCM +and LESs. The initialization and forcing data for each case is stored in +a NetCDF (version 4) file, although these files are not by default +included in the CCPP SCM repository. To access these cases you need to +clone the DEPHY-SCM repository, and provide the DEPHY-SCM file location +to the SCM. For example: + +.. code:: bash + + cd [...]/ccpp-scm/scm/data + git clone https://github.com/GdR-DEPHY/DEPHY-SCM DEPHY-SCM + cd [...]/ccpp-scm/scm/bin + ./run_scm.py -c MAGIC_LEG04A --case_data_dir [...]/ccpp-scm/scm/data/DEPHY-SCM/MAGIC/LEG04A -v + +Each DEPHY file has three dimensions (``time``, ``t0``, ``levels``) and contains the initial +conditions (``t0``, ``levels``) and forcing data (``time``, ``levels``). Just as when using the CCPP-SCM +formatted inputs, :numref:`Subsection %s `, not all fields +are required for all cases. More information on the DEPHY format +requirements can be found at +`DEPHY `__. + +.. _`case input dephy`: +.. literalinclude:: dephy_case_header.txt + :name: lst_case_input_netcdf_header_dephy + :caption: example NetCDF file (DEPHY format) header for case initialization and forcing data + +Included Cases +-------------- + +Several cases are included in the repository to serve as examples for +users to create their own and for basic research. All case configuration +namelist files for included cases can be found in ``ccpp-scm/scm/etc/case_config`` and represent the +following observational field campaigns: + +- Tropical Warm Pool – International Cloud Experiment (TWP-ICE) + maritime deep convection + +- Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) + Summer 1997 continental deep convection + +- Atlantic Stratocumulus Transition EXperiment (ASTEX) maritime + stratocumulus-to-cumulus transition + +- Barbados Oceanographic and Meteorological EXperiment (BOMEX) maritime + shallow convection + +- Large eddy simulation ARM Symbiotic Simulation and Observation + (LASSO) for May 18, 2016 (with capability to run all LASSO dates - + see :numref:`Section %s `) continental shallow convection + +For the ARM SGP case, several case configuration files representing +different time periods of the observational dataset are included, +denoted by a trailing letter. The LASSO case may be run with different +forcing applied, so three case configuration files corresponding to +these different forcing are included. In addition, two example cases are +included for using UFS Atmosphere initial conditions: + +- UFS initial conditions for 38.1 N, 98.5 W (central Kansas) for 00Z on + Oct. 3, 2016 with Noah variables on the C96 FV3 grid (``fv3_model_point_noah.nc``) + +- UFS initial conditions for 38.1 N, 98.5 W (central Kansas) for 00Z on + Oct. 3, 2016 with NoahMP variables on the C96 FV3 grid (``fv3_model_point_noahmp.nc``) + +See :numref:`Section %s ` for information on how to generate these +files for other locations and dates, given appropriate UFS Atmosphere +initial conditions and output. + +How to set up new cases +----------------------- + +Setting up a new case involves preparing the two types of files listed +above. For the case initialization and forcing data file, this typically +involves writing a custom script or program to parse the data from its +original format to the format that the SCM expects, listed above. An +example of this type of script written in Python is included in ``ccpp-scm/scm/etc/scripts/twpice_forcing_file_generator.py``. The +script reads in the data as supplied from its source, converts any +necessary variables, and writes a NetCDF (version 4) file in the format +described in subsections :numref:`Subsection %s ` and +:numref:`Subsection %s `. For reference, the following +formulas are used: + +.. math:: \theta_{il} = \theta - \frac{\theta}{T}\left(\frac{L_v}{c_p}q_l + \frac{L_s}{c_p}q_i\right) + +.. math:: q_t = q_v + q_l + q_i + +where :math:`\theta_{il}` is the ice-liquid water potential temperature, +:math:`\theta` is the potential temperature, :math:`L_v` is the latent +heat of vaporization, :math:`L_s` is the latent heat of sublimation +:math:`c_p` is the specific heat capacity of air at constant pressure, +:math:`T` is absolute temperature, :math:`q_t` is the total water +specific humidity, :math:`q_v` is the water vapor specific humidity, +:math:`q_l` is the suspended liquid water specific humidity, and +:math:`q_i` is the suspended ice water specific humidity. + +As shown in the example NetCDF header, the SCM expects that the vertical +dimension is pressure levels (index 1 is the surface) and the time +dimension is in seconds. The initial conditions expected are the height +of the pressure levels in meters, and arrays representing vertical +columns of :math:`\theta_{il}` in K, :math:`q_t`, :math:`q_l`, and +:math:`q_i` in kg kg\ :math:`^{-1}`, :math:`u` and :math:`v` in m +s\ :math:`^{-1}`, turbulence kinetic energy in m\ :math:`^2` +s\ :math:`^{-2}` and ozone mass mixing ratio in kg kg\ :math:`^{-1}`. + +For forcing data, the SCM expects a time series of the following +variables: latitude and longitude in decimal degrees [in case the +column(s) is moving in time (e.g., Lagrangian column)], the surface +pressure (Pa) and surface temperature (K). If surface fluxes are +specified for the new case, one must also include a time series of the +kinematic surface sensible heat flux (K m s\ :math:`^{-1}`) and +kinematic surface latent heat flux (kg kg\ :math:`^{-1}` m +s\ :math:`^{-1}`). The following variables are expected as 2-dimensional +arrays (vertical levels first, time second): the geostrophic u (E-W) and +v (N-S) winds (m s\ :math:`^{-1}`), and the horizontal and vertical +advective tendencies of :math:`\theta_{il}` (K s\ :math:`^{-1}`) and +:math:`q_t` (kg kg\ :math:`^{-1}` s\ :math:`^{-1}`), the large scale +vertical velocity (m s\ :math:`^{-1}`), large scale pressure vertical +velocity (Pa s\ :math:`^{-1}`), the prescribed radiative heating rate (K +s\ :math:`^{-1}`), and profiles of u, v, T, :math:`\theta_{il}` and +:math:`q_t` to use for nudging. + +Although it is expected that all variables are in the NetCDF file, only +those that are used with the chosen forcing method are required to be +nonzero. For example, the following variables are required depending on +the values of ``mom_forcing_type`` and ``thermo_forcing_type`` specified in the case configuration file: + +- ``mom_forcing_type = 1`` + + - Not implemented yet + +- ``mom_forcing_type = 2`` + + - geostrophic winds and large scale vertical velocity + +- ``mom_forcing_type = 3`` + + - u and v nudging profiles + +- ``thermo_forcing_type = 1`` + + - horizontal and vertical advective tendencies of + :math:`\theta_{il}` and :math:`q_t` and prescribed radiative + heating (can be zero if radiation scheme is active) + +- ``thermo_forcing_type = 2`` + + - horizontal advective tendencies of :math:`\theta_{il}` and + :math:`q_t`, prescribed radiative heating (can be zero if + radiation scheme is active), and the large scale vertical pressure + velocity + +- ``thermo_forcing_type = 3`` + + - :math:`\theta_{il}` and :math:`q_t` nudging profiles and the large + scale vertical pressure velocity + +For the case configuration file, it is most efficient to copy an +existing file in ``ccpp-scm/scm/etc/case_config`` and edit it to suit one’s case. Recall from subsection +:numref:`Subsection %s ` that this file is used to configure +the SCM framework parameters for a given case. Be sure to check that +model timing parameters such as the time step and output frequency are +appropriate for the physics suite being used. There is likely some +stability criterion that governs the maximum time step based on the +chosen parameterizations and number of vertical levels (grid spacing). +The ``case_name`` parameter should match the name of the case input data file that was +configured for the case (without the file extension). The parameter +should be less than or equal to the length of the forcing data unless +the desired behavior of the simulation is to proceed with the last +specified forcing values after the length of the forcing data has been +surpassed. The initial date and time should fall within the forcing +period specified in the case input data file. If the case input data is +specified to a lower altitude than the vertical domain, the remainder of +the column will be filled in with values from a reference profile. There +is a tropical profile and mid-latitude summer profile provided, although +one may add more choices by adding a data file to ``ccpp-scm/scm/data/processed_case_input`` and adding a parser +section to the subroutine ``get_reference_profile`` in ``scm/src/scm_input.f90``. Surface fluxes can either be specified in +the case input data file or calculated using a surface scheme using +surface properties. If surface fluxes are specified from data, set ``sfc_flux_spec`` to ``.true.`` +and specify ``sfc_roughness_length_cm`` for the surface over which the column resides. Otherwise,` +specify a ``sfc_type``. In addition, one must specify a ``column_area`` for each column. + +To control the forcing method, one must choose how the momentum and +scalar variable forcing are applied. The three methods of Randall and +Cripe (1999, JGR) have been implemented: “revealed forcing” where total +(horizontal :math:`+` vertical) advective tendencies are applied (type +1), “horizontal advective forcing” where horizontal advective tendencies +are applied and vertical advective tendencies are calculated from a +prescribed vertical velocity and the calculated (modeled) profiles (type +2), and “relaxation forcing” where nudging to observed profiles replaces +horizontal advective forcing combined with vertical advective forcing +from prescribed vertical velocity (type 3). If relaxation forcing is +chosen, a ``relax_time`` that represents the timescale over which the profile would +return to the nudging profiles must be specified. + +.. _`lasso`: + +Using other LASSO cases +----------------------- + +In order to use other LASSO cases than the one provided, perform the +following steps: + +#. Access http://archive.arm.gov/lassobrowser and use the navigation on + the left to choose the dates for which you would like to run a SCM + simulation. Pay attention to the “Large Scale Forcing” tab where you + can choose how the large scale forcing was generated, with options + for ECMWF, MSDA, and VARANAL. All are potentially valid, and it is + likely worth exploring the differences among forcing methods. Click + on Submit to view a list of simulations for the selected criteria. + Choose from the simulations (higher skill scores are preferred) and + check the “Config Obs Model Tar” box to download the data. Once the + desired simulations have been checked, order the data (you may need + to create an ARM account to do so). + +#. Once the data is downloaded, decompress it. From the ``config`` directory, copy + the files ``input_ls_forcing.nc``, ``input_sfc_forcing.nc``, and ``wrfinput_d01.nc`` into their own directory under ``ccpp-scm/scm/data/raw_case_input/``. + +#. Modify ``ccpp-scm/scm/etc/scripts/lasso1_forcing_file_generator_gjf.py`` to point to the input files listed above. Execute the script + in order to generate a case input file for the SCM (to be put in ``ccpp-scm/scm/data/processed_case_input/``): + + .. code:: bash + + ./lasso1_forcing_file_generator_gjf.py + +#. Create a new case configuration file (or copy and modify an existing + one) in ``ccpp-scm/scm/etc/case_config``. Be sure that the ``case_name`` variable points to the newly + created/processed case input file from above. + +.. _`UFSreplay`: + +Using UFS Output to Create SCM Cases: UFS-Replay +------------------------------------------------ + +.. _`pydepend_replay`: + +Python Dependencies +~~~~~~~~~~~~~~~~~~~ + +The scripts here require a few python packages that may not be found by +default in all python installations. There is a YAML file with the +python environment needed to run the script in ``ccpp-scm/environment-ufsreplay.yml``. To create and activate +this environment using conda: + +Create environment (only once): + +.. code:: bash + + > conda env create -f environment-ufsreplay.yml + +This will create the conda environment ``env_ufsreplay`` + +Activate environment: + +.. code:: bash + + > conda activate env_ufsreplay + +.. _`ufsicgenerator`: + +UFS_IC_generator.py +~~~~~~~~~~~~~~~~~~~ + +A script exists in ``scm/etc/scripts/UFS_IC_generator.py`` to read in UFS history (output) files and their +initial conditions to generate a SCM case input data file, in DEPHY +format. + +.. code:: bash + + ./UFS_IC_generator.py [-h] (-l LOCATION LOCATION | -ij INDEX INDEX) -d + DATE -i IN_DIR -g GRID_DIR -f FORCING_DIR -n + CASE_NAME [-t {1,2,3,4,5,6,7}] [-a AREA] [-oc] + [-lam] [-sc] [-near] + +Mandatory arguments: + +#. ``--location (-l)`` OR ``--index (-ij)``: Either longitude and latitude in decimal degrees east and north + of a location OR the UFS grid index with the tile number + + - -l 261.51 38.2 (two floating point values separated by a space) + + - -ij 8 49 (two integer values separated by a space; this option + must also use the ``--tile (-t)`` argument to specify the tile number) + +#. ``--date (-d)`` YYYYMMDDHHMMSS: date corresponding to the UFS initial conditions + +#. ``--in_dir (-i)``: path to the directory containing the UFS initial conditions + +#. ``--grid_dir (-g)``: path to the directory containing the UFS supergrid files (AKA "fix" + directory) + +#. ``--forcing_dir (-f)``: path to the directory containing the UFS history files + +#. ``--case_name (-n)``: name of case + +Optional arguments: + +#. ``--tile (-t)``: if one already knows the correct tile for the given longitude and + latitude OR one is specifying the UFS grid index (``--index`` argument) + +#. ``--area (-a)``: area of grid cell in :math:`m^2` (if known or different than the + value calculated from the supergrid file) + +#. ``--old_chgres (-oc)``: flag if UFS initial conditions were generated using older version + of chgres (global_chgres); might be the case for pre-2018 data + +#. ``--lam (-lam)``: flag to signal that the ICs and forcing is from a limited-area + model run + +#. ``--save_comp (-sc)``: flag to create UFS reference file for comparison + +#. ``--use_nearest (-near)``: flag to indicate using the nearest UFS history file gridpoint + +.. _`ufsforcingensemblegenerator`: + +UFS_forcing_ensemble_generator.py +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There is an additional script in ``scm/etc/scripts/UFS_forcing_ensemble_generator.py`` to create UFS-replay case(s) starting +with output from UFS Weather Model (UWM) Regression Tests (RTs). + +.. code:: bash + + UFS_forcing_ensemble_generator.py [-h] -d DIR -n CASE_NAME + (-lonl LON_1 LON_2 -latl LAT_1 LAT_2 -nens NENSMEMBERS | + -lons [LON_LIST] -lats [LAT_LIST]) + [-dt TIMESTEP] [-cres C_RES] [-sdf SUITE] [-sc] [-near] + +Mandatory arguments: + +#. ``--dir (-d)``: path to UFS Regression Test output + +#. ``--case_name (-n)``: name of cases + +#. Either: (see examples below) + + - ``--lon_limits (-lonl)`` AND ``--lat_limits (-latl)`` AND ``--nensmembers (-nens)``: longitude range, latitude range, and number of cases to + create + + - ``--lon_list (-lons)`` AND ``--lat_list (-lats)``: longitude and latitude of cases + +Optional arguments: + +#. ``--timestep (-dt)``: SCM timestep, in seconds + +#. ``--C_res (-cres)``: UFS spatial resolution + +#. ``--suite (-sdf)``: CCPP suite definition file to use for ensemble + +#. ``--save_comp (-sc)``: flag to create UFS reference file for comparison + +#. ``--use_nearest (-near)``: flag to indicate using the nearest UFS history file gridpoint + +Examples to run from within the ``scm/etc/scripts`` directory to create SCM cases starting +with the output from a UFS Weather Model regression test(s): + +On the supported platforms Cheyenne (NCAR) and Hera (NOAA), there are +staged UWM RTs located at: + +- Cheyenne ``/glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs`` +- Hera ``/scratch1/BMC/gmtb/CCPP-SCM/UFS_RTs`` + +.. _`example1`: + +Example 1: UFS-replay for single point +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +UFS regression test, ``control_c192``, for single point. + +.. code:: bash + + ./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_c192/ -sc --C_RES 192 -dt 360 -n control_c192 -lons 300 -lats 34 + +Upon successful completion of the script, the command to run the case(s) +will print to the screen. For example, + +.. code:: bash + + ./run_scm.py --npz_type gfs --file scm_ufsens_control_c192.py --timestep 360 + +The file ``scm_ufsens_control_c192.py`` is created in ``ccpp-scm/scm/bin/``, where the SCM run script is to be exectued. + +.. _`example2`: + +Example 2: UFS-replay for list of points +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +UFS regression test, ``control_c384``, for multiple points. + +.. code:: bash + + ./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_c384/ -sc --C_RES 384 -dt 225 -n control_c384 -lons 300 300 300 300 -lats 34 35 35 37 + +Upon successful completion of the script, the command to run the case(s) +will print to the screen. For example, + +.. code:: bash + + ./run_scm.py --npz_type gfs --file scm_ufsens_control_c384.py --timestep 225 + +The file ``scm_ufsens_control_c384.py`` contains ALL of the cases created. Each case created will have the +naming convention ``case_name_nXXX``, where the suffix ``XXX`` is the case number from 0 to the +number of points provided. The contents of the file should look like: + +.. code:: bash + + run_list = [{"case": "control_c384_n000", "suite": "SCM_GFS_v16"}, + {"case": "control_c384_n001", "suite": "SCM_GFS_v16"}, + {"case": "control_c384_n002", "suite": "SCM_GFS_v16"}, + {"case": "control_c384_n003", "suite": "SCM_GFS_v16"}] + +.. _`example3`: + +Example 3: UFS-replay for an ensemble of points +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +UFS regression test, ``control_p8``, for an ensemble (10) of randomly selected points +over a specified longitude (:math:`300-320^oW`) and latitude +(:math:`40-50^oN`) range + +But first, to use the ``control_p8`` test we need to rerun the regression test to +generate UFS history files with a denser and constant output interval. +First, in ``control_p8/model_configure``, change ``--output_fh`` to ``"interval -1"``, where ``interval`` is the UFS history file output frequency +(in hours), see `UFS Weather Model Users +Guide `__ +for more details. + +For the purposes of this example the ``control_p8`` test has already been rerun, but if +starting from your own UWM RTs, you can rerun the UWM regression test, +on Cheyenne for example, by running the following command in the RT +directory: ``qsub job_card`` + +Now the cases can be generated with the following command: + +.. code:: bash + + ./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_p8/ -sc --C_RES 96 -dt 720 -n control_p8 -lonl 300 320 -latl 40 50 -nens 10 -sdf SCM_GFS_v17_p8 + +Upon successful completion of the script, the command to run the case(s) +will print to the screen. For example, + +.. code:: bash + + ./run_scm.py --npz_type gfs --file scm_ufsens_control_p8.py --timestep 720 + +The file ``scm_ufsens_control_p8.py`` contains ten cases (n000-n009) to be run. The contents of the +file should look like: + +.. code:: bash + + run_list = [{"case": "control_p8_n000", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n001", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n002", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n003", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n004", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n005", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n006", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n007", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n008", "suite": "SCM_GFS_v17_p8"}, + {"case": "control_p8_n009", "suite": "SCM_GFS_v17_p8"}] diff --git a/scm/doc/TechGuide/chap_cases.tex b/scm/doc/TechGuide/chap_cases.tex deleted file mode 100644 index e12b75ead..000000000 --- a/scm/doc/TechGuide/chap_cases.tex +++ /dev/null @@ -1,364 +0,0 @@ -\chapter{Cases} -\label{chapter: cases} - -\section{How to run cases} -Only two files are needed to set up and run a case with the SCM. The first is a configuration namelist file found in \execout{ccpp-scm/scm/etc/case\_config} that contains parameters for the SCM infrastructure. The second necessary file is a NetCDF file containing data to initialize the column state and time-dependent data to force the column state. The two files are described below. - -\subsection{Case configuration namelist parameters} -\label{subsection: case config} -The \execout{case\_config} namelist expects the following parameters: -\begin{itemize} -\item \execout{case\_name} - \begin{itemize} - \item Identifier for which dataset (initialization and forcing) to load. This string must correspond to a dataset included in the directory \execout{ccpp-scm/scm/data/processed\_case\_input/} (without the file extension). - \end{itemize} -\item \execout{runtime} - \begin{itemize} - \item Specify the model runtime in seconds (integer). This should correspond with the forcing dataset used. If a runtime is specified that is longer than the supplied forcing, the forcing is held constant at the last specified values. - \end{itemize} -\item \execout{thermo\_forcing\_type} - \begin{itemize} - \item An integer representing how forcing for temperature and moisture state variables is applied (1 $=$ total advective tendencies, 2 $=$ horizontal advective tendencies with prescribed vertical motion, 3 $=$ relaxation to observed profiles with vertical motion prescribed) - \end{itemize} -\item \execout{mom\_forcing\_type} - \begin{itemize} - \item An integer representing how forcing for horizontal momentum state variables is applied (1 $=$ total advective tendencies; not implemented yet, 2 $=$ horizontal advective tendencies with prescribed vertical motion, 3 $=$ relaxation to observed profiles with vertical motion prescribed) - \end{itemize} -\item \execout{relax\_time} - \begin{itemize} - \item A floating point number representing the timescale in seconds for the relaxation forcing (only used if \execout{thermo\_forcing\_type} $=$ \exec{3} or \execout{mom\_forcing\_type} $=$ \exec{3}) - \end{itemize} -\item \execout{sfc\_flux\_spec} - \begin{itemize} - \item A boolean set to \exec{.true.} if surface flux are specified from the forcing data (there is no need to have surface schemes in a suite definition file if so) - \end{itemize} -\item \execout{sfc\_roughness\_length\_cm} - \begin{itemize} - \item Surface roughness length in cm for calculating surface-related fields from specified surface fluxes (only used if \execout{sfc\_flux\_spec} is True). - \end{itemize} -\item \execout{sfc\_type} - \begin{itemize} - \item An integer representing the character of the surface (0 $=$ sea surface, 1 $=$ land surface, 2 $=$ sea-ice surface) - \end{itemize} -\item \execout{reference\_profile\_choice} - \begin{itemize} - \item An integer representing the choice of reference profile to use above the supplied initialization and forcing data (1 $=$ ``McClatchey'' profile, 2 $=$ mid-latitude summer standard atmosphere) - \end{itemize} -\item \execout{year} - \begin{itemize} - \item An integer representing the year of the initialization time - \end{itemize} -\item \execout{month} - \begin{itemize} - \item An integer representing the month of the initialization time - \end{itemize} -\item \execout{day} - \begin{itemize} - \item An integer representing the day of the initialization time - \end{itemize} -\item \execout{hour} - \begin{itemize} - \item An integer representing the hour of the initialization time - \end{itemize} -\item \execout{column\_area} - \begin{itemize} - \item A list of floating point values representing the characteristic horizontal domain area of each atmospheric column in square meters (this could be analogous to a 3D model's horizontal grid size or the characteristic horizontal scale of an observation array; these values are used in scale-aware schemes; if using multiple columns, you may specify an equal number of column areas) - \end{itemize} -\item \execout{model\_ics} - \begin{itemize} - \item A boolean set to \exec{.true.} if UFS atmosphere initial conditions are used rather than field campaign-based initial conditions - \end{itemize} -\item \execout{C\_RES} - \begin{itemize} - \item An integer representing the grid size of the UFS atmosphere initial conditions; the integer represents the number of grid points in each horizontal direction of each cube tile - \end{itemize} -\item \execout{input\_type} - \begin{itemize} - \item 0 => original DTC format, 1 => DEPHY-SCM format. - \end{itemize} -\end{itemize} - -Optional variables (that may be overridden via run script command line arguments) are: -\begin{itemize} -\item \execout{vert\_coord\_file} - \begin{itemize} - \item File containing FV3 vertical grid coefficients. - \end{itemize} -\item \execout{n\_levels} - \begin{itemize} - \item Specify the integer number of vertical levels. - \end{itemize} -\end{itemize} - -\subsection{Case input data file (CCPP-SCM format)} -\label{subsection: case input} - -The initialization and forcing data for each case is stored in a NetCDF (version 4) file within the \execout{ccpp-scm/scm/data/processed\_case\_input} directory. Each file has at least two dimensions (\execout{time} and \execout{levels}, potentially with additions for vertical snow and soil levels) and is organized into 3 groups: scalars, initial, and forcing. Not all fields are required for all cases. For example the fields \execout{sh\_flux\_sfc} and \execout{lh\_flux\_sfc} are only needed if the variable \execout{sfc\_flx\_spec} $=$ \exec{.true.} in the case configuration file and state nudging variables are only required if \execout{thermo\_forcing\_type} $=$ \exec{3} or \execout{mom\_forcing\_type} $=$ \exec{3}. Using an active LSM (Noah, NoahMP, RUC) requires many more variables than are listed here. Example files for using with Noah and NoahMP LSMs are included in \execout{ccpp-scm/scm/data/processed\_case\_input/fv3\_model\_point\_noah[mp].nc}. - -\lstinputlisting[ - basicstyle=\scriptsize\ttfamily, - label=lst_case_input_netcdf_header, - caption=example NetCDF file (CCPP-SCM format) header for case initialization and forcing data - ]{./arm_case_header.txt} - -\subsection{Case input data file (DEPHY format)} -\label{subsection: case input dephy} -The Development and Evaluation of Physics in atmospheric models (DEPHY) format is an internationally-adopted data format intended for use by SCM and LESs. The initialization and forcing data for each case is stored in a NetCDF (version 4) file, although these files are not by default included in the CCPP SCM repository. To access these cases you need to clone the DEPHY-SCM repository, and provide the DEPHY-SCM file location to the SCM. For example: -\begin{lstlisting}[language=bash] -cd [...]/ccpp-scm/scm/data -git clone https://github.com/GdR-DEPHY/DEPHY-SCM DEPHY-SCM -cd [...]/ccpp-scm/scm/bin -./run_scm.py -c MAGIC_LEG04A --case_data_dir [...]/ccpp-scm/scm/data/DEPHY-SCM/MAGIC/LEG04A -v -\end{lstlisting} -Each DEPHY file has three dimensions (\execout{time}, \execout{t0}, \execout{levels}) and contains the initial conditions (\execout{t0}, \execout{levels}) and forcing data (\execout{time}, \execout{levels}). Just as when using the CCPP-SCM formatted inputs, \ref{subsection: case input}, not all fields are required for all cases. More information on the DEPHY format requirements can be found at \href{https://github.com/GdR-DEPHY/DEPHY-SCM}{DEPHY}. - -\lstinputlisting[ - basicstyle=\scriptsize\ttfamily, - label=lst_case_input_netcdf_header, - caption=example NetCDF file (DEPHY format) header for case initialization and forcing data - ]{./dephy_case_header.txt} - -\section{Included Cases} -Several cases are included in the repository to serve as examples for users to create their own and for basic research. All case configuration namelist files for included cases can be found in \execout{ccpp-scm/scm/etc/case\_config} and represent the following observational field campaigns: -\begin{itemize} -\item Tropical Warm Pool -- International Cloud Experiment (TWP-ICE) maritime deep convection -\item Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) Summer 1997 continental deep convection -\item Atlantic Stratocumulus Transition EXperiment (ASTEX) maritime stratocumulus-to-cumulus transition -\item Barbados Oceanographic and Meteorological EXperiment (BOMEX) maritime shallow convection -\item Large eddy simulation ARM Symbiotic Simulation and Observation (LASSO) for May 18, 2016 (with capability to run all LASSO dates - see \ref{sec:lasso}) continental shallow convection -\end{itemize} -For the ARM SGP case, several case configuration files representing different time periods of the observational dataset are included, denoted by a trailing letter. The LASSO case may be run with different forcing applied, so three case configuration files corresponding to these different forcing are included. In addition, two example cases are included for using UFS Atmosphere initial conditions: -\begin{itemize} -\item UFS initial conditions for 38.1 N, 98.5 W (central Kansas) for 00Z on Oct. 3, 2016 with Noah variables on the C96 FV3 grid (\execout{fv3\_model\_point\_noah.nc}) -\item UFS initial conditions for 38.1 N, 98.5 W (central Kansas) for 00Z on Oct. 3, 2016 with NoahMP variables on the C96 FV3 grid (\execout{fv3\_model\_point\_noahmp.nc}) -\end{itemize} -See \ref{sec:UFSreplay} for information on how to generate these files for other locations and dates, given appropriate UFS Atmosphere initial conditions and output. - -\section{How to set up new cases} - -Setting up a new case involves preparing the two types of files listed above. For the case initialization and forcing data file, this typically involves writing a custom script or program to parse the data from its original format to the format that the SCM expects, listed above. An example of this type of script written in Python is included in \execout{/ccpp-scm/scm/etc/scripts/twpice\_forcing\_file\_generator.py}. The script reads in the data as supplied from its source, converts any necessary variables, and writes a NetCDF (version 4) file in the format described in subsections \ref{subsection: case input} and \ref{subsection: case input dephy}. For reference, the following formulas are used: -\begin{equation} -\theta_{il} = \theta - \frac{\theta}{T}\left(\frac{L_v}{c_p}q_l + \frac{L_s}{c_p}q_i\right) -\end{equation} -\begin{equation} -q_t = q_v + q_l + q_i -\end{equation} -where $\theta_{il}$ is the ice-liquid water potential temperature, $\theta$ is the potential temperature, $L_v$ is the latent heat of vaporization, $L_s$ is the latent heat of sublimation $c_p$ is the specific heat capacity of air at constant pressure, $T$ is absolute temperature, $q_t$ is the total water specific humidity, $q_v$ is the water vapor specific humidity, $q_l$ is the suspended liquid water specific humidity, and $q_i$ is the suspended ice water specific humidity. - -As shown in the example NetCDF header, the SCM expects that the vertical dimension is pressure levels (index 1 is the surface) and the time dimension is in seconds. The initial conditions expected are the height of the pressure levels in meters, and arrays representing vertical columns of $\theta_{il}$ in K, $q_t$, $q_l$, and $q_i$ in kg kg$^{-1}$, $u$ and $v$ in m s$^{-1}$, turbulence kinetic energy in m$^2$ s$^{-2}$ and ozone mass mixing ratio in kg kg$^{-1}$. - -For forcing data, the SCM expects a time series of the following variables: latitude and longitude in decimal degrees [in case the column(s) is moving in time (e.g., Lagrangian column)], the surface pressure (Pa) and surface temperature (K). If surface fluxes are specified for the new case, one must also include a time series of the kinematic surface sensible heat flux (K m s$^{-1}$) and kinematic surface latent heat flux (kg kg$^{-1}$ m s$^{-1}$). The following variables are expected as 2-dimensional arrays (vertical levels first, time second): the geostrophic u (E-W) and v (N-S) winds (m s$^{-1}$), and the horizontal and vertical advective tendencies of $\theta_{il}$ (K s$^{-1}$) and $q_t$ (kg kg$^{-1}$ s$^{-1}$), the large scale vertical velocity (m s$^{-1}$), large scale pressure vertical velocity (Pa s$^{-1}$), the prescribed radiative heating rate (K s$^{-1}$), and profiles of u, v, T, $\theta_{il}$ and $q_t$ to use for nudging. - -Although it is expected that all variables are in the NetCDF file, only those that are used with the chosen forcing method are required to be nonzero. For example, the following variables are required depending on the values of \execout{mom\_forcing\_type} and \execout{thermo\_forcing\_type} specified in the case configuration file: - -\begin{itemize} -\item \execout{mom\_forcing\_type} $=$ \exec{1} - \begin{itemize} - \item Not implemented yet - \end{itemize} -\item \execout{mom\_forcing\_type} $=$ \exec{2} - \begin{itemize} - \item geostrophic winds and large scale vertical velocity - \end{itemize} -\item \execout{mom\_forcing\_type} $=$ \exec{3} - \begin{itemize} - \item u and v nudging profiles - \end{itemize} -\item \execout{thermo\_forcing\_type} $=$ \exec{1} - \begin{itemize} - \item horizontal and vertical advective tendencies of $\theta_{il}$ and $q_t$ and prescribed radiative heating (can be zero if radiation scheme is active) - \end{itemize} -\item \execout{thermo\_forcing\_type} $=$ \exec{2} - \begin{itemize} - \item horizontal advective tendencies of $\theta_{il}$ and $q_t$, prescribed radiative heating (can be zero if radiation scheme is active), and the large scale vertical pressure velocity - \end{itemize} -\item \execout{thermo\_forcing\_type} $=$ \exec{3} - \begin{itemize} - \item $\theta_{il}$ and $q_t$ nudging profiles and the large scale vertical pressure velocity - \end{itemize} -\end{itemize} - -For the case configuration file, it is most efficient to copy an existing file in \execout{ccpp-scm/scm/etc/case\_config} and edit it to suit one's case. Recall from subsection \ref{subsection: case config} that this file is used to configure the SCM framework parameters for a given case. Be sure to check that model timing parameters such as the time step and output frequency are appropriate for the physics suite being used. There is likely some stability criterion that governs the maximum time step based on the chosen parameterizations and number of vertical levels (grid spacing). The \execout{case\_name} parameter should match the name of the case input data file that was configured for the case (without the file extension). The \execout{runtime} parameter should be less than or equal to the length of the forcing data unless the desired behavior of the simulation is to proceed with the last specified forcing values after the length of the forcing data has been surpassed. The initial date and time should fall within the forcing period specified in the case input data file. If the case input data is specified to a lower altitude than the vertical domain, the remainder of the column will be filled in with values from a reference profile. There is a tropical profile and mid-latitude summer profile provided, although one may add more choices by adding a data file to \execout{ccpp-scm/scm/data/processed\_case\_input} and adding a parser section to the subroutine \execout{get\_reference\_profile} in \execout{-scm/scm/src/scm\_input.f90}. Surface fluxes can either be specified in the case input data file or calculated using a surface scheme using surface properties. If surface fluxes are specified from data, set \execout{sfc\_flux\_spec} to \exec{.true.} and specify \execout{sfc\_roughness\_length\_cm} for the surface over which the column resides. Otherwise, specify a \execout{sfc\_type}. In addition, one must specify a \execout{column\_area} for each column. - -To control the forcing method, one must choose how the momentum and scalar variable forcing are applied. The three methods of Randall and Cripe (1999, JGR) have been implemented: ``revealed forcing'' where total (horizontal $+$ vertical) advective tendencies are applied (type 1), ``horizontal advective forcing'' where horizontal advective tendencies are applied and vertical advective tendencies are calculated from a prescribed vertical velocity and the calculated (modeled) profiles (type 2), and ``relaxation forcing'' where nudging to observed profiles replaces horizontal advective forcing combined with vertical advective forcing from prescribed vertical velocity (type 3). If relaxation forcing is chosen, a \execout{relax\_time} that represents the timescale over which the profile would return to the nudging profiles must be specified. - -\section{Using other LASSO cases} -\label{sec:lasso} - -In order to use other LASSO cases than the one provided, perform the following steps: -\begin{enumerate} -\item Access \url{http://archive.arm.gov/lassobrowser} and use the navigation on the left to choose the dates for which you would like to run a SCM simulation. Pay attention to the ``Large Scale Forcing'' tab where you can choose how the large scale forcing was generated, with options for ECMWF, MSDA, and VARANAL. All are potentially valid, and it is likely worth exploring the differences among forcing methods. Click on Submit to view a list of simulations for the selected criteria. Choose from the simulations (higher skill scores are preferred) and check the ``Config Obs Model Tar'' box to download the data. Once the desired simulations have been checked, order the data (you may need to create an ARM account to do so). -\item Once the data is downloaded, decompress it. From the \execout{config} directory, copy the files \execout{input\_ls\_forcing.nc}, \execout{input\_sfc\_forcing.nc}, and \execout{wrfinput\_d01.nc} into their own directory under \execout{ccpp-scm/scm/data/raw\_case\_input/}. -\item Modify \execout{ccpp-scm/scm/etc/scripts/lasso1\_forcing\_file\_generator\_gjf.py} to point to the input files listed above. Execute the script in order to generate a case input file for the SCM (to be put in \execout{ccpp-scm/scm/data/processed\_case\_input/}): -\begin{lstlisting}[language=bash] -./lasso1_forcing_file_generator_gjf.py -\end{lstlisting} -\item Create a new case configuration file (or copy and modify an existing one) in \execout{ccpp-scm/scm/etc/case\_config}. Be sure that the \execout{case\_name} variable points to the newly created/processed case input file from above. -\end{enumerate} - -\section{Using UFS Output to Create SCM Cases: UFS-Replay} -\label{sec:UFSreplay} - - -\subsection{Python Dependencies} -\label{subsection: pydepend} -The scripts here require a few python packages that may not be found by default in all python installations. There is a YAML file with the python environment needed to run the script in \execout{ccpp-scm/environment-ufsreplay.yml}. To create and activate this environment using conda: - -Create environment (only once): - -\execout{> conda env create -f environment-ufsreplay.yml} - -This will create the conda environment \execout{env\_ufsreplay} - - -Activate environment: - -\execout{> conda activate env\_ufsreplay} - - -\subsection{UFS\_IC\_generator.py} -\label{subsection: ufsicgenerator} -A script exists in \execout{scm/etc/scripts/UFS\_IC\_generator.py} to read in UFS history (output) files and their initial conditions to generate a SCM case input data file, in DEPHY format. - - -\begin{lstlisting}[language=bash] -./UFS_IC_generator.py [-h] (-l LOCATION LOCATION | -ij INDEX INDEX) -d -DATE -i IN_DIR -g GRID_DIR -f FORCING_DIR -n -CASE_NAME [-t {1,2,3,4,5,6,7}] [-a AREA] [-oc] -[-lam] [-sc] [-near] -\end{lstlisting} - -Mandatory arguments: -\begin{enumerate} -\item \exec{-{}-location (-l)} OR \exec{-{}-index (-ij)}: Either longitude and latitude in decimal degrees east and north of a location OR the UFS grid index with the tile number - \begin{itemize} - \item -l 261.51 38.2 (two floating point values separated by a space) - \item -ij 8 49 (two integer values separated by a space; this option must also use the \exec{-{}-tile (-t)} argument to specify the tile number) - \end{itemize} -\item \exec{-{}-date (-d)} YYYYMMDDHHMMSS: date corresponding to the UFS initial conditions -\item \exec{-{}-in\_dir (-i)}: path to the directory containing the UFS initial conditions -\item \exec{-{}-grid\_dir (-g)}: path to the directory containing the UFS supergrid files (AKA "fix" directory) -\item \exec{-{}-forcing\_dir (-f)}: path to the directory containing the UFS history files -\item \exec{-{}-case\_name (-n)}: name of case -\end{enumerate} - -Optional arguments: -\begin{enumerate} -\item \exec{-{}-tile (-t)}: if one already knows the correct tile for the given longitude and latitude OR one is specifying the UFS grid index (\exec{-{}-index} argument) -\item \exec{-{}-area (-a)}: area of grid cell in $m^2$ (if known or different than the value calculated from the supergrid file) -\item \exec{-{}-old\_chgres (-oc)}: flag if UFS initial conditions were generated using older version of chgres (global\_chgres); might be the case for pre-2018 data -\item \exec{-{}-lam (-lam)}: flag to signal that the ICs and forcing is from a limited-area model run -\item \exec{-{}-save\_comp (-sc)}: flag to create UFS reference file for comparison -\item \exec{-{}-use\_nearest (-near)}: flag to indicate using the nearest UFS history file gridpoint -\end{enumerate} - -\subsection{UFS\_forcing\_ensemble\_generator.py} -\label{subsection: ufsforcingensemblegenerator} -There is an additional script in \execout{scm/etc/scripts/UFS\_forcing\_ensemble\_generator.py} to create UFS-replay case(s) starting with output from UFS Weather Model (UWM) Regression Tests (RTs). - -\begin{lstlisting}[language=bash] -UFS_forcing_ensemble_generator.py [-h] -d DIR -n CASE_NAME -(-lonl LON_1 LON_2 -latl LAT_1 LAT_2 -nens NENSMEMBERS | --lons [LON_LIST] -lats [LAT_LIST]) -[-dt TIMESTEP] [-cres C_RES] [-sdf SUITE] [-sc] [-near] -\end{lstlisting} - -Mandatory arguments: -\begin{enumerate} -\item \exec{-{}-dir (-d)}: path to UFS Regression Test output -\item \exec{-{}-case\_name (-n)}: name of cases -\item Either: (see examples below) - \begin{itemize} - \item \exec{-{}-lon\_limits (-lonl)} AND \exec{-{}-lat\_limits (-latl)} AND \exec{-{}-nensmembers (-nens)}: longitude range, latitude range, and number of cases to create - \item \exec{-{}-lon\_list (-lons)} AND \exec{-{}-lat\_list (-lats)}: longitude and latitude of cases - \end{itemize} -\end{enumerate} - -Optional arguments: -\begin{enumerate} -\item \exec{-{}-timestep (-dt)}: SCM timestep, in seconds -\item \exec{-{}-C\_res (-cres)}: UFS spatial resolution -\item \exec{-{}-suite (-sdf)}: CCPP suite definition file to use for ensemble -\item \exec{-{}-save\_comp (-sc)}: flag to create UFS reference file for comparison -\item \exec{-{}-use\_nearest (-near)}: flag to indicate using the nearest UFS history file gridpoint -\end{enumerate} - -Examples to run from within the \exec{scm/etc/scripts} directory to create SCM cases starting with the output from a UFS Weather Model regression test(s): - -On the supported platforms Cheyenne (NCAR) and Hera (NOAA), there are staged UWM RTs located at: -\begin{itemize} -\item \execout{Cheyenne /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS\_RTs} -\item \execout{Hera /scratch1/BMC/gmtb/CCPP-SCM/UFS\_RTs} -\end{itemize} - -\subsection{Example 1: UFS-replay for single point} -\label{subsection: example1} - -UFS regression test, \execout{control\_c192}, for single point. -\begin{lstlisting}[language=bash] -./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_c192/ -sc --C_RES 192 -dt 360 -n control_c192 -lons 300 -lats 34 -\end{lstlisting} - -Upon successful completion of the script, the command to run the case(s) will print to the screen. For example, - -\begin{lstlisting}[language=bash] -./run_scm.py --npz_type gfs --file scm_ufsens_control_c192.py --timestep 360 -\end{lstlisting} - -The file \exec{scm\_ufsens\_control\_c192.py} is created in \exec{ccpp-scm/scm/bin/}, where the SCM run script is to be exectued. - -\subsection{Example 2: UFS-replay for list of points} -\label{subsection: example2} - -UFS regression test, \execout{control\_c384}, for multiple points. -\begin{lstlisting}[language=bash] -./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_c384/ -sc --C_RES 384 -dt 225 -n control_c384 -lons 300 300 300 300 -lats 34 35 35 37 -\end{lstlisting} - -Upon successful completion of the script, the command to run the case(s) will print to the screen. For example, - -\begin{lstlisting}[language=bash] -./run_scm.py --npz_type gfs --file scm_ufsens_control_c384.py --timestep 225 -\end{lstlisting} - -The file \exec{scm\_ufsens\_control\_c384.py} contains \exec{ALL} of the cases created. Each case created will have the naming convention \exec{case\_name\_nXXX}, where the suffix \exec{XXX} is the case number from 0 to the number of points provided. The contents of the file should look like: -\begin{lstlisting}[language=bash] -run_list = [{"case": "control_c384_n000", "suite": "SCM_GFS_v16"}, - {"case": "control_c384_n001", "suite": "SCM_GFS_v16"}, - {"case": "control_c384_n002", "suite": "SCM_GFS_v16"}, - {"case": "control_c384_n003", "suite": "SCM_GFS_v16"}] -\end{lstlisting} - -\subsection{Example 3: UFS-replay for an ensemble of points} -\label{subsection: example3} - -UFS regression test, \execout{control\_p8}, for an ensemble (10) of randomly selected points over a specified longitude ($300-320^oW$) and latitude ($40-50^oN$) range - -But first, to use the \execout{control\_p8} test we need to rerun the regression test to generate UFS history files with a denser and constant output interval. First, in \execout{control\_p8/model\_configure}, change \exec{-{}-output\_fh} to \execout{"interval -1"}, where \execout{interval} is the UFS history file output frequency (in hours), see \href{https://ufs-weather-model.readthedocs.io/en/latest/InputsOutputs.html}{UFS Weather Model Users Guide} for more details. - -For the purposes of this example the \execout{control\_p8} test has already been rerun, but if starting from your own UWM RTs, you can rerun the UWM regression test, on Cheyenne for example, by running the following command in the RT directory: \execout{qsub job\_card} - -Now the cases can be generated with the following command: -\begin{lstlisting}[language=bash] -./UFS_forcing_ensemble_generator.py -d /glade/scratch/epicufsrt/GMTB/CCPP-SCM/UFS_RTs/control_p8/ -sc --C_RES 96 -dt 720 -n control_p8 -lonl 300 320 -latl 40 50 -nens 10 -sdf SCM_GFS_v17_p8 -\end{lstlisting} - -Upon successful completion of the script, the command to run the case(s) will print to the screen. For example, - -\begin{lstlisting}[language=bash] -./run_scm.py --npz_type gfs --file scm_ufsens_control_p8.py --timestep 720 -\end{lstlisting} - -The file \exec{scm\_ufsens\_control\_p8.py} contains ten cases (n000-n009) to be run. The contents of the file should look like: -\begin{lstlisting}[language=bash] -run_list = [{"case": "control_p8_n000", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n001", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n002", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n003", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n004", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n005", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n006", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n007", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n008", "suite": "SCM_GFS_v17_p8"}, - {"case": "control_p8_n009", "suite": "SCM_GFS_v17_p8"}] -\end{lstlisting} diff --git a/scm/doc/TechGuide/chap_ccpp.rst b/scm/doc/TechGuide/chap_ccpp.rst new file mode 100644 index 000000000..1f4e9f829 --- /dev/null +++ b/scm/doc/TechGuide/chap_ccpp.rst @@ -0,0 +1,383 @@ +.. _`chapter: ccpp_interface`: + +CCPP Interface +============== + +Chapter 6 of the CCPP v6 Technical Documentation +(https://ccpp-techdoc.readthedocs.io/en/v6.0.0/) provides a wealth of +information on the overall process of connecting a host model to the +CCPP framework for calling physics. This chapter describes the +particular implementation within this SCM, including how to set up, +initialize, call, and change a physics suite using the CCPP framework. + +Setting up a suite +------------------ + +Setting up a physics suite for use in the CCPP SCM with the CCPP +framework involves three steps: preparing data to be made available to +physics through the CCPP, running the ``ccpp_prebuild.py`` script to reconcile SCM-provided +variables with physics-required variables, and preparing a suite +definition file. + +Preparing data from the SCM +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described in sections 6.1 and 6.2 of the `CCPP Technical +Documentation `__ a host +model must allocate memory and provide metadata for variables that are +passed into and out of the schemes within the physics suite. As of this +release, in practice this means that a host model must do this for all +variables needed by all physics schemes that are expected to be used +with the host model. For this SCM, all variables needed by the physics +schemes are allocated and documented in the file ``ccpp-scm/scm/src/scm_type_defs.f90`` and are contained +within the ``physics`` derived data type. This derived data type initializes its +component variables in a ``create`` type-bound procedure. As mentioned in section +6.2 of the `CCPP Technical +Documentation `__, files +containing all required metadata was constructed for describing all +variables in the derived data type. These files are ``scm/src/GFS_typedefs.meta,``, ``scm/src/CCPP_typedefs.meta``, and ``scm_physical_constants.meta``. Further, ``scm_type_defs.meta`` +exists to provide metadata for derived data type definitions and their +instances, which is needed by the ccpp-framework to properly reference +the data. The standard names of all variables in this table must match +with a corresponding variable within one or more of the physics schemes. +A list of all standard names used can be found in ``ccpp/framework/doc/DevelopersGuide/CCPP_VARIABLES_SCM.pdf``. + +Editing and running ``ccpp_prebuild.py`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +General instructions for configuring and running the ``ccpp_prebuild.py`` script can be found +in chapter 8 of the `CCPP Technical +Documentation `__. The +script expects to be run with a host-model-dependent configuration file, +passed as argument ``–config=path_to_config_file``. Within this configuration file are variables that +hold paths to the variable definition files (where metadata tables can +be found on the host model side), the scheme files (a list of paths to +all source files containing scheme entry points), the auto-generated +physics schemes makefile snippet, the auto-generated physics scheme caps +makefile snippet, and the directory where the auto-generated physics +caps should be written out to. As mentioned in :numref:`Section %s `, this script must be run +to reconcile data provided by the SCM with data required by the physics +schemes before compilation – this is done automatically by ``cmake``. + +Preparing a suite definition file +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The suite definition file is a text file read by the model at compile +time. It is used to specify the physical parameterization suite, and +includes information about the number of parameterization groupings, +which parameterizations that are part of each of the groups, the order +in which the parameterizations should be run, and whether subcycling +will be used to run any of the parameterizations with shorter timesteps. + +In addition to the six or so major parameterization categories (such as +radiation, boundary layer, deep convection, resolved moist physics, +etc.), the suite definition file can also have an arbitrary number of +additional interstitial schemes in between the parameterizations to +prepare or postprocess data. In many models, this interstitial code is +not known to the model user but with the suite definition file, both the +physical parameterizations and the interstitial processing are listed +explicitly. + +For this release, supported suite definition files used with this SCM +are found in ``ccpp-scm/ccpp/suites`` and have default namelist, tracer configuration, and +timesteps attached in ``ccpp-scm/scm/src/suite_info.py``. For all of these suites, the physics schemes +have been organized into 3 groupings following how the physics are +called in the UFS Atmosphere model, although no code is executed in the +SCM time loop between execution of the grouped schemes. Several +“interstitial” schemes are included in the suite definition file to +execute code that previously was part of a hard-coded physics driver. +Some of these schemes may eventually be rolled into the schemes +themselves, improving portability. + +Initializing/running a suite +---------------------------- + +The process for initializing and running a suite in this SCM is +described in sections +:numref:`%s ` and :numref:`%s