Skip to content

Commit

Permalink
Merge pull request #151 from MetOffice/documentation-restructure
Browse files Browse the repository at this point in the history
Documentation restructure
  • Loading branch information
jfrost-mo authored Jul 11, 2023
2 parents 6e8af27 + fff3fc4 commit 732d5e2
Show file tree
Hide file tree
Showing 23 changed files with 257 additions and 205 deletions.
Empty file removed docs/source/_static/PLACEHOLDER
Empty file.
Empty file removed docs/source/_templates/PLACEHOLDER
Empty file.
10 changes: 10 additions & 0 deletions docs/source/background/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Background Information
======================

This section contains explanations and discussions to give background and
context to parts of CSET, such as why something is the way it is.

.. toctree::
:maxdepth: 2

why-cset
22 changes: 22 additions & 0 deletions docs/source/background/why-cset.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
Why use CSET?
=============

When evaluating weather models you are trying to understand the processes that
lead to biases in our model configurations, and how that compares to other
models and observations. This can be a very iterative process, and each step of
evaluation unveils more questions that need investigations.

CSET aids in this by providing a flexible way to interrogate model data, using
diagnostics that can be quickly created by the combination of operators in
:doc:`/usage/operator-recipes`. Common operations such as reading, writing, and
regridding are provided to reduce duplication of effort.

CSET is a centralised place for custom diagnostics to live, with well defined
:doc:`/contributing/index` to ensure that they stay maintained. By contributing
diagnostics to CSET you ensure that they outlive the paper they were written
for, and benefit the entire modelling community. Furthermore, any operators that
the community finds especially useful will be contributed back to
METplus_, so they can serve an
even wider community.

.. _METplus: https://dtcenter.org/community-code/metplus
8 changes: 2 additions & 6 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,14 @@
"sphinx.ext.doctest",
]

templates_path = ["_templates"]
exclude_patterns = []

# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = "furo"
html_static_path = ["_static"]

# -- LaTeX output configuration ----------------------------------------------
# https://www.sphinx-doc.org/en/master/latex.html

# latex_engine = 'xelatex'
latex_engine = "xelatex"
latex_elements = {"papersize": "a4paper"}
# latex_show_urls = 'footnote'
latex_show_urls = "footnote"
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ maintain documentation within the version control system, and keep it up to
date. The `Sphinx website`_ has a useful primer to using reStructuredText for
documentation.

The documentation is organised intro sections following the `Divio documentation
system`_.

You can build the documentation of CSET with the following command:

.. code-block:: bash
Expand All @@ -18,6 +21,8 @@ You can build the documentation of CSET with the following command:
The documentation will be output to the docs/build directory.

.. _Divio documentation system: https://documentation.divio.com/

Docstrings
----------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,28 +1,12 @@
Working Practices
=================

To aid in the development of CSET we are adopting some working practices so
everyone involved knows what to expect. On this page is everything you need to
get started, and the links below go into more detail on specific topics.

.. toctree::
:maxdepth: 2

git
testing
documentation
code-review
releases

.. _working_practices_getting_started:

Getting Started
---------------
===============

Before you can get to coding, there are a few steps you need to do.

Getting the code
~~~~~~~~~~~~~~~~
----------------

Git is a version control software, which makes collaborating on software much
easier. If you are new to git have a look at the :doc:`git` page to get started.
Expand All @@ -44,7 +28,7 @@ of its history onto your computer. From there we need to setup a few things
before we can develop.

Setting up tooling
~~~~~~~~~~~~~~~~~~
------------------

To be able to run the tests or build the documentation you need some
prerequisite software. The easiest way to get this is with `conda_`. The
Expand Down
File renamed without changes.
16 changes: 16 additions & 0 deletions docs/source/contributing/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Contributing to CSET
====================

To aid in the development of CSET we are adopting some working practices so
everyone involved knows what to expect. Linked from this page is everything you
need to get started, and the links below go into more detail on specific topics.

.. toctree::
:maxdepth: 2

getting-started
git
testing
documentation
code-review
releases
File renamed without changes
File renamed without changes.
File renamed without changes.
10 changes: 10 additions & 0 deletions docs/source/getting-started/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Getting Started
===============

This section contains tutorials suitable for someone getting started with CSET.
By following them you should learn the basics of using CSET.

.. toctree::
:maxdepth: 2

installation
15 changes: 15 additions & 0 deletions docs/source/getting-started/installation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Installation
============

.. Tutorial saying how to install CSET. For edge cases should link elsewhere.
The recommended way to install CSET is via conda. It is packaged on
`conda-forge`_ and can be installed with a simple ``conda install -c conda-forge
cset``.

If you instead want to run a development version that has yet to be released,
the easiest way is via an editable install. You can learn how to do this in the
:ref:`working_practices_getting_started` section of the
:doc:`/contributing/index`.

.. _conda-forge: https://anaconda.org/conda-forge/cset
52 changes: 20 additions & 32 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,44 +4,32 @@ CSET Documentation
.. toctree::
:hidden:

self
getting-started/index
usage/index
working-practices/index
operators
genindex
GitHub <https://github.com/MetOffice/CSET>

:abbr:`CSET (Convective Scale Evaluation Tool)` is a tool to aid in verifying
and evaluating convective-scale and turbulence-scale (regional and increasingly
global) model configurations. It aims to replace the RMED RES and Toolbox and
the collections of bespoke scripts littering people's home directories, reducing
effort wasted on duplicating already existing code. This centralisation of
diagnostics should also make evaluations more consistent, reproducible and
comparable. Development takes place in the `CSET repository on GitHub`_.
reference/index
background/index
contributing/index

:abbr:`CSET (Acronym meaning contested)` is a tool to aid in verifying and
evaluating :abbr:`ESM (Earth System Model)` configurations. It aims to replace
the :abbr:`RES (Regional Evaluation Suite)`, :abbr:`RMED (Regional Model
Evaluation & Development)` Toolbox and the collections of bespoke scripts
littering people's home directories. By centralising these in CSET the effort
spent on duplicate code can be reduced, and give a chance for code review to
catch any bugs. This will make evaluations more consistent, reproducible and
comparable. For more, see :doc:`/background/why-cset`

Use the side bar to the left to access other pages of the documentation.

Why use CSET?
-------------
Useful Links
------------

When evaluating weather models you are trying to understand the processes that
lead to biases in our model configurations, and how that compares to other
models and observations. This can be a very iterative process, and each step of
evaluation unveils more questions that need investigations.
`Source Code`_ | `Issue Tracker`_ | Releases_

CSET aids in this by providing a flexible way to interrogate model data, using
diagnostics that can be quickly created by the combination of :doc:`operators`
in an :doc:`usage/operator-recipes`. Common operations such as reading, writing,
and regridding are provided to reduce duplication of effort.

CSET is a centralised place for custom diagnostics to live, with well defined
:doc:`working-practices/index` to ensure that they stay maintained. By
contributing diagnostics to CSET you ensure that they outlive the paper they
were written for, and benefit the entire modelling community. Furthermore, any
operators that the community finds especially useful will be contributed back to
[METplus](https://dtcenter.org/community-code/metplus), so they can serve an
even wider community.

.. _CSET repository on GitHub: https://github.com/MetOffice/CSET
.. _Source Code: https://github.com/MetOffice/CSET
.. _Issue Tracker: https://github.com/MetOffice/CSET/issues
.. _Releases: https://github.com/MetOffice/CSET/releases

Licence
-------
Expand Down
71 changes: 71 additions & 0 deletions docs/source/reference/cli.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
CLI Usage
=========

.. _cset-bake-command:

cset bake
~~~~~~~~~

Used to run a recipe on some data. The recipe, input file, and a path to an
output file must be provided. Additional help is available with the ``--help``
option. The recipe format is described on the :doc:`/usage/operator-recipes` page. If
recipe_file is omitted from the command line, the recipe will be read from the
``CSET_RECIPE`` environment variable instead. This environment variable should
contain the YAML text that you want executed.

.. code-block:: text
usage: cset bake [-h] input_file output_file [recipe_file]
positional arguments:
input_file input file to read
output_file output file to write
recipe_file recipe file to execute. If omitted reads from CSET_RECIPE environment variable
options:
-h, --help show this help message and exit
.. _cset-cookbook-command:

cset cookbook
~~~~~~~~~~~~~

Saves the included recipe files to a local directory. This allows access to
pre-created recipes for many common tasks. See [LINK TO DOCUMENTATION PAGE ON
RECIPES] for descriptions of available recipes.

.. code-block:: text
usage: cset cookbook [-h] [recipe_dir]
positional arguments:
recipe_dir directory to save recipes. If omitted uses $PWD/recipes
options:
-h, --help show this help message and exit
.. _cset-graph-command:

cset graph
~~~~~~~~~~

Visualise a recipe. This renders a graph representing the recipe, and saves it
to a temporary file, which is automatically opened. If the ``--detailed`` option
is specified it also includes operator arguments in the output, so you can see
exactly what ran.

.. image:: recipe-graph.svg
:alt: A graph visualising a recipe. The nodes are linked with directed edges showing the flow of data.

.. code-block:: text
usage: cset graph [-h] [-o [OUTPUT_PATH]] [-d] [recipe]
positional arguments:
recipe recipe file to read. If omitted reads from CSET_RECIPE environment variable
options:
-h, --help show this help message and exit
-o [OUTPUT_PATH], --output_path [OUTPUT_PATH]
file in which to save the graph image, otherwise uses a temporary file. When specified the file is not automatically opened
-d, --detailed include operator arguments in output
12 changes: 12 additions & 0 deletions docs/source/reference/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Technical Reference
===================

This section contains comprehensive descriptions of CSET, and its various
components.

.. toctree::
:maxdepth: 2

cli
operators
recipe-format
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ CSET Operators

This page details the operators contained within CSET. It is automatically
generated from the code and its docstrings. Operators should be used via
:doc:`usage/operator-recipes`.
:doc:`/usage/operator-recipes`.

CSET.operators.constraints
--------------------------
Expand All @@ -17,7 +17,6 @@ CSET.operators.filters
.. automodule:: CSET.operators.filters
:members:


CSET.operators.misc
-------------------

Expand Down
60 changes: 60 additions & 0 deletions docs/source/reference/recipe-format.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
Recipe Format
=============

The recipes are text files written in `YAML 1.2`_, a configuration language that
is widely used. They are saved with the ``.yaml`` extension. Below is a
commented example recipe:

.. code-block:: yaml
name: Name of recipe
description: |
Extended description that can
go across multiple lines.
steps:
# Specify the operator to run in each step.
- operator: read.read_cubes
- operator: filters.filter_cubes
# Can specify extra keyword arguments as sub-maps.
constraint:
# Can nest in another operator to use its output as an argument.
operator: generate_constraints.generate_stash_constraints
# Input implicitly taken from the previous step, but can be overridden
# by using the appropriate keyword argument.
stash: m01s03i236
- operator: write.write_cube_to_nc
# Specify the name of the argument, and its value.
file_path: CSET_OUTPUT_PATH
# "CSET_OUTPUT_PATH" is special and becomes the runtime output file path.
The name and description keys provide a human readable description of what the
recipe does. They are currently not used anywhere, but it is good practice to
include them for future users of the recipe.

The steps keys specifies a list of processing steps. The steps are run from top
to bottom, with each step specifying an operator to run, and optionally any
additional inputs to that operator. A step is denoted by a ``-`` under the
``steps:`` key. The operators are specified on the operator key. Its value
should be a string of the form ``file.function``. For additional inputs the key
should be the name of the argument.

The below code block shows how you can nest operators multiple levels deep. For
details of the specific operators involved, and the arguments that can take, see
the :doc:`/reference/operators` page.

.. code-block:: yaml
- operator: filters.filter_cubes
constraint:
operator: constraints.combine_constraints
constraint:
operator: constraints.generate_stash_constraint
stash: m01s03i236
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
.. _YAML 1.2: https://yaml.org/
File renamed without changes
Loading

0 comments on commit 732d5e2

Please sign in to comment.