Update documentation to reflect changes

docs/source/contributing/code-review.rst

@@ -194,13 +194,11 @@ For a plotting routine, specific questions to consider include:
 * Is the labelling present and appropriate?
 * Is the plot legible?
 
-
 Portability Review
 ------------------
 
 .. attention::
 
     🚧 Section under construction. 🚧
 
-
 Something about the portability review…

docs/source/contributing/documentation.rst

@@ -10,8 +10,8 @@ maintain documentation within the version control system, and keep it up to
 date. The `Sphinx website`_ has a useful primer to using reStructuredText for
 documentation.
 
-The documentation is organised intro sections following the `Divio documentation
-system`_.
+The documentation is organised intro sections following the `Diátaxis
+documentation system`_.
 
 You can build the documentation of CSET with the following command:
 
@@ -21,7 +21,7 @@ You can build the documentation of CSET with the following command:
 
 The documentation will be output to the docs/build directory.
 
-.. _Divio documentation system: https://documentation.divio.com/
+.. _Diátaxis documentation system: https://diataxis.fr/
 
 Docstrings
 ----------

docs/source/getting-started/run-recipe.rst

@@ -1,32 +1,24 @@
 Run a pre-existing recipe
 =========================
 
-.. Tutorial on running a pre-existing recipe, covering cookbook and bake.
+.. Tutorial on running a pre-existing recipe, covering cset bake.
 
 CSET works by running recipes that describe how to transform and and visualise
 data. It comes with a collection of pre-written recipes to get you started. In
 this tutorial you will use CSET to plot the mean surface air temperature of a
 forecast.
 
-We will create a basic spatial plot of the mean surface air
-temperature. There is a pre-existing recipe for this that can be retrieved with
-the CSET cookbook command.
-
-Try the following:
-
-.. code-block:: bash
-
-    cset cookbook -o recipes mean_surface_air_temperature_spatial_plot.yaml
-
-This will write out a recipes folder containing recipe ``.yaml`` file to your
-current directory. We will use the
-``mean_surface_air_temperature_spatial_plot.yaml`` recipe.
+We will create a basic spatial plot of the mean surface air temperature, using a
+pre-existing recipe. You can `download the recipe file here`_.
 
 Now you need to find some data to process. You can `download an example file
 here`_, or with the following command.
 
 .. code-block:: bash
 
+    # Download recipe file.
+    curl -LO https://gist.githubusercontent.com/jfrost-mo/6e539d5be20bfa28342bf4ff82f24dea/raw/air_temperature_spatial_plot.yaml
+    # Download example data.
     curl -LO https://github.com/MetOffice/CSET/raw/main/tests/test_data/air_temp.nc
 
 Now we are ready to run our recipe. This is where we use the ``cset bake``
@@ -35,7 +27,7 @@ output should be a directory, but it will be created if it does not exist.
 
 .. code-block:: bash
 
-    cset bake -i air_temp.nc -o output/ -r recipes/mean_surface_air_temperature_spatial_plot.yaml
+    cset bake -i air_temp.nc -o output/ -r air_temperature_spatial_plot.yaml
 
 This will run the recipe and leave its output in the specified output directory.
 
@@ -45,4 +37,5 @@ The most interesting output will be the plot, which you can look at with
 You've now successfully run CSET with a pre-existing recipe. In the next
 tutorial we will see what is going on inside.
 
+.. _download the recipe file here: https://gist.githubusercontent.com/jfrost-mo/6e539d5be20bfa28342bf4ff82f24dea/raw/air_temperature_spatial_plot.yaml
 .. _download an example file here: https://github.com/MetOffice/CSET/raw/main/tests/test_data/air_temp.nc

docs/source/getting-started/visualise-recipe.rst

@@ -6,15 +6,14 @@ Visualising a recipe graphically
 In this tutorial we will investigate what is going on inside of a recipe, and
 visualise the *operators* inside.
 
-As in the previous tutorial use the ``cset cookbook`` command to find the "Mean
-Air Temperature Spatial Plot" recipe.
+As in the previous tutorial `download this example recipe file`_.
 
 We will now visualise the steps inside the recipe using the ``cset graph``
 command.
 
 .. code-block:: bash
 
-    cset graph -r recipes/mean-air-temp-spatial-plot.yaml
+    cset graph -r air_temperature_spatial_plot.yaml
 
 This should open an image of a visualisation of the recipe. Each node is a step,
 or an *operator*, which does a single processing task. You can see that later
@@ -38,20 +37,20 @@ Now we can see the structure of the recipe graphically, we can delve into what
 each operator is doing. The ellipses represent the operators, and the arrows
 between them show where they pass their output to the next operators.
 
-The first operator in the recipe is ``read.read_cubes``, however it takes a
-constraint on a STASH code, which is itself created by another operator,
-``constraints.generate_stash_constraint``.
+The first operator in the recipe is ``read.read_cubes``, which loads the data
+cubes from a file into a CubeList, which it passes onto the next step.
 
 This operators-running-operators behaviour is further used in the next step,
 where the read CubeList is filtered down to a single air temperature cube. There
-are two constraints used here, the STASH code, and the cell methods. These are
-combined into a single constraint by the ``constraints.combine_constraints``
-operator before being used by the ``filters.filter_cubes`` operator.
+are two constraints used here, the variable's STASH code, and the cell methods.
+These are combined into a single constraint by the
+``constraints.combine_constraints`` operator before being used by the
+``filters.filter_cubes`` operator.
 
-Afterwards the cube has its time dimension removed by the mean method applied by
-the ``collapse.collapse`` operator, so it becomes two-dimensional. Then it
-passes to the ``plot.spatial_contour_plot`` and ``write.write_cube_to_nc``
-operators to be plotted and saved.
+Afterwards the cube passes to the ``plot.spatial_contour_plot`` and
+``write.write_cube_to_nc`` operators to be plotted and saved.
 
 You now know how to visualise a recipe, and a little about the operators it is
-made up of. In the next tutorial you will learn to make your own.
+made up of. In the next tutorial you will learn to make your own recipe.
+
+.. _download this example recipe file: https://gist.githubusercontent.com/jfrost-mo/6e539d5be20bfa28342bf4ff82f24dea/raw/air_temperature_spatial_plot.yaml

docs/source/reference/cli.rst

@@ -57,7 +57,7 @@ or use ``--details`` for descriptions of available recipes.
 
     options:
     -h, --help            show this help message and exit
-    -d, --details         list available recipes. Supplied recipes are detailed.
+    -d, --details         list available recipes. Supplied recipes are detailed
     -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                             directory to save recipes. If omitted uses $PWD
 

docs/source/reference/recipe-format.rst

@@ -12,62 +12,60 @@ Below is a commented example recipe:
 
 .. code-block:: yaml
 
-    category: Category of recipe
-    title: Name of recipe
-    description: |
-      Extended description that can
-      go across multiple lines.
-
-    parallel:
-      # Specify the operator to run in each step.
-      - operator: read.read_cubes
-
-      - operator: filters.filter_cubes
-        # Can specify extra keyword arguments as sub-maps.
-        constraint:
-          # Can nest in another operator to use its output as an argument.
-          operator: generate_constraints.generate_stash_constraints
+  # Name of the recipe.
+  title: Surface air temperature spatial plot
+  # Category of recipe, used to group together multiple recipes in output.
+  category: Quick look
+  # Description will be displayed alongside output.
+  description: |
+    Extended description that can go across multiple lines. It is written in
+    [Markdown](https://commonmark.org/help/) and can thus contain links and
+    _formatting_.
+
+  # Sequence of steps to run.
+  steps:
+    # Specify the operator to run in each step.
+    - operator: read.read_cubes
+      # Specify the name of the argument, and its value.
+      filename_pattern: "*.nc"
+
+    - operator: filters.filter_cubes
+      # Can specify extra keyword arguments as sub-maps.
+      constraint:
+        operator: constraints.combine_constraints
+        var_constraint:
+          # Can nest in another operator to use its output as an argument,
+          # multiple levels deep if needed.
+          operator: constraints.generate_var_constraint
           # Input implicitly taken from the previous step, but can be overridden
           # by using the appropriate keyword argument.
-          stash: m01s03i236
-
-      - operator: write.write_cube_to_nc
-        # Specify the name of the argument, and its value.
-        filename: intermediate/processed_data
-        # intermediate is a slightly special folder for partially processed data
-        # that needs collating.
+          varname: m01s03i236
+        cell_method_constraint:
+          operator: constraints.generate_cell_methods_constraint
+          # Values can be more than just strings, such as this empty list.
+          cell_methods: []
 
-    # Steps to collate processed data into output.
-    collate:
-      - operator: read.read_cube
-        filename: intermediate/*.nc
+    # Save a sequence of plots, one per time.
+    - operator: plot.spatial_pcolormesh_plot
 
-      # Save a sequence of plots, one per time.
-      - operator: plot.plot_spatial_plot
-
-      # Save a single cube with all the processed data.
-      - operator: write.write_cube_to_nc
+    # Save a single cube with all the processed data.
+    - operator: write.write_cube_to_nc
+      overwrite: True
 
 The ``title`` and ``description`` keys provide a human readable description of
-what the recipe does. The ``title`` is also used to derive the ID of the running
-recipe, used when running the recipe in a workflow. The ``category`` is used to
-group the produced diagnostics in the output website.
-
-The ``parallel`` and ``collate`` keys specify lists of processing steps. The
-steps are run from top to bottom, with each step specifying an operator to run,
-and optionally any additional inputs to that operator. A parallel step is
-denoted by a ``-`` under the ``parallel:`` key. The operators are specified on
-the operator key. Its value should be a string of the form ``module.function``.
-For additional inputs the key should be the name of the argument.
-
-The ``collate:`` key is used for collating together the output of the
-parallel steps to produce the final output. This allows for the expensive
-processing to be parallelised over many compute nodes, with just the final
-visualisation of the data done in a single job to ensure it has all of the data.
+what the recipe does. The ``category`` is used to group the produced diagnostics
+in the output website.
+
+The ``steps`` key lists the processing steps. The steps are run from top to
+bottom, with each step specifying an operator to run, and optionally any
+additional inputs to that operator. Each separate step is denoted by a ``-``
+under the ``steps:`` key. The operators are specified on the operator key. Its
+value should be a string of the form ``module.function``. For additional inputs
+the key should be the name of the argument to that operator.
 
 The below code block shows how you can nest operators multiple levels deep. For
-details of the specific operators involved, and the arguments that can take, see
-the :doc:`/reference/operators` page.
+details of the specific operators involved, and the arguments that they can
+take, see the :doc:`/reference/operators` page.
 
 .. code-block:: yaml
 
@@ -98,7 +96,7 @@ case letters and underscores. For example:
 
 .. code-block:: yaml
 
-    parameter: $MY_VARIABLE
+    key: $MY_VARIABLE
 
 When the recipe is run with ``cset bake`` the variable is replaced with a value
 given on the command line. This is done using the variable name as an option,
@@ -114,8 +112,16 @@ Alternatively a space can be used between the variable and value:
 
     cset bake -i input -o output -r recipe.yaml --MY_VARIABLE value
 
-The given value will be templated into the parameter so what runs is actually:
+The given value will be templated into the variable so what runs is actually:
+
+.. code-block:: yaml
+
+    key: value
+
+This can also be used to template into existing values, such as the title.
 
 .. code-block:: yaml
 
-    parameter: value
+    title: $VARNAME spatial plot
+    # When VARNAME is "air_temperature" becomes:
+    title: air_temperature spatial plot

docs/source/usage/add-diagnostic.rst

@@ -175,7 +175,6 @@ Push code into remote repository branch with:
     # Just "git push" will suggest the correct command.
     git push --set-upstream origin <insert branch name>
 
-
 If the remote branch is behind changes on trunk then you update the remote
 branch on the GitHub repository by merging in changes from main. While on your
 branch:
@@ -185,7 +184,6 @@ branch:
     # Merge in the changes from the copy of main on GitHub.
     git merge origin/main
 
-
 Don't forget to push your changes back up to GitHub.
 
 .. code-block:: bash
@@ -198,7 +196,7 @@ Start the review process
 Make a :ref:`pull-request` on GitHub to propose your changes for inclusion in
 the trunk.
 
-Once you have satisfied the steps in the :doc:`Developer's guide
-</contributing/index>` go ahead and request a review on GitHub.
+Once you have satisfied the steps in the :doc:`/contributing/index` go ahead and
+request a review on GitHub.
 
 .. _recipe file: https://metoffice.github.io/CSET/usage/operator-recipes

docs/source/usage/workflow-installation.rst

@@ -47,20 +47,23 @@ the config.
 .. image:: rose-edit.png
     :alt: rose edit GUI, showing the environment setup options.
 
-* The Data and Cycling section contains settings for the input data, and the
-  workflow cycling controls.
-
-* The Diagnostic section contains settings for enabling the various diagnostics
-  included with CSET. Some general diagnostics will take from a list of variable
-  names or STASH codes, and operate on all of them.
-
 * The Environment section contains settings for controlling how CSET integrates
   with your compute environment. This includes things like activating modules,
   and choosing a site profile.
 
 * The General section contains whole-workflow configuration for things like
   logging and housekeeping.
 
+* The Models and Cases section contains settings for the input data. Here you
+  need to setup how you want to cycle (over case studies or a trial period),
+  how many models you want, and where to find the data for them.
+
+* The Diagnostic section contains settings for enabling the various diagnostics
+  included with CSET. The diagnostics are split into vague science area. Some
+  general diagnostics will have a table in their section for setting which model
+  fields to process. Diagnostics may also take additional options after being
+  enabled.
+
 Help for each variable can be viewed by clicking on the variable's name. Once
 you have configured CSET you can save and close rose edit.