python-data/solutions/ex04_cf_python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0cd6019e-c5b3-4dc6-bf86-6fcf10b06851",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# Practical for 'introduction to the NCAS CF Data Tools, cf-python and cf-plot'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e8d7e64-fd9a-4e9b-a369-61d5635cbb62",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<i>Practical instructions:</i> these green boxes provide instructions and tips about doing this practical (blue boxes are the same as in the teaching notebook and provide useful information). As guidance and for reference, the following are provided below before the practical material starts:\n",
    "<ul>\n",
    "    <li>the context and learning objectives from the main/presented Notebook below - you are advised to re-read this as a reminder;</li>\n",
    "    <li>a copy of the final section from the main Notebook which provides links to further information - you might find the documentation links especially useful here;</li>\n",
    "    <li>the note on terminology from section one is included also in the reminder as a guide to terms used throughout - read this if useful.</li>\n",
    "</ul>\n",
    "</div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4378ad0-8448-44ef-80e2-cd6e67346a1b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## A reminder: context, learning objectives and guidance links\n",
    "\n",
    "### What are the NCAS CF Data Tools and why do they all have 'cf' in the name?\n",
    "\n",
    "The _NCAS CF Data Tools_ are a suite of complementary Python libraries which are designed to facilitate working with data for research in the earth sciences and aligned domains. The two that are of most relevance to the average user, and those wanting to process, analyse and visualise atmospheric data, are *cf-python* (https://ncas-cms.github.io/cf-python/) and *cf-plot* (https://ncas-cms.github.io/cf-plot/build/). We will be focusing on use of cf-python and cf-plot today.\n",
    "\n",
    "The 'cf' in the names of the NCAS CF Data Tools corresponds to the _CF Conventions_, a metadata standard, because they are built around this standard in the form of using the CF Data Model, which as well as performance is considered a 'unique selling point' of the tools.\n",
    "\n",
    "\n",
    "### What are the CF Conventions?\n",
    "\n",
    "The _CF Conventions_, usually referred to in this way but also know by the full name of the **C**limate and **F**orecast (CF) metadata conventions, are a metadata standard which is becoming the de-facto convention to cover the description of geoscientific data so that sharing and intercomparison is simpler. See https://cfconventions.org/ for more information.\n",
    "\n",
    "\n",
    "### What are we going to learn in this session?\n",
    "\n",
    "Our **learning aim** is to be able to use the NCAS CF Data Tools Python libraries, namely cf-python and cf-plot to process, analyse and visualise netCDF and PP datasets, whilst appreciating the context and 'unique selling point' of the libraries as being built to use the CF Conventions, a metadata standard for earth science data, to make it simpler to do what you want to do with the datasets, by working on top of a Data Model for CF.\n",
    "\n",
    "We have **six distinct objectives**, matching the sections in this notebook and in the practical notebook you will work through. By the end of this lesson you should be familiar and have practiced using cf-python and cf-plot to:\n",
    "\n",
    "1. read dataset(s) and view the (meta)data at different detail levels;\n",
    "2. edit the (meta)data and write out the edited version to file;\n",
    "3. reduce datasets by subspacing and collapsing;\n",
    "4. visualise datasets as contour and vector plots;\n",
    "5. analyse data: applying mathematical and statistical operations and plotting trends;\n",
    "6. change the underlying grid of data through regridding."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26aa2a28-22ce-4e9f-bc1e-6b0d76ab807c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### Guidance: where to find more information and resources on the NCAS CF Data Tools\n",
    "\n",
    "Here are some links relating to the NCAS CF Data Tools and this training.\n",
    "\n",
    "* This training, with further material, is hosted online and there are instructions for setting up the environment so you can work through it in your own time: https://github.com/NCAS-CMS/cf-tools-training.\n",
    "* **The cf-python documentation lives at https://ncas-cms.github.io/cf-python/.**\n",
    "* The cf-python code lives on GitHub at https://github.com/NCAS-CMS/cf-python. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-python/issues.\n",
    "* **The cf-plot documentation lives at https://ncas-cms.github.io/cf-plot/build/.**\n",
    "* The cf-plot code lives on GitHub at https://github.com/NCAS-CMS/cf-plot. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-plot/issues.\n",
    "* There is a technical presentation about the NCAS CF Data Tools avaialble from https://hps.vi4io.org/_media/events/2020/summer-school-cfnetcdf.pdf.\n",
    "* The website of the CF Conventions can be found at https://cfconventions.org/.\n",
    "* The landing page for training into the CF Conventions is found here within the website above: https://cfconventions.org/Training/.\n",
    "\n",
    "If you have any queries after this course, please either use the Issue Trackers linked above or you can email me at: sadie.bartholomew@ncas.ac.uk."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ca3e79a-e39f-4a04-9dc5-3b56036bef5e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<i>Note:</i> In cf-python and when discussing related code and datasets, we use terminology from the CF Data Model (for more detail see: <a href=\"url\">https://ncas-cms.github.io/cf-python/cf_data_model.html</a>). For example cf-python methods are named in relation to concepts from this data model. We don't have time to cover this in detail but for this session it is useful to know the following terms:\n",
    "\n",
    "<ul>\n",
    "    <li><b>field</b>: a self-contained cf-python object corresponding to a netCDF data variable with all of its (CF) metadata attached;</li>\n",
    "    <li><b>field list</b>: a list of lields (see above), stored as its own cf-python object 'FieldList' which is similar to a Python list;</li>\n",
    "    <li><b>coordinate</b>: a (CF) metadata concept which corresponds to netCDF coordinate variables. One or more coordinates are defined on every field as either 'dimension' or 'auxiliary' coordinate objects in cf-python.</li>\n",
    "</ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c3beb87-7b0d-44ad-9da3-fa32494b795c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df666543-52ee-402a-8fa4-cb4d8389f3a8",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<i>Practical instructions:</i> run all of the cells in this section to do the set up.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "196e9e0a-83d1-464c-b53c-a6fc2e0c19ff",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Setting up\n",
    "\n",
    "**In this section we set up this Notebook, import the libraries and check the data we will work with, ready to use the libraries within this notebook.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d073760a-6734-4a1d-9832-a70a8e0dd089",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Import cf-python and cf-plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4b0c984-1adc-4515-b192-19da265e1f9a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import cfplot as cfp\n",
    "import cf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cfa3e24-6400-4de0-b155-aab85f701441",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Run some set up for nice outputs in this Jupyter Notebook (not required in interactive Python or a script):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aadf1e3e-382f-47c5-aeee-b52f3cd2868d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b00d47b-4497-4540-b1bd-6fa8fa8e3226",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Inspect the versions of cf-python and cf-plot and the version of the CF Conventions those are matched to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea75b1b6-ba78-428d-b9b2-ca29a8cf6f0d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(\"cf-python version is:\", cf.__version__)\n",
    "print(\"cf-plot version is:\", cfp.__version__)\n",
    "print(\"CF Conventions version is:\", cf.CF())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "628af6cc-24f1-4e85-bf8b-654268e72bd3",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<i>Note:</i> you can work with data compliant by any other version of the CF Conventions, or without (much) compliance, but the CF Conventions version gives the maximum version that these versions of the tools understand the features of.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8135f5c6-3d3e-45b6-a84a-3dcea754372f",
   "metadata": {},
   "source": [
    "Finally, see what datasets we have to explore:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfab63fe-088d-4ca8-ab95-40f61f0c2fd3",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<i>Note:</i> in a Jupyter Notebook, '!' preceeeds a shell command, so this is a terminal command and not Python\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a45afce7-abd8-4edd-b14c-eb033566bd52",
   "metadata": {
    "editable": true,
    "scrolled": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!ls ../data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7640cdb-0d6f-4933-86d7-e86e7b9acbe4",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef80e6de-8e62-468b-93c8-b41ac768375a",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<i>Practical instructions:</i> now we can start the practical. We will follow the same sectioning as in the teaching notebook, so please consult the notes there in the matching section for guidance and you can also consult the cf-python and cf-plot documentation linked above.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b7a3c0a-262d-4ac4-8c10-6fa3a42ae73a",
   "metadata": {},
   "source": [
    "## 1. Reading dataset(s) and viewing the (meta)data at different detail levels\n",
    "\n",
    "**In this section we look at a basic use of cf-python, reading in one or more datasets from file and inspecting the data and the metadata at different levels of detail to suit the amount of information you want to see.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9da08bf9-a15b-4d65-8c8e-632897daec0b",
   "metadata": {},
   "source": [
    "### a) Reading in data and extracting the _field_ of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45be7d09-fe3e-46b3-b35b-dec82b32cc01",
   "metadata": {},
   "source": [
    "**1.a.1)** Use `cf` to read in the netCDF dataset `qbo.nc` which is found (as shown at the end of the section above) under the directory `../data`, assigning it to a variable called 'fieldlist'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0a43e35-8627-4a08-ae1e-4fa9ec4f400d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "fieldlist = cf.read(\"../data/qbo.nc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "674d8543-fb55-4b2e-bd26-5a006dc9181d",
   "metadata": {},
   "source": [
    "**1.a.2)** Use the standard Python function `len` to see how long the read-in fieldlist is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e59c1a6-53ea-43c2-bf57-a405d1f64be9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "len(fieldlist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dfdb8c4-ac20-4b0c-87b6-5bc3829dee61",
   "metadata": {},
   "source": [
    "**1.a.3)** Access the first field in the fieldlist and assign it to the variable name 'field'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d92286c4-e62e-4e35-a934-809bc1afe02d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field = fieldlist[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4ed8112-fc35-496c-84ae-e8a05334c33c",
   "metadata": {},
   "source": [
    "### b) Inspecting the _field_ of interest with different amounts of detail"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e3ad7d7-2657-43d4-95a4-601429b16575",
   "metadata": {},
   "source": [
    "**1.b.1)** View the field from (1.a.3) above in minimal detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41528fdd-7f56-4801-95b9-c92b910e3a7d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a158a41-03f8-47f9-a84e-fd2eaac8a132",
   "metadata": {},
   "source": [
    "**1.b.2)** Now try viewing the field from (1.a.3) above at a medium detail level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0916e895-c503-47b8-a280-62404bb4d4a1",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4653be8e-e953-4a75-a343-93413cee09a6",
   "metadata": {},
   "source": [
    "**1.b.3)** OK, finally let's see it in its full glory - with maximal detail. Take a minute or two to compare these outputs and familiarise yourself with the formats of the different views and how they present the metadata (and preview of the data) of a field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b07e729d-3ecd-4c1e-b26b-9ff61fc04a70",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field.dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0af01357-b0b2-4581-a830-437337565d0d",
   "metadata": {},
   "source": [
    "### c) Inspecting a metadata _construct_ e.g. _coordinate_ from the _field_ of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03ce0882-35c1-4f41-9280-33d8b633572b",
   "metadata": {},
   "source": [
    "**1.c.1)** Let's assume we want to know about a specific metadata construct, in this case we are intereted in the pressure. Assign to a new variable called 'pressure' the pressure coordinate of the field stored in the variable 'field' from section (1a) as just inspected in section (1b)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99b9db17-713f-47c9-82a6-ec9af2213212",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure = field.coordinate(\"pressure\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf10ee1b-d4eb-43ac-839f-1f8066e703ae",
   "metadata": {},
   "source": [
    "**1.c.2)** View this coordinate with minimal detail level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "753ff7ac-2de7-49c7-b501-aee40ddb9090",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23fe4e99-326b-4209-b905-8ca211582b08",
   "metadata": {},
   "source": [
    "**1.c.3)** Now use the standard approach to view it with medium detail level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef442fa1-bfe4-4b5c-bfc7-4453c0d32e08",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(pressure)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b03ffb4-62d7-4e08-a1e5-28591c3fb602",
   "metadata": {},
   "source": [
    "**1.c.4)** Finally, let's use the approach for full detail level and see everything about this coordinate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee0cb3a0-5129-456e-8e9e-5f24a598d34d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure.dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd7fd831-8362-49eb-a03a-281d75596309",
   "metadata": {},
   "source": [
    "### d) Inspecting a data array of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16b595ee-6e54-40ec-ba3f-cc28f710eab1",
   "metadata": {},
   "source": [
    "**1.d.1)** Access the underlying data of the pressure coordinate from the previous sub-section, (1c), assigning it to a variable called 'pressure_data'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2f748b5-09c3-4382-a6d6-e50a31fca61f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure_data = pressure.data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7855240-2667-4883-a3fd-8d8b1759122a",
   "metadata": {},
   "source": [
    "**1.d.2)** Inspect the pressure coordinate data with minimal detail, noticing the units."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "539107da-25fe-4259-9fb2-0f6419954424",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7288c197-fd6e-405e-b49a-6740bc41992e",
   "metadata": {},
   "source": [
    "**1.d.3)** Access the data array of the pressure coordinate. Note that, because it is small, it is not computationally expensive to access this and similarly with other metadata data arrays, but accessing the underlying data array of the whole field (i.e. its main stored variable) could be intensive because for datasets in real usage the data can be very large and/or multi-dimensional."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e25482db-8342-49aa-bdc1-0911e842759d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure_array = pressure_data.array"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e2a3d9a-31ec-4f70-964c-212deb4709cc",
   "metadata": {},
   "source": [
    "**1.d.4)** Use the standard Python `print` function to view the pressure array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aba215ff-f5a7-4b97-9bdb-5a80961cd38f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(pressure_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09d04ff2-8998-48fb-a0a6-bc5deafe8fc6",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "932d2d38-b8d7-46cc-848f-b6f50b877310",
   "metadata": {},
   "source": [
    "## 2. Editing the (meta)data and writing out the edited version to file\n",
    "\n",
    "**In this section we demonstrate how to change the data that has been read-in from file, both in terms of the data arrays and the metadata that describes it, and then how to write data back out to file with a chosen name, so that you can see how cf-python can be used to edit data or to make new data.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc7c654e-38e5-4b06-a7c0-01828701d92f",
   "metadata": {},
   "source": [
    "### a) Changing the underlying data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d6b6412-42b4-4d70-9df2-1fd6b0349a56",
   "metadata": {},
   "source": [
    "**2.a.1)** Access the data (*not* the data array of the data) of the full field and assign it to a variable called 'data'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad3da187-8515-4a84-95e4-a9c1190dd08d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "data = field.data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b625ea77-a1db-4598-b1c4-6bba6319d1e0",
   "metadata": {},
   "source": [
    "**2.a.2)** Inspect the field data with medium detail use the `size` method on it to see its shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b6e33da-fad2-49a8-86e2-e94331e7e8dd",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(data)\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86474dfe-a2ac-4b52-a962-eda937b0f58d",
   "metadata": {},
   "source": [
    "**2.a.3)** Use the `size` method on it to see how many items (in this case, numbers) there are in it. Can you see how this relates to the `shape` above, and to the structure of the coordinates from the field inspection in (1.b.2)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "caaf9db2-c33d-434b-a3fe-ad91e9fc681e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "data.size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf00a0cd-ab33-4c08-94d0-a7c25a2df606",
   "metadata": {},
   "source": [
    "**2.a.4)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "394dc1e4-3e88-4370-b568-27f031fe1d5d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "first_time_subarray = data[0, :, :, :]\n",
    "first_time_subarray"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a97f4ab-dfba-4879-ac78-9c0edb36cf2d",
   "metadata": {},
   "source": [
    "**2.a.5)** Change all of the values in the first time subarray to the value '-50.0'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c167273d-16b2-4ea8-bc74-be8ac28d9820",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "first_time_subarray[0] = -50\n",
    "first_time_subarray"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5693c393-4c96-4841-8e4c-f336f9bfebb6",
   "metadata": {},
   "source": [
    "**2.a.6)** Access the index item `[1, :, 0, 0]` of the full data array from (2.a.1) and assign it to a variable called 'a_subarray'. Then check what shape it is and try to understand the size that emerges for that sub-array given that specific index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "beb5f4b6-5b5e-4761-8c66-2096edc8bfde",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "a_subarray = data[1, 0, :, 0]\n",
    "a_subarray.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9f9e1b8-1951-4545-94c1-9d567a0722d5",
   "metadata": {},
   "source": [
    "**2.a.7)** Change the values for this sub-array item to all ones i.e. `1.0`. Note to create an array populated all with the value one of the required shape to match the shape of the subarray from (2.a.6), you can use `numpy.ones(<desired shape>)` (you will need to import `numpy` first, and let's call the module `np` i.e. use `import numpy as np`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd65f487-c6b0-45e0-b093-1ac39f11b2d7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.ones((1, 1, 2, 1))\n",
    "data[1, 0, :, 0] = np.ones((1, 1, 2, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe2fe7a5-73b8-4021-8748-e88b3b0d6e5c",
   "metadata": {},
   "source": [
    "**2.a.8)** Inspect 'a_subarray' again to confirm it has been set with values of all one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5629a7f3-5fd8-4bfd-a6d7-eb141951a2c7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "a_subarray"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f2d2721-58c2-44d9-b422-3dd81ee8b333",
   "metadata": {},
   "source": [
    "### b) Changing some metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad17957e-9d70-4fcb-9954-e0ff331ac689",
   "metadata": {},
   "source": [
    "**2.b.1)** Take the 'pressure' variable coordinate we created in (1.c) and inspected in full detail in (1.c.4). Let's say we want to reverse the direction of this axes, so that the pressures increase in value along it (in order) rather than decreases. In `cf` we can do this using the `flip` method, which will adjust the data accordingly so that it is flipped in the corresponding way to match up to the new direction of the pressure axis.\n",
    "\n",
    "Reassign `pressure` to itself with the `flip` method applied (note you can, alternatively and equivalently, call the method `flip` with the argument `inplace=True` set without re-assignment because that argument tells `cf` to adjust the variable in-place)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f08327b2-51a1-404b-97fa-e753255eb252",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure = pressure.flip()\n",
    "# Or: pressure.flip(inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f240a543-5676-439e-b53e-4384ee5d9171",
   "metadata": {},
   "source": [
    "**2.b.2)** Now full inspect the pressure coordinate again with full detail to confirm that the flip of axis direction happened."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4311a56-3252-4b10-9b92-01f00c7e1cdb",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure.dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20031178-9faa-4501-b00f-70b0d1c1ae64",
   "metadata": {},
   "source": [
    "**2.b.3)** Notice the `standard_name`, which is a CF attribute used to identify the physical quantity in question, that must be one from the thousands of controlled options available in the CF Standard Name table (see https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html, you are encouraged to spend some minutes exploring this resource). At the moment the value in the field is not a valid standard name and it is also not specific enough to identify the variable well. \n",
    "\n",
    "Let's change it to the more descriptive valid CF Standard Name of 'air_pressure'. To do this, access the `standard_name` attribute of the pressure coordinate and re-set it to the string value desired."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6bfddb6-223a-46b0-8adf-725361fe1c64",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure.standard_name = \"air_pressure\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0e0d297-4e71-4a73-a194-f9b151f7a4a3",
   "metadata": {},
   "source": [
    "**2.b.4)** Once again view the pressure coordinate with maximal detail to see the two changes we made in this section. Also view the overall field in medium detail to see that these changes propagate through to the field object by Python object rules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "155bf824-7ebc-4178-9377-7be91b03393f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "pressure.dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf7cf826-2d23-40cd-bfca-648de59fd02c",
   "metadata": {},
   "source": [
    "### c) Writing a (list of) fields out to a file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f375ce40-ba55-45cd-83d2-9c5f58df3506",
   "metadata": {},
   "source": [
    "**2.c.1)** Let's write out a few fields to disk. We'll write our field from the previous sections as well as a new field all into the same file.\n",
    "\n",
    "First let's get a new field. We could read in another file to access some new fields, but let's use one of the cf-python ready-made 'example fields', which are designed for exploring cf-python capability without the need for real (and large/memory-intensive) data. Run `cf.example_field()` with any integer from 0 to 11 inclusive to get an example field, assigning it to the variable called 'new_field'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fb3f0c6-6638-4535-8729-b50d1f70fc29",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "new_field = cf.example_field(1)  # Or: any integer from 0 to 11 works fine as the argument"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e061bbd-836c-44fb-9614-358585b4c1bf",
   "metadata": {},
   "source": [
    "**2.c.2)** Inspect this new field with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b199ddb-00c6-456e-bb51-58117037c50b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(new_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19c295d5-9e45-49a8-bc53-fd9a139aacf8",
   "metadata": {},
   "source": [
    "**2.c.3)** Create a two-field FieldList called 'new_fieldlist' containing the field from (1.a.3) that we have been exploring in the previous sections (1.a) and (1.b) along with this new 'new_field' from (2.c.2). You can use the `cf` function `cf.FieldList` with the fields it should contain in a normal Python list as an arugment to set this up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a55c4a55-10bd-4269-a232-32e0d3737a94",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "new_fieldlist = cf.FieldList([field, new_field])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c155c131-5367-497f-8590-16bbfdc7c124",
   "metadata": {},
   "source": [
    "**2.c.4)** Call `print` with this new fieldlist as an argument to view it in medium detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2c71af9-a032-458a-b091-39d776c11025",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "print(new_fieldlist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aee0230d-681c-476e-800e-fa2f83b419ea",
   "metadata": {},
   "source": [
    "**2.c.4)** Now we have constructed our desired FieldList, we can write it out to file. Use `cf`'s `write` function to write it to a file called `two_fields.nc` in the `../data` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9775ffb-6830-4102-b3b0-eb11403ad806",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cf.write(new_fieldlist, \"../data/two_fields.nc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb5aabd2-6e6d-4e28-8e9f-3da1537d981a",
   "metadata": {},
   "source": [
    "**2.c.5)** Use the shell command `!ls ../data` to see the contents of that directory to confirm that the file was written there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f535282-e936-4254-9bc5-fc096bf69e6b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "!ls ../data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dd6be4b-60d3-4ca8-8108-fcda62bb7d95",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ccfd6a0-567d-43f3-9f06-15bb21afae6a",
   "metadata": {},
   "source": [
    "## 3. Reducing datasets by subspacing and collapsing\n",
    "\n",
    "**In this section we show how multi-dimensional data can be tamed using cf-python so that you can get a reduced form that can be analysed or plotted, by reducing the dimensions by selecting a subset of point(s) along the axes or collapsing down according to some statistic such as the mean or an extrema.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "410ac6d4-cd8f-43ed-a3f5-eaf8d1d8755a",
   "metadata": {},
   "source": [
    "### a) Subspacing using metadata conditions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cec90546-f200-42b3-af27-6508f367d305",
   "metadata": {},
   "source": [
    "**3.a.1)** Read in the file `ggas2014121200_00-18.nc` which is under `../data` and save the corresponding FieldList to a variable called `fieldlist_3`. Inspect it with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4af5c1c-cdc4-441d-8d2c-17d9cfd6b746",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "fieldlist_3 = cf.read(\"../data/ggas2014121200_00-18.nc\")\n",
    "print(fieldlist_3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3077b7a8-6f93-4a95-ad6d-58e8b3fc15ec",
   "metadata": {},
   "source": [
    "**3.a.2)** Extract the field representing the `cloud_area_fraction` to a variable which we will call `cloud_field`. Inspect that also with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91818c63-a229-491b-aee2-8882ff554746",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field = fieldlist_3[4]\n",
    "print(cloud_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df537a09-9cc3-43c0-9e6a-7d8d7902344c",
   "metadata": {},
   "source": [
    "**3.a.3)** Save the data of the `cloud_field` to a new variable we will call `cloud_field_data`. Do a `print` on the `shape` method of this to confirm the shape of the data, and compare this to the insepction report from the previous cell to see the same information represented in different ways."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d54f5edd-abc3-40e7-a2c1-971fb52e12ca",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_data = cloud_field.data\n",
    "print(cloud_field_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "862a5eb4-6a63-4c11-a188-f048c3d27926",
   "metadata": {},
   "source": [
    "**3.a.4)** Make a subspace of the `cloud_field` from the cells above to subspace on the *first* time point in order. Note: doing date-time subspaces requires an extra step due to the nature of specifying dates and times which can be ambiguous otherwise: you need to wrap a quoted datetime string in the call `cf.dt()` to notify cf-python that you are providing a valid datetime string, e.g. `field1.subspace(time=cf.dt(\"2020-01-01 12:15:00\"))`.\n",
    "\n",
    "Assign the subspace operation resulting field to a variable `cloud_field_subspace1` and inspect it with medium detail.\n",
    "\n",
    "*Extra task, for those who have studied section 4 before doing this practical**: make a contour plot of this subspace of the field to see what it looks like.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0cdd409-e75d-43bb-96fd-c79b1f5e575b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_subspace1 = cloud_field.subspace(time=cf.dt(\"2014-12-12 00:00:00\"))\n",
    "print(cloud_field_subspace1)\n",
    "print(cloud_field_subspace1.data.shape)\n",
    "# Extra part: cfp.con(cloud_field_subspace1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58adbc35-a63a-4c8a-973d-c28fea109ed3",
   "metadata": {},
   "source": [
    "**3.a.5** Make a subspace of the `cloud_field` from the cells above to subspace on the *last* point on the latitude axis.\n",
    "\n",
    "Assign the subspace operation resulting field to a variable `cloud_field_subspace2` and inspect it with medium detail.\n",
    "\n",
    "*Extra task, for those who have studied section 4 before doing this practical: make a contour plot of this subspace of the field to see what it looks like.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6db795e1-8eb4-42b8-a6d2-4ad554f7fbb7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_subspace2 = cloud_field.subspace(latitude=-89.46282196044922)\n",
    "print(cloud_field_subspace2)\n",
    "print(cloud_field_subspace2.data.shape)\n",
    "# Extra part: cfp.con(cloud_field_subspace2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb1db98b-27b3-4c8a-bf63-f67f83dd2bb8",
   "metadata": {},
   "source": [
    "### b) Subspacing using indexing, including equivalency to the above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d95810b-eb02-4862-938e-91e6634bb2b9",
   "metadata": {},
   "source": [
    "**3.b.1)** Take the cloud field from (3.a.2) which we have been subspacing in the previous cells and make a subspace which takes the first time point, leaving all other axes unchanged, but this time do it using indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d6ccf81-0a51-44f4-948c-95a243c751cd",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_subspace1_by_index = cloud_field[0, :, :, :]\n",
    "print(cloud_field_subspace1_by_index)\n",
    "cloud_field_subspace1_by_index.equals(cloud_field_subspace1)\n",
    "# or with the fields swapped i.e. cloud_field_subspace1.equals(cloud_field_subspace1_by_index) is also correct"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e2cc4bf-2520-4136-82af-48ac6f974043",
   "metadata": {},
   "source": [
    "**3.b.2)** Now make a subspace on the original `cloud_field`, leaving all other axes unchanged, to subspace on the *last* point on the latitude axis, like before, but this time use subspacing by indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f15ef080-2d79-4686-8f67-00baee8a98f2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_subspace2_by_index = cloud_field[:, :, -1, :]\n",
    "print(cloud_field_subspace2_by_index)\n",
    "cloud_field_subspace2_by_index.equals(cloud_field_subspace2)\n",
    "# or with the fields swapped i.e. cloud_field_subspace2.equals(cloud_field_subspace2_by_index) is also correct"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e65d7951-2d10-40d3-8858-87b7210079a8",
   "metadata": {},
   "source": [
    "**3.b.3)** Using indexing, do both of the subspaces from the previous sub-questions in one call on the original cloud field.\n",
    "\n",
    "Extra: do the same operation using the 'subspace by metadata' approach and use the `equals` method to show that the results are the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b83671d-442f-41c4-981a-764581a358ec",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_subspace2_by_index = cloud_field[0, :, -1, :]\n",
    "print(cloud_field_subspace2_by_index)\n",
    "cloud_field_subspace2_by_index.equals(cloud_field_subspace2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e05393-4c31-4505-8b80-4b8b2bdeba5d",
   "metadata": {},
   "source": [
    "**3.b.4)** Do a single subspace on the original cloud field that takes the first 100 latitude and the first 200 longitude values. Use whichever method (subspacing by metadata, or indexing) you prefer, in order to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd33a050-117c-4327-931c-ea8dd2372a70",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field[:, :, :100, :200]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b870563e-6f45-46ca-9dad-6d8a5cb69ec5",
   "metadata": {},
   "source": [
    "### c) Statistical collapses"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05d9602f-744d-4ce7-a4a0-1880a30db393",
   "metadata": {},
   "source": [
    "**3.c.1)** Take the original `cloud_field` from (3.a.2) and do a collapse over the time axis to reduce it down to the minimum value. Assign that to the variable name `cloud_field_collapse1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "843d1148-25e1-4cc2-91e3-28085b0ac7f3",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_collapse1 = cloud_field.collapse(\"time: minimum\")\n",
    "print(cloud_field_collapse1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1527c1bc-12e3-4708-a544-39fef8d0062b",
   "metadata": {},
   "source": [
    "**3.c.2)** Take the original `cloud_field` from (3.a.2) and do a collapse over the latitude axis to reduce it down to the mean value. Assign that to the variable name `cloud_field_collapse2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f44e6b5e-c3bf-48bf-81b4-d082a437d3c2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_collapse2 = cloud_field.collapse(\"latitude: mean\")\n",
    "print(cloud_field_collapse2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "592f58a8-77a3-4cc2-beb3-f6247f5f0310",
   "metadata": {},
   "source": [
    "**3.c.3)** Take the original `cloud_field` from (3.a.2) and do a collapse over the longitude axis to reduce it down to the maximum value. Assign that to the variable name `cloud_field_collapse3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62c236eb-c360-47b2-ac20-6d1250ed7a78",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_collapse3 = cloud_field.collapse(\"longitude: maximum\")\n",
    "print(cloud_field_collapse3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22357974-11e2-4820-b0d6-90273cbc4646",
   "metadata": {},
   "source": [
    "**3.c.4)** Finally, take the original `cloud_field` from (3.a.2) again and do a collapse over all horizontal space via the pair of horizontal spatial axes, latitude and longitude, to reduce them down to the standard deviation value. Assign that to the variable name `cloud_field_collapse4`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "532ae8bf-09a1-4a2c-8800-2802fe186202",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cloud_field_collapse4 = cloud_field.collapse(\"longitude: latitude: standard_deviation\")\n",
    "print(cloud_field_collapse4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59b69a9d-a94c-464c-aa3c-482c699bc404",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80edfa76-7f87-42b9-8cef-058304ca902a",
   "metadata": {},
   "source": [
    "## 4. Visualising datasets as contour and vector plots\n",
    "\n",
    "**In this section we demonstrate how to plot using cf-plot the data we have read and then processed and/or analysed using cf-python, notably showing how to create contour plots and vector plots as examples of some of the available plot types.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "794c4940-43fd-45a3-ba9a-69084c0e5ccd",
   "metadata": {},
   "source": [
    "### a) Making a contour plot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8508032-181e-4a04-ba70-0f0dfd24f601",
   "metadata": {},
   "source": [
    "**4.a.1)** Read in the file `alpine_precip_DJF_means.nc` which is in the `../data` directory and assign the first field it contains to a variable called 'field_4'. Inspect that field with medium detail level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b39603-c768-472b-bd2b-8aec63e525c0",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field_4 = cf.read(\"../data/alpine_precip_DJF_means.nc\")[0]\n",
    "print(field_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f780f284-697e-41a3-8984-db53ac970235",
   "metadata": {},
   "source": [
    "**4.a.2)** Make a contour plot of the field, noting that because it is already 2D over X and Y axes corresponding to its projection so it can be plotted directly, without any need to reduce via subspacing or collapsing. Just get a plot working - there is no need to customise it yet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94613743-f648-409e-b7a3-22bbf5849c1c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.con(field_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b19186dc-3480-47d1-a859-79008841bd58",
   "metadata": {},
   "source": [
    "### b) Customising the (contour) plot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "017e176a-67ab-4580-82e3-245bf30cc585",
   "metadata": {},
   "source": [
    "**4.b.1)** Customise the contour plot made in section (4a) to remove the contour lines by running the same call with a new argument set appropriately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4509ae8-c96c-4d9d-9b41-83e3828d03b7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.con(field_4, lines=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b403c8c-5991-44b2-89db-660c58f0abdd",
   "metadata": {},
   "source": [
    "**4.b.2)** Customise the contour plot further so, as well as having the contour lines removed as applied above, it is shown using a different colour map to the default colour map, 'viridis'. You can use the link https://ncas-cms.github.io/cf-plot/build/colour_scales.html#colour-scales to explore the different options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e4e3a79-9e31-4779-9aa3-7cb8d1832819",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.cscale(\"precip4_11lev\")  # or another choice of valid colour map string argument\n",
    "cfp.con(field_4, lines=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "422253a2-28ab-4d6a-8841-d9cdb0ea527c",
   "metadata": {},
   "source": [
    "**4.b.3)** Customise the contour plot further by adding the argument and value `resolution=\"10m\"` to the call to `mapset`, which will increase the resolution of the country borders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7945c4ee-53ed-4c26-b468-b4cd6b0edc88",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.mapset(resolution=\"10m\")\n",
    "cfp.con(field_4, lines=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d01534c9-8c56-4f6f-a550-2e86a5749bed",
   "metadata": {},
   "source": [
    "**4.b.4)** Customise the contour plot further so that it focuses in on a smaller region over the Alps, specifically from longitude of 5 to 10 degrees east and from latitude of 45 to 47 degrees. Note, a call to `mapset` will reset any of the previous calls, so you will need to re-apply previous arguments to it that you want to preserve for future plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70bf4f42-61de-4255-acbc-9235a36f403e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.mapset(lonmin=5, lonmax=10, latmin=45, latmax=47, resolution=\"10m\")\n",
    "cfp.con(field_4, lines=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6964260d-84fb-41d9-a1b9-a74e85a79c20",
   "metadata": {},
   "source": [
    "### c) Making a vector plot with basic customisation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a7610d7-2654-4759-9083-0ae921546f63",
   "metadata": {},
   "source": [
    "**4.c.1)** Read in the netCDF file `sea_currents.nc` stored in the usual directory, `../data`, assigning it to a variable 'irish_sea_fieldlist' and inspect it with medium detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1ba3732-e318-4e4e-a5e7-8ca17db707b9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "irish_sea_fieldlist = cf.read(\"../data/sea_currents.nc\")\n",
    "print(irish_sea_fieldlist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "920060c3-46e1-420d-a400-f110e9c08b97",
   "metadata": {},
   "source": [
    "**4.c.2)** Note that the first two fieds represent perpendicular components of the same directional variable for sea water velocity. The first field is the eastward and the second field is the northward, component of this.\n",
    "\n",
    "In order to ensure they are compatible to plot together as vectors, we first need to ensure they are defined on the same gridpoints - in cf-python (from CF Data Model) terminology, we say they need to have the same *domain*. Assign to variable names 'eastward_component_field' and 'northward_component_field' the first and second fields respectively, and inspect them with minimal detail as a first step check to see if there is the same number of axes and points along each upon which the data is defined (there are two code blocks here, one for each field since when you inspect ith minimal detail it will only show if it is the final call in the cell)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0946df9-087f-4460-8519-4b2b4dea2d6f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "eastward_component_field = irish_sea_fieldlist[0]\n",
    "eastward_component_field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49cd847e-4c92-46d6-9f31-f81b8ec3b4e2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "northward_component_field = irish_sea_fieldlist[1]\n",
    "northward_component_field"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8018e3d9-685b-4a12-93c3-77f3dfcb035e",
   "metadata": {},
   "source": [
    "**4.c.3)** You should be able to see from the metadata that the two fields are perpendicular components of the same wind field. But, for us to be sure these represent components of the same vector field, we need to check that they are defined at the same grid points - else the components are defined at different location points in space and therefore can't be combined into a single vector. In cf-python we call the locations that the data is representative of the *domain*.\n",
    "\n",
    "We can check whether the two fields have the same domain by taking the `domain` attribute of each field and comparing those using the `equals` method, e.g. `field1.domain.equals(field2.domain)` for fields `field1` and `field2`. Do this for the two fields we defined in the pair of cells above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33efb918-44f1-46cc-9018-d7775a108382",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "eastward_component_field.domain.equals(northward_component_field.domain)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b170dab7-7a23-47e9-8dc1-04b448c04ab5",
   "metadata": {},
   "source": [
    "**4.c.4)** We know now that the fields are the same variable, with perpendicular northward and eastward components, with the same domain i.e. defined on the same gridpoints. This means they are compatible vector components and can be plotted together to form a vector plot for the variable in question, the sea water velocity. However, we currently have the time coordinate with more than one point, so we need to reduce both fields down in the correpsonding way to get a particular 2D plot.\n",
    "\n",
    "Let's take the final time point in the series of both of these fields to plot. Subspace both fields down in time to that final time axis point, using indexing - using section (3b) for a reminder about subspacing by index if you need guiance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f94aaa31-8468-4803-869e-878d542a061e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "eastward_component_field_last_time = eastward_component_field[-1, :, :, :]\n",
    "print(eastward_component_field_last_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c1640c3-2174-472d-994a-b6a3d7c54ae2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "northward_component_field_last_time = northward_component_field[-1, :, :, :]\n",
    "print(northward_component_field_last_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90055498-9f1a-463e-aec1-1cf22cd4b333",
   "metadata": {},
   "source": [
    "**4.c.5)** Make a vector plot using the two subspaced component fields you found and defined in the previous section to form the corresponding vectors. Do not worry about customising it yet, just get a plot working even if the vectors are not optimised for readability, however do run `cfp.mapset(resolution=\"10m\")` before you make the call to make the vector plot to reset the X and Y limits we specified in the previous section.\n",
    "\n",
    "Note that the `u` keyword argument is for eastward vector components and the `v` keyword argument is for northward vector components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3249180-3c84-4a96-a8f8-f40340769bc2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.mapset(resolution=\"10m\")\n",
    "cfp.vect(u=eastward_component_field_last_time, v=northward_component_field_last_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f99f557c-a4dd-42e4-8b7d-5d9bcc1d5ad9",
   "metadata": {},
   "source": [
    "**4.c.6)** Customise the plot you just made, in particular so that the vector lines are legible through adjusting the spacing and size at which they are plotted. This usually takes some trial and error - guess some values for the keyword argument values and increase or decrease them until you find a good balance so the vector lines can be distinguished and clearly interpreted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "783e8212-335b-4276-8014-2be54f95ca4b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.mapset(resolution=\"10m\")\n",
    "cfp.vect(u=eastward_component_field_last_time, v=northward_component_field_last_time, scale=3, stride=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b191c8b2-a038-4601-b046-997a4566c2f6",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72524bb7-a643-4753-ad8b-f419656883fa",
   "metadata": {},
   "source": [
    "## 5. Analysing data: applying mathematical and statistical operations and plotting trends\n",
    "\n",
    "**In this section we demonstrate how to do some data analysis including performing arithmetic and statistical calculations on the data, showing how cf-python's CF Conventions metadata awareness means that the metadata is automatically updated to account for the operations that are performed.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59637071-edcb-40fe-970a-1d89e0209aa7",
   "metadata": {},
   "source": [
    "### a) Applying mathematics e.g. arithmetic and trigonometry on fields"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ce63f35-c0a3-4c5b-a58c-98856f26ce25",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "**5.a.1)** Take the `fieldlist_3` we have already defined in (3.a.1). This time, we'll work with the *fourth* field in order in the list. Extract this field to a variable called `field_5` and inspect it with medium detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8846ce13-3ac6-4586-9318-bcf96abe2b97",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field_5 = fieldlist_3[3]\n",
    "print(field_5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfdf903e-1a4b-46e2-9ba5-4118307329c7",
   "metadata": {},
   "source": [
    "**5.a.2)** Take a subspace of the first time point, assigning it to a new variable called 'field_5_subspace', and plot it on a contour plot to see what the underlying data is like. Making a plot with cf-plot early on in this way is, for example, a good way to get a quick look at a new dataset to get a feel for the overall scope and patterns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c7be26b-51d7-4f43-aaa5-ce916b97a468",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field_5_subspace = field_5[0, :, :, :]\n",
    "cfp.con(field_5_subspace)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d0733cc-b6fa-4889-87ba-c0161b29d8eb",
   "metadata": {},
   "source": [
    "**5.a.3)** To normalise data representing a variable `x`, there is the general formula `normalized_x = (x - minimum_x) / x-range`. Use the cf-python methods available on a field, namely `minimum()` and `range()`, to normalise the data in the subspace of `field_5` we defined as `field_5_subspace` from the previous step. Call this result `norm_field_5` as a new Python variable. Hint: you can apply these methods directly on the field which knows to apply them to its underlying data, and the whole data array will be operated on at once, as with array operations in NumPy and similar Python array libraries.\n",
    "\n",
    "Finally, re-plot the contour plot at the subspace on the first time point to see that the data has been normalised appropriately: the colour bar range should go from 0 to 1 now but the data plot itself (contour pattern and colours) should not have changed because the normalisation adjusts all values to the 0 to 1 range but preserves the relative magnitudes between them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e770cba8-88db-44ac-a143-51e369b9695b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "norm_field_5 = (field_5_subspace - field_5_subspace.minimum()) / field_5_subspace.range()\n",
    "print(norm_field_5)\n",
    "cfp.con(norm_field_5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3636fd39-4aa6-4b86-8eaf-0154b285fc2d",
   "metadata": {},
   "source": [
    "**5.a.4)** Finally, to explore basic field arithmetic and see the influence on the data via a plot, multiply `field_5_subspace` by negative one and plot this result as a contour plot. Describe in a sentence comment the difference in the plot to the original from ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aed0f47d-cad3-4a80-8ef6-2878ec963fd8",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "field_5_reversed = -1 * field_5_subspace\n",
    "cfp.con(field_5_reversed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "561caf84-0824-4b69-8ad9-82ce28d5bfd1",
   "metadata": {},
   "source": [
    "### b) Line plotting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a42040e2-0bc1-4e25-9168-5dc474e97ab2",
   "metadata": {},
   "source": [
    "**5.b.1)** Let's make some line plots. To set this up, read in the file `IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc` under the usual directory `../data` and assign the *first* field in that FieldList to a variable called `monthly_field`. Inspect it with medium detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "274d1d90-5be7-46ff-8532-5de882998187",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "monthly_field = cf.read(\"../data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc\")[0]\n",
    "print(monthly_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5de191f4-d755-43f7-8718-e04a8376b647",
   "metadata": {},
   "source": [
    "**5.b.2)** Notice that the `monthly_field` is three-dimensional (3D) with axes sizes all over one for each of the three dimensions. In order to do a lineplot, we need a one-dimensional series, so we need to reduce the field down to size one in two of the dimensions to make a line plot.\n",
    "\n",
    "Let's do a collapse to reduce the two spatial axes, latitude and longitude, at once. Do a collapse to the maximum over those two axes, assigning it to a variable we'll call `spatial_max_monthly_field`. You can specify both of those axes by name, or you can use `\"area\"` as a shorthand string which means both latitude and longitude at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34f9b25c-c249-4868-b262-ecadf69367b8",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "spatial_max_monthly_field = monthly_field.collapse(\"area: maximum\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0e10b7f-b3b5-4dd3-b79b-1d8b6e1bb190",
   "metadata": {},
   "source": [
    "**5.b.3)** Use cf-plot to make a line plot of the collapsed field `spatial_max_monthly_field` from the cell above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "537ec534-e25e-4dc6-88b6-57e7797a1273",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.lineplot(spatial_max_monthly_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "682335b3-5eba-416c-bbd2-182f5915df7d",
   "metadata": {},
   "source": [
    "### c) Calculating seasonal trends"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9799e66c-0107-4dd0-8652-5a851a795c8f",
   "metadata": {},
   "source": [
    "**5.c.1)** Define a new variable `get_mam_season` and set it to the cf-python function `mam()` which represents a specific collapse over the months of March, April and May, just like `cf.djf()` represents a collapse over December, January and February etc. - there are four such methods to cover each trio of months closely aligned with the seasons, two of which were demonstrated in the teaching Notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f483867-c35f-456b-8314-7c9c006b163a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "get_mam_season = cf.mam()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cbc9cf7-cfd7-49fc-8244-91612c006cdb",
   "metadata": {},
   "source": [
    "**5.c.2)** Do a grouped collapse over the months of March, April and May only using the `group` argument set to `get_mam_season`, for a *maximum* collapse over the *time* axis on the `spatial_max_monthly_field`. Make a line plot of this collapsed field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8763a854-82cf-4fc5-80b5-c0214310fae9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "mam_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_mam_season)\n",
    "cfp.lineplot(mam_season_max)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcb98f04-a03d-4c1c-b0cd-ea153daf6f16",
   "metadata": {},
   "source": [
    "**5.c.3)** Define three more variables representing specific collapse over the DJF, JJA and SON trio of months using the appropriate cf-python methods to complete the set across the calendar year (`cf.son()` is for the SON months and use the teaching Notebook for guidance on the other two methods if required but their names follow the same pattern so you can possibly guess them), assigning them to variables `get_djf_season`, `get_jja_season` and `get_son_season` respectively.\n",
    "\n",
    "Then, using those three variables one-by-one, define three grouped collapses, also for a *maximum* collapse over the *time* axis as above, on the `spatial_max_monthly_field` field using these as the `group`, calling them `djf_season_max`, `jja_season_max` and `son_season_max` corresponding to the collapses in order above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea2e67a9-1ec6-4d3b-a673-2f06c9f7ae11",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "get_djf_season = cf.djf()\n",
    "get_jja_season = cf.jja()\n",
    "get_son_season = cf.son()\n",
    "djf_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_djf_season)\n",
    "jja_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_jja_season)\n",
    "son_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_son_season)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b094573f-f690-48da-a8e9-93481ead73fb",
   "metadata": {},
   "source": [
    "### d) Plotting the seasonal trends on one (line)plot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "428b074b-d2af-4804-9131-2985d69bf4d9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "**5.d.1)** Let's make a line plot which shows all of the fields above: the four season maximum trends `djf_season_max`, `mam_season_max`, `jja_season_max` and `son_season_max`, along with the `spatial_max_monthly_field`. Use the code block from the teaching Notebook as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas. You do not need to set positions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83a29404-cc39-47eb-9909-b2af9b1ab7e9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.gopen()\n",
    "cfp.lineplot(spatial_max_monthly_field)\n",
    "cfp.lineplot(djf_season_max)\n",
    "cfp.lineplot(mam_season_max)\n",
    "cfp.lineplot(jja_season_max)\n",
    "cfp.lineplot(son_season_max)\n",
    "cfp.gclose()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "422daf4c-731d-48db-812e-53298d8d80a7",
   "metadata": {},
   "source": [
    "**5.d.2)** Finally, copy your code from the previous cell and use the `label` argument to `lineplot` to assign labels to your individual line plot calls, which will result in a legend emerging on the final plot with the labels to identify each line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6396957-b64b-4c79-9fcd-067463457842",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.gopen()\n",
    "cfp.lineplot(spatial_max_monthly_field, label=\"Original monthly data (spatial mean)\")\n",
    "cfp.lineplot(djf_season_max, label=\"Maximum over the DJF months of the original spatial maximum\")\n",
    "cfp.lineplot(mam_season_max, label=\"Maximum over the MAM months of the original spatial maximum\")\n",
    "cfp.lineplot(jja_season_max, label=\"Maximum over the JJA months of the original spatial maximum\")\n",
    "cfp.lineplot(son_season_max, label=\"Maximum over the SON months of the original spatial maximum\")\n",
    "cfp.gclose()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b746f6b-ef82-4aa9-8d39-ca179f15db62",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e4c72ef-8631-4749-b4fd-6c79c8d6e455",
   "metadata": {},
   "source": [
    "## 6. Changing the underlying grid of data through regridding\n",
    "\n",
    "**In this section we demonstrate how to change the underlying grid of the data to another grid which could be a higher- or lower- resolution one, or a completely different grid, which is called regridding or interpolation, and indicate various options cf-python supports for doing this.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "854d7fd4-6d65-4827-b77c-4a89d42da6b0",
   "metadata": {},
   "source": [
    "### a) Getting a _source_ field ready to regrid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65ee428b-cf95-423c-b92e-4cd644b7df41",
   "metadata": {},
   "source": [
    "**6.a.1)** Define a new variable 'yearly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_yearly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6ec5550-61a1-47ff-99f1-339f6dff0fa9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "yearly_field = cf.read(\"../data/precip_1D_yearly.nc\")[0]\n",
    "print(yearly_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7955e4b0-a23e-4398-9c09-48375a37e2e2",
   "metadata": {},
   "source": [
    "**6.a.2)** This will be our source field for regridding. Note it is a time series, not a dataset defined across different points in space! We can regrid data on any axis or axes, they do not have to represent spatial coordinates, including 1D series. In this case, we have an effectively 1D (3D but with two of the axes size one, hence of size which can be ignored) time series and we are going to regrid it. For a time series, visually this\n",
    "\n",
    "Make a line plot of the time series in `yearly_field` to see it. Add the argument `marker=\"o\"` to the call which plots markers against the individual data points forming the line. With the markers on we can see the discrete sample points that make up the data represented by the continuous line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17a5908e-ea5a-4b8b-bbf2-980f7bcf0796",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.lineplot(yearly_field, marker=\"o\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "628e91df-76f0-485e-8288-19ee932a7fb6",
   "metadata": {},
   "source": [
    "### b) Getting the _destination_ field: another field in order to regrid the previous _onto its grid_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff212883-d45d-4b3a-8d60-5f8816c9acce",
   "metadata": {},
   "source": [
    "**6.b.1)** Let's get another time series field, but one with a different sampling resolution. The sampling of time points across the axes is the *grid* in this context. For our destination field, we will take a series which is sampled at higher resolution.\n",
    "\n",
    "Define a new variable 'monthly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_monthly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f1f0704-f409-4ae5-b6d1-acadd22790d1",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "# Read in another precipitation field, with more time axis points, and inspect it\n",
    "monthly_field = cf.read(\"../data/precip_1D_monthly.nc\")[0]\n",
    "print(monthly_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9f9d588-5bf4-4911-ba4e-e8e2ece77809",
   "metadata": {},
   "source": [
    "**6.b.2)** As the name of the corresponding read-in files suggests, this `dest_field` represents monthly data and the `source_field` represents yearly data, so `dest_field` is around 12 times the resolution of the `source_field`.\n",
    "\n",
    "Make a line plot of the `dest_field` showing the markers for the discrete data points. Compare this line plot to the one above with respect to the time axis (on the x-axis) to confirm the difference in sampling frequencies of around x12. Note we don't care about the values of the data at each point and whether it is different to the data of the source grid we are considering! All we want to extract from this destination field time series is the time point sampling, our destination grid. (Likewise, in the teaching Notebook where we did horizontal spatial grid regridding, the data defined on the destination grid `lower_res_field` did not matter, just the grid points it was defined on.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a414b073-0c90-498e-bd71-62bdbc3cc69d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.lineplot(monthly_field, marker=\"o\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dde6c34c-37a9-48c9-8e02-a70913411d7a",
   "metadata": {},
   "source": [
    "### c) Performing the regrid operation from the source to the destination fields"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce49aba-e16f-4278-ac9f-4fa3e8d4c8c5",
   "metadata": {},
   "source": [
    "**6.c.1)** Before we do the regridding operation, let's demonstrate briefly how to remove any size one axes from a field. Note this is not necessary in order to do the regridding, but it can 'tidy up' the fields so they don't have size one axes attached, which can be useful in cases like this because it means the axes are both 1D so it is simpler to interpret the regridding we are doing. To remove the size one axes of a field, apply the `squeeze()` method: you can either re-assign the field with `squeeze()` applied to the same variable name, or use the `inplace=True` argument to the method which changes the field in-place i.e. to the variable storing it without the need to re-assign.\n",
    "\n",
    "Apply the `squeeze` method to the `monthly_field` and the `yearly_field` and inspect the results of each with medium level of detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7d637a8-bdb9-43ad-9e05-0e2abcb58fb9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "monthly_field = monthly_field.squeeze()  # or just monthly_field.squeeze(inplace=True)\n",
    "print(monthly_field)\n",
    "yearly_field = yearly_field.squeeze()  # or just yearly_field.squeeze(inplace=True)\n",
    "print(yearly_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc533850-f32c-4441-9a4f-f14c3206db14",
   "metadata": {},
   "source": [
    "**6.c.2)** Now to do the actual regridding operation! (You could call all the field methods such as reading, indexing and squeezing in the same line as the actual regrid operation, but doing so would be harder to interpret and to debug in case something wasn't speciied correctly, or some other issue, so we have worked these out step-by-step as better practice.)\n",
    "\n",
    "In the teaching Notebook we used the `regrids` method which performs *spherical* regridding (`s` for spherical) because longitude and latitude are defined on the Earth's surface which forms a spherical coordinate system (you can read more about spherical regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#spherical-regridding).\n",
    "\n",
    "But that is not appropriate here because we aren't considering coordinates defined on a spherical surface, so we need to use *Cartesian* regridding. We use the `regridc` method of cf-python to do this, where the `c` in the name stands for the 'C' of Cartesian, which is the appropriate type here and in general you should use when considering any axes in Euclidean (ie. flat/non-curved) space (you can read more about Cartesian regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#cartesian-regridding).\n",
    "\n",
    "With that in mind, take the source and destinations fields we set up in the previous sub-sections and apply the cf-python Cartesian regridding operation to regrid `yearly_field` to the grid (i.e. the sampling resolution in this 1D case) of `monthly_field`, assigning the output to a new variable `linear_regridded_field`. Use the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, this time. You will need to set a further argument for Cartesian regridding, namely `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d428cede-8d0f-4c5e-b921-4ab7eb30d6f1",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "linear_regridded_field = yearly_field.regridc(monthly_field, method=\"linear\", axes=\"T\")\n",
    "print(linear_regridded_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01084a42-6564-40cc-9b96-0a945e03715c",
   "metadata": {},
   "source": [
    "**6.c.3)** Make a line plot of `linear_regridded_field`, including the argument `marker=\"o\"` so that we can see the discrete points the line is based upon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20e4025c-fe01-4ab4-b92f-74806a30d486",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.lineplot(linear_regridded_field, marker=\"o\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7071b0f4-c833-4fe4-a279-55c773979c75",
   "metadata": {},
   "source": [
    "**6.c.4)** This time, regrid the inverse way to before, so that our `yearly_field` this time is our destination field and our `monthly_field` is our source field. Again, use `regridc` with the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, and set `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis.\n",
    "\n",
    "Call this `inverse_linear_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32cabd25-e5c6-48ae-858e-9495751f70e0",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "inverse_linear_regridded_field = monthly_field.regridc(yearly_field, method=\"linear\", axes=\"T\")\n",
    "print(inverse_linear_regridded_field)\n",
    "cfp.lineplot(inverse_linear_regridded_field, marker=\"o\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3aaa456-411f-437d-92eb-aba7800113d1",
   "metadata": {},
   "source": [
    "**6.c.5)** Regrid the reverse way to before, like in (6.c.4), but this time use the `\"nearest_stod\"` method of `regridc` which stands for 'nearest neighbour source to destination' interpolation. This will do the same regridding but using a different approach to the underlying calculation based upon mapping each destination point to the closest source point.\n",
    "\n",
    "Call this `inverse_nearest_stod_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5e67ce3-f7eb-47bf-8c85-f1f9da6f4883",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "inverse_nearest_stod_regridded_field = monthly_field.regridc(yearly_field, method=\"nearest_stod\", axes=\"T\")\n",
    "print(inverse_nearest_stod_regridded_field)\n",
    "cfp.lineplot(inverse_nearest_stod_regridded_field, marker=\"o\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1286c96e-02f2-406e-a70d-914ae00389b5",
   "metadata": {},
   "source": [
    "### d) Finally, some more advanced cf-plot plotting to compare the source, destination, and regridded results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2196667-b7dd-4548-89e0-5194aa6170a5",
   "metadata": {},
   "source": [
    "**6.d.1)** Use cf-plot to make a plot with *two separate line plots* on one canvas (in either separate rows or columns, your choice), one of the first original field `yearly_field` and the linearly-regridded version of it, `linear_regridded_field`, so we can compare the original (un-regridded) and regridded result.\n",
    "\n",
    "Use the code block from the teaching Notebook section (6b) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas and call `cfp.gpos(N)` with an integer starting at `N=1` to tell cf-plot to move on to the next position in the plot before making a `lineplot` call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "910689b2-26d4-4973-bb4e-97cf1cfeb10a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.gopen(rows=1, columns=2)  # or could use rows=2, columns=1 for one column instead\n",
    "cfp.gpos(1)\n",
    "cfp.lineplot(\n",
    "    yearly_field, marker=\"o\", color=\"red\", title=\"Original time series... before regridding\")\n",
    "cfp.gpos(2)\n",
    "cfp.lineplot(linear_regridded_field, marker=\"o\", color=\"blue\", title=\"... and after regridding\")\n",
    "cfp.gclose()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7561067c-b9a0-45a8-9c37-87318528d39b",
   "metadata": {},
   "source": [
    "**6.d.2)** This time, use cf-plot to make a plot with *multiple line plots plotted together on the same (x- and y-) axes*. Plot both original fields along with the two regridded `inverse_*_regridded_field`, this time, and plot them in different colours of your choosing with sensible descriptive labels:\n",
    "1. `monthly_field`\n",
    "2. `yearly_field`\n",
    "3. `inverse_linear_regridded_field`\n",
    "4. `inverse_nearest_stod_regridded_field`\n",
    "\n",
    "Use the code block from the teaching Notebook section (5d) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` and by, not calling `cfp.gpos()` with position integers in the last question, all of the lines will be plotted onto the same canvas to share axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e80586f-08a7-44e9-bdde-8898f0a68650",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "clear_answer_cell"
    ]
   },
   "outputs": [],
   "source": [
    "cfp.gopen()\n",
    "cfp.lineplot(monthly_field, marker=\"o\", color=\"black\", label=\"Before regridding - source field\")\n",
    "cfp.lineplot(yearly_field, marker=\"o\", color=\"red\", label=\"Regridding to - destination field\")\n",
    "cfp.lineplot(inverse_linear_regridded_field, marker=\"o\", color=\"blue\", label=\"Linearly regridded\")\n",
    "cfp.lineplot(\n",
    "    inverse_nearest_stod_regridded_field, marker=\"o\", color=\"green\", label=\"Nearest source to destination method regridded\")\n",
    "cfp.gclose()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "877fa655-484a-4d24-a31d-b36d93a96ad8",
   "metadata": {},
   "source": [
    "**6.d.3)** Finally, study the two compound plots created in this section, in particular paying attention to the marker spacings representing the time data point sampling i.e. resolution which forms our grid in this context, to help you to ensure you understand the process of regridding.\n",
    "\n",
    "Also, look back to the horizontal spatial regridding example plot in (6d) of the teaching Notebook and pay particular attention to the cell block sizes from before and after regridding, to remind yourself the output from a spatial regridding operation.\n",
    "\n",
    "We've practiced regridding of fields in two contexts: spherical regridding in a 2D spatial context and Cartesian regridding in a 1D time series context. Hopefully these two examples have allowed you to understand what regridding does, but at least how cf-python can enable you to do it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08fa89e2-6c21-4118-9cf9-21958bdfa00e",
   "metadata": {},
   "source": [
    "***"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 + Jaspy",
   "language": "python",
   "name": "jaspy"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}