embryonic essay about weather prediction production.

docs/source/Contribution/Philosophy/AboutTime.ipynb

@@ -0,0 +1,146 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "6395a549",
+   "metadata": {},
+   "source": [
+    "# It´s About Time\n",
+    "\n",
+    "Say you are a weather forecaster, charged with producing a prediction of the weather, perhaps for the public, or perhaps for a vertical domain, such as aviation, or transport. In normal situations, the forecaster will be asked to produce a forecast at a specific time of day, to help her clients plan their business.\n",
+    "\n",
+    "How does a forecaster build a forecast? \n",
+    "\n",
+    "Assuming the forecaster is in the americas, one could use a browser:\n",
+    "\n",
+    "* look at a picture of the sky from a GOES satellite image (visible spectrum), to see where it is cloudy at the moment.\n",
+    "* look at a different web site for closer in shots from polar orbiting satellites (e.g. HRDPS) that cover a smaller area.\n",
+    "* look at a United States Geological Survey, or Water Survey of Canada web site to see the current stream flows.\n",
+    "* look at a some weather web sites, to see the current observations from surface stations.\n",
+    "* look at some Aviation (FAA or NAVCANADA web sites) to find weather observations at airports (which tend to have a tightly controlled quality, and so are relatively reliable.)\n",
+    "* Find RADAR imagery on the net for the areas of interest (it might be that two or three RADAR stations have coverage of the area one is looking at.\n",
+    "\n",
+    "If you want to look at all that information together... you build a really large spreadsheet where each cell describes one part of the sky, and for each cell, you would store something like temperature, barometric pressure, and wind (direction and speed.)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2502a307",
+   "metadata": {},
+   "source": [
+    "Now to get movement, you need to be looking at a trend, not just a snapshot.  \n",
+    "From one picture, you don´t know the direction that anything is moving. \n",
+    "So you really need at least two snapshots to get speed.\n",
+    "\n",
+    "Cloud is a one location, p0 at an initial time, and and has moved to p1 at time t1."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "703ed5c2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#illustration with two points."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8d6d3a94",
+   "metadata": {},
+   "source": [
+    "\n",
+    "Now to predict the future, with just two points in time, you have to assume stuff is always moving at the same speed. If you think air masses are accellerating or slowing down, you need another picture. With three of them you can compare the speeds between the first two pictures, and the last two pictures, and see a change in speed.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5c1a4040",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#illustration with three points."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e5006a41",
+   "metadata": {},
+   "source": [
+    "Now with three points, you have constant accelleration... but if the acceleration itself is varying, then you need more again."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eb70daaa",
+   "metadata": {},
+   "source": [
+    "The more points in time you have, the better you can understand the movement to establish the wind speed and direction at a given point in time, and how it will change in future.\n",
+    "\n",
+    "So once you have good information about the clouds that exist, and the rate they are moving at, and whether they are speeding up, or building up, or dissipating... then a forecaster is trained in physics, and applies the rules of physics to understand how clouds will move in the future.   Essentially it uses the spreadsheet to calculate what the sky should look like one small step later in time, and then another step, and another, until you have arrived as far in time as needed for the forecast product.\n",
+    "\n",
+    "At this point, the result is a spreadsheet. The forecaster then can use the spreadsheet and write descriptions of weather for their client, or have an automated process do that, or produce simlated ¨satellite\" imagery to show where the clouds till in the future.\n",
+    "\n",
+    "Restating things:\n",
+    "\n",
+    "* step 1: Acquisiion: gather a time series of data for kinds of data.\n",
+    "* step 2: Assimilation: put all the different data into a pile of spreadsheets.\n",
+    "* step 3: Numerical Model: hit calculate on the spreadsheet for the number of timestesp you need.\n",
+    "* step 4: Services: translating the spreadsheets back into things people can understand.\n",
+    "\n",
+    "\n",
+    "Looking at all these steps, it is obvious that they are extremely tedious for a human to do, and things that a computer, in principle should be great at. Rather than having a human look at web sites, and extract data,\n",
+    "have a computer do it, and present the forecaster with a best first guess product.\n",
+    "\n",
+    "## Let the Computer Do It\n",
+    "\n",
+    "Instead of a human browsing dozens of web sites and mapping stuff into a spreadsheet manually, and then hitting calculate, and then mapping back the spreadsheet back to something his clients will understand, the forecaster hits a button, and has the computer does all the tedious work. So when the forecaster pushes the button, the computer:\n",
+    "\n",
+    "* scans the entire world for a number of hours of satellite, RADAR, and point observations from weather stations, airports, ships, and planes.  looking for information for the last hour or two to get a good trend.\n",
+    "* stuffs that into a spreadsheet.\n",
+    "* runs the spreadsheet.\n",
+    "* extracts the interesting numbers or pictures from the spreadsheet.\n",
+    "\n",
+    "Great, how long will that take, and how good will the result be?\n",
+    "\n",
+    "Well the quality of the result will vary with the quality of the input. To make a spreadsheet, the forecaster decides how big an volume of space to cover with each cell. The bigger the volume covered by each cell, the more you are taking different data points and averaging them to get one value for the the cell, so the \"fuzzier\" the pictures that result.  \n",
+    "\n",
+    "How big are the spreadsheets? 30 years ago, the \"high resolution\" spreadsheet covering North America had each cell was 150 km. on a side, At that time, computers were not big enough to cover the whole world. Today, the high resolution models are around 10km on a side (100 sq. km.) As the area of the earth is around 500 million sq. km. that means that the grid today should be about 5 million cells. per level, models typically have 25 levels representing different heights of air in the atmosphere, so that means 125 million cells to calculate.\n",
+    "\n",
+    "Each cell has information about different phenomena, called variables, say 25 of them. so to get a full picture, one needs 125 million x 25 variables x 8 bytes per variable = 6.2 billion variables in 25 gbytes of memory.  To calculate the what the values of those variables are at the next unit in time would involve \n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "64fb191a",
+   "metadata": {},
+   "source": [
+    "* https://en.wikipedia.org/wiki/History_of_numerical_weather_prediction#cite_note-RFE-43 note on Weather model from 1989\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}