2023 edition - update materials (#64)

jorisvandenbossche · Sep 19, 2023 · 1e888a7 · 1e888a7
1 parent 6ae2cac
commit 1e888a7
Show file tree

Hide file tree

Showing 50 changed files with 674 additions and 592 deletions.
diff --git a/_solved/00-jupyter_introduction.ipynb b/_solved/00-jupyter_introduction.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -548,7 +548,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "nav_menu": {},
   "toc": {

diff --git a/_solved/01-introduction-tabular-data.ipynb b/_solved/01-introduction-tabular-data.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -231,6 +231,22 @@
     "countries.sort_values(by='pop_est')"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div style=\"border: 5px solid #3776ab; border-radius: 2px; padding: 2em;\">\n",
+    "\n",
+    "## Python recap\n",
+    "\n",
+    "Python objects have **attributes** and **methods**:\n",
+    "\n",
+    "* Attribute: `obj.attribute` (no parentheses!) -> property of the object (pandas examples: `dtypes`, `columns`, `shape`, ..)\n",
+    "* Method: `obj.method()` (function call with parentheses) -> action (pandas examples: `mean()`, `sort_values()`, ...)\n",
+    "\n",
+    "</div>"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -524,7 +540,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Often, you want to select rows based on a certain condition. This can be done with *'boolean indexing'* (like a where clause in SQL). \n",
+    "Often, you want to select rows based on a certain condition. This can be done with *'boolean indexing'* (like a WHERE clause in SQL).\n",
     "\n",
     "The indexer (or boolean mask) should be 1-dimensional and the same length as the thing being indexed."
    ]
@@ -1886,6 +1902,7 @@
  "metadata": {
   "celltoolbar": "Nbtutor - export exercises",
   "jupytext": {
+   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow",
    "formats": "ipynb,md:myst"
   },
   "kernelspec": {
@@ -1903,7 +1920,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

diff --git a/_solved/02-introduction-geospatial-data.ipynb b/_solved/02-introduction-geospatial-data.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -377,7 +377,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from shapely.geometry import Point, Polygon, LineString"
+    "from shapely import Point, Polygon, LineString"
    ]
   },
   {
@@ -1061,6 +1061,9 @@
  ],
  "metadata": {
   "celltoolbar": "Nbtutor - export exercises",
+  "jupytext": {
+   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow"
+  },
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",
@@ -1076,7 +1079,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

diff --git a/_solved/03-coordinate-reference-systems.ipynb b/_solved/03-coordinate-reference-systems.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -533,6 +533,10 @@
  ],
  "metadata": {
   "celltoolbar": "Nbtutor - export exercises",
+  "jupytext": {
+   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow",
+   "notebook_metadata_filter": "-jupytext.cell_metadata_filter"
+  },
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",
@@ -548,7 +552,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

diff --git a/_solved/04-spatial-relationships-joins.ipynb b/_solved/04-spatial-relationships-joins.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -726,7 +726,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "france = countries.loc[countries['name'] == 'France', 'geometry'].squeeze()"
+    "france = countries.loc[countries['name'] == 'France', 'geometry'].item()"
    ]
   },
   {
@@ -1110,6 +1110,9 @@
   }
  ],
  "metadata": {
+  "jupytext": {
+   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow"
+  },
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",
@@ -1125,7 +1128,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

diff --git a/_solved/05-spatial-operations-overlays.ipynb b/_solved/05-spatial-operations-overlays.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey  (<mailto:[email protected]>, <mailto:[email protected]>). Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -301,7 +301,7 @@
    },
    "outputs": [],
    "source": [
-    "print(str(africa)[:1000])"
+    "print(str(africa)[:100])"
    ]
   },
   {
@@ -1411,9 +1411,6 @@
  ],
  "metadata": {
   "celltoolbar": "Nbtutor - export exercises",
-  "jupytext": {
-   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow"
-  },
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",
@@ -1429,7 +1426,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

diff --git a/_solved/10-introduction-raster.ipynb b/_solved/10-introduction-raster.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey. Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey. Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -221,7 +221,7 @@
     "    - `x`: the x coordinates of the data set\n",
     "- Each of these dimensions are defined by a __coordinate__ (1D) array\n",
     "- It contains 304407 (227\\*447\\*3) data values (stored as float32)\n",
-    "- Other metadata, such as the spatial information) provided by the `tiff` are stored in the __`Attributes`__\n",
+    "- Other metadata, such as some of the metadata provided by the `tiff` file are stored in the __`Attributes`__. The attributes is a dictionary.\n",
     "\n",
     "Looking to the data itself (click on the icons on the right), we can see these are Numpy arrays..."
    ]
@@ -546,6 +546,13 @@
     "\n",
     "</div>"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -568,7 +575,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   }
  },
  "nbformat": 4,

diff --git a/_solved/11-xarray-intro.ipynb b/_solved/11-xarray-intro.ipynb
@@ -8,9 +8,9 @@
     "\n",
     "\n",
     "> *DS Python for GIS and Geoscience*  \n",
-    "> *November, 2022*\n",
+    "> *September, 2023*\n",
     ">\n",
-    "> *© 2022, Joris Van den Bossche and Stijn Van Hoey. Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
+    "> *© 2023, Joris Van den Bossche and Stijn Van Hoey. Licensed under [CC BY 4.0 Creative Commons](http://creativecommons.org/licenses/by/4.0/)*\n",
     "\n",
     "---"
    ]
@@ -426,14 +426,15 @@
     "\n",
     "**EXERCISE**:\n",
     "\n",
-    "* Read in the file `./data/gent/raster/2020-09-17_Sentinel_2_L1C_True_color.tiff` with xarray and assign the data to a new variable `tc_data`.  \n",
+    "* Read in the file `./data/gent/raster/2020-09-17_Sentinel_2_L1C_True_color.tiff` with xarray in the original raw data dtype and assign the data to a new variable `tc_data`.  \n",
     "* Inspect the display of `tc_data`. What are the different dimensions of the array? \n",
     "* Select only the *second* layer of `tc_data` and assign the output to a new variable `tc_g`.\n",
     "* Plot the second (green) layer.\n",
     "* Create a new array with each of the elements in the `tc_g` array with a value above 15000 set to a new value of 65535.\n",
     "    \n",
     "<details><summary>Hints</summary>\n",
     "\n",
+    "* Use the `mask_and_scale` parameter in the reader function to make sure the data type of tc_data is the same as the raw data.\n",
     "* To select a specific subset for a certain dimension of a DataArray, use the `.sel()` method. The argument name to use is the name of the dimension.\n",
     "* You can mask an array and set the masked values to another value with the `.where()` method. Check the help of the method for information on the keyword arguments.    \n",
     "\n",
@@ -595,7 +596,7 @@
    },
    "outputs": [],
    "source": [
-    "b4_data = xr.open_dataarray(\"./data/gent/raster/2020-09-17_Sentinel_2_L1C_B04.tiff\", \n",
+    "b4_data = xr.open_dataarray(\"./data/gent/raster/2020-09-17_Sentinel_2_L1C_True_color.tiff\", \n",
     "                            engine=\"rasterio\", mask_and_scale=False)"
    ]
   },
@@ -662,11 +663,18 @@
    "outputs": [],
    "source": [
     "# Create the histogram plots\n",
-    "fig, (ax0, ax1) = plt.subplots(1, 2, sharey=True)\n",
+    "fig, (ax0, ax1) = plt.subplots(1, 2, sharey=True, figsize=(12, 4))\n",
     "b4_data.plot.hist(bins=30, log=True, ax=ax0)\n",
     "b4_data_f.plot.hist(bins=30, log=True, ax=ax1);"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This is what the `mask_and_scale` option also does: it reads the nodata value from the metadata in the tiff file and uses this to mask the no-data value with a Nan value."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1156,7 +1164,7 @@
     "\n",
     "The data set `./data/herstappe/raster/2020-09-17_Sentinel_2_L1C_True_color.tiff` (assign to variable `herstappe_data`) has values ranging in between 0.11325, 0.8575. To improve the quality of the visualization, stretch __each of the layers individually__ to the values to the range 0. to 1. with a linear transformation: \n",
     "    \n",
-    "$$z_i=\\frac{x_i-\\min(x)}{\\max(x)-\\min(x)}$$\n",
+    "$$z_{i,scaled}=\\frac{z_i-\\min(z)}{\\max(z)-\\min(z)}$$\n",
     "\n",
     "Make a plot of the end result and compare with a plot of the original data. \n",
     "\n",
@@ -1251,7 +1259,6 @@
     "- Read the data file and assign to a variable `gent`.\n",
     "- Try to plot it with the `imshow()` method.\n",
     "- Convert the array to a float array and call it `gent_f`.\n",
-    "- Convert the max value of 65536 to np.nan (as we have done before).\n",
     "- Now divide the array by 65536 to get our data in a [0-1] range.\n",
     "- Plot the result with the `imshow()` method.\n",
     "\n",
@@ -1303,9 +1310,8 @@
    },
    "outputs": [],
    "source": [
-    "# Convert to float and make 65535 equal to Nan\n",
-    "gent_f = gent.astype(float)\n",
-    "gent_f = gent_f.where(gent != 65535)"
+    "# Convert to float\n",
+    "gent_f = gent.astype(float)"
    ]
   },
   {
@@ -1648,7 +1654,8 @@
  "metadata": {
   "celltoolbar": "Nbtutor - export exercises",
   "jupytext": {
-   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow"
+   "cell_metadata_filter": "-run_control,-deletable,-editable,-jupyter,-slideshow",
+   "notebook_metadata_filter": "-jupytext.cell_metadata_filter"
   },
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
@@ -1665,7 +1672,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.11.5"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {