Merge pull request #95 from cloudnativegeo/staging

New Python Notebook doing a deeper dive into COG Overview Resampling
cloudnativegeo · Jan 4, 2024 · a54ed18 · a54ed18
2 parents 9509048 + 838eeb4
commit a54ed18
Show file tree

Hide file tree

Showing 5 changed files with 715 additions and 48 deletions.
diff --git a/_quarto.yml b/_quarto.yml
@@ -38,6 +38,7 @@ website:
           - cloud-optimized-geotiffs/intro.qmd
           - cloud-optimized-geotiffs/cogs-details.qmd
           - cloud-optimized-geotiffs/cogs-examples.ipynb
+          - cloud-optimized-geotiffs/cogs-overview_resampling.ipynb
         - section: Zarr
           contents:
           - zarr/intro.qmd

diff --git a/cloud-optimized-geotiffs/cogs-examples.ipynb b/cloud-optimized-geotiffs/cogs-examples.ipynb
@@ -9,6 +9,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "793aee06",
    "metadata": {},
@@ -18,28 +19,37 @@
     "The packages needed for this notebook can be installed with `conda` or `mamba`. Using the [`environment.yml` from this folder](./environment.yml) run:\n",
     "\n",
     "```bash\n",
-    "conda create -f environment.yml\n",
+    "conda env create -f environment.yml\n",
     "```\n",
     "\n",
     "or\n",
     "\n",
     "```bash\n",
-    "mamba create -f environment.yml\n",
+    "mamba env create -f environment.yml\n",
     "```\n",
     "\n",
-    "This notebook has been tested to work with the listed Conda environment."
+    "Finally, you may activate and select the kernel in the notebook (running in Jupyter)\n",
+    "\n",
+    "```bash\n",
+    "conda activate coguide-cog\n",
+    "```\n",
+    "\n",
+    "The notebook has been tested to work with the listed Conda environment."
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "5a8aa8b4",
    "metadata": {},
    "source": [
     "## Setup\n",
     "\n",
-    "For demonstrating some COG concepts, we will download a regular GeoTIFF, create a Cloud-Optimized GeoTIFF and explore how they are different.\n",
+    "To demonstrate some COG concepts, we will download a regular GeoTIFF, create a Cloud-Optimized GeoTIFF, and explore their differences.\n",
+    "\n",
+    "To access and integrate NASA Earth data into your Jupyter Notebook, you can create an account by visiting [NASA's Earthdata Login page](https://urs.earthdata.nasa.gov/users/new). This will enable you to register for an account and retrieve the datasets used in the notebook.\n",
     "\n",
-    "First we use the [earthaccess](https://github.com/nsidc/earthaccess) library to setup credentials to fetch data from NASA's EarthData catalog."
+    "First, we use the [earthaccess](https://github.com/nsidc/earthaccess) library to set up credentials to fetch data from NASA's EarthData catalog."
    ]
   },
   {
@@ -66,43 +76,23 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "id": "ad220208",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "EARTHDATA_USERNAME and EARTHDATA_PASSWORD are not set in the current environment, try setting them or use a different strategy (netrc, interactive)\n",
-      "You're now authenticated with NASA Earthdata Login\n",
-      "Using token with expiration date: 10/24/2023\n",
-      "Using .netrc file for EDL\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "<earthaccess.auth.Auth at 0x10427d390>"
-      ]
-     },
-     "execution_count": 2,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "earthaccess.login()"
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "8ecd2c7c",
    "metadata": {},
    "source": [
     "## Download a GeoTIFF from EarthData\n",
     "\n",
-    "Note: The whole point of  is that we _don't_ download data. So in future examples, we will demonstrate how to access just subsets of data using COG and compare that with a GeoTIFF."
+    "Note: The whole point is that we _don't_ download data. So, in future examples, we will demonstrate how to access just subsets of data using COG and compare that with a GeoTIFF."
    ]
   },
   {
@@ -182,13 +172,14 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "451dbe01",
    "metadata": {},
    "source": [
     "## Is it a valid COG?\n",
     "\n",
-    "We can use `rio_cogeo.cog_validate` to check. It returns `is_valid`, `errors` and `warnings`."
+    "We can use `rio_cogeo.cog_validate` to check. It returns `is_valid`, `errors`, and `warnings`:"
    ]
   },
   {
@@ -226,25 +217,27 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "25a82b89",
    "metadata": {},
    "source": [
-    "Return values:\n",
+    "Here's some more context on the output message:\n",
     "    \n",
     "1. `is_valid` is `False`: this is not a valid COG.\n",
     "2. `errors` are `'The file is greater than 512xH or 512xW, but is not tiled'`. To be a valid COG, the file should be tiled since it has a height and width both greater than 512.\n",
     "3. `warnings` are `'The file is greater than 512xH or 512xW, it is recommended to include internal overviews'`. It is recommended to provide overviews."
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "f2826f57-6df4-458f-98b2-f639f25450ba",
    "metadata": {},
    "source": [
     "## Converting a GeoTIFF to COG\n",
     "\n",
-    "We can use `rio_cogeo.cog_create` to convert a GeoTIFF into a Cloud Optimized GeoTIFF"
+    "We can use `rio_cogeo.cog_create` to convert a GeoTIFF into a Cloud Optimized GeoTIFF:"
    ]
   },
   {
@@ -293,25 +286,25 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "35f1787d",
    "metadata": {},
    "source": [
-    "This is a valid COG, so we will use it to compare with our GeoTIFF."
+    "The file is a valid COG, so we will use it to compare with our GeoTIFF."
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "ff2cc531",
    "metadata": {},
    "source": [
     "## Data Structure\n",
     "\n",
-    "**Dimensions**\n",
-    "Dimensions are the number of bands, rows and columns stored in a GeoTIFF. [More Info](intro.qmd#dimensions)\n",
+    "**Dimensions**: the number of bands, rows, and columns stored in a GeoTIFF. [More Info](intro.qmd#dimensions)\n",
     "\n",
-    "**Internal Blocks** (aka chunks or internal tiles)\n",
-    "Internal blocks are required if the dimensions of data are over 512x512. [More Info](intro.qmd#internal-blocks)\n",
+    "**Internal Blocks (chunks or internal tiles)**: Internal blocks are required if the data dimensions are over 512x512. [More Info](intro.qmd#internal-blocks)\n",
     "\n",
     "Let's check out the dimensions and blocks of our GeoTIFF and Cloud-Optimized GeoTIFF."
    ]
@@ -357,13 +350,14 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "51828845-3531-4813-b694-25a4563a58f5",
    "metadata": {},
    "source": [
-    "They have the same dimensions which is what we expect, so that is good!\n",
+    "They have the exact dimensions that we expected, which is good!\n",
     "\n",
-    "We can also print information about the GeoTIFF's IFD (Internal File Directory). Only one item is returned because the GeoTIFF doesn't have overviews. When we print the IFD info for the COG, which has overviews, we see more items returned."
+    "We can also print information about the GeoTIFF's IFD (Internal File Directory). Only one item is returned because the GeoTIFF needs overviews. We see more items returned when we print the IFD info for the COG, which has overviews."
    ]
   },
   {
@@ -420,6 +414,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "12c0b3e4",
    "metadata": {},
@@ -428,14 +423,14 @@
     "\n",
     "Overviews are downsampled (aggregated) data intended for visualization.\n",
     "\n",
-    "The smallest size overview should match the tiling components’ fetch size, typically 256x256. Due to aspect ratio variation just aim to have at least one dimension at or slightly less than 256. \n",
-    "> The COG driver in GDAL, or rio cogeo tools should do this.\n",
+    "The most miniature size overview should match the tiling components’ fetch size, typically 256x256. Due to aspect ratio variation, aim to have at least one dimension at or slightly less than 256. \n",
+    "> The COG driver in GDAL or rio cogeo tools should do this.\n",
     "\n",
-    "There are many resampling algorithms for generating overviews. The best resampling algorithm depends on the range, type, and distribution of the data. When creating overviews several options should be compared before deciding which resampling method to apply. \n",
+    "There are many resampling algorithms for generating overviews. The best resampling algorithm depends on the data's range, type, and distribution. When creating overviews, several options should be compared before deciding which resampling method to apply. \n",
     "\n",
     "GDAL >= 3.2 allows for the overview resampling method to be set directly.\n",
     "\n",
-    "<!-- TODO: need to add hints on how to check which resampling method to use for overviews. Possibly provide code for comparing. Alex has a draft of this to add.-->"
+    "For more information on overviews, see the [COG overview resampling notebook](cogs-overview_resampling.ipynb)."
    ]
   },
   {