mapreader-autumn-workshop-2024-part2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MapReader Autumn Workshop (2024)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up for Google Colab\n",
    "\n",
    ">**NOTE**: Skip this section if you are not using Google Colab!\n",
    "\n",
    "The below cells will:\n",
    "\n",
    "- Mount your Google Drive\n",
    "- Create a directory for the workshop\n",
    "- Change the working directory to the workshop directory\n",
    "- Download and install the required packages and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# mount your drive\n",
    "from google.colab import drive\n",
    "drive.mount('/content/drive')\n",
    "\n",
    "# set up MapReader_Autumn_Workshop directory\n",
    "!mkdir /content/drive/MyDrive/MapReader_Autumn_Workshop\n",
    "%cd /content/drive/MyDrive/MapReader_Autumn_Workshop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/maps-as-data/mapreader-autumn-workshop-2024.git\n",
    "!pip install mapreader[dev]\n",
    "!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'\n",
    "!git clone https://github.com/maps-as-data/DeepSolo.git\n",
    "!python -m pip install 'git+https://github.com/maps-as-data/DeepSolo.git'\n",
    "!wget https://huggingface.co/rwood-97/DeepSolo_ic15_res50/resolve/main/ic15_res50_finetune_synth-tt-mlt-13-15-textocr.pth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# enable custom widgets in colab\n",
    "from google.colab import output\n",
    "output.enable_custom_widget_manager()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define root directory\n",
    "\n",
    ">**NOTE**: Start from here!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "\timport google.colab\n",
    "\tROOT = './mapreader-autumn-workshop-2024'\n",
    "except ImportError:\n",
    "\tROOT = '.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download maps\n",
    "\n",
    "https://mapreader.readthedocs.io/en/latest/using-mapreader/step-by-step-guide/1-download.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this workshop, we have pre-selected and downloaded two maps from the OS 6-inch 1st edition map series.\n",
    "These were downloaded from a tile layer hosted by the NLS. You can find more information about NLS tile layers [here](https://maps.nls.uk/).\n",
    "\n",
    "The two maps and their metadata are saved in the `maps` directory of the `mapreader-autumn-workshop-2024` repository (which we cloned earlier).\n",
    "\n",
    "We will use one to demonstrate annotating and training, and the other to demonstrate inference.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load maps and patchify\n",
    "\n",
    "https://mapreader.readthedocs.io/en/latest/using-mapreader/step-by-step-guide/2-load.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now load one map and its metadata using the `loader`.\n",
    "\n",
    "From here, we can patchify our map, visualise metadata and add further information about our map/patches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mapreader import loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_maps = loader(f\"{ROOT}/maps/map_74427695.png\") # load just one map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_maps.add_metadata(f\"{ROOT}/maps/metadata.csv\", ignore_mismatch=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_maps) # see which maps you have loaded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run text spotting, we will slice our maps into 1000x1000 pixel patches:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_maps.patchify_all(method=\"pixel\", patch_size=1000) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> If you now look in your files you will see a `patches_1000_pixel` directory which contains all the patches of your map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_maps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show a sample of the patches\n",
    "my_maps.show_sample(num_samples=3, tree_level=\"patch\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have these patches, we can create dataframes containing parent and patch information using the `convert_images()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_df, patch_df = my_maps.convert_images()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_df.head() # parent information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "patch_df.head() # patch information (showing only first 5 rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spot text\n",
    "\n",
    "https://mapreader.readthedocs.io/en/latest/using-mapreader/step-by-step-guide/6-spot-text.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MapReader offers three different text spotting frameworks:\n",
    "\n",
    "1. DPText-DETR (detection only)\n",
    "2. DeepSolo (detection and recognition)\n",
    "3. MapTextPipeline (detection and recognition)\n",
    "\n",
    "For this workshop, we will use the DeepSolo framework to spot text on our patches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mapreader import DeepSoloRunner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# paths to our config and weights files for the text spotting model\n",
    "cfg_file = \"./DeepSolo/configs/R_50/IC15/finetune_150k_tt_mlt_13_15_textocr.yaml\"\n",
    "weights_file = \"./ic15_res50_finetune_synth-tt-mlt-13-15-textocr.pth\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner = DeepSoloRunner(\n",
    "    patch_df,\n",
    "    parent_df = parent_df,\n",
    "    cfg_file = cfg_file,\n",
    "    weights_file = weights_file,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to time constraints, we will run the DeepSolo model on just one patch for the workshop.\n",
    "This is done using the `run_on_image()` method.\n",
    "\n",
    "We will also use the `return_dataframe` argument to return a dataframe which makes looking at the results easier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "patch_predictions = my_runner.run_on_image(\"./patches_1000_pixel/patch-0-0-1000-1000-#map_74427695.png#.png\", return_dataframe=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "patch_predictions.head() # view the predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view the results using the `show()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner.show(\n",
    "    \"patch-0-0-1000-1000-#map_74427695.png#.png\", # patch id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've saved the predictions for the rest of our patches in the `text_predictions` directory of the `mapreader-autumn-workshop-2024` repository.\n",
    "\n",
    "These are saved as a `pkl` file, which we can load using the `pickle` library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "with open(f\"{ROOT}/text_predictions/patch_predictions.pkl\", \"rb\") as f:\n",
    "\tmy_runner.patch_predictions = pickle.load(f) # load these as the patch predictions attribute"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we've loaded these, we can use the `show()` method to visualise the predictions for any patch we like:\n",
    "\n",
    "> **NOTE**: You can change the patch id to view the predictions for different patches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner.show(\n",
    "    \"patch-0-4000-1000-5000-#map_74427695.png#.png\", # patch id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can convert the patch pixel bounds to parent pixel bounds using the `convert_to_parent_pixel_bounds()` method.\n",
    "\n",
    "This rescales the pixel bounds of the text predictions to the parent image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_predictions = my_runner.convert_to_parent_pixel_bounds(return_dataframe=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_predictions.head() # view the predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can view the results for a whole parent image using the `show()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner.show(\n",
    "    \"map_74427695.png\", # parent id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we added metadata to our map earlier, we can also convert pixel bounds to geographic coordinates using the `convert_to_coords()` method. \n",
    "\n",
    "Once we have this, we can export our results to a GeoJSON file and load them into a GIS software to visualize them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "geo_predictions = my_runner.convert_to_coords(return_dataframe=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "geo_predictions.head() # view the predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner.save_to_geojson(\"deepsolo_text_predictions.geojson\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Search in results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can search our results using the `search_preds()` method. \n",
    "\n",
    "This will search in the parent predictions NOT the georeferenced predictions so we will need to convert our search results to coordinates later!\n",
    "\n",
    "> **NOTE**: This method accepts regex patterns. Feel free to try this out if you are familiar with regex."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "search_results = my_runner.search_preds(\"church\", ignore_case=True, return_dataframe=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "search_results.head() # view the search results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also view these results on our parent images using the `show_search_results()` method.\n",
    "\n",
    "Since we only have two parent maps, we can pick one to show results for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_runner.show_search_results(\n",
    "    \"map_74427695.png\", # parent id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And lastly, we can convert our search results to coordinates and save them using the `save_search_results_to_geojson()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "geo_search_results = my_runner.save_search_results_to_geojson(\"search_results.geojson\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mr_py312",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}