updating demo notebooks

demo_sfm_pipeline.ipynb

@@ -0,0 +1,342 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# RUN SfM Pipeline with Deep-Image-Matching\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Initialization\n",
+        "\n",
+        "Let's first import DIM and load a logger to see what's going on\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 1,
+      "metadata": {},
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "Deep Image Matching loaded in 1.891 seconds.\n"
+          ]
+        }
+      ],
+      "source": [
+        "from pprint import pprint\n",
+        "\n",
+        "import deep_image_matching as dim\n",
+        "\n",
+        "logger = dim.setup_logger(\"dim\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Define the configuration\n",
+        "\n",
+        "Get the list of possible pipelines and matching strategy and chose one of them."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(\"Available pipelines:\")\n",
+        "pprint(dim.Config.get_pipelines())\n",
+        "print(\"Available matching strategy:\")\n",
+        "pprint(dim.Config.get_matching_strategies())"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Now you have to build a dictionary with the input processing parameters (they are the same as the input parameters for the CLI) and pass it to the Config class to initialize the configuration object. Refer to the [documentation](https://3dom-fbk.github.io/deep-image-matching/) for more information about the parameters.\n",
+        "\n",
+        "Note that the `dir` defines the project directory, where the images are stored and the results will be saved.\n",
+        "Deep-Image-Matching will search for the images inside an 'image' subdirectory and will save the results in a 'results\\_{processing_params}' subdirectory, where {processing_params} are some information on the processing parameters used.\n",
+        "\n",
+        "By default DIM will not run if the output directory already exists, to avoid overwriting previous results. If you want to overwrite the results, you can set the `force` parameter to True. We have not implemented the possibility to recover the previous results yet (e.g., by using existing extracted features), but we may add it in the future.\n",
+        "\n",
+        "The `config_file` parameter is the path to the configuration file (optional). In this file you can specify all the parameters that you need for controlling the feature extraction and matching. Refer to the [documentation](https://3dom-fbk.github.io/deep-image-matching/advanced_configuration/) for more information about how to write this file. Note that this file il optional, if you don't pass it, the default parameters will be used.\n",
+        "\n",
+        "If you use set `verbose` to True, DIM will log all the processing steps and it will save some figures with the extracted features and the matches in a `debug` folder inside the results directory. Note that this will slow down the processing and will create a lot of files if the dataset is big. It is reccomended to use it only for testing or debugging purposes. \n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "params = {\n",
+        "    \"dir\": \"../assets/example_cyprus\",\n",
+        "    \"pipeline\": \"superpoint+lightglue\",\n",
+        "    \"config_file\": \".../assets/example_cyprus/config_superpoint+lightglue.yaml\",\n",
+        "    \"strategy\": \"matching_lowres\",\n",
+        "    \"quality\": \"high\",\n",
+        "    \"tiling\": \"preselection\",\n",
+        "    \"skip_reconstruction\": False,\n",
+        "    \"force\": True,\n",
+        "    \"camera_options\": \"../config/cameras.yaml\",\n",
+        "    \"openmvg\": None,\n",
+        "    \"verbose\": False,\n",
+        "}\n",
+        "config = dim.Config(params)\n",
+        "\n",
+        "# Save the configuration to a json file for later use\n",
+        "config.save()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can check the configuration object.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(\"Config general:\")\n",
+        "pprint(config.general)\n",
+        "print(\"Config extractor:\")\n",
+        "pprint(config.extractor)\n",
+        "print(\"Config matcher:\")\n",
+        "pprint(config.matcher)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Run the extraction and matching\n",
+        "\n",
+        "First, you have to create an instance of the ImageMatching class and pass the configuration object to it.\n",
+        "\n",
+        "Then you can run the extraction and matching by calling the `run` method.\n",
+        "This method will automatically run all the steps needed to extract the features and match the images. It will return the path to the h5 files containing the features and the matches.\n",
+        "The `features.h5` file contains the features extracted from each images, while the `matches.h5` file contains the indices of the features matched.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Initialize ImageMatcher class\n",
+        "matcher = dim.ImageMatcher(config)\n",
+        "\n",
+        "# Run image matching\n",
+        "feature_path, match_path = matcher.run()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "#### The `features.h5` file\n",
+        "\n",
+        "#### The `matches.h5` file\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# # print features_h5 content\n",
+        "# with h5py.File(features_h5, \"r\") as f:\n",
+        "#     print(f.keys())\n",
+        "#     print(f[images[0].name].keys())\n",
+        "#     print(f[images[0].name][\"keypoints\"][:])\n",
+        "#     print(f[images[1].name].keys())\n",
+        "#     print(f[images[1].name][\"keypoints\"][:])\n",
+        "\n",
+        "# # Print matches.h5 content\n",
+        "# with h5py.File(matches_h5, \"r\") as f:\n",
+        "#     print(f.keys())\n",
+        "#     g0 = f[images[0].name]\n",
+        "#     print(g0.keys())\n",
+        "#     g1 = g0[images[1].name]\n",
+        "#     print(g1.__array__())"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Export in colmap format\n",
+        "\n",
+        "Then you can use the `export_to_colmap` function that will read the features and the matches from the h5 files and will save them in a COLMAP sqliite database. \n",
+        "DIM assigns camera models to images based on the options defined in `cameras.yaml` file."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Read camera options\n",
+        "with open(config.general[\"camera_options\"], \"r\") as file:\n",
+        "    camera_options = yaml.safe_load(file)\n",
+        "\n",
+        "database_path = config.general[\"output_dir\"] / \"database.db\"\n",
+        "dim.io.export_to_colmap(\n",
+        "    img_dir=config.general[\"image_dir\"],\n",
+        "    feature_path=feature_path,\n",
+        "    match_path=match_path,\n",
+        "    database_path=database_path,\n",
+        "    camera_options=camera_options,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Alternatively you can assign camera models with a dictionary.\n",
+        "\n",
+        "For images not assigned to specific `cam<x>` camera groups, the options specified under `general` are applied. The `camera_model` can be selected from `[\"simple-pinhole\", \"pinhole\", \"simple-radial\", \"opencv\"]`. It's worth noting that it's easily possible to extend this to include all the classical COLMAP camera models. Cameras can either be shared among all images (`single_camera == True`), or each camera can have a different camera model (`single_camera == False`).\n",
+        "\n",
+        "A subset of images can share intrinsics using `cam<x>` key, by specifying the `camera_model` along with the names of the images separated by commas. There's no limit to the number of `cam<x>` entries you can use.\n",
+        "\n",
+        "**Note**: Use the SIMPLE-PINHOLE camera model if you want to export the solution to Metashape later, as there are some bugs in COLMAP (or pycolamp) when exportingthe solution in the Bundler format.\n",
+        "e.g., using FULL-OPENCV camera model, the principal point is not exported correctly and the tie points are wrong in Metashape.\n",
+        "\n",
+        "```python\n",
+        "camera_options = {\n",
+        "    \"general\": {\n",
+        "        \"camera_model\": \"pinhole\",  # [\"simple-pinhole\", \"pinhole\", \"simple-radial\", \"opencv\"]\n",
+        "        \"single_camera\": True,\n",
+        "    },\n",
+        "    \"cam0\": {\n",
+        "        \"camera_model\": \"pinhole\",\n",
+        "        \"images\": \"DSC_6468.JPG,DSC_6468.JPG\",\n",
+        "    },\n",
+        "    \"cam1\": {\n",
+        "        \"camera_model\": \"pinhole\",\n",
+        "        \"images\": \"\",\n",
+        "    },\n",
+        "}\n",
+        "```\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Run reconstruction\n",
+        "\n",
+        "You can run the reconstruction with pycolmap, OpenMVG or MICMAC. The suggested method is pycolmap, as it is the most integrated with DIM.\n",
+        "\n",
+        "Note that pycolmap is not installed by default with DIM, so you have to install it manually. You can find the installation instructions [here](https://3dom-fbk.github.io/deep-image-matching/installation/).\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "try:\n",
+        "    import pycolmap\n",
+        "except ImportError:\n",
+        "    logger.error(\"Pycomlap is not available.\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "If the pycolmap module is imported, you can define all the parameters for the COLAMP reconstruction.\n",
+        "You can check all the available parameters with:\n",
+        "\n",
+        "```python\n",
+        "print(pycolmap.IncrementalPipelineOptions().summary())\n",
+        "```\n",
+        "\n",
+        "Alternatevely, you can leave the dictionary empty for using the default confiuration\n",
+        "\n",
+        "```python\n",
+        "reconst_opts = {}\n",
+        "```\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Run reconstruction\n",
+        "opt = dict(\n",
+        "    triangulation=dict(\n",
+        "        ignore_two_view_tracks=False,\n",
+        "        min_angle=0.5,\n",
+        "    ),\n",
+        "    mapper=dict(filter_min_tri_angle=0.5, filter_max_reproj_error=5.0),\n",
+        ")\n",
+        "refine_intrinsics = False\n",
+        "verbose = False\n",
+        "\n",
+        "model = dim.reconstruction.pycolmap_reconstruction(\n",
+        "    database_path=config.general[\"output_dir\"] / \"database.db\",\n",
+        "    sfm_dir=config.general[\"output_dir\"],\n",
+        "    image_dir=config.general[\"image_dir\"],\n",
+        "    refine_intrinsics=refine_intrinsics,\n",
+        "    options=opt,\n",
+        "    verbose=verbose,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Print COLMAP camera values\n",
+        "print(list(model.cameras.values()))"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "deep_image_matching",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.14"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}