From 7bbd835c5225c94ccd3ee2c64f3829e8b20941e9 Mon Sep 17 00:00:00 2001 From: Aleksandra Badaczewska Date: Tue, 12 Sep 2023 14:41:17 -0500 Subject: [PATCH] added info about setting up .conda on HPC --- .../OpenDroneMap/03-ODM-georeferencing.md | 49 ++++++++++++++++--- 1 file changed, 43 insertions(+), 6 deletions(-) diff --git a/IntroPhotogrammetry/OpenDroneMap/03-ODM-georeferencing.md b/IntroPhotogrammetry/OpenDroneMap/03-ODM-georeferencing.md index 76b5e3b..1a63282 100644 --- a/IntroPhotogrammetry/OpenDroneMap/03-ODM-georeferencing.md +++ b/IntroPhotogrammetry/OpenDroneMap/03-ODM-georeferencing.md @@ -149,6 +149,7 @@ _____ _____ _____ ____ ____ __________ __________ ________ ________ geo_x geo_y geo_z im_x im_y image_name [gcp_name] [extra1] [extra2] ``` +--- # Software for manual detection of (any) GCPs @@ -156,6 +157,7 @@ geo_x geo_y geo_z im_x im_y image_name [gcp_name] [extra1] [extra2] Use any software for tagging GCPs, e.g., the [GCP Editor Pro](https://uav4geo.com/software/gcpeditorpro) is a good match for the ODM. Download source code from the [GitHub](https://github.com/uav4geo/GCPEditorPro). +--- # Automatic detection of ArUco targets @@ -206,7 +208,7 @@ You can generate GCP (Ground Control Point) markers ranging from 1 to up to 1000 **2. Printing and Placement:** Once generated, these markers are printed on sturdy material to withstand outdoor conditions. They're then placed strategically at known positions within the area to be surveyed. -
+
PRO TIP:
When recording the exact positions of the placed Ground Control Points (GCPs), always note which GCP corresponds to which ArUco marker ID (e.g., create GCP_reference.txt file); this association will be crucial when pinpointing them on the images during post-processing.
Expected format of the GCP_reference.txt is 4-columns: ArUco_ID x y z
@@ -237,6 +239,8 @@ Expected format of the GCP_reference.txt is 4-columns: ArUco_ID x y z **5. Georeferencing:** Knowing the real-world coordinates of each ArUco marker, the captured images can be georeferenced (assigned to a specific location in a spatial reference system). This ensures that the imagery aligns accurately with geographic coordinates. +--- + # Create env for geospatial analysis Creating a Conda environment for geospatial analyses streamlines your workflow by isolating installations and dependencies specific to selected geospatial tools. This offers the advantage of consistency, compatibility and convenience, beneficial for both High-Performance Computing (HPC) and local machines. @@ -261,16 +265,50 @@ Isolating installations and dependencies ensures that you can **effortlessly int ``` 4. install required libraries: ``` - pip install opencv-python opencv-contrib-python Pillow numpy matplotlib + pip install numpy==1.22.2 opencv-python==4.8.0.76 opencv-contrib-python==4.8.0.76 Pillow==10.0.0 pyproj==3.6.0 Shapely==1.8.1.post1 svgwrite==1.4.1 ``` - This command installs a foundational set of dependencies crucial for the Python utilities detailed below (sourced from GitHub repos). With these dependencies in place and the environment activated, these tools are set to operate immediately post-cloning, eliminating the need of further setup.

- In the future, if you seek to augment this environment with more packages, you can effortlessly do so at any point using either `conda install` or `pip install` commands. + This command installs a foundational set of dependencies crucial for the Python utilities detailed below (sourced from GitHub repos). With these dependencies in place and the environment activated, these tools are set to operate immediately post-cloning, eliminating the need of further setup.
+ +
+ PRO TIP: +
+ In the future, if you seek to augment this environment with more packages, you can effortlessly do so at any point using either conda install or pip install commands in the activated environment. + +
5. deactivate this environment *(if you no longer intend to use it for this session)*: ``` conda deactivate ``` +
+WARNING: +
+Before starting to use Conda on HPC cluster (e.g. Atlas or Ceres), it’s advisable to change the default location (your home directory) where Conda will install your customized libraries. Installing a lot of Python libraries may contribute to the default 5G soft limit quota on your home directory being surpassed. To overcome this issue you can move .conda directory from your home directory to your project directory and create a symbolic link to the new location. + +
+ +**Change storage location of your Conda envs:**
+( Do it only once! All your Conda envs are stored in .conda dir by default.)
+*Remember to replace the placeholders in <> with the appropriate paths from your file system.* +``` +cd ~ +mkdir /project// +mv .conda /project/// +chgrp -R proj- /project///.conda +chmod -R g+s /project///.conda +ln -s /project///.conda .conda +``` +*The* `mv` *and* `chgrp` *commands may take longer time depending on how much data you have in the .conda directory.* + +
+PRO TIP: +
+If you're unsure whether you've moved your .conda directory from home to the project, run ls -lha in your home directory to see the actual locations of all files, including the (eventually) soft-linked .conda. + +


+ + **B. On your local machine (alternatively):**
* If you already have the Conda environment manager installed, skip step 1 and proceed with the instructions outlined above. * **NOTE:** On a local machine you will use `conda activate geospatial` instead of `source` command. @@ -292,10 +330,9 @@ Isolating installations and dependencies ensures that you can **effortlessly int * [gcp_images_picker.py](https://github.com/ISUgenomics/geo_utils/tree/main#gcp-images-picker) - automatically selects the representative images for each GCP, minimizing manual inspection - ## **Find-GCP** python utility: installation -**[Find-GCP](https://github.com/zsiki/Find-GCP)** is a Python tool leveraging the OpenCV library, designed to detect ArUco Ground Control Points in imagery and generate the corresponding GCP file required for photogrammetric programs such as Open Drone Map. The GitHub repo contains a few small utilities useful in Land Surveying Tasks: +**[Find-GCP](https://github.com/zsiki/Find-GCP)** is a Python tool leveraging the OpenCV library, designed to detect ArUco Ground Control Points in imagery and generate the corresponding GCP file required for photogrammetric programs such as Open Drone Map. The GitHub repo contains a few small utilities useful in land surveying tasks: * [aruco_make.py](https://github.com/zsiki/Find-GCP#aruco_makepy) - generates aruco marker images using different standard dictionaries * [gcp_find.py](https://github.com/zsiki/Find-GCP#gcp_findpy) - identifies Ground Control Points (GCP) in imagery