A Hough-Space-based Nearest Neighbor Object Recognition Pipeline for Point Clouds
The maneki-neko (lucky cat) mesh model is intellectual property of user bs3 (taken from thingiverse and converted to a point cloud).
Point-cloud-donkey is a customizable pipeline based on the Point Cloud Library (PCL) for point cloud classification. The development started as an adaptation of the Implicit Shape Model (ISM) [1] algorithm to point cloud data. There are still many references to ISM in the code. Over time, the framework was extended and became a general, local feature, Hough-Space voting approach for point cloud classification. It allows to classify point cloud objects and localize them amongst clutter (the latter part still being in development).
The following table summarizes classification results on some datasets. For more information please refer to the publications linked in the Citing section.
| 3D-Data Set | Base Pipeline2 | Base Pipeline3 | Extended Pipeline4 |
|---|---|---|---|
| Aim@Shape1 | 85.0 | 90.0 | 93.5 |
| McGill 3D Shape Benchmark | - | 85.2 | 86.6 |
| Princeton Shape Benchmark | 61.7 | 67.0 | 68.4 |
| Shrec 2012 | - | 70.2 | 74.5 |
| ModelNet 10 | - | 62.4 | 83.8 |
| ModelNet 40 | - | 71.9 | 75.4 |
1: Dataset no longer online
2: Pipeline excluding steps marked with a red star in the image above [bibtex] [PDF] and [bibtex] [PDF]
3: Optimized parameters in training [bibtex] [PDF]
4: Pipeline including orange and green steps marked with a red star in the image above [bibtex] [PDF]
| RGB-D Data Set | Base Pipeline3 | Base Pipeline with Short CSHOT5 |
|---|---|---|
| class6 / instance accuracy | class6 / instance accuracy | |
| Washington RGB-D (partial) | 91.6 / 83.4 | 91.0 / 82.8 |
| BigBird | 84.0 / 71.2 | 91.0 / 81.7 |
| YCB | 87.9 / 73.2 | 81.4 / 68.6 |
5: Shortened Color SHOT Descriptors [in press]
6: Self-assigned instances to classes, see [ICARSC21 supplements]
This framework was tested with various versions of Ubuntu LTS. The installation instructions can be found on the Wiki pages.
For older versions, please consult the Wiki pages for older Ubuntu versions.
To quickly start training a classifier to classify isolated point clouds, refer to the instructions on the following Wiki pages.
Quick Start Using the Command Line
Documentation for different tools, config files and other required artifacts can be found on the following Wiki page.
The development of this framework started during the Master's thesis of Norman Link, that I supervised. I am very thankful to Norman for his contributions. Please check out other projects of Norman on GitHub:
Further, I would like to thank the developers of third party libraries used in this project:
- Point Cloud Library
- Compact Geometric Features
- cnpy Library
- gdiam Library
- lzf Library
- B-SHOT Descriptor
If you are using this repository for academic work, please consider citing the publication listed below. If you consider citing a specific contribution contained in this repository, please refer to the following Wiki page:
- Paper introducing codebook cleaning and global verification
- This extended pipeline is the generic point cloud processing pipeline depicted in the image on top of this page including the orange and green steps marked with a red star.
- PDF: Seib2019B3S
@inproceedings{Seib2019B3S,
author = {Seib, Viktor and Theisen, Nick and Paulus, Dietrich},
editor = {Tremeau, Alain and Farinella, Giovanni Maria and Braz, Jose},
title = {Boosting 3D Shape Classification with Global Verification and Redundancy-free Codebooks},
booktitle = {Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications},
publisher = {SciTePress},
volume = {5},
pages = {257 - 264},
year = {2019},
url = {http://www.uni-koblenz.de/~agas/Documents/Seib2019B3S.pdf},
isbn = {978-989-758-354-4},
}
Point-cloud-donkey is released under the BSD-3-Clause license. See LICENSE for details. Point-cloud-donkey also includes some 3rd party code which might be subject to other licenses.
[1] Leibe, Bastian and Leonardis, Ales and Schiele, Bernt; "Combined Object Categorization and Segmentation with an Implicit Shape Model", Workshop on statistical learning in computer vision, ECCV, 2004