This is the code for our ICML paper on topology-driven singularity analysis:
@inproceedings{vonRohrscheidt23a,
title = {Topological Singularity Detection at Multiple Scales},
author = {von Rohrscheidt, Julius and Rieck, Bastian},
year = 2023,
booktitle = {Proceedings of the 40th International Conference on Machine Learning},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
number = 202,
pages = {35175--35197},
editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
abstract = {The manifold hypothesis, which assumes that data lies on or close to an unknown manifold of low intrinsic dimension, is a staple of modern machine learning research. However, recent work has shown that real-world data exhibits distinct non-manifold structures, i.e. singularities, that can lead to erroneous findings. Detecting such singularities is therefore crucial as a precursor to interpolation and inference tasks. We address this issue by developing a topological framework that (i) quantifies the local intrinsic dimension, and (ii) yields a Euclidicity score for assessing the `manifoldness' of a point along multiple scales. Our approach identifies singularities of complex spaces, while also capturing singular structures and local geometric complexity in image data.}
}Our code has been tested with Python 3.8 and Python 3.9 under Mac OS X and Linux. Other Python versions may not support all dependencies.
The recommended way to install the project is via poetry.
If this is available, installation should work very quickly:
$ poetry install
Recent versions of pip should also be capable of installing the
project directly:
$ pip install .
To reproduce the main experiments in our paper, we ship synthetic data
sets in the repository and offer the automated capability to download
the computer vision data sets (MNIST and FashionMNIST). For
reasons of simplicity, we suggest to reproduce the experiments with
synthetic point clouds first as they run quickly even on a standard
desktop computer.
All experiments make use of the script cli.py, which provides
a command-line interface to our framework. Given input parameters for
the local annuli, this script will calculate Euclidicity values as
described in the paper. For reasons of simplicity, all output is
provided to stdout, i.e. the standard output of your terminal, and
needs to be redirected to a file for subsequent analysis.
We will subsequently provide the precise commands to reproduce the
experiments; readers are invited to take a look at the code in cli.py
or call python cli.py --help in order to see what additional options
are available for processing data.
Run the following commands from the root directory of the repository:
$ cd tardis
$ python cli.py ../data/Pinched_torus.txt.gz -q 500 -r 0.05 -R 0.45 -s 0.2 -S 0.6 > ../output/Pinched_torus.txt
This will create a point cloud of 500 sample points with
Warning: this example might require a long runtime on an ordinary
machine. We ran this on our cluster (see also the scripts
folder in the root directory).
Run the following commands from the root directory of the repository:
$ cd tardis
$ python cli.py -k 100 -q 2000 -d 2 --num-steps 20 ../data/Wedged_spheres_2D.txt.gz > ../output/Wedged_spheres_2D.txt
This will make use of the automated parameter selection procedure based on nearest neighbours. Notice that this example uses more query points; it is of course possible to adjust this parameter.
Check out the examples folder for some code snippets that demonstrate how to use TARDIS in your own code. They all make use of the preliminary API.
Our code is released under a BSD-3-Clause license. This license essentially permits you to freely use our code as desired, integrate it into your projects, and much more---provided you acknowledge the original authors. Please refer to LICENSE.md for more information.
This project is maintained by members of the AIDOS Lab. Please open an issue in case you encounter any problems.