Custom runners (singularity/slurm/...)

[maumueller]

This PR makes it possible to specify custom runners to run ann-benchmarks in containerized, non-Docker environments. My use case is that I (and many others) have to use SLURM on an hpc cluster to run jobs. The current setup doesn't allow me to work with ann-benchmarks at all because docker is usually not supported in these environments.

My solution is a bit hacky:

1. All docker containers are converted to singularity images (python templates/custom_runners/convert_docker_to_singularity.py) in a local setup that supports docker.
2. All images are moved to the hpc cluster.
3. Set up ann-benchmarks in the usual way, but skip the installation step for the libraries. Interface-wise, jobs are run in the usual way but through the custom runner, e.g., python3 run.py --dataset random-xs-20-euclidean --custom-container templates/custom_runner/slurm.template.

This works great for me, but I'd love to hear some thoughts on improving this solution.

[erikbern]

Interesting! Are you able to fan out and run all the ann-benchmarks much faster using this?

[maumueller]

Well, the main reason is to be able to use it at all in that environment. But technically, I could just spawn all individual runs for all datasets at once and led the scheduler take care of the rest, yes.

[erikbern]

I can take a look once this is ready – let me know if/once it is!