SEVN contains around 5,000 full panoramic images and labels for house numbers, doors, and street name signs, which can be used for several different navigation tasks. Agents trained with SEVN have access to variable-resolution images, visible text, and simulated GPS data to navigate the environment. The SEVN Simulator is OpenAI Gym-compatible to allow the use of state-of-the-art deep reinforcement learning algorithms. An instance of the simulator using low-resolution imagery can be run at 400-800 frames per second on a machine with 2 CPU cores and 2 GB of RAM.
| Low Resolution Views (84x84px) | High Resolution Views (1280x1280px) |
|---|---|
 |
 |
4,988 panoramic images across 6.3 kilometers with 3,259 labels on house numbers, doors, and street name signs.
A longer introduction can be found here: Creating a Navigation Assistant for the Visually Impaired
In order to setup the environment, do something like the following. If using a fresh Ubuntu install, ensure that build-essential is installed (i.e., sudo apt-get build-essential). We'll need GCC for this, and that installs it.
# Install the code
git clone https://github.com/mweiss17/SEVN.git
cd SEVN
# Create a new conda environment for the depenencies
conda create -n sevn python=3.7
# Install the dependencies
conda activate sevn
pip install -e .
# Download the low resolution image data, labels, and spatial graph
python scripts/download.py
# Test that your environment is correctly setup
python scripts/01-play.py
# WARNING! Running this step downloads 28GB of image data and is not required to run the model or play with the environment.
# python scripts/download.py --high-res
# python scripts/01-play.py --high-res
You can manually download the dataset here (in case you don't want to follow the installation instructions above).
- Coordinates of Panoramas
- Connectvity Graph
- Image Labels
- Panoramas (Low resolution)
- Panoramas (High resolution) (Warning! 48 GB of images in a zip file)
For more information about the data-preprocessing and the data format consult the README in the SEVN-data github repository.
For more information about how to train an agent on SEVN consult the README in the SEVN Model Github repo.
This work was accepted at Conference on Robot Learning (CoRL) 2019.
The paper was also published to Arxiv: Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments
If you use this work, please cite us. Here's the Bibtex for our paper.
@misc{weiss2019navigation,
title={Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments},
author={Martin Weiss and Simon Chamorro and Roger Girgis and Margaux Luck and
Samira E. Kahou and Joseph P. Cohen and Derek Nowrouzezahrai and
Doina Precup and Florian Golemo and Chris Pal},
year={2019},
eprint={1910.13249},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
- Martin Weiss, PhD student (Mila)
- Simon Chamorro, Undergrad student (USherbrooke)
- Roger Girgis, PhD student (Mila)
- Margaux Luck, Postdoctoral fellow (Mila)
- Samira Ebrahimi Kahou, Postdoctoral fellow (Mila)
- Joseph Paul Cohen, Postdoctoral fellow (Mila)
- Derek Nowrouzezahrai, Professor (Mila & McGill)
- Doina Precup, Professor (Mila & McGill)
- Florian Golemo, Postdoctoral fellow (Mila & ElementAI)
- Chris Pal, Professor (Mila, Polytechnique Montreal & ElementAI)
- OpenAI Gym - A toolkit for developing and comparing reinforcement learning algorithms
This project is licensed under the MIT licence - please see the LICENCE file in the repository.