CO3D: Common Objects In 3D

This repository contains a set of tools for working with the Common Objects in 3D (CO3D) dataset. The dataset has been introduced in our ICCV'21 paper: Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction.

Download the dataset

The links to all dataset files are present in this repository in co3d_links.txt.

Automatic batch-download

We also provide a python script that allows downloading all dataset files at once. In order to do so, execute the download script: python ./download_dataset.py --download_folder DOWNLOAD_FOLDER where DOWNLOAD_FOLDER is a local target folder for downloading the dataset files. Make sure to create this folder before commencing the download.

Installation

This is a Python 3 / PyTorch codebase.

1. Install PyTorch.
2. Install PyTorch3D.
3. Install the remaining dependencies in requirements.txt:

pip install lpips visdom tqdm requests

Note that the core data model in dataset/types.py is independent of PyTorch and can be imported and used with other machine-learning frameworks.

Dependencies

• PyTorch
• PyTorch3D
• tqdm
• visdom
• lpips
• requests

Getting started

1. Install dependencies - See Installation above.
2. Download the dataset here to a given root folder DATASET_ROOT_FOLDER.
3. In dataset/dataset_zoo.py set the DATASET_ROOT variable to your DATASET_ROOT_FOLDER`:

   dataset_zoo.py:25: DATASET_ROOT = DATASET_ROOT_FOLDER
   

4. Run eval_demo.py:

   python eval_demo.py
   

   Note that eval_demo.py runs an evaluation of a simple depth-based image rendering (DBIR) model on the same data as in the paper. Hence, the results are directly comparable to the numbers reported in the paper.

Running tests

Unit tests can be executed with:

python -m unittest

Reproducing results

Implicitron is our open-source framework used to train all implicit shape learning methods from the CO3D paper. Please visit the following link for more details: https://github.com/facebookresearch/pytorch3d/tree/main/projects/implicitron_trainer

Reference

If you use our dataset, please use the following citation:

@inproceedings{reizenstein21co3d,
	Author = {Reizenstein, Jeremy and Shapovalov, Roman and Henzler, Philipp and Sbordone, Luca and Labatut, Patrick and Novotny, David},
	Booktitle = {International Conference on Computer Vision},
	Title = {Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction},
	Year = {2021},
}

License

The CO3D codebase is released under the BSD License.

Overview video

The following presentation of the dataset was delivered at the Extreme Vision Workshop at CVPR 2021: