Skip to content

Latest commit

 

History

History
110 lines (78 loc) · 5.15 KB

README.md

File metadata and controls

110 lines (78 loc) · 5.15 KB
Raw

RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering

Paper|Project Page|Dataset

​ teaser

This repository contains the implementation of our paper RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering.

You can find detailed usage instructions for training your own models below.

If you find our code useful, please cite:

@inproceedings{zhang2025rise,
	title={RISE-SDF: A Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering},
	author={Zhang, Deheng and Wang, Jingyu and Wang, Shaofei and Mihajlovic, Marko and Prokudin, Sergey and Lensch, Hendrik and Tang, Siyu},
	booktitle={International Conference on 3D Vision (3DV)},
	year={2025}
}

Requirements

Note:

Install

git clone --recursive [email protected]:dehezhang2/RISE-SDF.git

Environments

  • Create python 3.10 environment

    conda create --name rise-sdf python=3.10
conda activate rise-sdf

  • Install PyTorch>=1.13 here based the package management tool you used and your cuda version (older PyTorch versions may work but have not been tested)

    conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

  • Install Nvdiffrast

    git clone https://github.com/NVlabs/nvdiffrast.git
cd nvdiffrast
pip install .

  • Install tiny-cuda-nn PyTorch extension and other dependencies (make sure you are in the CUDA available mode):

    pip install --global-option="--no-networks" git+https://github.com/NVlabs/tiny-cuda-nn#subdirectory=bindings/torch
pip install -r requirements.txt
    • if you find a problem with nerfacc, uninstall your current nerfacc and build it again. 
        pip uninstall nerfacc
  pip install git+https://github.com/nerfstudio-project/nerfacc.git

Datasets & Preparation

  • Download the pre-integrated BSDF into the ./load/bsdf/ folder.
  • Download the environment maps from TensorIR Dataset into the ./load/ folder.
  • Download the TensoIR Dataset, save the files to the ./load/TensoIR_synthetic/ folder (for TensoIR, you need to change the config file albedo_format = 'png', has_roughness=false).
  • Download Shiny Inverse Rendering Dataset, save the files to the ./load/TensoIR_synthetic/ folder.

Physically Based Inverse Rendering (PBIR)

Training

Run the launch script with --train, specifying the config file, the GPU(s) to be used (GPU 0 will be used by default), and the scene name:

# on Shiny Inverse Rendering Dataset
python launch.py --config configs/split-mixed-occ-tensoir.yaml --gpu 0 --train dataset.scene=toaster_disney

The config snapshots, checkpoints, and experiment outputs are saved to exp/[name]/[tag]@[timestamp]. You can change any configuration in the YAML file by specifying arguments without --, for example:

python launch.py --config configs/split-mixed-occ-tensoir.yaml --gpu 0 --train dataset.scene=toaster_disney tag=iter50k seed=0 trainer.max_steps=50000

Relighting

The training procedure is by default followed by testing, which computes metrics on test data, generates animations, and exports the geometry as triangular meshes. If you want to do testing alone, just resume the pre-trained model and replace --train with --test, for example:

python launch.py --config exp/split-mixed-occ-tensoir-toaster_disney/your_experiment_directory/config/parsed.yaml --resume exp/split-mixed-occ-tensoir-toaster_disney/your_experiment_directory/ckpt/epoch\=0-step\=40000.ckpt --gpu 0 --test models.phys_kick_in_step=0

Note that you can change the relighting environment map easily in the config file

dataset
  relight_list:  ['bridge', 'city']
  hdr_filepath: ./load/high_res_envmaps_2k/

Acknowledgement

Our repo is developed based on instant-nsr_pl, IntrinsicAvatar, NeRO, Nvdiffrec. Please also consider citing the corresponding papers. We thank authors of these papers for their wonderful works which greatly facilitates the development of our project.

LICENSE

The code is released under the GPL-3.0 license.