LiCROcc

This repository contains the implementation of the paper.

If you find our work useful, Please give us a star 🌟!

LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera
Yukai Ma^1,2, Jianbiao Mei^1,2, Xuemeng Yang², Licheng Wen², Weihua Xu¹, Jiangning Zhang¹, Xingxing Zuo³, Botian Shi^{2,^}, Yong Liu^{1,^
1}ZJU ²PJLab³TUM
^{^}Corresponding Authors

News

🎉 [2024/11/06] LiCROcc was accepted by RAL!!!

Getting Started

Installation

# Create a conda virtual environment and activate it.
conda create -n licrocc python=3.8 -y
conda activate licrocc
# Install torch
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f ttps://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
# Install mmdet3d
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v0.17.1 # Other versions may not be compatible.
pip install -v -e .
# 
cd ../projects
pip install -v -e .

Prepare Dataset

-Please refer to OpenOccupancy to prepare nuScenes dataset.

• Please refer to CRN to generate radar point cloud in BEV view.

python scripts/gen_radar_bev.py  # accumulate sweeps and transform to LiDAR coords

Folder structure:

LiCROcc
├── data/
│   ├── nuscenes/
│   │   ├── maps/
│   │   ├── samples/
│   │   ├── sweeps/
│   │   ├── lidarseg/
│   │   ├── v1.0-test/
│   │   ├── v1.0-trainval/
│   │   ├── nuscenes_occ_infos_train.pkl
│   │   ├── nuscenes_occ_infos_val.pkl
│   │   ├── radar_bev_filter/
│   ├── nuScenes-Occupancy/
│   ├── depth_gt/

Run and Eval

Download Teacher model here

Train RC-LiCROcc

./tools/dist_train.sh ./projects/configs/ssc_rs/ssc_rs_base_nuscenes_LC2LR123.py N_GPUs

Train R-LiCROcc

./tools/dist_train.sh ./projects/configs/ssc_rs/ssc_rs_base_nuscenes_LC2radar12.py N_GPUs

If an error is reported during training：

TypeError: FormatCode() got an unexpected keyword argument 'verify'

Simply remove ‘verify=True’

Eval RC-LiCROcc

./tools/dist_test.sh ./projects/configs/ssc_rs/ssc_rs_base_nuscenes_LC2LR123.py ./path/to/ckpts.pth N_GPUs

Eval R-LiCROcc

./tools/dist_test.sh ./projects/configs/ssc_rs/ssc_rs_base_nuscenes_LC2radar12.py ./path/to/ckpts.pth N_GPUs

Visualization

Temporarily only support saving occupancy predictions (refer to MonoScene for visualization tools)

./tools/dist_test.sh $PATH_TO_CFG $PATH_TO_CKPT $GPU_NUM --show --show-dir $PATH

Model Zoo

Subset	Checkpoint	Logs	Note
RC-LiCROcc	link	link	train on 8 A100
R-LiCROcc	link	link	train on 8 A100

Cite Us

@ARTICLE{10777549,
  author={Ma, Yukai and Mei, Jianbiao and Yang, Xuemeng and Wen, Licheng and Xu, Weihua and Zhang, Jiangning and Zuo, Xingxing and Shi, Botian and Liu, Yong},
  journal={IEEE Robotics and Automation Letters}, 
  title={LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction Using LiDAR and Camera}, 
  year={2025},
  volume={10},
  number={1},
  pages={852-859},
  keywords={Radar;Semantics;Radar imaging;Three-dimensional displays;Laser radar;Feature extraction;Cameras;Sensors;Meteorology;Point cloud compression;Sensor fusion;semantic scene completion;knowledge distillation},
  doi={10.1109/LRA.2024.3511427}}

Credit

We adopt the following open-sourced projects:

• SSC-RS
• FlashOCC
• MonoOcc