Skip to content

Latest commit

 

History

History
59 lines (45 loc) · 2.68 KB

FIDNet.md

File metadata and controls

59 lines (45 loc) · 2.68 KB
Raw

Robo3D Benchmark

The following metrics are consistently used in our benchmark:

  • Mean Corruption Error (mCE):

    • The Corruption Error (CE) for model $A$ under corruption type $i$ across 3 severity levels is: $\text{CE}_i^{\text{Model}A} = \frac{\sum((1 - \text{mIoU})^{\text{Model}A})}{\sum((1 - \text{mIoU})^{\text{Baseline}})}$.
    • The average CE for model $A$ on all $N$ corruption types, i.e., mCE, is calculated as: $\text{mCE} = \frac{1}{N}\sum\text{CE}_i$.

  • Mean Resilience Rate (mRR):

    • The Resilience Rate (RR) for model $A$ under corruption type $i$ across 3 severity levels is: $\text{RR}_i^{\text{Model}A} = \frac{\sum(\text{mIoU}^{\text{Model}A})}{3\times (\text{clean-mIoU}^{\text{Model}A})} .$
    • The average RR for model $A$ on all $N$ corruption types, i.e., mRR, is calculated as: $\text{mRR} = \frac{1}{N}\sum\text{RR}_i$.

FIDNet

SemanticKITTI-C

Corruption Light Moderate Heavy Average $\text{CE}_i$ $\text{RR}_i$
Fog 45.49 44.98 40.51 43.66 127.67 74.25
Wet Ground 55.67 50.22 48.99 51.63 105.13 87.81
Snow 48.10 49.82 51.11 49.68 107.71 84.49
Motion Blur 45.18 40.37 35.59 40.38 88.88 68.67
Beam Missing 55.65 49.31 43.00 49.32 116.03 83.88
Crosstalk 51.77 49.43 47.18 49.46 121.32 84.12
Incomplete Echo 49.46 48.29 46.77 48.17 113.74 81.92
Cross-Sensor 40.85 30.73 17.97 29.85 130.03 50.77
  • Summary: $\text{mIoU}_{\text{clean}} =$ 58.80%, $\text{mCE} =$ 113.81%, $\text{mRR} =$ 76.99%.

nuScenes-C

Corruption Light Moderate Heavy Average $\text{CE}_i$ $\text{RR}_i$
Fog 66.31 65.56 62.52 64.80
Wet Ground 69.65 68.44 65.98 68.02
Snow
Motion Blur 58.53 48.80 39.38 48.90
Beam Missing 57.44 47.42 39.56 48.14
Crosstalk
Incomplete Echo 52.08 48.47 45.73 48.76
Cross-Sensor 29.91 20.83
  • Summary: $\text{mIoU}_{\text{clean}} =$ 71.38%, $\text{mCE} =$ %, $\text{mRR} =$ %.

References

@inproceedings{zhao2021fidnet,
  title = {FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding},
  author = {Zhao, Yiming and Bai, Lin and Huang, Xinming},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
}