Recent advancements in real-time neural rendering using point-based techniques have paved the way for the widespread adoption of 3D representations. However, foundational approaches like 3D Gaussian Splatting come with a substantial storage overhead caused by growing the SfM points to millions, often demanding gigabyte-level disk space for a single unbounded scene, posing significant scalability challenges and hindering the splatting efficiency.
To address this challenge, we introduce LightGaussian, a novel method designed to transform 3D Gaussians into a more efficient and compact format. Drawing inspiration from the concept of Network Pruning, LightGaussian identifies Gaussians that are insignificant in contributing to the scene reconstruction and adopts a pruning and recovery process, effectively reducing redundancy in Gaussian counts while preserving visual effects. Additionally, LightGaussian employs distillation and pseudo-view augmentation to distill spherical harmonics to a lower degree, allowing knowledge transfer to more compact representations while maintaining reflectance. Furthermore, we propose a hybrid scheme, VecTree Quantization, to quantize all attributes, resulting in lower bitwidth representations with minimal accuracy losses.
In summary, LightGaussian achieves an averaged compression rate over 15x while boosting the FPS from 139 to 215, enabling an efficient representation of complex scenes on Mip-NeRF 360, Tank and Temple datasets.
最近在实时神经渲染中使用基于点的技术取得的进步，为3D表示的广泛采用铺平了道路。然而，像3D高斯喷溅这样的基础方法带来了大量的存储开销，因为它将SfM点增长到数百万，通常需要单个无限制场景的千兆字节级磁盘空间，这对可扩展性构成了重大挑战，并阻碍了喷溅效率。
为了应对这一挑战，我们引入了LightGaussian，这是一种旨在将3D高斯转换成更高效、更紧凑格式的新方法。从网络剪枝的概念中获得灵感，LightGaussian识别出在场景重建中贡献不大的高斯，并采用剪枝和恢复过程，有效减少了高斯数量的冗余，同时保留了视觉效果。此外，LightGaussian运用蒸馏和伪视图增强，将球面谐波蒸馏到更低的程度，允许知识转移到更紧凑的表示中，同时保持反射率。此外，我们提出了一种混合方案，VecTree量化，用于对所有属性进行量化，从而实现更低位宽表示，同时将准确性损失降至最低。
总结来说，LightGaussian实现了平均压缩率超过15倍，同时将FPS从139提升到215，使得在Mip-NeRF 360、Tank和Temple数据集上高效地表示复杂场景成为可能。