We present the first application of 3D Gaussian Splatting to incremental 3D reconstruction using a single moving monocular or RGB-D camera. Our Simultaneous Localisation and Mapping (SLAM) method, which runs live at 3fps, utilises Gaussians as the only 3D representation, unifying the required representation for accurate, efficient tracking, mapping, and high-quality rendering. Several innovations are required to continuously reconstruct 3D scenes with high fidelity from a live camera. First, to move beyond the original 3DGS algorithm, which requires accurate poses from an offline Structure from Motion (SfM) system, we formulate camera tracking for 3DGS using direct optimisation against the 3D Gaussians, and show that this enables fast and robust tracking with a wide basin of convergence. Second, by utilising the explicit nature of the Gaussians, we introduce geometric verification and regularisation to handle the ambiguities occurring in incremental 3D dense reconstruction. Finally, we introduce a full SLAM system which not only achieves state-of-the-art results in novel view synthesis and trajectory estimation, but also reconstruction of tiny and even transparent objects.
我们首次将3D高斯喷溅应用于使用单个移动的单眼或RGB-D摄像机进行增量式3D重建。我们的同时定位与地图构建（SLAM）方法实时运行，速度为每秒3帧，仅使用高斯作为唯一的3D表示，统一了精确、高效跟踪、绘图和高质量渲染所需的表示。要从实时摄像机不断重建高保真度的3D场景，需要多项创新。首先，为了超越原始的3D高斯喷溅（3DGS）算法，该算法需要离线结构运动（SfM）系统提供精确的姿态，我们为3DGS制定了针对3D高斯进行直接优化的摄像机跟踪方法，并展示了这使得快速且稳健的跟踪成为可能，并具有广泛的收敛盆地。其次，通过利用高斯的显性特性，我们引入了几何验证和规范化来处理增量式3D密集重建中出现的歧义。最后，我们引入了一个完整的SLAM系统，它不仅在新视图合成和轨迹估计方面实现了最新技术水平，而且还能重建微小甚至透明的物体。