Real-time 3D reconstruction of surgical scenes plays a vital role in computer-assisted surgery, holding a promise to enhance surgeons' visibility. Recent advancements in 3D Gaussian Splatting (3DGS) have shown great potential for real-time novel view synthesis of general scenes, which relies on accurate poses and point clouds generated by Structure-from-Motion (SfM) for initialization. However, 3DGS with SfM fails to recover accurate camera poses and geometry in surgical scenes due to the challenges of minimal textures and photometric inconsistencies. To tackle this problem, in this paper, we propose the first SfM-free 3DGS-based method for surgical scene reconstruction by jointly optimizing the camera poses and scene representation. Based on the video continuity, the key of our method is to exploit the immediate optical flow priors to guide the projection flow derived from 3D Gaussians. Unlike most previous methods relying on photometric loss only, we formulate the pose estimation problem as minimizing the flow loss between the projection flow and optical flow. A consistency check is further introduced to filter the flow outliers by detecting the rigid and reliable points that satisfy the epipolar geometry. During 3D Gaussian optimization, we randomly sample frames to optimize the scene representations to grow the 3D Gaussian progressively. Experiments on the SCARED dataset demonstrate our superior performance over existing methods in novel view synthesis and pose estimation with high efficiency.
实时的手术场景三维重建在计算机辅助手术中扮演着重要角色，有望提升外科医生的可视性。最近在三维高斯斑点化（3DGS）方面的进展显示出在一般场景的实时新视角合成中具有巨大潜力，这依赖于通过运动结构（SfM）生成的精确姿态和点云进行初始化。然而，由于手术场景中纹理极少且光度不一致性的挑战，3DGS与SfM在恢复准确的摄像机姿态和几何方面存在困难。 为了解决这个问题，本文提出了首个基于无SfM的3DGS方法，用于手术场景重建，通过联合优化摄像机姿态和场景表达。根据视频连续性，我们方法的关键在于利用即时光流先验来引导从3D高斯中导出的投影光流。与大多数依赖光度损失的先前方法不同，我们将姿态估计问题形式化为最小化投影光流和光流之间的流动损失。进一步引入一致性检查，通过检测满足对极几何的刚性和可靠点来过滤流异常值。在3D高斯优化过程中，我们随机采样帧来逐步优化场景表达以扩展3D高斯。 在SCARED数据集上的实验表明，我们的方法在新视角合成和姿态估计方面具有显著的性能优势，并且具有高效率。