In this paper, we propose a 3D geometry-aware deformable Gaussian Splatting method for dynamic view synthesis. Existing neural radiance fields (NeRF) based solutions learn the deformation in an implicit manner, which cannot incorporate 3D scene geometry. Therefore, the learned deformation is not necessarily geometrically coherent, which results in unsatisfactory dynamic view synthesis and 3D dynamic reconstruction. Recently, 3D Gaussian Splatting provides a new representation of the 3D scene, building upon which the 3D geometry could be exploited in learning the complex 3D deformation. Specifically, the scenes are represented as a collection of 3D Gaussian, where each 3D Gaussian is optimized to move and rotate over time to model the deformation. To enforce the 3D scene geometry constraint during deformation, we explicitly extract 3D geometry features and integrate them in learning the 3D deformation. In this way, our solution achieves 3D geometry-aware deformation modeling, which enables improved dynamic view synthesis and 3D dynamic reconstruction. Extensive experimental results on both synthetic and real datasets prove the superiority of our solution, which achieves new state-of-the-art performance.
在这篇论文中，我们提出了一种用于动态视图合成的3D几何感知可变形高斯喷涂方法。现有的基于神经辐射场（NeRF）的解决方案以隐式方式学习变形，无法融入3D场景几何信息。因此，学习到的变形不一定在几何上是连贯的，这导致了动态视图合成和3D动态重建的结果不尽人意。最近，3D高斯喷涂提供了3D场景的新表示方式，基于此可以在学习复杂的3D变形时利用3D几何信息。具体来说，场景被表示为一系列3D高斯体，其中每个3D高斯体随时间移动和旋转以模拟变形。为了在变形过程中强制执行3D场景几何约束，我们显式提取3D几何特征并将其集成到学习3D变形中。通过这种方式，我们的解决方案实现了3D几何感知的变形建模，从而使动态视图合成和3D动态重建得到改进。在合成和真实数据集上的大量实验结果证明了我们解决方案的优越性，实现了新的最先进性能。