We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to accurately represent specular surfaces. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy that significantly enhances both scene geometry and specular color prediction. We demonstrate that our model outperforms prior methods for view synthesis of scenes containing dynamic specular objects and that it is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes.
我们提出了 SpectroMotion，一种将三维高斯喷涂 (3D Gaussian Splatting, 3DGS) 与基于物理的渲染 (PBR) 和变形场相结合，用于重建动态的高光场景。先前将 3DGS 拓展到动态场景建模的方法在准确表现高光表面方面存在困难。我们的方法通过引入残差校正技术来准确计算变形过程中的表面法向量，并配合可变形的环境贴图以适应随时间变化的光照条件，从而解决了这一局限。我们采用由粗到细的训练策略，大幅提升了场景几何结构和高光颜色的预测效果。实验表明，我们的模型在包含动态高光物体的场景视图合成方面优于现有方法，并且是唯一能够合成逼真动态高光真实场景的 3DGS 方法，在渲染复杂的动态高光场景方面超越了最新技术。