Reconstructing dynamic scenes with large-scale and complex motions remains a significant challenge. Recent techniques like Neural Radiance Fields and 3D Gaussian Splatting (3DGS) have shown promise but still struggle with scenes involving substantial movement. This paper proposes RelayGS, a novel method based on 3DGS, specifically designed to represent and reconstruct highly dynamic scenes. Our RelayGS learns a complete 4D representation with canonical 3D Gaussians and a compact motion field, consisting of three stages. First, we learn a fundamental 3DGS from all frames, ignoring temporal scene variations, and use a learnable mask to separate the highly dynamic foreground from the minimally moving background. Second, we replicate multiple copies of the decoupled foreground Gaussians from the first stage, each corresponding to a temporal segment, and optimize them using pseudo-views constructed from multiple frames within each segment. These Gaussians, termed Relay Gaussians, act as explicit relay nodes, simplifying and breaking down large-scale motion trajectories into smaller, manageable segments. Finally, we jointly learn the scene's temporal motion and refine the canonical Gaussians learned from the first two stages. We conduct thorough experiments on two dynamic scene datasets featuring large and complex motions, where our RelayGS outperforms state-of-the-arts by more than 1 dB in PSNR, and successfully reconstructs real-world basketball game scenes in a much more complete and coherent manner, whereas previous methods usually struggle to capture the complex motion of players.
重建具有大规模复杂运动的动态场景仍是一个显著的挑战。尽管神经辐射场（NeRF）和三维高斯散点（3D Gaussian Splatting, 3DGS）等技术在该领域表现出潜力，但在处理具有显著运动的场景时仍显不足。 本文提出了RelayGS，一种基于3DGS的新方法，专为表示和重建高度动态场景而设计。RelayGS通过三阶段学习，构建了完整的四维表示，其中包含规范的三维高斯和紧凑的运动场。第一阶段，我们从所有帧中学习基础的3DGS模型，忽略时间上的场景变化，并利用可学习掩码将剧烈运动的前景与微动的背景分离。第二阶段，我们从第一阶段分离的前景高斯生成多个副本，每个副本对应一个时间段，并通过利用每段内多个帧构建的伪视角进行优化。这些高斯被称为Relay Gaussians，作为显式的中继节点，将大规模运动轨迹分解为更小且可控的片段。第三阶段，我们联合学习场景的时间运动，并对前两阶段学习的规范高斯进行细化优化。 在包含大规模复杂运动的两个动态场景数据集上的实验表明，RelayGS在PSNR上比现有最先进方法提高了1 dB以上，并成功重建了真实世界中的篮球比赛场景。相比之下，现有方法通常难以捕捉球员复杂的运动，而RelayGS能够以更完整和连贯的方式进行重建，展现了卓越的性能和适应性。