Novel view synthesis of dynamic scenes has been an intriguing yet challenging problem. Despite recent advancements, simultaneously achieving high-resolution photorealistic results, real-time rendering, and compact storage remains a formidable task. To address these challenges, we propose Spacetime Gaussian Feature Splatting as a novel dynamic scene representation, composed of three pivotal components. First, we formulate expressive Spacetime Gaussians by enhancing 3D Gaussians with temporal opacity and parametric motion/rotation. This enables Spacetime Gaussians to capture static, dynamic, as well as transient content within a scene. Second, we introduce splatted feature rendering, which replaces spherical harmonics with neural features. These features facilitate the modeling of view- and time-dependent appearance while maintaining small size. Third, we leverage the guidance of training error and coarse depth to sample new Gaussians in areas that are challenging to converge with existing pipelines. Experiments on several established real-world datasets demonstrate that our method achieves state-of-the-art rendering quality and speed, while retaining compact storage. At 8K resolution, our lite-version model can render at 60 FPS on an Nvidia RTX 4090 GPU.
动态场景的新视角合成一直是一个有趣但具有挑战性的问题。尽管近期取得了一些进展，但要同时实现高分辨率的逼真结果、实时渲染和紧凑存储仍然是一个艰巨的任务。为了解决这些挑战，我们提出了时空高斯特征涂抹作为一种新的动态场景表征，它由三个关键组成部分构成。首先，我们通过增强3D高斯模型与时间不透明度和参数化运动/旋转，构建了表现力强的时空高斯。这使时空高斯能够捕捉场景内的静态、动态以及瞬时内容。其次，我们引入了涂抹特征渲染，用神经特征替代球形谐波。这些特征有助于建模视角和时间依赖的外观，同时保持小尺寸。第三，我们利用训练误差和粗略深度的指导，在现有管道难以收敛的区域采样新的高斯模型。在几个已建立的真实世界数据集上的实验表明，我们的方法在渲染质量和速度方面达到了最先进水平，同时保持了紧凑的存储。在8K分辨率下，我们的轻量版模型可以在Nvidia RTX 4090 GPU上以60 FPS的速度渲染。