Recent advances in 3D Gaussian Splatting (3DGS) have garnered significant attention in computer vision and computer graphics due to its high rendering speed and remarkable quality. While extant research has endeavored to extend the application of 3DGS from static to dynamic scenes, such efforts have been consistently impeded by excessive model sizes, constraints on video duration, and content deviation. These limitations significantly compromise the streamability of dynamic 3D Gaussian models, thereby restricting their utility in downstream applications, including volumetric video, autonomous vehicle, and immersive technologies such as virtual, augmented, and mixed reality. This paper introduces SwinGS, a novel framework for training, delivering, and rendering volumetric video in a real-time streaming fashion. To address the aforementioned challenges and enhance streamability, SwinGS integrates spacetime Gaussian with Markov Chain Monte Carlo (MCMC) to adapt the model to fit various 3D scenes across frames, in the meantime employing a sliding window captures Gaussian snapshots for each frame in an accumulative way. We implement a prototype of SwinGS and demonstrate its streamability across various datasets and scenes. Additionally, we develop an interactive WebGL viewer enabling real-time volumetric video playback on most devices with modern browsers, including smartphones and tablets. Experimental results show that SwinGS reduces transmission costs by 83.6% compared to previous work with ignorable compromise in PSNR. Moreover, SwinGS easily scales to long video sequences without compromising quality.
近年来，3D Gaussian Splatting (3DGS) 因其高渲染速度和卓越的质量在计算机视觉和计算机图形学领域引起了广泛关注。尽管现有研究已经努力将3DGS的应用从静态场景扩展到动态场景，但这些尝试一直受到模型规模过大、视频时长限制以及内容偏差的阻碍。这些局限性大大削弱了动态3D高斯模型的流媒体能力，从而限制了其在体积视频、自动驾驶车辆以及虚拟现实、增强现实和混合现实等沉浸式技术中的应用。 本文提出了SwinGS，这是一种用于实时流媒体方式训练、传输和渲染体积视频的新框架。为了解决上述挑战并增强流媒体能力，SwinGS 将时空高斯与马尔可夫链蒙特卡洛（MCMC）相结合，使模型能够在不同帧之间适应各种3D场景。同时，采用滑动窗口方法，以累积方式为每一帧捕捉高斯快照。我们实现了SwinGS的原型，并展示了其在多个数据集和场景中的流媒体能力。此外，我们开发了一个交互式WebGL查看器，能够在包括智能手机和平板电脑在内的大多数设备上的现代浏览器中实现实时体积视频播放。实验结果表明，与之前的工作相比，SwinGS 在传输成本上减少了83.6%，且几乎不影响PSNR质量。此外，SwinGS 可以轻松扩展到长视频序列而不影响质量。