GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis
We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting. Unlike the original Gaussian Splatting or neural implicit rendering methods that necessitate per-subject optimizations, we introduce Gaussian parameter maps defined on the source views and regress directly Gaussian Splatting properties for instant novel view synthesis without any fine-tuning or optimization. To this end, we train our Gaussian parameter regression module on a large amount of human scan data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable and experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.
我们提出了一种新的方法，名为GPS-Gaussian，用于实时合成角色的新视图。所提出的方法能够在稀疏视图摄像头设置下实现2K分辨率渲染。与原始的高斯喷溅或神经隐式渲染方法需要对每个主题进行优化不同，我们在源视图上引入了定义的高斯参数图，并直接回归高斯喷溅属性以实现即时新视图合成，无需任何微调或优化。为此，我们在大量人类扫描数据上训练我们的高斯参数回归模块，与深度估计模块联合使用，将2D参数图提升到3D空间。所提出的框架完全可微分，且在几个数据集上的实验表明，我们的方法在渲染速度上超越了最先进的方法。