We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We disentangle the motion and appearance of a virtual human with explicit mesh geometry and implicit appearance modeling with Gaussian Splatting. The Gaussians are defined by barycentric coordinates and displacement on a triangle mesh as Phong surfaces. We extend lifted optimization to simultaneously optimize the parameters of the Gaussians while walking on the triangle mesh. SplattingAvatar is a hybrid representation of virtual humans where the mesh represents low-frequency motion and surface deformation, while the Gaussians take over the high-frequency geometry and detailed appearance. Unlike existing deformation methods that rely on an MLP-based linear blend skinning (LBS) field for motion, we control the rotation and translation of the Gaussians directly by mesh, which empowers its compatibility with various animation techniques, e.g., skeletal animation, blend shapes, and mesh editing. Trainable from monocular videos for both full-body and head avatars, SplattingAvatar shows state-of-the-art rendering quality across multiple datasets.
我们介绍了SplattingAvatar，一种混合3D表示的真实感人类化身，将高斯Splatting嵌入到三角形网格中，它在现代GPU上的渲染速度超过300 FPS，在移动设备上为30 FPS。我们解开了虚拟人类的运动和外观，使用显式的网格几何和隐式的外观建模与高斯Splatting。高斯定义为三角网格上的重心坐标和位移，作为Phong表面。我们扩展了提升优化，以同时优化高斯参数，同时在三角形网格上行走。SplattingAvatar是虚拟人类的混合表示，其中网格代表低频运动和表面形变，而高斯接管高频几何和详细外观。与依赖于MLP基线性混合蒙皮（LBS）场的现有变形方法不同，我们通过网格直接控制高斯的旋转和平移，这增强了其与各种动画技术的兼容性，例如，骨骼动画、混合形状和网格编辑。SplattingAvatar能够从单目视频中训练，适用于全身和头部化身，显示出跨多个数据集的最先进渲染质量。