In this paper, we propose a novel semantic splatting approach based on Gaussian Splatting to achieve efficient and low-latency. Our method projects the RGB attributes and semantic features of point clouds onto the image plane, simultaneously rendering RGB images and semantic segmentation results. Leveraging the explicit structure of point clouds and a one-time rendering strategy, our approach significantly enhances efficiency during optimization and rendering. Additionally, we employ SAM2 to generate pseudo-labels for boundary regions, which often lack sufficient supervision, and introduce two-level aggregation losses at the 2D feature map and 3D spatial levels to improve the view-consistent and spatial continuity.
本文提出了一种基于高斯点云（Gaussian Splatting）的新型语义投影方法，以实现高效、低延迟的渲染与优化。我们的方法将点云的RGB属性和语义特征投影到图像平面，同时生成RGB图像和语义分割结果。通过利用点云的显式结构和一次性渲染策略，我们的方法显著提升了优化和渲染过程的效率。 此外，为了解决边界区域监督不足的问题，我们引入了 SAM2 生成伪标签，用于增强边界区域的语义信息。与此同时，我们在二维特征图和三维空间两个层次上引入了双重聚合损失，以改善视角一致性和空间连续性。 这一方法不仅在效率上有显著提升，还在语义分割的准确性和连续性上取得了优异表现，为实时场景渲染和语义理解提供了一种有效的解决方案。