3D representation is essential to the significant advance of 3D generation with 2D diffusion priors. As a flexible representation, NeRF has been first adopted for 3D representation. With density-based volumetric rendering, it however suffers both intensive computational overhead and inaccurate mesh extraction. Using a signed distance field and Marching Tetrahedra, DMTet allows for precise mesh extraction and real-time rendering but is limited in handling large topological changes in meshes, leading to optimization challenges. Alternatively, 3D Gaussian Splatting (3DGS) is favored in both training and rendering efficiency while falling short in mesh extraction. In this work, we introduce a novel 3D representation, Tetrahedron Splatting (TeT-Splatting), that supports easy convergence during optimization, precise mesh extraction, and real-time rendering simultaneously. This is achieved by integrating surface-based volumetric rendering within a structured tetrahedral grid while preserving the desired ability of precise mesh extraction, and a tile-based differentiable tetrahedron rasterizer. Furthermore, we incorporate eikonal and normal consistency regularization terms for the signed distance field to improve generation quality and stability. Critically, our representation can be trained without mesh extraction, making the optimization process easier to converge. Our TeT-Splatting can be readily integrated in existing 3D generation pipelines, along with polygonal mesh for texture optimization. Extensive experiments show that our TeT-Splatting strikes a superior tradeoff among convergence speed, render efficiency, and mesh quality as compared to previous alternatives under varying 3D generation settings.
三维表示对于利用二维扩散先验进行三维生成的重大进展至关重要。作为一种灵活的表示方式，NeRF首先被用于三维表示。尽管采用基于密度的体积渲染，但它在计算开销大和网格提取不准确方面都有所不足。使用有符号距离场和行进四面体算法，DMTet能够实现精确的网格提取和实时渲染，但在处理网格的大规模拓扑变化时存在局限，导致优化挑战。另一方面，3D高斯涂抹（3DGS）在训练和渲染效率上受到青睐，但在网格提取方面略显不足。在这项工作中，我们引入了一种新颖的三维表示方式，四面体涂抹（TeT-Splatting），它支持在优化过程中易于收敛、精确的网格提取和实时渲染。这是通过在结构化的四面体网格中整合基于表面的体积渲染，同时保留精确网格提取的所需能力，并采用基于瓦片的可微四面体光栅化器实现的。此外，我们为有符号距离场引入了等值面和法线一致性的正则化项，以提高生成质量和稳定性。关键的是，我们的表示可以在不进行网格提取的情况下进行训练，使优化过程更易于收敛。我们的TeT-Splatting可以容易地集成到现有的三维生成流程中，并与多边形网格一起用于纹理优化。广泛的实验表明，与之前的替代方案相比，我们的TeT-Splatting在收敛速度、渲染效率和网格质量之间取得了更优的平衡，适用于不同的三维生成设置。