RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Image
The generation of high-quality 3D car assets is essential for various applications, including video games, autonomous driving, and virtual reality. Current 3D generation methods utilizing NeRF or 3D-GS as representations for 3D objects, generate a Lambertian object under fixed lighting and lack separated modelings for material and global illumination. As a result, the generated assets are unsuitable for relighting under varying lighting conditions, limiting their applicability in downstream tasks. To address this challenge, we propose a novel relightable 3D object generative framework that automates the creation of 3D car assets, enabling the swift and accurate reconstruction of a vehicle's geometry, texture, and material properties from a single input image. Our approach begins with introducing a large-scale synthetic car dataset comprising over 1,000 high-precision 3D vehicle models. We represent 3D objects using global illumination and relightable 3D Gaussian primitives integrating with BRDF parameters. Building on this representation, we introduce a feed-forward model that takes images as input and outputs both relightable 3D Gaussians and global illumination parameters. Experimental results demonstrate that our method produces photorealistic 3D car assets that can be seamlessly integrated into road scenes with different illuminations, which offers substantial practical benefits for industrial applications.
高质量3D汽车资产的生成对于包括视频游戏、自动驾驶和虚拟现实在内的多种应用至关重要。目前使用NeRF或3D-GS作为3D物体表示的生成方法，通常在固定光照下生成朗伯体对象，并且缺乏对材质和全局光照的独立建模。因此，生成的资产在不同光照条件下无法重新照明，限制了其在下游任务中的适用性。为了解决这一挑战，我们提出了一种全新的可重新照明3D物体生成框架，能够自动创建3D汽车资产，实现从单张输入图像快速且精确地重建车辆的几何结构、纹理和材质属性。我们的方法首先引入了一个大规模合成汽车数据集，包含超过1000个高精度3D车辆模型。我们使用结合BRDF参数的全局光照和可重新照明的3D高斯基元来表示3D对象。在此表示基础上，我们引入了一个前馈模型，能够将图像作为输入并输出可重新照明的3D高斯和全局光照参数。实验结果表明，我们的方法能够生成逼真的3D汽车资产，并可无缝集成到不同光照条件的道路场景中，这为工业应用提供了巨大的实用价值。