We propose RelitLRM, a Large Reconstruction Model (LRM) for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images captured under unknown static lighting. Unlike prior inverse rendering methods requiring dense captures and slow optimization, often causing artifacts like incorrect highlights or shadow baking, RelitLRM adopts a feed-forward transformer-based model with a novel combination of a geometry reconstructor and a relightable appearance generator based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture design enables to effectively decompose geometry and appearance, resolve the ambiguity between material and lighting, and capture the multi-modal distribution of shadows and specularity in the relit appearance. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines while being significantly faster.
我们提出了RelitLRM，一种大型重建模型（LRM），用于在新的照明条件下从稀疏（4-8张）姿态图像生成高质量的3D物体高斯点表示，这些图像是在未知的静态照明下捕获的。与先前的需要密集采集和缓慢优化的逆渲染方法不同，RelitLRM采用了一种前馈的基于Transformer的模型，结合了几何重建器和基于扩散的可重光照外观生成器的创新架构。该模型在具有已知不同照明条件的合成多视角渲染图像上进行端到端训练。此架构设计能够有效分解几何和外观，解决材料与光照之间的模糊性问题，并捕捉重光照外观中的阴影和高光的多模态分布。我们展示了稀疏视角的前馈式RelitLRM在重光照效果上与最先进的基于密集视角优化的基准方法具有竞争力，同时显著加快了处理速度。