While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics.
尽管人类能够轻松识别内在动态并适应新场景，现代AI系统却常常面临挑战。当前的视觉动态定锚方法要么使用纯神经网络模拟器（黑箱），这可能违反物理定律，要么依赖传统的物理模拟器（白箱），这些方法依赖专家定义的方程，可能无法完全捕捉真实的动态。我们提出了一种名为Neural Material Adaptor (NeuMA) 的方法，它将现有的物理定律与学习到的修正相结合，能够在保持物理先验的普适性和可解释性的同时，精确学习实际的动态。此外，我们还提出了Particle-GS，这是一种基于粒子的3D高斯点云变体，用于连接模拟和观测到的图像，允许通过反向传播图像梯度来优化模拟器。在粒子精度、动态渲染质量和泛化能力等多方面的动态实验中，NeuMA展现了其对内在动态的准确捕捉能力。