Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to generate visualization mesh real time #200

Open
kai-lan opened this issue May 2, 2024 · 2 comments
Open

How to generate visualization mesh real time #200

kai-lan opened this issue May 2, 2024 · 2 comments

Comments

@kai-lan
Copy link

kai-lan commented May 2, 2024

Is there a way to generate body mesh based on model output at real-time? Isn't the model output compatible with SMPL paramters and we just need a formula like Vertices_new = Vertices_template + shape_param * beta + pose_param * pose because we have the pose? Why do we need python -m visualize.render_mesh, which seems to construct mesh from the output .mp4 video, and would take quite a few minutes?

Maybe somewhere I understood wrong?

@GuyTevet
Copy link
Owner

GuyTevet commented May 7, 2024

The SMPL mesh is currently constructed from the HumanML pose locations according to https://github.com/EricGuo5513/HumanML3D , hence there is a SMPLlify optimization in the loop that is not real-time. It should be possible to extract SMPL from the HumanML rotations (with some tweaks since they don't have the same rest pose), this should make the mesh visualization run in real-time.

@XGraphKhipu
Copy link

XGraphKhipu commented Aug 11, 2024

But SMPLify only converts joints to SMPL. Which tool can I use to obtain axis-angle data or directly convert to SMPL-X vertex?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants