Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It's already July, where are codes and dataset? #4

Open
JZArray opened this issue Jul 1, 2024 · 10 comments
Open

It's already July, where are codes and dataset? #4

JZArray opened this issue Jul 1, 2024 · 10 comments

Comments

@JZArray
Copy link

JZArray commented Jul 1, 2024

No description provided.

@KleinXin
Copy link

KleinXin commented Jul 1, 2024

looking forward

@guahunyo
Copy link

guahunyo commented Jul 4, 2024

https://github.com/KwaiVGI/LivePortrait There's a similar one here

@johndpope
Copy link

johndpope commented Jul 4, 2024

just an observeration -
Megaportrait paper dropped by samsung ai - russia in 2022-07 (2207.07621
then MetaPortrait https://arxiv.org/abs/2212.08062 by microsoft in december - 2212 - used DenseMotionNetwork
then VASA paper says ( 2404.10667) - says dont use metaportrait - use megaportrait (doesnt use densemotionnetwork) for faster fps - more details here johndpope/MegaPortrait-hack#16
(livePortrait is using densemotionnetwork)

it will be good when this EmoPortrait code drops (as @neeek2303 worked on the original megaportraits code) - as we can clear up details on warping with resnet50...

but..... some researchers in china released paper in 2405 may - which claims to achieve simpler / cleaner faster disentanglement - there's no warp function -
i've been coding this - and wasnt getting anywhere - then switched to gan architecture - believe after a few hours - it managed to disentangle the pose see the turned faces on the right.
https://github.com/johndpope/speak-hack
image
(once i get architecture right, will throw it at a gpu farm)

@neeek2303
Copy link
Owner

@johndpope @guahunyo @KleinXin @JZArray
Hey guys. Thank you for commenting and sorry for the long wait. At the moment I’m working on a project that is very time-consuming; I literally don’t have time to sit down, put things in order in the code, prepare the data and put everything in the public domain. This work was done as part of my work at Imperial College London, not as part of my work at Meta. So I can't do this during work hours. But rest assured, I'm still going to do it!

I will try to post the code in July as promised (and the dataset too). I will also have a vacation in August, and if I don’t have time to do everything in July, then at least I’ll post it in the first half of August. Thank you for your understanding!

@JZArray
Copy link
Author

JZArray commented Jul 4, 2024

https://github.com/KwaiVGI/LivePortrait There's a similar one here

@guahunyo thx,already tried

@JZArray
Copy link
Author

JZArray commented Jul 4, 2024

@johndpope @guahunyo @KleinXin @JZArray Hey guys. Thank you for commenting and sorry for the long wait. At the moment I’m working on a project that is very time-consuming; I literally don’t have time to sit down, put things in order in the code, prepare the data and put everything in the public domain. This work was done as part of my work at Imperial College London, not as part of my work at Meta. So I can't do this during work hours. But rest assured, I'm still going to do it!

I will try to post the code in July as promised (and the dataset too). I will also have a vacation in August, and if I don’t have time to do everything in July, then at least I’ll post it in the first half of August. Thank you for your understanding!

thx, actually we have already successfully reimplement MegaPortraits, and problems(such as ID leakage, closing eyes, lip movement and so on) have been solved. We only want to compare our reimplementation with the original one, and to see where we can further refine our self-built model.

@johndpope
Copy link

johndpope commented Jul 15, 2024

@KleinXin
Copy link

https://x.com/purzbeats/status/1812287664240107969?s=46&t=-tkSIrsyNobBjIvQ2IAQwQ

Got speed @neeek2303

Amazing! Is the face on the bottom left corner yours?It seems not the one from the paper.

@johndpope
Copy link

Hi @KleinXin - no - thats not me - https://www.youtube.com/watch?v=zLSfnO0Ob-I

@FurkanGozukara
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants