New features and feature requests #87
Replies: 21 comments 31 replies
-
not sure what you mean by that exactly, but sounds interesting ;-) can you elaborate further? excited about further config-options for blending etc., too! well, you know what's on @anxiousottergames ' and my wishlist.. ;-p but like I already said: you focus on what you deem necessary, first. it's coming along nicely :-) |
Beta Was this translation helpful? Give feedback.
-
Sounds good to me. I'm not sure if you had them listed in order of importance but I personally feel like even the order of the planned features listed is perfect. |
Beta Was this translation helpful? Give feedback.
-
With Roop development now shut down, this is the only place where we can expect improvements on this nice tool. After a lot of tests, I figured out (at least within my test results) that the biggest flip side is the eye movement. It is totally missing in a video swap. The model seems to be looking straight forward, irrespective of the actions. Hence, the model looks like visually impaired or blind. This is not noticeable in a still image swap because it is a still image, there is no action. But in a video, this lack of eye movement is very easily noticeable. You can tell within 4-5 seconds that the video is fake. Above all of the other enhancements discussed, I think we first need to focus on how to copy the eye-movements of the target to the output while we swap the face from the source. Basically, the face will come from the source, we need to detect the eyeballs, copy the eyeball direction from the target (not using the source eyeball directions). That will add life to our end product. This technology is already there because Adobe Photoshop can do this. We can change eye direction in a photo using Photoshop. We have to do that same thing with the output image after detecting the direction from the target frames. But I am not sure if there is an open source project already available for this. A close match would be Gaze Correction, but it has limited features. It may be used as a start point. |
Beta Was this translation helpful? Give feedback.
-
That's why I linked https://github.com/ai-forever/ghost which especially cared about gaze and emotions. From my limited testing it did this better than the rest but the face mask was lacking or rather looked too artificial. |
Beta Was this translation helpful? Give feedback.
-
can we get a fps changer (probably doable with ffmpeg even without reencoding) this is useful for some high resolution videos that are 60fps and because of this takes a lot of time but can be 30fps cutting processing time in half , maybe processing every second frame instead of every frame ? Also like in stable diffusion extension can we select faces by number they appear on screen ? (1 and 3 for example out of 4 faces) Both of these are especially useful for 180vr videos where a persons face appear two times but since they are in different angles app doesnt recognize both as the same almost always. This is not a problem if only one person is in the video just select all faces BUT if there are more than one person on screen we either need to change all faces or just one persons face on one side of the video. |
Beta Was this translation helpful? Give feedback.
-
I tested Fast Segment-Anything yesterday and am quite disappointed. It's faster than the current Clip2Seg but surprisingly very bad when using low-resolution inputs. If it can identify objects, the resulting masks are way better though. I don't know if it's worth the hassle. |
Beta Was this translation helpful? Give feedback.
-
Suggestion on the Side: What about an Upload Source for an Audio File (mp3) for "directly" a new Lip Sync? -> Overwrits the Lip Sync of the Target Video and Change it with the "Manuellextra located Audio File. :-) |
Beta Was this translation helpful? Give feedback.
-
Controlnet?Sent from my iPhoneOn Aug 21, 2023, at 6:24 AM, JeetGuhaThakurta ***@***.***> wrote:
With Roop development now shut down, this is the only place where we can expect improvements on this nice tool. After a lot of tests, I figured out (at least within my test results) that the biggest flip side is the eye movement. It is totally missing in a video swap. The model seems to be looking straight forward, irrespective of the actions. Hence, the model looks like visually impaired or blind.
This is not noticeable in a still image swap because it is a still image, there is no action. But in a video, this lack of eye movement is very easily noticeable. You can tell within 4-5 seconds that the video is fake. Above all of the other enhancements discussed, I think we first need to focus on how to copy the eye-movements of the target to the output while we swap the face from the source.
Basically, the face will come from the source, we need to detect the eyeballs, copy the eyeball direction from the target (not using the source eyeball directions). That will add life to our end product. This technology is already there because Adobe Photoshop can do this. We can change eye direction in a photo using Photoshop. We have to do that same thing with the output image after detecting the direction from the target frames. But I am not sure if there is an open source project already available for this. A close match would be Gaze Correction, but it has limited features. It may be used as a start point.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
I'm searching for the possibility to erode and blur the face mask but on the last roop-unleashed (3.3.4) I can't find the options, is that possible? I'm asking that 'cause i'm still obtain a slight ghost-box around my swapped face and i want to improve the results blurring the box a bit. |
Beta Was this translation helpful? Give feedback.
-
Since I had a potato computer, I integrated gpen 256 thinking it was better than nothing. But I couldn't get the Original/Enhanced image blend ratio part to work correctly. It only works between 0.90 and 1. When it goes below that, it adds strange blurs and distorts the image. what would be the reason ? |
Beta Was this translation helpful? Give feedback.
-
@C0untFloyd quick question. I noticed the TEMP folder gets really chunky after extended use and keeps facesets duplicated from the UI. Can i periodically delete the contents of the temp folder? |
Beta Was this translation helpful? Give feedback.
-
Transfered here from the discussion started by @aripsam Since this version is pretty stable and has been running well for a while now, I'm just wondering if there are any plans to add any more features? Anything in the pipeline? Some features I can think of -
|
Beta Was this translation helpful? Give feedback.
-
i also have a suggestion, maybe its caused by its limitation? |
Beta Was this translation helpful? Give feedback.
-
Btw. I forked a new promising hairstyle transfer repo today, which would be a great addition to roop unleashed once it is working: I can't get it to work so far though, not in Colab and not on my machine. Perhaps one of you can enlighten me, the problem is the dynamic compilation of python modules which I can't seem to get right. Also there is a lot of hopefully unnecessary Jupyter stuff in it, this would have to be removed for our own local needs. |
Beta Was this translation helpful? Give feedback.
-
This app is so great. My new addiction. Only problem in my path to achieving perfection are the mouths. Not going to mask a mouth manual frame by frame, im doing 40k frame scenes. Clip2seg with trigger word: Open mouth does the trick, but the whole process falls back to cpu on my AMD system. So we go from 4 frames per sec, to 10 sec per frame. :P I found this screenshot at " It also comes with a 'Mouth Mask' mode that allows the mouth to interact with objects." This is the only thing roop unleashed is missing, a specific mouth masking tool. |
Beta Was this translation helpful? Give feedback.
-
So now i have a few really awesome video's made with roop-unleashed. Sometimes a short video takes like 2 hours of tweaking to get perfect. Sometimes restoreformer is magic, sometimes gpen. And the values are different for every source. I didnt keep a template/settings journal so now i want to re do some things i've made with different faces i'm stuck tweaking and trying again. Could we have like an settings/options to filename thing? or maybe like stable diffusion that we add meta data for each run. something like this would be awesome: Filename Output Template |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Unfortunately I can't actively access the UI components so this impossible. |
Beta Was this translation helpful? Give feedback.
-
Hi @C0untFloyd i have the following request or question. im now doing batches of pictures. With multilpe persons on them, but every picture has different people. On the left side i've added 10/15 different face sets. And what i would like to do is do big batches with multiple faces without having to put the faces in the target by hand for every picture. So an option to: input = face A - Face B - Face C and for every picture find all faces if fround 2 faces on picture automaticly do face A to found face A - Face B to found face B If this is to hard to implement. A way to change the first found from left to right to right to left or to 2nd found left to right would also fix my issue. I can then do the batch of pictures more then once. so run 1 would swap face (first found left to right) atm the only option is manualy select faces for 1000 pic's, or do all faces and have 2 or 3 clones :P |
Beta Was this translation helpful? Give feedback.
-
Hi @C0untFloyd I have a feature request. It is kind of a quality of life update. So when I choose Specify face selection for swapping to "selected faces" and set the action on no face detected to "skip frame" there is a lot of flickering in faces at weird angles. could it be possible to remove the entire frame length if there are lot of fickering frames in between good continuos frames. example - Suppose out of 10 frames 3 frames are skipped because no similar face was detected. I would love to skip those entire 10 frames all together so i dont see any flicker. Creating a smooth video without. I would rather have a portion of video frames to be cut out with a solid transition then have flickering faces frames that ruins immersion of the whole video. I wolud love to keep continuos good frames that dont have any skipped frames between them. This will eliminate flickering issue on many of my projects. A slider would be great that lets you set the number of skiped frames before the entire length of 10,20,30,40 etc continuos frames which ever the number you set in the UI are discarded. If i set slider to 2 then it will discard the entire length of 10 frames if it encounters 2 skipped frames in between those 10 frames. eliminating the flicker. It can be hard to explain what i mean but i know you will understand what i am trying to say. like in a video if i have 5 seconds of good face swap and then 2 seconds of face swape that contains many skipped frame then i would love to skip all those frames in those 2 seconds and only keep those 5 sec of good swaped frames. Thanks |
Beta Was this translation helpful? Give feedback.
-
Hello @C0untFloyd! Thank you for all of your work. Feature idea: Setting for auto clearing temp files or maybe an option to have a prompt pop up upon initial start to ask if you want to clear them Not sure what would work best or easiest but the temp folder definitely gets very large. I'm down to 100gbs on my drive and I'm running into a full drive error more often now as I forget to clear the temp folder. Super minor as a whole though and probably lower priority compared to other features. |
Beta Was this translation helpful? Give feedback.
-
Here I would like to collect and show planned features for roop unleashed.
My wishlist so far would be:
Beta Was this translation helpful? Give feedback.
All reactions