Why has post-processing (even in previous versions as early as 3.67 take so long that it basically doesn't work anymore? Is it just me?

[Bakulat]

I've tried the newest version, and I've gone back several updates and if I add any post-processing at all like Codeformer, GPEN, etc. I'm trying to figure out if there's something I can do to get its effectiveness and speed back!

[C0untFloyd]

It's just you. On the contrary it should be a lot faster since converting them from pytorch to onnx models. Stupid question but did you verify you're running on cuda? Or is your vram exhausted when using post-processors (most likely)?

[Bakulat]

I have Cuda selected in the web-ui. The only reason I'm so confused is because it has been working absolutely perfect and lightning fast until recently. What could cause a sudden drop like that? I mean I have made a lot of videos (like a lot of videos), with post-processing, tweaking and experimenting, and this has been working perfectly for at least the 2 months since I've been using it. and all of a sudden its just completely unusable. I'd really love to try to figure this out, and thank you for responding. I'm not extremely knowledgeable in this area but I'm determined to figure this out.

I did delete some large unreleated folders in the same folder where I had roop installed, but I didn't think I messed with anything. And I've since reinstalled several times trying to figure out why the performance has fallen off. Is it possible cuda isn't installed properly or I should try to update it? Do you know the command line for that?

I was trying version 3.9 at at the moment and at the top of the ui it says:
python: 3.10.14 • torch: 2.1.2+cu118 • gradio: 4.26.0

I'm running a faceswap without a post processor right now and I see this line a few times
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}

Does that mean it isn't using my GPU?
I found this just now too: No CLI args supported - use Settings Tab instead
Using provider ['CPUExecutionProvider'] - Device:cpu

[Bakulat]

I'm tellin ya! Somethin ain't right!! I just ordered a 12gb 3060 that will be here ina couple days and I have a gut feeling that it still isn't gonna be working right. I might just end up reinstalling windows and starting fresh, been a while anyways. But I know something's going on that is causing the major drop in performance and it isn't my specs. I don't mean to imply it isn't on my end, but its just driving me crazy I haven't been able to figure out what it is.

[C0untFloyd]

Just keep calm this isn't the end of the world (yet) 😏

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
Does that mean it isn't using my GPU?

Yes and I very much doubt

I have Cuda selected in the web-ui.

If you would have it selected and it's not installed you would get a onnxruntime error. I think the solution might be just as easy as selecting cuda in the combobox in the settings. Apply settings after changing and restart everything.
Running on your nvidia card would look like:
Using provider ['CUDAExecutionProvider'] - Device:cuda

And when swapping:

Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'has_user_compute_stream': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'enable_cuda_graph': '0', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'prefer_nhwc': '0', 'use_ep_level_unified_stream': '0'}, 'CPUExecutionProvider': {}}

CLI arguments aren't supported for a very long time. You can configure everything in the settings tab.
Oh btw. how did you install? Just doing a git pull isn't sufficient, you also need to do a pip install -r requirements.txt most of the time. Of course with activated conda env.

[Bakulat]

I can promise you I have Cuda selected

te_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}}
find model: C:\Roop\roop-unleashed_win3.9.0_installer\roop-unleashed\roop-unleashed\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}}
find model: C:\Roop\roop-unleashed_win3.9.0_installer\roop-unleashed\roop-unleashed\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}}
find model: C:\Roop\roop-unleashed_win3.9.0_installer\roop-unleashed\roop-unleashed\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}}
model ignore: C:\Roop\roop-unleashed_win3.9.0_installer\roop-unleashed\roop-unleashed\models\buffalo_l\genderage.onnx genderage
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}}
find model: C:\Roop\roop-unleashed_win3.9.0_installer\roop-unleashed\roop-unleashed\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
Processing: 6%|████▍

That's my last one, I've reinstalled again in a different folder and deleted lots of leftover things from previous installations to try and clear out what might be wrong, I ran the requirements.txt, its still not working on the newest version or previous versions

I feel like its something from my python cache that's screwing this up, but I am pretty much out of ideas other than a full reinstall of windows. I'm not a coder by any means but I think an older installation of roop is messing up the post processing, I really couldn't tell you.

This code you linked:
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'has_user_compute_stream': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'enable_cuda_graph': '0', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'prefer_nhwc': '0', 'use_ep_level_unified_stream': '0'}, 'CPUExecutionProvider': {}}

Am I supposed to open command in the installation folder and run that?

[Bakulat]

Update: I used pip cache purge inside the roop folder and it removed about 1700 items, I did a fresh install, and now post-processing seems to be working again!
Processing: 100%|████████████████████▉| 362/363 [06:06<00:01, 1.01s/frames, memory_usage=03.29GB, execution_threads=2]

I feel like I've had much better results than this, but there could be a couple factors behind that. Once this 3060 gets delivered with that meaty 12gb, I think I'll have much better results. Currently did this with 3.9, I'm afraid to upgrade now! lol

[Bakulat]

although I'm not sure, next attempt didn't go super well and 2 minutes per 100 frames doesn't seem to match my earlier performance.

What do you make of this?

Ignoring torch: markers 'sys_platform == "darwin"' don't match your environment
Ignoring torchvision: markers 'sys_platform == "darwin"' don't match your environment
Ignoring onnxruntime: markers 'sys_platform == "darwin" and platform_machine != "arm64"' don't match your environment
Ignoring onnxruntime-silicon: markers 'sys_platform == "darwin" and platform_machine == "arm64"' don't match your environment

[Bakulat]

all hope is lost Processing: 6%|█▎ | 18/301 [02:49<43:23, 9.20s/frames, memory_usage=02.75GB, execution_threads=3]

edit: seems like running nevc264 was apart of the problem. went back to libx264 and now its fast again.

[C0untFloyd]

Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider':

Looks good to me. The only thing left for you to tweak are the number of threads. Perhaps 3 is already too much currently.
There is a benchmark thread here
#461
and a general FAQ Entry
https://github.com/C0untFloyd/roop-unleashed/wiki/FAQ#why-is-my-graphics-card-gpu-not--hardly-being-utilized

You know, the most important thing when using a gpu is not running out of vram! Because if that happens, the memory is shuffled between your regular ram and the gpu video memory. This can slow processing down to a crawl!

[Bakulat]

Noted! It seems like if I try to run it on anything other than the lib codec, like hevnc, its not going to work. and the lib is giving me very low quality results. I have to use the cuda 11.8 right? What if I maybe installed multiple versions of it?

[Bakulat]

I will check all that out - any reason why hvenc would specifically be giving me problems?

Edit: probably some of the best results I've had since I ran into trouble:
Processing: 100%|████████████████████▉| 425/426 [02:44<00:00, 2.58frames/s, memory_usage=01.72GB, execution_threads=1]
Running ffmpeg

Processing Watchvid took 164.56353497505188 secs
Finished

I used force cpu for facial analyssis and h264 4gb cut off for memory on my gpu (6gb total currently 2060)

[Bakulat]

Just wanna wrap this thread up by saying I got my Roopy back! Thanks for the help.

To anyone out there with similar issues, I used a different drive altogether to do a fresh install, and I used pip cache purge to clear out the old components that were messing with my build. I had uninstalled and reinstalled so many times I think the cache had components that weren't updated/compatible. I also double checked I had the right cuda version installed 8.11.

[C0untFloyd]

Glad you could sort it out. I still believe the most important change was reducing the num threads because 6 Gb VRAM is really just too little for postprocessing in parallel.