Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add low VRAM mode, CPU-only mode + image pre-loading fix #407

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

FlyingFathead
Copy link

@FlyingFathead FlyingFathead commented Sep 7, 2024

  1. Low VRAM mode implemented: Introduced the unimplemented option lingering around in streamlit_helpers.py to run the model with half-precision (float16). Allows efficient usage on GPUs with limited memory, lowering memory requirements by half. I've tested this on a RTX 3060 w/ 12GB VRAM.

  2. CPU-only mode: added an option to run the model on CPUs for users without the necessary CUDA GPU hardware, though at slower speeds.

Both can be set with True/False in streamlit_helpers.py, although a more viable solution in the future would be to i.e. add a separate config.ini w/ configparser for the entire framework.

  1. Image Loading Fix: Fixed an issue where the app would throw an error if an image was not loaded before model execution. The model now uses a dummy tensor until the image is loaded in. See: video_sampling.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant