Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize Pygmalion on CPU to support lower ram #4

Open
HRNPH opened this issue Mar 31, 2023 · 4 comments
Open

Optimize Pygmalion on CPU to support lower ram #4

HRNPH opened this issue Mar 31, 2023 · 4 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@HRNPH
Copy link
Owner

HRNPH commented Mar 31, 2023 •
edited
Loading

Interestingly we may be able to implement C optimized version of Pygmalion6B into our project
@HRNPH HRNPH added enhancement New feature or request help wanted Extra attention is needed labels Mar 31, 2023
@beyabay
Copy link

beyabay commented Mar 31, 2023

does it require 12gb RAM or VRAM?

@HRNPH
Copy link
Owner Author

HRNPH commented Mar 31, 2023 •
edited
Loading

Just RAM not VRAM for CPU inference it worked perfectly on 16GB RAM

but 12GB should be sufficient (in case it isn't please tell me) still I recommend 16GB RAM

as for GPU inference(Which will be faster obviously) I didn't do it yet since I didn't have GPU with enough VRAM to try it out

@endymion114
Copy link

What is missing to run it on a gpu assuming you had a few 24g m40 gpus laying around

@HRNPH
Copy link
Owner Author

HRNPH commented Apr 5, 2023 •
edited
Loading

Notting's missing it's just a few lines of code I just can't test it :D
I'll just do some update today then :b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@endymion114 @HRNPH @beyabay