-
Notifications
You must be signed in to change notification settings - Fork 406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: Allow loading .gguf and .bin files #1114
Comments
That works already the |
hi, |
You have to add them with .bin, else they wont get loaded into the UI. |
gaby
changed the title
the ability to run gguf moddles?
bug: Allow loading .gguf and .bin files
Feb 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
would it be possible to add support to run gguf moddles like this?
https://huggingface.co/TheBloke/Kunoichi-7B-GGUF
I have been testing, and simply changeing the file extention from .gguf to .bin loads it ok.
but having support for the .gguf extention would help.
image I am using:
ghcr.io/serge-chat/serge:main
debian unstable with all updates installed.
docker docker compose installed and up-to-date.
using chromium from the debian repo.
thank you for considdering my request.
Checklist
docker-compose.yml
✓ f408cac Editdocker-compose.yml
✓ Editcharts/serge/values.yaml
✓ 105f0fd Editcharts/serge/values.yaml
✓ Editscripts/model_loader.py
✓ eadd93c Editscripts/model_loader.py
✓ EditREADME.md
✓ 761e0dc EditREADME.md
✓ EditThe text was updated successfully, but these errors were encountered: