Auto load models on startup #3338
Unanswered
CiraciNicolo
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
I was reading through the doc and the issues, but it seems there is no way to load models in GPU at startup. Currently the model is loaded in VRAM after the first request causing some wait. It is possibile to autoload custom models in VRAM? Am I missing something?
The only reference to autoload is this, but I'm using a ConfigMap to create model config files.
Beta Was this translation helpful? Give feedback.
All reactions