Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

zhangqingwu · 2024-08-21T09:54:53Z

Inference acceleration, can the trained model use some inference framework?
The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

zhangqingwu commented Aug 21, 2024

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

Comments

zhangqingwu commented Aug 21, 2024