Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mllama3.2-V-11b support text-only mode? #2680

Open
AnyangAngus opened this issue Oct 29, 2024 · 5 comments
Open

mllama3.2-V-11b support text-only mode? #2680

AnyangAngus opened this issue Oct 29, 2024 · 5 comments
Assignees

Comments

@AnyangAngus
Copy link

Can mllama3.2-V-11b support text only mode?
transformers code like this: https://github.com/huggingface/transformers/blob/655bec2da7120a8681acc2ce951f8d58c6f0e6ef/src/transformers/models/mllama/modeling_mllama.py#L1645

@AllentDan
Copy link
Collaborator

Yes, text only is supported.

@AnyangAngus
Copy link
Author

Yes, text only is supported.

@AllentDan cool !
how to use the text only infer on pipeline? set the image = None?

from lmdeploy import pipeline
from lmdeploy.vl import load_image

pipe = pipeline('meta-llama/Llama-3.2-11B-Vision-Instruct')

#image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
image=None
response = pipe(('who won the world cup in 2006?', image))
print(response)

@AllentDan
Copy link
Collaborator

No, just like normal LLM inference. response = pipe('who won the world cup in 2006?')

@AnyangAngus
Copy link
Author

No, just like normal LLM inference. response = pipe('who won the world cup in 2006?')

@AllentDan found log lmdeploy - WARNING - archs.py:53 - Fallback to pytorch engine because /models/Llama3.2-11B-Vision-Instruct/ not supported by turbomind engine.

How long can turbomind engine support ?

@AllentDan
Copy link
Collaborator

There is no such a plan in the short future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants