mllama3.2-V-11b support text-only mode?

[AnyangAngus]

Can mllama3.2-V-11b support text only mode?
transformers code like this: https://github.com/huggingface/transformers/blob/655bec2da7120a8681acc2ce951f8d58c6f0e6ef/src/transformers/models/mllama/modeling_mllama.py#L1645

[AllentDan]

Yes, text only is supported.

[AnyangAngus]

Yes, text only is supported.

@AllentDan cool !
how to use the text only infer on pipeline? set the image = None?

from lmdeploy import pipeline
from lmdeploy.vl import load_image

pipe = pipeline('meta-llama/Llama-3.2-11B-Vision-Instruct')

#image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
image=None
response = pipe(('who won the world cup in 2006?', image))
print(response)

[AllentDan]

No, just like normal LLM inference. response = pipe('who won the world cup in 2006?')

[AnyangAngus]

No, just like normal LLM inference. response = pipe('who won the world cup in 2006?')

@AllentDan found log lmdeploy - WARNING - archs.py:53 - Fallback to pytorch engine because /models/Llama3.2-11B-Vision-Instruct/ not supported by turbomind engine.

How long can turbomind engine support ?

[AllentDan]

There is no such a plan in the short future.