fix config saving when check on misplaced args broken

[eaidova]

What does this PR do?

some model configs does not expect be loaded without arguments that required for checking misplaced arguments in 4.45 release (example https://huggingface.co/OpenGVLab/InternVL2-1B). This fix allows ignoring such cases for loading and saving config files

from pathlib import Path
import requests
from optimum.intel.openvino import OVModelForVisualCausalLM
from PIL import Image
from transformers import AutoTokenizer


model_id = "OpenGVLab/InternVL2-1B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model_path = Path(".models/InternVL2-1B/FP32")
if not model_path.exists():
    model = OVModelForVisualCausalLM.from_pretrained(model_id, trust_remote_code=True)
    model.save_pretrained(model_path)
    tokenizer.save_pretrained(model_path)
model = OVModelForVisualCausalLM.from_pretrained(model_path, trust_remote_code=True)


image_file = "http://images.cocodataset.org/val2017/000000039769.jpg"
raw_image = Image.open(requests.get(image_file, stream=True).raw).convert("RGB")

question = "Please describe the image shortly."
inputs = model.preprocess_inputs(
    tokenizer=tokenizer,
    image=raw_image,
    text=question,
)
generation_output = model.generate(**inputs, max_new_tokens=10)

print(tokenizer.batch_decode(generation_output[:, inputs["input_ids"].shape[1]:]))

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[AlexKoff88]

I am fine with the code in the main part but the test is blowing off due to the need to replicate the model-specific preprocessing. I wonder if there is a more elegant solution for that?

[nikita-savelyevv]

Could you please provide an example in the PR description on how to run the InternVL2 model? Similar examples for (nano)llava and minicpmv were quite useful.

[eaidova]

Could you please provide an example in the PR description on how to run the InternVL2 model? Similar examples for (nano)llava and minicpmv were quite useful.

example of model inference added in description

[AlexKoff88]

@eaidova, LGTM. Please fix the test: tests/openvino/test_modeling.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_4_internvl2

[eaidova]

@eaidova, LGTM. Please fix the test: tests/openvino/test_modeling.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_4_internvl2

@AlexKoff88 it is fixed on model code level https://huggingface.co/katuni4ka/tiny-random-internvl2/commit/35d8860bba682fdbc190914c38f8af37eb46196c

[nikita-savelyevv]

LGTM

Could you please update the code example in the description with the following one?

from pathlib import Path
import requests
from optimum.intel.openvino import OVModelForVisualCausalLM
from PIL import Image
from transformers import AutoTokenizer


model_id = "OpenGVLab/InternVL2-1B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model_path = Path(".models/InternVL2-1B/FP32")
if not model_path.exists():
    model = OVModelForVisualCausalLM.from_pretrained(model_id, trust_remote_code=True)
    model.save_pretrained(model_path)
    tokenizer.save_pretrained(model_path)
model = OVModelForVisualCausalLM.from_pretrained(model_path, trust_remote_code=True)


image_file = "http://images.cocodataset.org/val2017/000000039769.jpg"
raw_image = Image.open(requests.get(image_file, stream=True).raw).convert("RGB")

question = "Please describe the image shortly."
inputs = model.preprocess_inputs(
    tokenizer=tokenizer,
    image=raw_image,
    text=question,
)
generation_output = model.generate(**inputs, max_new_tokens=10)

print(tokenizer.batch_decode(generation_output[:, inputs["input_ids"].shape[1]:]))