model not matched in notebooks/generate_captions.ipynb ?

[YhQIAO]

run the code cell :
prefix_length = 10 model = ClipCaptionModel(prefix_length) model.load_state_dict(torch.load(model_path, map_location=CPU)) model = model.eval() device = CUDA(0) if is_gpu else "cpu" model = model.to(device)

and get error:
RuntimeError: Error(s) in loading state_dict for ClipCaptionModel:
Unexpected key(s) in state_dict: "gpt.transformer.h.0.attn.bias", "gpt.transformer.h.0.attn.masked_bias", "gpt.transformer.h.1.attn.bias", "gpt.transformer.h.1.attn.masked_bias", "gpt.transformer.h.2.attn.bias", "gpt.transformer.h.2.attn.masked_bias", "gpt.transformer.h.3.attn.bias", "gpt.transformer.h.3.attn.masked_bias", "gpt.transformer.h.4.attn.bias", "gpt.transformer.h.4.attn.masked_bias", "gpt.transformer.h.5.attn.bias", "gpt.transformer.h.5.attn.masked_bias", "gpt.transformer.h.6.attn.bias", "gpt.transformer.h.6.attn.masked_bias", "gpt.transformer.h.7.attn.bias", "gpt.transformer.h.7.attn.masked_bias", "gpt.transformer.h.8.attn.bias", "gpt.transformer.h.8.attn.masked_bias", "gpt.transformer.h.9.attn.bias", "gpt.transformer.h.9.attn.masked_bias", "gpt.transformer.h.10.attn.bias", "gpt.transformer.h.10.attn.masked_bias", "gpt.transformer.h.11.attn.bias", "gpt.transformer.h.11.attn.masked_bias".

[georghess]

Hi,
easiest thing is to set model.load_state_dict(model_weights, strict=False) . See rmokady/CLIP_prefix_caption#76 for reference.