Some layers are not loaded(‘encoder.embed_token.weight’ and 'shared.weight') when loading converted safetensors into transfomrers.MT5EncoderModel.

[WongGawa]

System Info

I converted pytorch_model.bin to model.safetensors locally for mt5-xxl model.
And I compared transfromers.MT5EncoderModel's last_hidden_states difference by mt5-xxl's pytorch_model.bin and safetensors.
And because encoder.embed_tokens.weight and shared.weight are not loaded into MT5EncoderModel when loading converted safetensors, which causes those layers to be newly initialized.

Information

• The official example scripts
• My own modified scripts

Reproduction

1. convert pytorch_model.bin to model.safetensors by convert.py
2. loading into transformers.MT5EncoderModel of pytorch_model.bin and converted model.safetensors
3. compute difference between las_hidden_state of pytorch_model.bin and model.safetensors

Expected behavior

the same output or at lease within a certain margin of error