MEGATHREAD: If the model conversion scripts fail, list them in this thread

[azuritecoin]

This thread is to consolidate all the current chat about failed model conversions. If you ran the conversion scripts and it failed please provide the following:

• the name of the model
• which script failed first
• the error logs including
  • the specific command that you use
  • All the text following Traceback

[Roknyses]

I'm very bad at programming, I wanted to convert a model from a novel ai leak that worked on webui from AUTOMATIC1111, with convert_original_stable_diffusion_to_diffusers.py . Added the address of the script, since without it an error appeared that it was not in the system32 folder.

C:\Windows\system32>python D:/random/Stable_diffusion/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="D:/random/Stable_diffusion/novel.ckpt" --dump_path="D:/random/Stable_diffusion/novel_diffusers"
Traceback (most recent call last):
  File "D:\random\Stable_diffusion\convert_original_stable_diffusion_to_diffusers.py", line 799, in <module>
    global_step = checkpoint["global_step"]
KeyError: 'global_step'

I tried to use the conversion script from here, the same error, but the description also says this:

C:\Windows\system32>python D:/random/Stable_diffusion/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="D:/random/Stable_diffusion/novel.ckpt" --dump_path="D:/random/Stable_diffusion/novel_diffusers"
global_step key not found in model
Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel:

after listing a bunch of weights...

- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "D:\random\Stable_diffusion\convert_original_stable_diffusion_to_diffusers.py", line 1006, in <module>
    text_model = convert_ldm_clip_checkpoint(checkpoint)
  File "D:\random\Stable_diffusion\convert_original_stable_diffusion_to_diffusers.py", line 647, in convert_ldm_clip_checkpoint
    text_model.load_state_dict(text_model_dict)
  File "C:\Users\yaroslav\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1671, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
        Missing key(s) in state_dict:

and again listing a large number of weights, i don't understand what it means, please help.

[azuritecoin]

As of now the conversion script does not work with novelAI leak. There might be a work around here: https://gist.github.com/sftblw/1586a5b0d962e606ac5e89754fdd24cd

I haven't tried it yet though.

[ItCameFr0mMars]

Trying to convert https://civitai.com/models/3950/art-and-eros-aeros-a-tribute-to-beauty
I first attempted to run the command shown in the Readme

C:\Users\username\OneDrive\Desktop\stable_diffusions>python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="./aEros.ckpt" --dump_path="./aErosDiff"
Traceback (most recent call last):
  File "C:\Users\username\OneDrive\Desktop\stable_diffusions\convert_original_stable_diffusion_to_diffusers.py", line 799, in <module>
    global_step = checkpoint["global_step"]
KeyError: 'global_step'

Then after updating the convert_original_stable_diffusion_to_diffusers.py script & running setup.bat - upgrade:

Traceback (most recent call last):
  File "C:\Users\username\Desktop\stable diff\convert_original_stable_diffusion_to_diffusers.py", line 19, in <module>
    from diffusers.pipelines.stable_diffusion.convert_from_ckpt import load_pipeline_from_original_stable_diffusion_ckpt
ModuleNotFoundError: No module named 'diffusers.pipelines.stable_diffusion.convert_from_ckpt'

After changing to a different diffusers version:

(virtualenv) C:\Users\username\Desktop\stable diff>python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="./aeros.ckpt" --dump_path="./aeros"
global_step key not found in model
Traceback (most recent call last):
  File "C:\Users\username\Desktop\stable diff\convert_original_stable_diffusion_to_diffusers.py", line 103, in <module>
    pipe = load_pipeline_from_original_stable_diffusion_ckpt(
  File "C:\Users\username\Desktop\stable diff\virtualenv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 881, in load_pipeline_from_original_stable_diffusion_ckpt
    original_config = OmegaConf.load(original_config_file)
  File "C:\Users\username\Desktop\stable diff\virtualenv\lib\site-packages\omegaconf\omegaconf.py", line 189, in load
    with io.open(os.path.abspath(file_), "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\username\\AppData\\Local\\Temp\\tmp8ng68_sy\\inferenc.yaml'

After adding --original_config_file=v1-inference.yaml to the command:

(virtualenv) C:\Users\username\Desktop\stable diff>python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="./aeros.ckpt" --dump_path="./aeros" --original_config_file=v1-inference.yaml
global_step key not found in model
Traceback (most recent call last):
  File "C:\Users\username\Desktop\stable diff\convert_original_stable_diffusion_to_diffusers.py", line 103, in <module>
    pipe = load_pipeline_from_original_stable_diffusion_ckpt(
  File "C:\Users\username\Desktop\stable diff\virtualenv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 953, in load_pipeline_from_original_stable_diffusion_ckpt
    converted_vae_checkpoint = convert_ldm_vae_checkpoint(checkpoint, vae_config)
  File "C:\Users\username\Desktop\stable diff\virtualenv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 485, in convert_ldm_vae_checkpoint
    new_checkpoint["encoder.conv_in.weight"] = vae_state_dict["encoder.conv_in.weight"]
KeyError: 'encoder.conv_in.weight'

No idea what to do next

[qwerkilo]

try to download the 7.63g model(artErosAerosATribute_aerosPruned.ckpt),it can be converted.then use this script(https://github.com/Amblyopius/AMD-Stable-Diffusion-ONNX-FP16/blob/main/diffusers_to_onnx_optim.py) and convert to onnx,with sd1.5 mse vae.

[ItCameFr0mMars]

@qwerkilo can you link me to that version of the model?

[qwerkilo]

https://civitai.com/api/download/models/4396?type=Model&format=PickleTensor

[ItCameFr0mMars]

@qwerkilo I got a not enough memory error: "RuntimeError: [enforce fail at ..\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 6553600 bytes." I only have 4 gigs of ram on this machine, is there a way to convert with that limit?

[azuritecoin]

I might have gotten it to work. Or at least it created a loadable ONNX model.

1. Download the attached convert script and replace the existing one
2. Download the .safetensor version of the model from Civitai
3. Download vae-ft-mse-840000-ema-pruned.ckpt from here: https://huggingface.co/stabilityai/sd-vae-ft-mse-original/blob/main/vae-ft-mse-840000-ema-pruned.ckpt
4. run the following: convert_original_stable_diffusion_to_diffusers.py --checkpoint_path=artErosAerosATribute_aerosNovae.safetensors --dump_path=areos_diffusers --vae_checkpoint_path=vae-ft-mse-840000-ema-pruned.ckpt --original_config_file=v1-inference.yaml --from_safetensors
5. run the second conversion script

convert_original_stable_diffusion_to_diffusers.txt

[morpheuskibbe]

"convert_stable_diffusion_checkpoint_to_onnx.py" does not apper to function correctly

It runs without error but the resulting files do not work for the diffuser. My first instance running into this was under the "Download Model and Convert to ONNX" section "Option 1" did not result in a functional situation, at least not when following the instructions as provided.

"Option 2" worked correctly and I am able to open the UI and generate images using the default stable diffusion model.

However the issue is that the same command in option 1 that didn't result in function is also the command called to convert 3rd party models from ckpt to onnx

Specifically:
python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="./name.ckpt" --dump_path="./name_diffusers"
python convert_stable_diffusion_checkpoint_to_onnx.py --model_path="./name_diffusers" --output_path="model/name_onnx"

results in a folder in models that does not contain things like the JSON file.
That files DOES exist in the diffusers folder, but moving the contents over doesn't solve the issue.

In any case as of now the current instructions do not function to convert cpkt properly. I would be happy if anyone could help me solve the issue.

[LarkTelby]

model: https://civitai.com/models/4201/realistic-vision-v20
2nd script failed first

Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
None

Traceback (most recent call last):
File "C:\stable_diff\convert_stable_diffusion_checkpoint_to_onnx.py", line 265, in
convert_models(args.model_path, args.output_path, args.opset, args.fp16)
File "C:\stable_diff\virtualenv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\stable_diff\convert_stable_diffusion_checkpoint_to_onnx.py", line 110, in convert_models
onnx_export(
File "C:\stable_diff\convert_stable_diffusion_checkpoint_to_onnx.py", line 58, in onnx_export
export(
File "C:\stable_diff\virtualenv\lib\site-packages\torch\onnx\utils.py", line 506, in export
_export(
File "C:\stable_diff\virtualenv\lib\site-packages\torch\onnx\utils.py", line 1548, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\stable_diff\virtualenv\lib\site-packages\torch\onnx\utils.py", line 1117, in _model_to_graph
graph = _optimize_graph(
File "C:\stable_diff\virtualenv\lib\site-packages\torch\onnx\utils.py", line 665, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\stable_diff\virtualenv\lib\site-packages\torch\onnx\utils.py", line 1901, in _run_symbolic_function
raise errors.UnsupportedOperatorError(
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.