Feature Request: Negative prompt in commandline

[ClashSAN]

Could you add the option to add negative prompting in commandline? I would really appreciate this option as using cpu, and the browser + gradio open ends up adding 10 seconds more per iteration. It's claimed that disabling the gradio loading bar improves inference speed here.

I'm might end up trying to install this on a lightweight linux distribution in order to generate a higher maximum size in one go without using swap memory. This kit is awesome by the way, the model conversion just works, it is only 18% slower than bes-dev/openvino , lower sizes are an option (finetuned miniSD), and I am able to use better sampling (DPM++ 2M) thanks to diffusers, in 6-10 steps.

[azuritecoin]

If you're going with Linux, you should use ROCm instead of ONNX. You'll get a 3-4x better performance.

Automatic1111's webUI works with AMD GPUs on Linux.

[ClashSAN]

I want to try getting sd to run on a portable usb, with non-gpu systems

[azuritecoin]

Added negative prompt to txt2img_onnx.py

BTW, if you're going to focus on non-gpu systems, you should take a look at the fp16 version of the model. It works in CPU ONNX, but not using DirectML.

[azuritecoin]

did the memory usage go down at least?

[ClashSAN]

it seems to for the lower sized generations

512x512
10.1 for fp32
7.9 but then later jumps and stays at 12.6 fp16

256x256
6.9 for fp32
5.4 for fp16

[ClashSAN]

@azuritecoin I was able to run your txt2img script with a quantized int8 onnx model, it does very well and uses very low memory. At 256x256, it uses 0.9 gb of ram, with 1.3 gb peak. This will be good to use with any model finetuned at 256x256. It takes the same amount of time for inference.

Do you happen to know how to use the onnxruntime-openvino acceleration with your txt2img script?

from the diffusers package, in line 145 in import_utils.py:

_onnxruntime_version = "N/A"
_onnx_available = importlib.util.find_spec("onnxruntime") is not None
if _onnx_available:
candidates = ("onnxruntime", "onnxruntime-gpu", "onnxruntime-directml", "onnxruntime-openvino")
_onnxruntime_version = None

[azuritecoin]

I don't know of any development to get diffusers to work with openvino. If I understand correctly, you will need to convert original stable diffusion/diffusers models to openvino models then run the model using ONNX.

If you got stable diffusion running on openvino then it's not that hard to create a UI interface using gradio interface.

[ClashSAN]

thanks for the info. I thought the direct-ml and openvino backends can both use regular onnx models, but I guess not. bes dev seemed to have converted the unet in a way that does not allow different sizes. Only 1 person was successful converting a model (wd1.3) for that openvino repo, I tried but ran into issues.