Error when attempting to optimize stable-diffusion with directML

[fragfeaster777]

When attempting: python stable_diffusion.py --optimize, I get a "TypeError: z_(): incompatible function arguments" error for "Optimizing text_encoder". Note that "Optimizing vae_encoder", "Optimizing vae_decoder" and "Optimizing unet" worked.

Log under "Optimizing text_encoder":

[DEBUG] [olive_evaluator.py:1153:validate_metrics] No priority is specified, but only one sub type metric is specified. Use rank 1 for single for this metric.
[INFO] [run.py:138:run_engine] Running workflow default_workflow
[INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[DEBUG] [run.py:179:run_engine] Registering pass OnnxConversion
[DEBUG] [run.py:179:run_engine] Registering pass OrtTransformersOptimization
[DEBUG] [accelerator_creator.py:130:_fill_accelerators] The accelerator device and execution providers are specified, skipping deduce.
[DEBUG] [accelerator_creator.py:169:_check_execution_providers] Supported execution providers for device gpu: ['DmlExecutionProvider', 'CPUExecutionProvider']
[DEBUG] [accelerator_creator.py:199:create_accelerators] Initial accelerators and execution providers: {'gpu': ['DmlExecutionProvider']}
[INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[DEBUG] [run.py:235:run_engine] Pass OnnxConversion already registered
[DEBUG] [run.py:235:run_engine] Pass OpenVINOConversion already registered
[DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered
[DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered
[INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[INFO] [engine.py:1085:_create_system] Creating target system ...
[DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local
[INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[INFO] [engine.py:1097:_create_system] Creating host system ...
[DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local
[INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[DEBUG] [engine.py:711:_cache_model] Cached model bebe0e3c to cache\default_workflow\models\bebe0e3c.json
[DEBUG] [engine.py:338:run_accelerator] Running Olive in no-search mode ...
[DEBUG] [engine.py:430:run_no_search] Running ['convert', 'optimize'] with no search ...
[INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
[DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\user_script.py is inferred to be of type file.
[DEBUG] [dummy_inputs.py:45:get_dummy_inputs] Using dummy_inputs_func to get dummy inputs
[DEBUG] [conversion.py:234:_export_pytorch_model] Converting model on device cpu with dtype None.
Traceback (most recent call last):
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 433, in
main()
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 370, in main
optimize(common_args.model_id, common_args.provider, unoptimized_model_dir, optimized_model_dir)
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 244, in optimize
run_res = olive_run(olive_config)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}")(name, value)
TypeError: z(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %340 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

Other information

• OS: Windows 10
• Olive version: Tried with 0.6.2 and 0.4.0
• onnxruntime-directml: 1.18.0

[xiaoyu-work]

@PatriceVignola can you please take a look?

[DominicSYHsieh]

I have some almost the same issues, here's the error log :

Optimizing text_encoder
[2024-08-08 02:36:04,730] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:36:04,733] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:36:04,733] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:36:04,733] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:36:04,735] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:36:04,735] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:36:04,735] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:36:04,773] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
main()
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 601, in main
optimize(
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 369, in optimize
olive_run(olive_config)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\Acer\Desktop\Olive\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\Acer\Desktop\Olive\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}")(name, value)
TypeError: z(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %341 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

as well as the text_encoder_2, error log:

Optimizing text_encoder_2
[2024-08-08 02:24:50,086] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:24:50,089] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:24:50,090] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:24:50,090] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:24:50,090] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:24:50,136] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
main()
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 601, in main
optimize(
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 369, in optimize
olive_run(olive_config)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\Acer\Desktop\Olive\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\Acer\Desktop\Olive\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}")(name, value)
TypeError: z(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %662 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModelWithProjection::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

System information :
OS: Windows 11
Olive : 0.6.2
onnxruntime-directml : tried with 1.18.0 / 1.18.1

[Jay19751103]

Use following can optimize the model, it needs to change json
accelerate==0.33.0
aiohappyeyeballs==2.3.4
aiohttp==3.10.1
aiosignal==1.3.1
alembic==1.13.2
annotated-types==0.7.0
async-timeout==4.0.3
attrs==24.2.0
certifi==2024.7.4
charset-normalizer==3.3.2
colorama==0.4.6
coloredlogs==15.0.1
colorlog==6.8.2
config==0.5.1
datasets==2.20.0
diffusers==0.29.2
dill==0.3.8
filelock==3.15.4
flatbuffers==24.3.25
frozenlist==1.4.1
fsspec==2024.5.0
greenlet==3.0.3
huggingface-hub==0.24.5
humanfriendly==10.0
idna==3.7
importlib_metadata==8.2.0
invisible-watermark==0.2.0
Jinja2==3.1.4
lightning-utilities==0.11.6
Mako==1.3.5
MarkupSafe==2.1.5
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.3
numpy==1.26.4
olive-ai==0.6.2
onnx==1.16.2
onnx-tool==0.9.0
onnx_opcounter==0.0.3
onnxruntime==1.18.0
onnxruntime-directml==1.18.0
opencv-python==4.10.0.84
optimum==1.21.3
optuna==3.6.1
packaging==24.1
pandas==2.2.2
pillow==10.4.0
protobuf==3.20.3
psutil==6.0.0
pyarrow==17.0.0
pyarrow-hotfix==0.6
pydantic==2.7.0
pydantic_core==2.18.1
pyreadline3==3.4.1
python-dateutil==2.9.0.post0
pytz==2024.1
PyWavelets==1.6.0
PyYAML==6.0.1
regex==2024.7.24
requests==2.32.3
safetensors==0.4.4
sentencepiece==0.2.0
six==1.16.0
SQLAlchemy==2.0.32
sympy==1.13.1
tabulate==0.9.0
tokenizers==0.19.1
torch==2.4.0
torchmetrics==1.4.1
tqdm==4.66.5
transformers==4.42.4
typing_extensions==4.12.2
tzdata==2024.1
urllib3==2.2.2
xxhash==3.4.1
yarl==1.9.4
zipp==3.19.2

OLIVE modifications
0001-Fix-optimization-sdxl-issues.patch

Note that weights.pb does not copied into optimized unet folder
find in from cache or footprint directory , the size close 5GB and copy it to the folder manually, then you can run it locally.

[Jay19751103]

Key for this issue is downgrade transformer to transformers==4.42.4

[saddam213]

I am also stuck on this issue

TypeError: z_(): incompatible function arguments. The following argument types are supported:
    1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %329 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

This has broken all stable-diffusion converter scripts

[DominicSYHsieh]

For this issue, downgrade transformers to 4.42.4 is work for me.