export .pte error with Macbook pro M4

[wangqiang58]

🐛 Describe the bug

when I export .pte from Llama3.2 1B, it is enterrupt.
the error is blow:

python -m examples.models.llama.export_llama
--checkpoint "/Users/qitmac001443/.llama/checkpoints/Llama3.1-8B/consolidated.00.pth"
--params "/Users/qitmac001443/.llama/checkpoints/Llama3.1-8B/params.json"
-kv
--use_sdpa_with_kv_cache
-X
-d bf16
--metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}'
--output_name="llama3_1.pte"

INFO:root:Applying quantizers: []
INFO:root:Loading model with checkpoint=/Users/qitmac001443/.llama/checkpoints/Llama3.1-8B/consolidated.00.pth, params=/Users/qitmac001443/.llama/checkpoints/Llama3.1-8B/params.json, use_kv_cache=True, weight_type=WeightType.LLAMA
INFO:root:model.to torch.bfloat16
INFO:root:Loading custom ops library: /Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.dylib
INFO:root:Model after source transforms: Transformer(
(tok_embeddings): Embedding(128256, 4096)
(layers): ModuleList(
(0-31): 32 x TransformerBlock(
(attention): Attention(
(wq): Linear(in_features=4096, out_features=4096, bias=False)
(wk): Linear(in_features=4096, out_features=1024, bias=False)
(wv): Linear(in_features=4096, out_features=1024, bias=False)
(wo): Linear(in_features=4096, out_features=4096, bias=False)
(kv_cache): KVCache()
(SDPA): SDPA(
(kv_cache): KVCache()
)
(apply_rotary_emb): RotaryEmbedding()
)
(feed_forward): FeedForward(
(w1): Linear(in_features=4096, out_features=14336, bias=False)
(w2): Linear(in_features=14336, out_features=4096, bias=False)
(w3): Linear(in_features=4096, out_features=14336, bias=False)
)
(attention_norm): RMSNorm()
(ffn_norm): RMSNorm()
)
)
(norm): RMSNorm()
(output): Linear(in_features=4096, out_features=128256, bias=False)
)
INFO:root:Exporting with:
INFO:root:inputs: (tensor([[2, 3, 4]]), tensor([0]))
INFO:root:kwargs: None
INFO:root:dynamic shapes: ({1: <class 'executorch.extension.llm.export.builder.token_dim'>}, {0: 1})
E1129 14:14:25.160000 32543 .venv/lib/python3.12/site-packages/torch/export/_trace.py:1017] always_classified is unsupported.
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/export_llama.py", line 32, in
main() # pragma: no cover
^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/export_llama.py", line 28, in main
export_llama(args)
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/export_llama_lib.py", line 508, in export_llama
builder = _export_llama(args)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/export_llama_lib.py", line 643, in _export_llama
builder_exported = _prepare_for_llama_export(args).export()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/executorch/extension/llm/export/builder.py", line 201, in export
exported_module = export_for_training(
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/init.py", line 168, in export_for_training
return _export_for_training(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/_trace.py", line 1031, in wrapper
raise e
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/_trace.py", line 1004, in wrapper
ep = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/exported_program.py", line 122, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/_trace.py", line 1808, in _export_for_training
export_artifact = export_func( # type: ignore[operator]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/_trace.py", line 1279, in _strict_export_lower_to_aten_ir
gm_torch_level = _export_to_torch_ir(
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/export/_trace.py", line 660, in _export_to_torch_ir
gm_torch_level, _ = torch._dynamo.export(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 1539, in inner
result_traced = opt_f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1740, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 556, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1740, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1395, in call
return self._torchdynamo_orig_callable(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 545, in call
return _compile(
^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 977, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 706, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_utils_internal.py", line 95, in wrapper_function
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 741, in _compile_inner
out_code = transform_code_object(code, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/bytecode_transformation.py", line 1348, in transform_code_object
transformations(instructions, code_options)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 229, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 658, in transform
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2912, in run
super().run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2418, in CALL
self._call(inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2412, in _call
self.call_function(fn, args, kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/nn_module.py", line 442, in call_function
return tx.inline_user_function_return(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1816, in CALL_FUNCTION_EX
self.call_function(fn, argsvars.items, kwargsvars)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 410, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 349, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 125, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2418, in CALL
self._call(inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2412, in _call
self.call_function(fn, args, kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 410, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 349, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 125, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2418, in CALL
self._call(inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2412, in _call
self.call_function(fn, args, kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/nn_module.py", line 442, in call_function
return tx.inline_user_function_return(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1816, in CALL_FUNCTION_EX
self.call_function(fn, argsvars.items, kwargsvars)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 410, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 349, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 125, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2418, in CALL
self._call(inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2412, in _call
self.call_function(fn, args, kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 410, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 349, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/functions.py", line 125, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 973, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3127, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/dynamo/symbolic_convert.py", line 3255, in inline_call
tracer.run()
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1120, in run
while self.step():
^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1032, in step
self.dispatch_table[inst.opcode](self, inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 640, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2418, in CALL
self._call(inst)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2412, in _call
self.call_function(fn, args, kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 967, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/misc.py", line 1022, in call_function
return self.obj.call_method(tx, self.name, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/tensor.py", line 589, in call_method
return wrap_fx_proxy(
^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 2108, in wrap_fx_proxy
return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 2174, in wrap_fx_proxy_cls
return _wrap_fx_proxy(
^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 2270, in _wrap_fx_proxy
example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2278, in get_fake_value
raise TorchRuntimeError(str(e)).with_traceback(e.traceback) from None
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2213, in get_fake_value
ret_val = wrap_fake_exception(
^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 1761, in wrap_fake_exception
return fn()
^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2214, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2346, in run_node
raise RuntimeError(make_error_message(e)).with_traceback(
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2330, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/utils/_stats.py", line 21, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 1271, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 1813, in dispatch
return self._cached_dispatch_impl(func, types, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 1381, in _cached_dispatch_impl
output = self._dispatch_impl(func, types, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 2267, in _dispatch_impl
op_impl_out = op_impl(self, func, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_impls.py", line 147, in dispatch_to_op_implementations_dict
return op_implementations_dict[func](fake_mode, func, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_impls.py", line 641, in multi_device_op_default
return run_and_return_new_tensor_of_input_device(fake_mode, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_impls.py", line 551, in run_and_return_new_tensor_of_input_device
out = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_ops.py", line 723, in call
return self._op(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/meta_registrations.py", line 400, in meta_copy
aten.expand_copy.default(intermediate, self.size())
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_ops.py", line 723, in call
return self._op(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_refs/init.py", line 2228, in _fn
result = fn(*args, out=out, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_prims_common/wrappers.py", line 291, in _fn
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1123, in call
return self._op(args, **(kwargs or {}))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/_refs/init.py", line 2994, in expand
torch._check(
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/init.py", line 1617, in _check
_check_with(RuntimeError, cond, message)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/init.py", line 1599, in _check_with
raise error_type(message_evaluated)
torch.dynamo.exc.TorchRuntimeError: Failed running call_method copy((FakeTensor(..., size=(1, 8, 8, 128), dtype=torch.bfloat16), FakeTensor(..., size=(1, 8, s0, 128), dtype=torch.bfloat16)), **{}):
expand: attempting to expand a dimension of length s0!

from user code:
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/llama_transformer.py", line 534, in forward
h = layer(
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/llama_transformer.py", line 442, in forward
h = self.attention.forward(
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/llama_transformer.py", line 343, in forward
output = self.SDPA(input_pos, q, k, v, bsz, seqlen, self.mask)
File "/Users/qitmac001443/Desktop/workspace/executorch/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in call_impl
return forward_call(*args, **kwargs)
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/llama_transformer.py", line 250, in forward
k, v = self.kv_cache.update(input_pos, k, v)
File "/Users/qitmac001443/Desktop/workspace/executorch/examples/models/llama/llama_transformer.py", line 202, in update
narrowed_k.copy(k_val)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

Versions

PyTorch version: 2.6.0.dev20241112
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.5 (arm64)
GCC version: Could not collect
Clang version: 16.0.0 (clang-1600.0.26.4)
CMake version: version 3.31.1
Libc version: N/A

Python version: 3.12.7 | packaged by Anaconda, Inc. | (main, Oct 4 2024, 08:22:19) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-14.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M3

Versions of relevant libraries:
[pip3] executorch==0.5.0a0+d679ad7
[pip3] numpy==1.26.4
[pip3] torch==2.6.0.dev20241112
[pip3] torchao==0.7.0+git75d0693
[pip3] torchaudio==2.5.0.dev20241112
[pip3] torchsr==1.0.4
[pip3] torchvision==0.20.0.dev20241112
[conda] No relevant packages

[JacobSzwejbka]

Seems like the error is from export_for_training. Youll probably get better support in the pytorch/pytorch repo on that step. pytorch/pytorch#141893 I opened an issue here and tagged a compiler eng.