Skip to content

Commit

Permalink
[ET] Fix serde handling of inf (#6832)
Browse files Browse the repository at this point in the history
Pull Request resolved: #6799

## Context

I was trying to inspect the ETDump of a Vulkan LLaMa inference run, and experienced an `Exception` stemming from a symbol range containing an infinity.

The full error log is

```
Traceback (most recent call last):
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 4035, in _mag
    mag_first_dig = int(ceil(log10(xpos)))
OverflowError: cannot convert float infinity to integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__run_lpar_main__.py", line 31, in <module>
    __invoke_main()
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__run_lpar_main__.py", line 28, in __invoke_main
    run_as_main(module, main_function)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__par__/meta_only/bootstrap.py", line 98, in run_as_main
    oss_run_as_main(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__par__/bootstrap.py", line 94, in run_as_main
    main()
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/backends/vulkan/fb/tools/inspector/inspector_cli.py", line 41, in main
    inspector = Inspector(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/devtools/inspector/_inspector.py", line 1025, in __init__
    self._etrecord = parse_etrecord(etrecord_path=etrecord)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/devtools/etrecord/_etrecord.py", line 318, in parse_etrecord
    edge_dialect_program = deserialize(serialized_artifact)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/serialize.py", line 683, in deserialize
    return ExportedProgramDeserializer(expected_opset_version).deserialize(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/serialize.py", line 598, in deserialize
    res = GraphModuleDeserializer().deserialize(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/export_serialize.py", line 1855, in deserialize
    lower = int(vr.lower)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 308, in __int__
    r = self.round(2)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 3856, in round
    digits_to_decimal = _mag(x)  # _mag(12) = 2, _mag(.012) = -1
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 4037, in _mag
    mag_first_dig = int(ceil(Float(mpf_log(xpos._mpf_, 53))/log(10)))
AttributeError: 'Infinity' object has no attribute '_mpf_'
```

This arises because the range of a symbol is `[-inf to inf]`. Since infinity cannot be converted to `int` in Python, an exception is raised.

## Changes

Check if a range bound is negative or positive infinity and set it explicitly instead of converted the symbolic integer to `int`.
ghstack-source-id: 253217725
@exported-using-ghexport

Differential Revision: [D65848116](https://our.internmc.facebook.com/intern/diff/D65848116/)

Co-authored-by: Stephen Jia <[email protected]>
  • Loading branch information
pytorchbot and SS-JIA authored Nov 13, 2024
1 parent ee32ea3 commit 93b202c
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion exir/serde/export_serialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -1847,7 +1847,13 @@ def deserialize(
self.symbol_name_to_range = {}
if symbol_name_to_range:
for k, vr in symbol_name_to_range.items():
lower = int(vr.lower)
if math.isinf(vr.lower) and vr.lower < 0:
lower = -math.inf
elif math.isinf(vr.lower):
lower = math.inf
else:
lower = int(vr.lower)

if vr.upper >= 2: # max is >= 2, not sym bool range
lower = max(2, lower)
self.symbol_name_to_range[k] = symbolic_shapes.ValueRanges(
Expand Down

0 comments on commit 93b202c

Please sign in to comment.