Debug Deadlock from new Legion

[goliaro]

Description of changes:

To reproduce, make sure you are running on cuda 12 or above on nvidia tesla t4 gpus, then run:

cd tests/inference
python ../../inference/python/incr_decoding.py -config-file ./python_test_configs/incr_dec-python-llama-2-7b-hf-full_prec-2_tp_2_pp.json

Stack trace:
trace.log

Related Issues:

Linked Issues:

• Issue #

Issues closed by this PR:

• Closes #

---

This change is