We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python benchmarks/dynamo/torchbench.py --accuracy --bfloat16 -d xpu -n10 --inference --only shufflenet_v2_x1_0 --backend=inductor
============ Summary for torchbench bfloat16 inference accuracy ============ Real failed models: 14 [['timm_nfnet', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy'], ['resnet152', 'fail_accuracy']] ============ Summary for torchbench bfloat16 training accuracy ============ Real failed models: 11 [['shufflenet_v2_x1_0', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_nfnet', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['resnet152', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy']] ============ Summary for torchbench float16 training accuracy ============ Real failed models: 5 [['timm_regnet', 'fail_accuracy'] ============ Summary for torchbench amp_bf16 inference accuracy ============ Real failed models: 17 [['resnet152', 'fail_accuracy'], ['resnet18', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_regnet', 'fail_accuracy'], ['resnext50_32x4d', 'fail_accuracy'], ['densenet121', 'fail_accuracy']] ============ Summary for torchbench amp_bf16 training accuracy ============ Real failed models: 14 [['resnet18', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['resnet50', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['densenet121', 'fail_accuracy'], ['timm_regnet', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['resnext50_32x4d', 'fail_accuracy'], ['resnet152', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy']] ============ Summary for torchbench amp_fp16 training accuracy ============ Real failed models: 5 [['timm_regnet', 'fail_accuracy']]
env: python: 3.10 XPU_OPS: 9ed0a1a TRITON_COMMIT_ID: e98b6fcb8df5b44eb0d0addb6767c573d37ba024 TORCH_COMMIT_ID: 4f8b7c4272db521f7ffc4070ce1bdece513d1183 TORCHBENCH_COMMIT_ID: 03cde49eba0580ed17f9ae2250832fd8af4ed756 TORCHVISION_COMMIT_ID: d23a6e1664d20707c11781299611436e1f0c104f TORCHAUDIO_COMMIT_ID: a6b0a140cc13216975e8922093459019537bb80a TRANSFORMERS_VERSION: 243e186efbf7fb93328dd6b34927a4e8c8f24395 TIMM_COMMIT_ID: ac3470188b914c5d7a5058a7e28b9eb685a62427 DRIVER_VERSION: 1.23.10.49.231129.50 KERNEL_VERSION: 5.15.0-73-generic #80-Ubuntu SMP Mon May 15 15:18:26 UTC 2023 BUNDLE_VERSION: 2025.0.1.20241113 OS_PRETTY_NAME: Ubuntu 22.04.2 LTS GCC_VERSION: 11
The text was updated successfully, but these errors were encountered:
Caused by e035f6b
Sorry, something went wrong.
No branches or pull requests
🐛 Describe the bug
============ Summary for torchbench bfloat16 inference accuracy ============
Real failed models: 14 [['timm_nfnet', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy'], ['resnet152', 'fail_accuracy']]
============ Summary for torchbench bfloat16 training accuracy ============
Real failed models: 11 [['shufflenet_v2_x1_0', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_nfnet', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['resnet152', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy']]
============ Summary for torchbench float16 training accuracy ============
Real failed models: 5 [['timm_regnet', 'fail_accuracy']
============ Summary for torchbench amp_bf16 inference accuracy ============
Real failed models: 17 [['resnet152', 'fail_accuracy'], ['resnet18', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['timm_regnet', 'fail_accuracy'], ['resnext50_32x4d', 'fail_accuracy'], ['densenet121', 'fail_accuracy']]
============ Summary for torchbench amp_bf16 training accuracy ============
Real failed models: 14 [['resnet18', 'fail_accuracy'], ['shufflenet_v2_x1_0', 'fail_accuracy'], ['resnet50', 'fail_accuracy'], ['timm_resnest', 'fail_accuracy'], ['mnasnet1_0', 'fail_accuracy'], ['densenet121', 'fail_accuracy'], ['timm_regnet', 'fail_accuracy'], ['mobilenet_v2', 'fail_accuracy'], ['resnext50_32x4d', 'fail_accuracy'], ['resnet152', 'fail_accuracy'], ['timm_vovnet', 'fail_accuracy']]
============ Summary for torchbench amp_fp16 training accuracy ============
Real failed models: 5 [['timm_regnet', 'fail_accuracy']]
Versions
env:
python: 3.10
XPU_OPS: 9ed0a1a
TRITON_COMMIT_ID: e98b6fcb8df5b44eb0d0addb6767c573d37ba024
TORCH_COMMIT_ID: 4f8b7c4272db521f7ffc4070ce1bdece513d1183
TORCHBENCH_COMMIT_ID: 03cde49eba0580ed17f9ae2250832fd8af4ed756
TORCHVISION_COMMIT_ID: d23a6e1664d20707c11781299611436e1f0c104f
TORCHAUDIO_COMMIT_ID: a6b0a140cc13216975e8922093459019537bb80a
TRANSFORMERS_VERSION: 243e186efbf7fb93328dd6b34927a4e8c8f24395
TIMM_COMMIT_ID: ac3470188b914c5d7a5058a7e28b9eb685a62427
DRIVER_VERSION: 1.23.10.49.231129.50
KERNEL_VERSION: 5.15.0-73-generic #80-Ubuntu SMP Mon May 15 15:18:26 UTC 2023
BUNDLE_VERSION: 2025.0.1.20241113
OS_PRETTY_NAME: Ubuntu 22.04.2 LTS
GCC_VERSION: 11
The text was updated successfully, but these errors were encountered: