Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AArch64 C_graph_ci_cpu test fails #2139

Open
Radu2k opened this issue Oct 1, 2024 · 0 comments
Open

AArch64 C_graph_ci_cpu test fails #2139

Radu2k opened this issue Oct 1, 2024 · 0 comments
Labels
bug A confirmed library bug platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64

Comments

@Radu2k
Copy link
Contributor

Radu2k commented Oct 1, 2024

Summary

This PR has enabled the test to run. This led to identifying that there are cases where ctest -R test_benchdnn_modeC_graph_ci_cpu fails.

We need to appropriately guard or remove the unsupported test cases.

Version

Report oneDNN version is main.

Environment

  • CPU: Neoverse V1
  • Compiler version: gcc-10 | g++-10
  • CMake build command:
    CC=gcc-10 CXX=g++-10 cmake ../../ \ -DDNNL_AARCH64_USE_ACL=ON \ -DCMAKE_BUILD_TYPE=Debug \ -DDNNL_CPU_RUNTIME=OMP \ -DDNNL_TEST_SET=CI \ -DDNNL_BUILD_FOR_CI=ON \ -DDNNL_BUILD_TESTS=ON

Steps to reproduce

Run ctest -R test_benchdnn_modeC_graph_ci_cpu in the build dir.

Observed behavior

Multiple tests cases fail.

run: --graph --case=complex_fusion/mha/MHA-GPT-inf-int8-bs1.json
[   0][0:0:0:0] exp_f32:          62 exp:          62 got:           4 diff:      58 rdiff:0.935484
[   1][0:0:0:1] exp_f32:           5 exp:           5 got:           3 diff:       2 rdiff:     0.4
[   2][0:0:0:2] exp_f32:          63 exp:          63 got:           2 diff:      61 rdiff:0.968254
[   3][0:0:0:3] exp_f32:          62 exp:          62 got:           5 diff:      57 rdiff:0.919355
[   4][0:0:0:4] exp_f32:         250 exp:         250 got:         128 diff:     122 rdiff:   0.488
[   5][0:0:0:5] exp_f32:           4 exp:           4 got:         130 diff:     126 rdiff:    31.5
[   6][0:0:0:6] exp_f32:           9 exp:           9 got:         132 diff:     123 rdiff: 13.6667
[   7][0:0:0:7] exp_f32:         200 exp:         200 got:           3 diff:     197 rdiff:   0.985
[   8][0:0:0:8] exp_f32:         196 exp:         196 got:         130 diff:      66 rdiff:0.336735
[   9][0:0:0:9] exp_f32:         199 exp:         199 got:         255 diff:      56 rdiff:0.281407

run: --graph --case=complex_fusion/mha/MHA-stable_diffusion-inf-bf16-bs1.json
[  80][0:0:1:0] exp_f32:    0.565094 exp:    0.566406 got:      0.5625 diff:0.00390625 rdiff:0.00689655
[  82][0:0:1:2] exp_f32:  -0.0144409 exp:  -0.0144653 got:  -0.0158691 diff:0.00140381 rdiff:0.0970464
[  87][0:0:1:7] exp_f32:     1.38038 exp:     1.38281 got:       1.375 diff:0.0078125 rdiff:0.00564972
[  89][0:0:1:9] exp_f32:    0.223952 exp:    0.223633 got:    0.226562 diff:0.00292969 rdiff:0.0131004
[  92][0:0:1:12] exp_f32:    0.962662 exp:    0.960938 got:    0.964844 diff:0.00390625 rdiff:0.00406504
[ 106][0:0:1:26] exp_f32:    -0.98731 exp:   -0.988281 got:   -0.992188 diff:0.00390625 rdiff:0.00395257
[ 108][0:0:1:28] exp_f32:   -0.739033 exp:   -0.738281 got:   -0.742188 diff:0.00390625 rdiff:0.00529101
[ 112][0:0:1:32] exp_f32:    0.840471 exp:    0.839844 got:     0.84375 diff:0.00390625 rdiff:0.00465116
[ 117][0:0:1:37] exp_f32:     1.20595 exp:     1.20312 got:     1.21094 diff:0.0078125 rdiff:0.00649351
[ 122][0:0:1:42] exp_f32:    -4.07635 exp:     -4.0625 got:    -4.09375 diff: 0.03125 rdiff:0.00769231

Expected behavior

Pass by completing successfully by skipping the unsupported cases.

@theComputeKid theComputeKid added bug A confirmed library bug platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64 labels Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A confirmed library bug platform:cpu-aarch64 Codeowner: @oneapi-src/onednn-cpu-aarch64
Projects
None yet
Development

No branches or pull requests

2 participants