Remove -v in CI tests to produce shorter logs
#6438
GitHub Actions / Test Results
failed
Apr 26, 2024 in 0s
7 fail, 1 475 skipped, 2 344 pass in 2h 48m 51s
28 files 28 suites 2h 48m 51s ⏱️
3 826 tests 2 344 ✅ 1 475 💤 7 ❌
526 248 runs 81 962 ✅ 444 173 💤 113 ❌
Results for commit d2a068a.
Annotations
Check warning on line 0 in tests.functions.gemmgelu_test.TestGemmGelu
github-actions / Test Results
1 out of 22 runs failed: test_gemmgelu (tests.functions.gemmgelu_test.TestGemmGelu)
artifacts/Test Results (py311-ort-nightly-macos-latest)/pytest.xml [took 0s]
Raw output
AssertionError:
Not equal to tolerance rtol=1e-07, atol=0
Mismatched elements: 1 / 16 (6.25%)
Max absolute difference: 2.3841858e-07
Max relative difference: 1.5581162e-07
x: array([[[1.608876, 1.258398, 1.868434, 1.530172, 1.502594, 1.577003,
0.930287, 1.438904],
[2.2129 , 1.367099, 2.42691 , 2.158696, 1.992608, 2.096078,
1.297177, 2.084625]]], dtype=float32)
y: array([[[1.608876, 1.258398, 1.868434, 1.530172, 1.502594, 1.577003,
0.930287, 1.438904],
[2.2129 , 1.367099, 2.42691 , 2.158696, 1.992608, 2.096078,
1.297177, 2.084625]]], dtype=float32)
tests/functions/gemmgelu_test.py:62: in test_gemmgelu
self.run_converter_test(case)
tests/common/onnx_script_test_case.py:255: in run_converter_test
np.testing.assert_allclose(actual, param.output, rtol=self.rtol)
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
return func(*args, **kwds)
E AssertionError:
E Not equal to tolerance rtol=1e-07, atol=0
E
E Mismatched elements: 1 / 16 (6.25%)
E Max absolute difference: 2.3841858e-07
E Max relative difference: 1.5581162e-07
E x: array([[[1.608876, 1.258398, 1.868434, 1.530172, 1.502594, 1.577003,
E 0.930287, 1.438904],
E [2.2129 , 1.367099, 2.42691 , 2.158696, 1.992608, 2.096078,
E 1.297177, 2.084625]]], dtype=float32)
E y: array([[[1.608876, 1.258398, 1.868434, 1.530172, 1.502594, 1.577003,
E 0.930287, 1.438904],
E [2.2129 , 1.367099, 2.42691 , 2.158696, 1.992608, 2.096078,
E 1.297177, 2.084625]]], dtype=float32)
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU
github-actions / Test Results
7 out of 28 runs failed: test_output_match_opinfo__native_layer_norm_cpu_float32 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU)
artifacts/Test Results (py310-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-onnx-weekly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py38-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py39-macos-latest)/pytest.xml [took 0s]
Raw output
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 1.154693603515625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.0036514615640044212 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 15.526153564453125 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.04909800738096237 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 4.3814697265625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.013855422846972942 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 3.34576416015625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.010580234229564667 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU
github-actions / Test Results
7 out of 28 runs failed: test_output_match_opinfo__native_layer_norm_cpu_float32 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU)
artifacts/Test Results (py310-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-onnx-weekly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py38-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py39-macos-latest)/pytest.xml [took 0s]
Raw output
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 1.154693603515625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.0036514615640044212 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 15.526153564453125 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.04909800738096237 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 4.3814697265625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.013855422846972942 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Tensor-likes are not close!
E
E Mismatched elements: 1 / 1 (100.0%)
E Greatest absolute difference: 3.34576416015625 at index (0,) (up to 0.00018 allowed)
E Greatest relative difference: 0.010580234229564667 at index (0,) (up to 3.7e-05 allowed)
The above exception was the direct cause of the following exception:
tests/function_libs/torch_lib/ops_test.py:293: in run_test_output_match
raise AssertionError(f"Output {j} mismatch") from e
E AssertionError: Output 2 mismatch
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU
github-actions / Test Results
All 4 runs failed: test_output_match_opinfo__nn_functional_cross_entropy_cpu_float16 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU)
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-torch-nightly-windows-latest)/pytest.xml [took 2s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py312-torch-nightly-windows-latest)/pytest.xml [took 1s]
Raw output
AssertionError: Scalars are not close!
Expected 1.2529296875 but got 1.2509765625.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.001558846453624318 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected 1.2529296875 but got 1.2509765625.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.001558846453624318 (up to 0.001 allowed)
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU
github-actions / Test Results
All 4 runs failed: test_output_match_opinfo__nn_functional_nll_loss_weight_cpu_float16 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU)
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-torch-nightly-windows-latest)/pytest.xml [took 1s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py312-torch-nightly-windows-latest)/pytest.xml [took 1s]
Raw output
AssertionError: Scalars are not close!
Expected -1.087890625 but got -1.08984375.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.0017953321364452424 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -0.0015745162963867188 but got 0.0006766319274902344.
Absolute difference: 0.002251148223876953 (up to 1e-05 allowed)
Relative difference: 1.429739551786796 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -1.4326171875 but got -1.4345703125.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.0013633265167007499 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -0.2271728515625 but got -0.228515625.
Absolute difference: 0.0013427734375 (up to 1e-05 allowed)
Relative difference: 0.005910800644814616 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected 21.34375 but got 21.3125.
Absolute difference: 0.03125 (up to 1e-05 allowed)
Relative difference: 0.0014641288433382138 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -11.90625 but got -11.9375.
Absolute difference: 0.03125 (up to 1e-05 allowed)
Relative difference: 0.0026246719160104987 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -1.087890625 but got -1.08984375.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.0017953321364452424 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -0.0015745162963867188 but got 0.0006766319274902344.
E Absolute difference: 0.002251148223876953 (up to 1e-05 allowed)
E Relative difference: 1.429739551786796 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -1.4326171875 but got -1.4345703125.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.0013633265167007499 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -0.2271728515625 but got -0.228515625.
E Absolute difference: 0.0013427734375 (up to 1e-05 allowed)
E Relative difference: 0.005910800644814616 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected 21.34375 but got 21.3125.
E Absolute difference: 0.03125 (up to 1e-05 allowed)
E Relative difference: 0.0014641288433382138 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -11.90625 but got -11.9375.
E Absolute difference: 0.03125 (up to 1e-05 allowed)
E Relative difference: 0.0026246719160104987 (up to 0.001 allowed)
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU
github-actions / Test Results
All 4 runs failed: test_output_match_opinfo__nn_functional_nll_loss_weight_cpu_float16 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU)
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py311-torch-nightly-windows-latest)/pytest.xml [took 3s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 0s]
artifacts/Test Results (py312-torch-nightly-windows-latest)/pytest.xml [took 1s]
Raw output
AssertionError: Scalars are not close!
Expected -1.087890625 but got -1.08984375.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.0017953321364452424 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -0.0015745162963867188 but got 0.0006766319274902344.
Absolute difference: 0.002251148223876953 (up to 1e-05 allowed)
Relative difference: 1.429739551786796 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -1.4326171875 but got -1.4345703125.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.0013633265167007499 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -0.2271728515625 but got -0.228515625.
Absolute difference: 0.0013427734375 (up to 1e-05 allowed)
Relative difference: 0.005910800644814616 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected 21.34375 but got 21.3125.
Absolute difference: 0.03125 (up to 1e-05 allowed)
Relative difference: 0.0014641288433382138 (up to 0.001 allowed)
AssertionError: Scalars are not close!
Expected -11.90625 but got -11.9375.
Absolute difference: 0.03125 (up to 1e-05 allowed)
Relative difference: 0.0026246719160104987 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -1.087890625 but got -1.08984375.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.0017953321364452424 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -0.0015745162963867188 but got 0.0006766319274902344.
E Absolute difference: 0.002251148223876953 (up to 1e-05 allowed)
E Relative difference: 1.429739551786796 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -1.4326171875 but got -1.4345703125.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.0013633265167007499 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -0.2271728515625 but got -0.228515625.
E Absolute difference: 0.0013427734375 (up to 1e-05 allowed)
E Relative difference: 0.005910800644814616 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected 21.34375 but got 21.3125.
E Absolute difference: 0.03125 (up to 1e-05 allowed)
E Relative difference: 0.0014641288433382138 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected -11.90625 but got -11.9375.
E Absolute difference: 0.03125 (up to 1e-05 allowed)
E Relative difference: 0.0026246719160104987 (up to 0.001 allowed)
Check warning on line 0 in tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU
github-actions / Test Results
All 4 runs failed: test_output_match_opinfo__nn_functional_cross_entropy_cpu_float16 (tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU)
artifacts/Test Results (py311-torch-nightly-macos-latest)/pytest.xml [took 2s]
artifacts/Test Results (py311-torch-nightly-windows-latest)/pytest.xml [took 4s]
artifacts/Test Results (py312-torch-nightly-macos-latest)/pytest.xml [took 2s]
artifacts/Test Results (py312-torch-nightly-windows-latest)/pytest.xml [took 5s]
Raw output
AssertionError: Scalars are not close!
Expected 1.2529296875 but got 1.2509765625.
Absolute difference: 0.001953125 (up to 1e-05 allowed)
Relative difference: 0.001558846453624318 (up to 0.001 allowed)
tests/function_libs/torch_lib/ops_test.py:279: in run_test_output_match
torch.testing.assert_close(
E AssertionError: Scalars are not close!
E
E Expected 1.2529296875 but got 1.2509765625.
E Absolute difference: 0.001953125 (up to 1e-05 allowed)
E Relative difference: 0.001558846453624318 (up to 0.001 allowed)
Loading