Skip to content

Commit

Permalink
Increanse TensorRT tolerance from default 1e-5 to 1e-3 after TRT 10.4 (
Browse files Browse the repository at this point in the history
…#22321)

### Description
Increanse TensorRT tolerance from default 1e-5 to 1e-3 after TRT 10.4

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
  • Loading branch information
jingyanwangms authored Oct 5, 2024
1 parent 004bd36 commit 5036e63
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion java/src/test/java/ai/onnxruntime/InferenceTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -770,8 +770,9 @@ private void runProvider(OrtProvider provider) throws OrtException {
if (provider == OrtProvider.CORE_ML) {
// CoreML gives slightly different answers on a 2020 13" M1 MBP
assertArrayEquals(expectedOutput, resultArray, 1e-2f);
} else if (provider == OrtProvider.CUDA) {
} else if (provider == OrtProvider.CUDA || provider == OrtProvider.TENSOR_RT) {
// CUDA gives slightly different answers on a H100 with CUDA 12.2
// Need larger tolerance since TRT 10.4
assertArrayEquals(expectedOutput, resultArray, 1e-3f);
} else {
assertArrayEquals(expectedOutput, resultArray, 1e-5f);
Expand Down

0 comments on commit 5036e63

Please sign in to comment.