the speed after using tensorRT is not faster than pytorch

[yilawu]

I have run main.py
and get the result like this

tensorRT inference time:
avg GPU time: 0.0072007338205973305
avg GPU compute time: 0.0065811554590861
avg pre time: 0.0005954186121622722
avg post time: 2.4159749348958332e-05
pytorch inference time:
avg GPU time: 0.007817573017544217
avg GPU compute time: 0.006660024325052897
avg pre time: 0.0005226506127251519
avg post time: 0.0006348980797661676
trt_result =============================
[array([[[ 415.59, 240.87, 506.79, 545.85, 0.83679, 0],
[ 49.603, 242.71, 167.21, 552.12, 0.81033, 0],
[ 18.173, 137.37, 515.33, 463.03, 0.71066, 5],
[ 149.82, 246.16, 230.95, 529.5, 0.69774, 0],
[ 16.668, 346.08, 62.551, 542.5, 0.45844, 0]]], dtype=float32)]
pytorch_result ==============================
[array([[ 415.47, 231.19, 495.51, 524.26, 0.8785, 0],
[ 147.73, 239.38, 221.98, 510.14, 0.86237, 0],
[ 46.126, 236.25, 158, 531.58, 0.84014, 0],
[ 44.223, 136.36, 494.82, 450.86, 0.63891, 5],
[ 16.669, 331.73, 57.432, 517.17, 0.52587, 0]], dtype=float32)]

tensorRT avg GPU time: 0.0072007338205973305
pytorch avg GPU time: 0.007817573017544217
which looks like it speed up just 7.7％.
while readme claims that it can speed up 4x

[TrojanXu]

Currently it depends on your GPU. On V100, it delivers a 4x speedup, while on other gpus, especially on pascal GPUs, it doesn't accelerate that much.

[yilawu]

Currently it depends on your GPU. On V100, it delivers a 4x speedup, while on other gpus, especially on pascal GPUs, it doesn't accelerate that much.

my gpu is rtx2070s

[makaveli10]

@wurongyuan can you share the working tensorrt code! the current repo seems to throw some onnx parsing error. Its kind of urgent. Thanks in advance.

[yilawu]

@wurongyuan can you share the working tensorrt code! the current repo seems to throw some onnx parsing error. Its kind of urgent. Thanks in advance.

my code is just from this repo