Skip to content

Latest commit

 

History

History
61 lines (50 loc) · 4.56 KB

BENCHMARK_INFER_en.md

File metadata and controls

61 lines (50 loc) · 4.56 KB
Raw

Inference Benchmark

一、Prepare the Environment

  • 1、Test Environment:
    • CUDA 10.1
    • CUDNN 7.6
    • TensorRT-6.0.1
    • PaddlePaddle v2.0.1
    • The GPUS are Tesla V100 and GTX 1080 Ti and Jetson AGX Xavier
  • 2、Test Method:
    • In order to compare the inference speed of different models, the input shape is 3x640x640, use demo/000000014439_640x640.jpg.
    • Batch_size=1
    • Delete the warmup time of the first 100 rounds and test the average time of 100 rounds in ms/image, including network calculation time and data copy time to CPU.
    • Using Fluid C++ prediction engine: including Fluid C++ prediction, Fluid TensorRT prediction, the following test Float32 (FP32) and Float16 (FP16) inference speed.

Attention: For TensorRT, please refer to the TENSOR tutorial for the difference between fixed and dynamic dimensions. Due to the imperfect support for the two-stage model under fixed size, dynamic size test was adopted for the Faster RCNN model. Fixed size and dynamic size do not support exactly the same OP for fusion, so the performance of the same model tested at fixed size and dynamic size may differ slightly.

二、Inferring Speed

1、Linux System

（1）Tesla V100

Model backbone Fixed size or not The net size paddle_inference trt_fp32 trt_fp16
Faster RCNN FPN ResNet50 no 640x640 27.99 26.15 21.92
Faster RCNN FPN ResNet50 no 800x1312 32.49 25.54 21.70
YOLOv3 Mobilenet_v1 yes 608x608 9.74 8.61 6.28
YOLOv3 Darknet53 yes 608x608 17.84 15.43 9.86
PPYOLO ResNet50 yes 608x608 20.77 18.40 13.53
SSD Mobilenet_v1 yes 300x300 5.17 4.43 4.29
TTFNet Darknet53 yes 512x512 10.14 8.71 5.55
FCOS ResNet50 yes 640x640 35.47 35.02 34.24

（2）Jetson AGX Xavier

Model backbone Fixed size or not The net size paddle_inference trt_fp32 trt_fp16
Faster RCNN FPN ResNet50 no 640x640 169.45 158.92 119.25
Faster RCNN FPN ResNet50 no 800x1312 228.07 156.39 117.03
YOLOv3 Mobilenet_v1 yes 608x608 48.76 43.83 18.41
YOLOv3 Darknet53 yes 608x608 121.61 110.30 42.38
PPYOLO ResNet50 yes 608x608 111.80 99.40 48.05
SSD Mobilenet_v1 yes 300x300 10.52 8.84 8.77
TTFNet Darknet53 yes 512x512 73.77 64.03 31.46
FCOS ResNet50 yes 640x640 217.11 214.38 205.78

2、Windows System

（1）GTX 1080Ti

Model backbone Fixed size or not The net size paddle_inference trt_fp32 trt_fp16
Faster RCNN FPN ResNet50 no 640x640 50.74 57.17 62.08
Faster RCNN FPN ResNet50 no 800x1312 50.31 57.61 62.05
YOLOv3 Mobilenet_v1 yes 608x608 14.51 11.23 11.13
YOLOv3 Darknet53 yes 608x608 30.26 23.92 24.02
PPYOLO ResNet50 yes 608x608 38.06 31.40 31.94
SSD Mobilenet_v1 yes 300x300 16.47 13.87 13.76
TTFNet Darknet53 yes 512x512 21.83 17.14 17.09
FCOS ResNet50 yes 640x640 71.88 69.93 69.52