Add inference percentile details to driver perf

[richagadgil]

Updated perf output:

Batch size: 1
Rate: 7247.13 inferences/sec
Total time: 0.137986ms
Min time: 0.108128ms
Max time: 0.298645ms
Mean time: 0.140612ms
Median time: 0.137811ms
Percentile(99%) time: 0.240435ms
Total instructions time: 0.191937ms
Overhead time: 0.00502366ms, -0.0539513ms
Overhead: 4%, 39%

[codecov]

Codecov Report

Attention: Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.06%. Comparing base (fc26f01) to head (ac26370).
Report is 10 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/program.cpp	95.83%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3504      +/-   ##
===========================================
+ Coverage    92.02%   92.06%   +0.03%     
===========================================
  Files          509      509              
  Lines        21014    21038      +24     
===========================================
+ Hits         19339    19368      +29     
+ Misses        1675     1670       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[migraphx-bot]

Test	Batch	Rate new ac2637	Rate old 0d07c2	Diff	Compare
torchvision-resnet50	64	3,259.68	3,260.52	-0.03%	✅
torchvision-resnet50_fp16	64	6,992.95	6,982.39	0.15%	✅
torchvision-densenet121	32	2,436.61	2,430.89	0.24%	✅
torchvision-densenet121_fp16	32	4,085.49	4,067.06	0.45%	✅
torchvision-inceptionv3	32	1,638.97	1,636.77	0.13%	✅
torchvision-inceptionv3_fp16	32	2,759.83	2,755.18	0.17%	✅
cadene-inceptionv4	16	775.92	775.67	0.03%	✅
cadene-resnext64x4	16	808.46	808.09	0.05%	✅
slim-mobilenet	64	7,536.18	7,531.83	0.06%	✅
slim-nasnetalarge	64	211.53	211.39	0.07%	✅
slim-resnet50v2	64	3,500.35	3,495.72	0.13%	✅
bert-mrpc-onnx	8	1,148.52	1,140.79	0.68%	✅
bert-mrpc-tf	1	467.30	447.09	4.52%	🔆
pytorch-examples-wlang-gru	1	421.13	357.24	17.89%	🔆
pytorch-examples-wlang-lstm	1	376.59	320.71	17.42%	🔆
torchvision-resnet50_1	1	787.02	776.99	1.29%	✅
cadene-dpn92_1	1	399.92	430.45	-7.09%	🔴
cadene-resnext101_1	1	381.66	378.42	0.86%	✅
onnx-taau-downsample	1	343.22	341.70	0.44%	✅
dlrm-criteoterabyte	1	33.33	32.05	3.97%	🔆
dlrm-criteoterabyte_fp16	1	52.75	50.61	4.21%	🔆
agentmodel	1	8,457.16	6,696.76	26.29%	🔆
unet_fp16	2	58.82	58.59	0.39%	✅
resnet50v1_fp16	1	946.14	840.65	12.55%	🔆
resnet50v1_int8	1	986.70	931.26	5.95%	🔆
bert_base_cased_fp16	64	1,171.94	1,170.54	0.12%	✅
bert_large_uncased_fp16	32	363.95	363.52	0.12%	✅
bert_large_fp16	1	199.09	196.02	1.57%	✅
distilgpt2_fp16	16	2,200.99	2,199.51	0.07%	✅
yolov5s	1	552.23	536.04	3.02%	🔆
tinyllama	1	43.44	43.33	0.27%	✅
vicuna-fastchat	1	173.29	171.81	0.86%	✅
whisper-tiny-encoder	1	418.56	416.70	0.45%	✅
whisper-tiny-decoder	1	434.27	418.85	3.68%	🔆

This build is not recommended to merge 🔴

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance