Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update csv output format #540

Merged
merged 4 commits into from
Mar 22, 2024
Merged

Update csv output format #540

merged 4 commits into from
Mar 22, 2024

Conversation

tgerdesnv
Copy link
Collaborator

Old format:

Statistic,avg,min,max,p99,p95,p90,p75,p50,p25
time_to_first_token(ns),4238735,3367978,6702240,6371118,5344958,5006259,4841394,4146648,3484484
inter_token_latency(ns),27202264,3849435,138324924,28283424,27737593,27469154,27067290,26979956,26926962
request_latency(ns),3363927003,3367978,14238834483,14091273510,13740917508,13692672723,3752510140,4846258,3612270
num_output_token,126,0,584,562,509,505,135,0,0
output_token_throughput(per sec),36.37
request_throughput(per sec),0.29

New format:

Metric,avg,min,max,p99,p95,p90,p75,p50,p25
Time to First Token (ns),4238735,3367978,6702240,6371118,5344958,5006259,4841394,4146648,3484484
Inter Token Latency (ns),27202264,3849435,138324924,28283424,27737593,27469154,27067290,26979956,26926962
Request Latency (ns),3363927003,3367978,14238834483,14091273510,13740917508,13692672723,3752510140,4846258,3612270
Num Output Token,126,0,584,562,509,505,135,0,0

Metric,Value
Output Token Throughput (per sec),36.37
Request Throughput (per sec),0.29

Also added unit testing to protect the output format

@dyastremsky
Copy link
Contributor

Fantastic work, Tim! This looks great. There is an ongoing conversation, but once that is resolved, happy to approve from my end.

Copy link
Contributor

@nv-hwoo nv-hwoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one small nit. Otherwise, looks good overall!

@tgerdesnv tgerdesnv requested a review from nv-hwoo March 22, 2024 15:43
@tgerdesnv tgerdesnv merged commit 2cb6062 into main Mar 22, 2024
3 checks passed
@tgerdesnv tgerdesnv deleted the tgerdes-fix-csv-output branch March 22, 2024 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants