Skip to content
Benchmark

Add DRY sampling parameters to gpt_params and server_context #180

Sign in to view logs

Add DRY sampling parameters to gpt_params and server_context

Add DRY sampling parameters to gpt_params and server_context #180

Run time

Learn about OS pricing on GitHub Actions
Job Run time
bench-server-baseline (phi-2, q4_0)
1s
bench-server-baseline (phi-2, q8_0)
1s
bench-server-baseline (phi-2, f16)
1s
3s