Skip to content

Commit

Permalink
Update leaderboard.md
Browse files Browse the repository at this point in the history
  • Loading branch information
harsha-simhadri authored Dec 8, 2023
1 parent 8785b9d commit 7d110da
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions neurips23/leaderboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ The recall of the baselines at this QPS threshold is listed [above](#measuring_y
For tasks "Filter", "Out-of-Distribution" and "Sparse" tracks, algorithms were ranked on the QPS they achieve on the track dataset, as long as the recall@10 is at least 90%.
These results files for [Azure D8lds_v5](Azure_D8lds_v5_table.md) and [AWS EC2 c6i.2xlarge](ec2_c6i.2xlarge_table.md) list the maximum QPS measured for each algorihtm with at least 90% recall@10.

For the Streaming track, algorithms will be ranked on recall@10, as long as each algorithm completes the runbook within the alloted 1 hour.
The [result file](streaming/res_final_runbook_AzureD8lds_v5.csv) lists measurements on Azure D8lds_v5.
For the Streaming track, algorithms were ranked on recall@10, as long as each algorithm completes the runbook within the alloted 1 hour. The leading entry had a recall of 0.9849.
The [result file](streaming/res_final_runbook_AzureD8lds_v5.csv) lists measurements for all streaming algorithms on Azure D8lds_v5.

QPS vs recall@10 plots for tracks based on public queries on Azure D8lds_v5:
**Filter track**
Expand All @@ -18,3 +18,5 @@ QPS vs recall@10 plots for tracks based on public queries on Azure D8lds_v5:

**Sparse track**
![sparse-full](sparse/plot_public_queries_AzureD8lds_v5.png)

More plots to follow.

0 comments on commit 7d110da

Please sign in to comment.