Skip to content

Commit

Permalink
new leaderboard announcement (#277)
Browse files Browse the repository at this point in the history
  • Loading branch information
ingberam authored Feb 5, 2024
1 parent 25f528d commit 9537af2
Show file tree
Hide file tree
Showing 2 changed files with 66 additions and 11 deletions.
73 changes: 62 additions & 11 deletions neurips23.html
Original file line number Diff line number Diff line change
Expand Up @@ -71,17 +71,68 @@ <h1>
<img src="assets/zilliz-logo.png" class="zilliz-logo" alt="Zilliz" />
</p>

<div class="table-container">
<p class="hero-text">
<b>New: the latest ongoing leaderboard has been released (Feb 1st, 2024). <br> Top entries:</b>
</p>

<table>
<th colspan="3" class="group-border">Filter track</th>
<th colspan="3" class="group-border">OOD track</th>
<th colspan="3">Sparse track</th>
</tr>
<tr>
<th>Rank</th>
<th>Algorithm</th>
<th class="group-border">QPS@90% recall</th>
<th>Rank</th>
<th>Algorithm</th>
<th class="group-border">QPS@90% recall</th>
<th>Rank</th>
<th>Algorithm</th>
<th>QPS@90% recall</th>
</tr>
<tr>
<td>1</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/filter/pinecone ">Pinecone-filter</a></td>
<td class="group-border">85,491</td>
<td>1</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/ood/pinecone-ood">Pinecone-ood</a></td>
<td class="group-border">31,643</td>
<td>1</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/sparse/zilliz ">Zilliz</a></td>
<td>9,933</td>
</tr>
<tr>
<td>2</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/filter/zilliz ">Zilliz</a></td>
<td class="group-border">84,596</td>
<td>2</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/ood/zilliz ">Zilliz</a></td>
<td class="group-border">27,585</td>
<td>2</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/sparse/pyanns">PyANNS</a></td>
<td>8,732</td>
</tr>
<tr>
<td>3</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/filter/parlayivf">ParlayANN IVF<sup>2</sup></a></td>
<td class="group-border">37,902</td>
<td>3</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/ood/mysteryann">RoarANN</a></td>
<td class="group-border">22,555</td>
<td>3</td>
<td><a href="https://github.com/harsha-simhadri/big-ann-benchmarks/tree/main/neurips23/sparse/pinecone_smips">Pinecone_smips</a></td>
<td>8,078</td>
</tr>
</table>
<p class="hero-text">
<b> Update:</b> we will maintain a new <i>ongoing leaderboard</i> to include
existing and new submissions to the big-ann repository,
that will be published periodically. The first updates are planned for 2/1/2024 and 3/1/2024.
Same rules apply as in the original competition, except:
(1) Open source is encouraged but not enforced. Closed-source entries will be marked as such.
In any case, a short description of the algorithm is required.
(2) We will only evaluate the public query sets in the new leaderboard
(on the same VM type: Azure Standard D8lds v5 with 8 vcpus, 16 GiB memory)
(3) To participate, simply send a PR with the new algorithm (no need for a CMT entry).
See here for the <a href="https://github.com/harsha-simhadri/big-ann-benchmarks/blob/main/neurips23/ongoing_leaderboard/leaderboard.md">full ongoing leaderboard and rules</a>. Next update: March 1st, 2024.
<br>
Note: entries by pinecone and zilliz are not open source.
</p>
</div>


<p class="hero-text">
This challenge is to encourage the development of indexing data
Expand All @@ -104,10 +155,10 @@ <h1>
NeurIPS'23 Session
</a>
<a href="https://github.com/harsha-simhadri/big-ann-benchmarks/blob/main/neurips23/leaderboard.md" class="button button-white">
Leaderboard
NeurIPS'23 Leaderboard
</a>
<a href="#winners" class="button button-alpha">
Track Winners
Track Winners and presentations
</a>
</div>
</div>
Expand Down
4 changes: 4 additions & 0 deletions style.css
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,10 @@ a:hover {
overflow-x: auto;
}

.group-border {
border-right: 3px solid #738fab;
}

table {
border-collapse: collapse;
font-size: 1rem;
Expand Down

0 comments on commit 9537af2

Please sign in to comment.