Rebuild regression docs for cosDPR-distil, tweak README sections (#2234)

castorini · Nov 2, 2023 · 2f3e7d5 · 2f3e7d5
1 parent 2b8dd35
commit 2f3e7d5
Show file tree

Hide file tree

Showing 4 changed files with 29 additions and 12 deletions.
diff --git a/README.md b/README.md
@@ -271,7 +271,10 @@ See individual pages for details!
 The experiments described below are not associated with rigorous end-to-end regression testing and thus provide a lower standard of reproducibility.
 For the most part, manual copying and pasting of commands into a shell is required to reproduce our results.
 
-### MS MARCO (V1)
+<details>
+<summary>MS MARCO V1</summary>
+
+### MS MARCO V1
 
 + Reproducing [BM25 baselines for MS MARCO Passage Ranking](docs/experiments-msmarco-passage.md)
 + Reproducing [BM25 baselines for MS MARCO Document Ranking](docs/experiments-msmarco-doc.md)
@@ -280,16 +283,28 @@ For the most part, manual copying and pasting of commands into a shell is requir
 + Reproducing [docTTTTTquery results](docs/experiments-docTTTTTquery.md) (MS MARCO Passage and Document Ranking)
 + Notes about reproduction issues with [MS MARCO Document Ranking w/ docTTTTTquery](docs/experiments-msmarco-doc-doc2query-details.md)
 
-### MS MARCO (V2)
+</details>
+<details>
+<summary>MS MARCO V2</summary>
+
+### MS MARCO V2
 
 + Reproducing [BM25 baselines on the MS MARCO V2 Collections](docs/experiments-msmarco-v2.md)
 
+</details>
+<details>
+<summary>TREC-COVID and CORD-19</summary>
+
 ### TREC-COVID and CORD-19
 
 + [Indexing AI2's COVID-19 Open Research Dataset](docs/experiments-cord19.md)
 + [Baselines for the TREC-COVID Challenge](docs/experiments-covid.md)
 + [Baselines for the TREC-COVID Challenge using doc2query](docs/experiments-covid-doc2query.md)
 
+</details>
+<details>
+<summary>Other Experiments and Features</summary>
+
 ### Other Experiments and Features
 
 + [Working with the 20 Newsgroups Dataset](docs/experiments-20newsgroups.md)
@@ -302,6 +317,8 @@ For the most part, manual copying and pasting of commands into a shell is requir
 + Runbook for [ECIR 2019 paper on cross-collection relevance feedback](docs/runbook-ecir2019-ccrf.md)
 + Support for [approximate nearest-neighbor search](docs/approximate-nearestneighbor.md) on dense vectors with inverted indexes
 
+</details>
+
 ## 🙋 How Can I Contribute?
 
 If you've found Anserini to be helpful, we have a simple request for you to contribute back.

diff --git a/docs/regressions/regressions-dl19-passage-cos-dpr-distil.md b/docs/regressions/regressions-dl19-passage-cos-dpr-distil.md
@@ -99,13 +99,13 @@ With the above commands, you should be able to reproduce the following results:
 
 | **AP@1000**                                                                                                  | **cosDPR-distil**|
 |:-------------------------------------------------------------------------------------------------------------|-----------|
-| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.460     |
+| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.458     |
 | **nDCG@10**                                                                                                  | **cosDPR-distil**|
-| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.722     |
+| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.717     |
 | **R@100**                                                                                                    | **cosDPR-distil**|
-| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.609     |
+| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.605     |
 | **R@1000**                                                                                                   | **cosDPR-distil**|
-| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.807     |
+| [DL19 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.805     |
 
 Note that due to the non-deterministic nature of HNSW indexing, results may differ slightly between each experimental run.
 Nevertheless, scores are generally stable to the third digit after the decimal point.

diff --git a/docs/regressions/regressions-dl20-passage-cos-dpr-distil.md b/docs/regressions/regressions-dl20-passage-cos-dpr-distil.md
@@ -103,9 +103,9 @@ With the above commands, you should be able to reproduce the following results:
 | **nDCG@10**                                                                                                  | **cosDPR-distil**|
 | [DL20 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.701     |
 | **R@100**                                                                                                    | **cosDPR-distil**|
-| [DL20 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.714     |
+| [DL20 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.712     |
 | **R@1000**                                                                                                   | **cosDPR-distil**|
-| [DL20 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.844     |
+| [DL20 (Passage)](https://trec.nist.gov/data/deep2020.html)                                                   | 0.843     |
 
 Note that due to the non-deterministic nature of HNSW indexing, results may differ slightly between each experimental run.
 Nevertheless, scores are generally stable to the third digit after the decimal point.

diff --git a/docs/regressions/regressions-msmarco-passage-cos-dpr-distil.md b/docs/regressions/regressions-msmarco-passage-cos-dpr-distil.md
@@ -95,13 +95,13 @@ With the above commands, you should be able to reproduce the following results:
 
 | **AP@1000**                                                                                                  | **cosDPR-distil**|
 |:-------------------------------------------------------------------------------------------------------------|-----------|
-| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.392     |
+| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.393     |
 | **RR@10**                                                                                                    | **cosDPR-distil**|
-| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.387     |
+| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.388     |
 | **R@100**                                                                                                    | **cosDPR-distil**|
-| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.900     |
+| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.903     |
 | **R@1000**                                                                                                   | **cosDPR-distil**|
-| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.970     |
+| [MS MARCO Passage: Dev](https://github.com/microsoft/MSMARCO-Passage-Ranking)                                | 0.974     |
 
 Note that due to the non-deterministic nature of HNSW indexing, results may differ slightly between each experimental run.
 Nevertheless, scores are generally stable to the third digit after the decimal point.