From 663b04e5d6212f683b1bccf1f210cf3d0193240c Mon Sep 17 00:00:00 2001 From: Yilin Zhang Date: Thu, 24 Aug 2023 19:13:30 -0400 Subject: [PATCH 1/3] Add an entry in the Passage Ranking Experiment Reproduction Log for ugrad onboarding --- docs/experiments-msmarco-passage.md | 3 ++- docs/start-here.md | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/experiments-msmarco-passage.md b/docs/experiments-msmarco-passage.md index 09ac119c3a..7a3f0aecef 100644 --- a/docs/experiments-msmarco-passage.md +++ b/docs/experiments-msmarco-passage.md @@ -421,4 +421,5 @@ The BM25 run with default parameters `k1=0.9`, `b=0.4` roughly corresponds to th + Results reproduced by [@Richard5678](https://github.com/Richard5678) on 2023-06-11 (commit [`2d484d3`](https://github.com/castorini/anserini/commit/2d484d330b6218852552901fa4dc62c441e7ff17)) + Results reproduced by [@pratyushpal](https://github.com/pratyushpal) on 2023-07-14 (commit [`17d5fc7`](https://github.com/castorini/anserini/commit/17d5fc7f338b511c4dc49de88e9b2ab7ea27f8aa)) + Results reproduced by [@sahel-sh](https://github.com/sahel-sh) on 2023-07-22 (commit [`4b8f051`](https://github.com/castorini/anserini/commit/4b8f051c25992a5d87ecf8d30d45a93aff17abc4)) -+ Results reproduced by [@Mofetoluwa](https://github.com/Mofetoluwa) on 2023-08-03 (commit [`7314128`](https://github.com/castorini/anserini/commit/73141282b62979e189ac3c87d9a902064f34a1c5)) \ No newline at end of file ++ Results reproduced by [@Mofetoluwa](https://github.com/Mofetoluwa) on 2023-08-03 (commit [`7314128`](https://github.com/castorini/anserini/commit/73141282b62979e189ac3c87d9a902064f34a1c5)) ++ Results reproduced by [@yilinjz](https://github.com/yilinjz) on 2023-08-24 (commit [`d4cb6fd`](https://github.com/castorini/anserini/commit/d4cb6fd1c0b5ed0a7eac4747af919823acc939fa)) diff --git a/docs/start-here.md b/docs/start-here.md index 02679fd859..f1df55b4dc 100644 --- a/docs/start-here.md +++ b/docs/start-here.md @@ -227,7 +227,7 @@ the fourth colum provides the relevance judgment itself. In this case, 0 means "not relevant" and 1 means "relevant". So, this entry says that the document with id 7187158 is relevant to the query with id 1048585. -Well, how do we get the actual contents of document 1048585? +Well, how do we get the actual contents of document 7187158? The simplest way is to grep through the collection itself: ```bash From 00385b098f9c9d846000c88e2876d68f87446193 Mon Sep 17 00:00:00 2001 From: yilinjz Date: Sat, 2 Sep 2023 16:33:06 -0400 Subject: [PATCH 2/3] reproduced experiment in student environment --- docs/regressions-msmarco-passage-cos-dpr-distil.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/regressions-msmarco-passage-cos-dpr-distil.md b/docs/regressions-msmarco-passage-cos-dpr-distil.md index 17bc104b76..19453fce6d 100644 --- a/docs/regressions-msmarco-passage-cos-dpr-distil.md +++ b/docs/regressions-msmarco-passage-cos-dpr-distil.md @@ -109,3 +109,4 @@ Nevertheless, scores are generally stable to the third digit after the decimal p ## Reproduction Log[*](reproducibility.md) To add to this reproduction log, modify [this template](../src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template) and run `bin/build.sh` to rebuild the documentation. ++ Results reproduced by [@yilinjz](https://github.com/yilinjz) on 2023-09-01 (commit [`4ae518b`](https://github.com/castorini/anserini/commit/4ae518bb284ebcba0b273a473bc8774735cb7d19)) \ No newline at end of file From b8f65e57693735ddfedad9e0439d1d5202e7d3fb Mon Sep 17 00:00:00 2001 From: yilinjz Date: Sat, 2 Sep 2023 17:49:24 -0400 Subject: [PATCH 3/3] moved repro log to template --- docs/regressions-msmarco-passage-cos-dpr-distil.md | 1 + .../docgen/templates/msmarco-passage-cos-dpr-distil.template | 2 ++ 2 files changed, 3 insertions(+) diff --git a/docs/regressions-msmarco-passage-cos-dpr-distil.md b/docs/regressions-msmarco-passage-cos-dpr-distil.md index 19453fce6d..a377ac302b 100644 --- a/docs/regressions-msmarco-passage-cos-dpr-distil.md +++ b/docs/regressions-msmarco-passage-cos-dpr-distil.md @@ -109,4 +109,5 @@ Nevertheless, scores are generally stable to the third digit after the decimal p ## Reproduction Log[*](reproducibility.md) To add to this reproduction log, modify [this template](../src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template) and run `bin/build.sh` to rebuild the documentation. + + Results reproduced by [@yilinjz](https://github.com/yilinjz) on 2023-09-01 (commit [`4ae518b`](https://github.com/castorini/anserini/commit/4ae518bb284ebcba0b273a473bc8774735cb7d19)) \ No newline at end of file diff --git a/src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template b/src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template index 7c022ae3f0..da6b0aa424 100644 --- a/src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template +++ b/src/main/resources/docgen/templates/msmarco-passage-cos-dpr-distil.template @@ -87,3 +87,5 @@ Nevertheless, scores are generally stable to the third digit after the decimal p ## Reproduction Log[*](reproducibility.md) To add to this reproduction log, modify [this template](${template}) and run `bin/build.sh` to rebuild the documentation. + ++ Results reproduced by [@yilinjz](https://github.com/yilinjz) on 2023-09-01 (commit [`4ae518b`](https://github.com/castorini/anserini/commit/4ae518bb284ebcba0b273a473bc8774735cb7d19)) \ No newline at end of file