About training with TREC 2019

[mhd0528]

Hi,
I'm trying to reproduce your result on TREC 2019 task and I have noticed that TREC actually includes several tasks while it's based on MS MARCO dataset. So may I ask which part of TREC 2019 did you use in your experiments? Thanks!

[jacklin64]

Hi, @mhd0528, thanks for your question. To clarify, here TREC 2019 means TREC 2019 DL (deep learning track: https://microsoft.github.io/msmarco/TREC-Deep-Learning-2019). In addition, we did not specifically train TCT for TREC 2019 since the query set is relative small for training. The most common approach is to train models on MS MARCO passage retrieval dataset and directly conduct retrieval on TREC DL queries.

[mhd0528]

Hi @jacklin64, thanks for the reply. Then may I ask if you can point me to the query file(s) you used for TREC DL? I found there are 3 test files on TREC-2019's page for passage retrieval task. Thanks again.

[jacklin64]

This is the one we use: (https://msmarco.blob.core.windows.net/msmarcoranking/msmarco-test2019-queries.tsv.gz and the according qrels https://trec.nist.gov/data/deep/2019qrels-pass.txt)

[mhd0528]

Ok, thanks so much for helping!