pyserini Development Install issue (on Windows)

[AreelKhan]

Hello!
My name is Areel, I am trying to complete the developer installation of pyserini but have run into some problems.
I am following the steps linked here.
Everything works fine until I get to the point where I have to clone the GitHub repo. It is not much of an issue. The SSH clone command does not work so I use the HTTPS clone command instead.
The command in the guide:

$ git clone [email protected]:castorini/pyserini.git --recurse-submodules

The command I used instead:

$ git clone https://github.com/castorini/pyserini.git --recurse-submodules

The pyserini repo has been successfully cloned. Moving on to the next command.
The guide tells me to run the following:

$ cd pyserini
$ cd tools/eval && tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4 && make && cd ../../..
$ cd tools/eval/ndeval && make && cd ../../..

However this command fails to run successfully. The beginning portions of the command run just fine. As in, the following commands execute smoothly.

$ cd pyserini
$ cd tools/eval && tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4

However, running the make command causes trouble. This is the error I get:

$ make
##########################################
OUTPUT:
gcc -g -I.  -Wall -DVERSIONID=\"9.0.4\"  -o trec_eval trec_eval.c formats.c meas_init.c meas_acc.c meas_avg.c meas_print_single.c meas_print_final.c get_qrels.c get_trec_results.c get_prefs.c get_qrels_prefs.c get_qrels_jg.c form_res_rels.c form_res_rels_jg.c form_prefs_counts.c utility_pool.c get_zscores.c convert_zscores.c measures.c  m_map.c m_P.c m_num_q.c m_num_ret.c m_num_rel.c m_num_rel_ret.c m_gm_map.c m_Rprec.c m_recip_rank.c m_bpref.c m_iprec_at_recall.c m_recall.c m_Rprec_mult.c m_utility.c m_11pt_avg.c m_ndcg.c m_ndcg_cut.c m_Rndcg.c m_ndcg_rel.c m_binG.c m_G.c m_rel_P.c m_success.c m_infap.c m_map_cut.c m_gm_bpref.c m_runid.c m_relstring.c m_set_P.c m_set_recall.c m_set_rel_P.c m_set_map.c m_set_F.c m_num_nonrel_judged_ret.c m_prefs_num_prefs_poss.c m_prefs_num_prefs_ful.c m_prefs_num_prefs_ful_ret.c m_prefs_simp.c m_prefs_pair.c m_prefs_avgjg.c m_prefs_avgjg_Rnonrel.c m_prefs_simp_ret.c m_prefs_pair_ret.c m_prefs_avgjg_ret.c m_prefs_avgjg_Rnonrel_ret.c m_prefs_simp_imp.c m_prefs_pair_imp.c m_prefs_avgjg_imp.c m_map_avgjg.c m_Rprec_mult_avgjg.c m_P_avgjg.c m_yaap.c -lm
process_begin: CreateProcess(NULL, gcc -g -I. -Wall -DVERSIONID=\"9.0.4\" -o trec_eval trec_eval.c formats.c meas_init.c meas_acc.c meas_avg.c meas_print_single.c meas_print_final.c get_qrels.c get_trec_results.c get_prefs.c get_qrels_prefs.c get_qrels_jg.c form_res_rels.c form_res_rels_jg.c form_prefs_counts.c utility_pool.c get_zscores.c convert_zscores.c measures.c m_map.c m_P.c m_num_q.c m_num_ret.c m_num_rel.c m_num_rel_ret.c m_gm_map.c m_Rprec.c m_recip_rank.c m_bpref.c m_iprec_at_recall.c m_recall.c m_Rprec_mult.c m_utility.c m_11pt_avg.c m_ndcg.c m_ndcg_cut.c m_Rndcg.c m_ndcg_rel.c m_binG.c m_G.c m_rel_P.c m_success.c m_infap.c m_map_cut.c m_gm_bpref.c m_runid.c m_relstring.c m_set_P.c m_set_recall.c m_set_rel_P.c m_set_map.c m_set_F.c m_num_nonrel_judged_ret.c m_prefs_num_prefs_poss.c m_prefs_num_prefs_ful.c m_prefs_num_prefs_ful_ret.c m_prefs_simp.c m_prefs_pair.c m_prefs_avgjg.c m_prefs_avgjg_Rnonrel.c m_prefs_simp_ret.c m_prefs_pair_ret.c m_prefs_avgjg_ret.c m_prefs_avgjg_Rnonrel_ret.c m_prefs_simp_imp.c m_prefs_pair_imp.c m_prefs_avgjg_imp.c m_map_avgjg.c m_Rprec_mult_avgjg.c m_P_avgjg.c m_yaap.c -lm, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [Makefile:54: trec_eval] Error 2
##########################################

Quite a long error :(
It is from my understanding that in this step we are trying to compile trec_eval. I found the README that outlines the steps for compiling trec_eval on a windows computer. However, following those steps (downloading cygwin and compiling trec_eval from a cygwin terminal) did not work either. In fact, I got the exact same error as earlier.
I am at a bit of a dead end. I would really appreciate any help!
Thanks in advance, and please feel free to contact me for any further clarifications.

[lintool]

hi @AreelKhan - thanks for your detailed issue...

Unfortunately, I don't have a Windows machine, so I can't help debug, but this seems more like an issue to file with trec_eval: https://github.com/usnistgov/trec_eval

@MXueguang do you have any tips on getting a dev install on a Windows machine?

[AreelKhan]

Hey Jimmy,

Thanks for your reply. If I am able to find a solution, I'll make sure to post a solution here.

[AreelKhan]

UPDATE:

I was able to fix the issue!
The problem had to do with correctly setting paths for environment variables. I'll briefly explain what I did in case someone else is/will face the same problem.
During the developer installation for pyserini and anserini you must use the make command to compile trec_eval. However, there are (possibly) extra steps if installing on a Windows machine, linked here. You must download cygwin and compile from a cygwin terminal.
This did not work for me, I was still getting the following error:

make (e=2): The system cannot find the file specified.
make: *** [Makefile:54: trec_eval] Error 2

Next, I added the cygwin bin directory to the PATH variable in my system environment variables. For my machine, the cygwin bin directory was as follows: C:\cygwin64\bin.
This should have worked, except I had make already installed on my machine using chocolatey. For that reason, when calling make, my machine would use the chocolatey installation instead of the cygwin one. To fix this I moved the C:\cygwin64\bin environment variable I added to PATH earlier to the top. Now, when my machine sees make it correctly uses the cygwin installation, instead of chocolatey.