From 771930ea4608edcf4f47c819b28529810a6072e3 Mon Sep 17 00:00:00 2001 From: harsha vardhan simhadri Date: Mon, 16 Oct 2023 00:32:35 +0000 Subject: [PATCH] make download_gt work --- neurips23/README.md | 6 +++--- neurips23/streaming/final_runbook.yaml | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/neurips23/README.md b/neurips23/README.md index 469f3cd5..e435ad34 100644 --- a/neurips23/README.md +++ b/neurips23/README.md @@ -119,9 +119,9 @@ python run.py --neurips23track streaming --algorithm diskann --dataset msturing- For streaming track, runbook specifies the order of operations to be executed by the algorithms. To download the ground truth for every search operation: (needs azcopy tool in your binary path): ``` -python benchmark/streaming/download_gt.py --runbook_file neurips23/streaming/simple_runbook.yaml --dataset msspacev-10M -python benchmark/streaming/download_gt.py --runbook_file neurips23/streaming/delete_runbook.yaml --dataset msturing-10M-clustered -python benchmark/streaming/download_gt.py --runbook_file neurips23/streaming/final_runbook.yaml --dataset msturing-30M-clustered +python -m benchmark.streaming.download_gt --runbook_file neurips23/streaming/simple_runbook.yaml --dataset msspacev-10M +python -m benchmark.streaming.download_gt --runbook_file neurips23/streaming/delete_runbook.yaml --dataset msturing-10M-clustered +python -m benchmark.streaming.download_gt --runbook_file neurips23/streaming/final_runbook.yaml --dataset msturing-30M-clustered ``` Alternately, to compute ground truth for an arbitrary runbook, [clone and build DiskANN repo](https://github.com/Microsoft/DiskANN) and use the command line tool to compute ground truth at various search checkpoints. The `--gt_cmdline_tool` points to the directory with DiskANN commandline tools. ``` diff --git a/neurips23/streaming/final_runbook.yaml b/neurips23/streaming/final_runbook.yaml index c54a9b26..1d098bb7 100644 --- a/neurips23/streaming/final_runbook.yaml +++ b/neurips23/streaming/final_runbook.yaml @@ -3840,3 +3840,4 @@ msturing-30M-clustered: end: 29991665 1280: operation: 'search' + gt_url: "https://comp21storage.blob.core.windows.net/publiccontainer/comp23/clustered_data/msturing-30M-clustered/final_runbook.yaml"