Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change ScaNN submission to pin version and use all available cores. #310

Merged
merged 1 commit into from
Sep 25, 2024

Conversation

arron2003
Copy link
Contributor

Change the scann configuration to work with benchmarking on bare-metal AMD machines (#306 from @sourcesync )

  1. Change the number of threads to be num_cpus.
  2. Pin ScaNN version.

The expected results in my run:

algorithm,parameters,dataset,count,qps,distcomps,build,indexsize,mean_ssd_ios,mean_latency,track,recall/ap
scann,"ScaNN,tree=27/40000,AH2,reorder=140",text2image-10M,10,120563.94772606545,0.0,2.384185791015625e-05,5194988.0,0,0,ood,0.8800800000000001
scann,"ScaNN,tree=34/40000,AH2,reorder=155",text2image-10M,10,104613.5888041144,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.8993499999999999
scann,"ScaNN,tree=35/40000,AH2,reorder=150",text2image-10M,10,102017.11204505162,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.90015
scann,"ScaNN,tree=35/40000,AH2,reorder=155",text2image-10M,10,102674.5089226592,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.900902
scann,"ScaNN,tree=36/40000,AH2,reorder=150",text2image-10M,10,104263.22144037379,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.901652
scann,"ScaNN,tree=37/40000,AH2,reorder=145",text2image-10M,10,106193.13800005772,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.9022270000000001
scann,"ScaNN,tree=38/40000,AH2,reorder=140",text2image-10M,10,103822.48977692406,0.0,2.09808349609375e-05,5194716.0,0,0,ood,0.902576
scann,"ScaNN,tree=42/40000,AH2,reorder=160",text2image-10M,10,91762.75642867034,0.0,2.288818359375e-05,5195112.0,0,0,ood,0.9140330000000001

@sourcesync
Copy link
Collaborator

Great! Do you guys think you can merge this today @magdalendobson @arron2003 ?

@sourcesync
Copy link
Collaborator

Great! Do you guys think you can merge this today @magdalendobson @arron2003 ?

Hey, nevermind my request to merge this PR today/soon. I ended up just merging @arron2003's remote main branch into my PR branch . Here are the updated rankings. Confirmed by @arron2003.

@harsha-simhadri harsha-simhadri merged commit 54b8894 into harsha-simhadri:main Sep 25, 2024
24 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants