Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variability in test size #541

Open
wahajali opened this issue Aug 12, 2024 · 0 comments
Open

Variability in test size #541

wahajali opened this issue Aug 12, 2024 · 0 comments

Comments

@wahajali
Copy link

I’m trying to understand the variability in the Test Size of the datasets provided and if that has an impact on the QPS. The test sizes for datasets listed vary from 500 to 50,000. I had a few of questions:

  • Is there any reason for the test size selection?
  • Will smaller test size impact recall?
  • In case we iterate more than once on this dataset for querying, i.e. run_count > 1, databases might have cached the results from earlier runs. In which case the impact on smaller test size should be more pronounced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant