Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install MS MARCO Passage V1 Parquet regressions #2640

Merged
merged 15 commits into from
Dec 7, 2024
Merged

Conversation

lintool
Copy link
Member

@lintool lintool commented Nov 27, 2024

Performed Faiss to Parquet conversion, setting up regressions from Parquet encoding of dense vectors.

@lintool lintool marked this pull request as draft November 27, 2024 19:12
Copy link

codecov bot commented Nov 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.59%. Comparing base (026ea0e) to head (5223ed4).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #2640   +/-   ##
=========================================
  Coverage     65.59%   65.59%           
  Complexity     1480     1480           
=========================================
  Files           225      225           
  Lines         13010    13010           
  Branches       1551     1551           
=========================================
  Hits           8534     8534           
  Misses         3949     3949           
  Partials        527      527           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lintool
Copy link
Member Author

lintool commented Dec 6, 2024

Removed this statement from docs because it isn't true anymore...

Note that here we are explicitly using Lucene's NoMergePolicy merge policy, which suppresses any merging of index segments.
This is because merging index segments is a costly operation and not worthwhile given our query set.

@lintool lintool requested review from ronakice and UShivani3 December 7, 2024 20:02
@lintool lintool marked this pull request as ready for review December 7, 2024 20:02
@ronakice
Copy link
Member

ronakice commented Dec 7, 2024

LGTM!

@lintool lintool merged commit f9f73c2 into master Dec 7, 2024
3 checks passed
@lintool lintool deleted the parquet-msmarco branch December 7, 2024 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants