Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Hugging Face Text Embeddings Interface's re-ranker API #728

Merged
merged 6 commits into from
Jul 3, 2024
Merged

Add support for Hugging Face Text Embeddings Interface's re-ranker API #728

merged 6 commits into from
Jul 3, 2024

Conversation

dyastremsky
Copy link
Contributor

@dyastremsky dyastremsky commented Jun 29, 2024 •
edited
Loading

Support Hugging Face Text Embeddings Interface's re-ranker API. This extends the ranking endpoint-type to Hugging Face TEI. If a user specifies "--extra-inputs rankings:tei", then the Hugging Face TEI is used.

Main changes:

  • Added documentation on how to run the ranking endpoint with Hugging Face's re-ranker API)
  • When rankings:tei is passed in as an extra inputs, GenAI-Perf logic routes to use the HuggingFace TEI re-ranker API instead for benchmarking ranking
  • The profile data JSON file now checks for the HuggingFace TEI re-ranker endpoint (reranker)
  • Added profile unit parsing unit tests for this endpoint
  • Updated our documentation to avoid the use of gerunds in the extra tutorials, consistent with Tim's PR for the main tutorial

Base automatically changed from feat-retriever to main June 29, 2024 04:49
@dyastremsky dyastremsky marked this pull request as ready for review July 2, 2024 22:16
@dyastremsky
Parse TEI rankings format
a6a396f 
Update table for TEI rankings
@dyastremsky dyastremsky force-pushed the dyas-tei-support branch 3 times, most recently from a7ab59d to 25ce3b2 Compare July 2, 2024 22:21
@dyastremsky
Remove gerunds in headings
6932a93 
Fix quotes

Remove gerunds in headings

Fix commands
@dyastremsky dyastremsky requested review from nv-hwoo and debermudez and removed request for nv-hwoo July 2, 2024 22:24
@dyastremsky dyastremsky self-assigned this Jul 2, 2024
@dyastremsky dyastremsky requested a review from nv-hwoo July 2, 2024 22:24
@dyastremsky dyastremsky merged commit d3fadc1 into main Jul 3, 2024
5 checks passed
@dyastremsky dyastremsky deleted the dyas-tei-support branch July 3, 2024 16:26
lkomali pushed a commit that referenced this pull request Jul 16, 2024
@dyastremsky @lkomali
Add support for Hugging Face Text Embeddings Interface's re-ranker API (
0662823 
#728)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@debermudez debermudez debermudez approved these changes

@nv-hwoo nv-hwoo Awaiting requested review from nv-hwoo

Assignees

@dyastremsky dyastremsky

Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dyastremsky @debermudez