Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add file input retriever class for reading from file source #86

Merged
merged 3 commits into from
Sep 18, 2024
Merged

Add file input retriever class for reading from file source #86

merged 3 commits into from
Sep 18, 2024

Conversation

nv-hwoo
Copy link
Contributor

@nv-hwoo nv-hwoo commented Sep 17, 2024

This PR

  • introduces the new class FileInputRetriever that handles reading input data from files
  • moves all the file related methods under InputRetrieverFactory class to the FileInputRetriever class
  • moves all the unit tests related to the FileInputRetriever to test_file_input_retriever.py

This PR does NOT

  • completely decouples endpoint logics from the retriever (this refining will be done in the follow up PR as there's a high change that will make the PR too big)

dyastremsky
dyastremsky approved these changes Sep 17, 2024
View reviewed changes
Copy link
Contributor

@dyastremsky dyastremsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spectacular work here, Hyunjae. This is a great reorganization of the code.

Added a few comments with discussions around long-term direction. The code looks good to merge into the feature branch.

genai-perf/genai_perf/inputs/file_input_retriever.py Show resolved Hide resolved
genai-perf/genai_perf/inputs/input_retriever_factory.py Show resolved Hide resolved
genai-perf/genai_perf/inputs/inputs.py Show resolved Hide resolved
@nv-hwoo nv-hwoo merged commit 4d0a921 into feature-create-input-retriever-APIs Sep 18, 2024
8 checks passed
@nv-hwoo nv-hwoo deleted the hwoo-add-file-retriever branch September 18, 2024 00:31
nv-hwoo added a commit that referenced this pull request Sep 23, 2024
@nv-hwoo
Add file input retriever class for reading from file source (#86)
4048c70 
* add file input retriever

* move input source check to inputs.py

* create unittests for file input retriever
dyastremsky pushed a commit that referenced this pull request Oct 3, 2024
@nv-hwoo @dyastremsky
Add file input retriever class for reading from file source (#86)
2d5b05a 
* add file input retriever

* move input source check to inputs.py

* create unittests for file input retriever
pvijayakrish pushed a commit that referenced this pull request Oct 8, 2024
@tanmayv25 @pskiran1 @mc-nv
Adding support for TensorRT 10 APIs in the backend. Keep TRT 8 suppor…
75d3361 
…t as well (#88) (#89)

* Replace binding index-based methods with name-based alternatives

* Remove unused variables

* Remove unused variables

* Remove allInput*Specified()

* Delete TRTV1Interface

* Replace getProfileShapeValues() with getProfileTensorValues()

* Remove buffer_bindings_

* Enhancements

* Replace isExecutionBinding()

* Add INT64 support

* Remove hasImplicitBatchDimension()

* Update Copyright

* Remove unused variables

* Undo copyright

* Undo Copyright

* Undo copyright

* Fix the handling in INT64 shape tensors output

* Fix data dependent output shapes

* Fix pre commit errors

* Update copyright

* Resolve review comments

* Include source for building on TRT 8 (#86) (#87)

* Include source for building on TRT 8

* Apply suggestions from code review



---------



* Fix envvar access in CMake

---------

Co-authored-by: Sai Kiran Polisetty <[email protected]>
Co-authored-by: Misha Chornyi <[email protected]>
pvijayakrish pushed a commit that referenced this pull request Oct 8, 2024
@pskiran1 @tanmayv25 @mc-nv
Support BF16 data type in TensorRT backend (#90)
fd867b7 
* Adding support for TensorRT 10 APIs in the backend. Keep TRT 8 support as well (#88)

* Replace binding index-based methods with name-based alternatives

* Remove unused variables

* Remove unused variables

* Remove allInput*Specified()

* Delete TRTV1Interface

* Replace getProfileShapeValues() with getProfileTensorValues()

* Remove buffer_bindings_

* Enhancements

* Replace isExecutionBinding()

* Add INT64 support

* Remove hasImplicitBatchDimension()

* Update Copyright

* Remove unused variables

* Undo copyright

* Undo Copyright

* Undo copyright

* Fix the handling in INT64 shape tensors output

* Fix data dependent output shapes

* Fix pre commit errors

* Update copyright

* Resolve review comments

* Include source for building on TRT 8 (#86) (#87)

* Include source for building on TRT 8

* Apply suggestions from code review



---------

Co-authored-by: Misha Chornyi <[email protected]>

* Fix envvar access in CMake

---------

Co-authored-by: Sai Kiran Polisetty <[email protected]>
Co-authored-by: Misha Chornyi <[email protected]>

* Add support for kBF16

---------

Co-authored-by: Tanmay Verma <[email protected]>
Co-authored-by: Misha Chornyi <[email protected]>
debermudez pushed a commit that referenced this pull request Oct 8, 2024
@nv-hwoo @debermudez
Add file input retriever class for reading from file source (#86)
6555337 
* add file input retriever

* move input source check to inputs.py

* create unittests for file input retriever
debermudez added a commit that referenced this pull request Oct 9, 2024
@debermudez @nv-hwoo @dyastremsky
create input retriever apis (#118)
59e1a32 
* Add retriever protocol and dataset format transformer (#71)

* Add synthetic data retriever class (#77)

* add synthetic data retriever

* fix pytest

* Add file input retriever class for reading from file source (#86)

* add file input retriever

* move input source check to inputs.py

* create unittests for file input retriever

* Refactor non-chat LLM endpoint converters into template style design (#93)

* refactor non-chat LLM endpoint converters into template style design

* fix bug and add unit tests

* fix codeql

* Refactor embeddings/rankings/tensorrtllm_engine converters to template-syle design (#101)

* refactor converters

* clean up and add more unit tests

* add test for tensorrtllm_engine_converter

* address feedback

* Fix misaligned payload bug in vllm converter (#107)

* Use generic retriever functions for embeddings endpoint (#109)

* Create separate batch size option for text and images (#111)

* Refactor OpenAI Chat converter into template-style design (#114)

* add unit tests for openai chat

* move image retrieval unittest to chat converter

* convert to template style design

* fix codeql

* remove dataset support (#100)

* Remove dataset option

* Fix Readme

* Update READMe command back to single command

* Use text, not text_input, as field name (#113)

* refactor retriever file locations (#116)

* Move retriever files into a separate directory

* Move source images and fix import paths

* Fix import sorting and missed moving farewell file

* Fix codeql

---------

Co-authored-by: Hyunjae Woo <[email protected]>
Co-authored-by: David Yastremsky <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dyastremsky dyastremsky dyastremsky approved these changes

@debermudez debermudez Awaiting requested review from debermudez

@matthewkotila matthewkotila Awaiting requested review from matthewkotila

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nv-hwoo @dyastremsky