LlmInputs - Updated API + generic framework + convert to vLLM format (#…

…486) * Initial API changes. Unit tests passing * Creating and using generic format with openai * Initial code to support vllm output * Refactoring * General cleanup + todos * Fixing codeql * Fix codeql issue * Removing output_filename define
triton-inference-server · Mar 5, 2024 · 9444bb1 · 9444bb1
1 parent ae5d5b6
commit 9444bb1
Show file tree

Hide file tree

Showing 3 changed files with 409 additions and 76 deletions.
diff --git a/src/c++/perf_analyzer/genai-pa/genai_pa/constants.py b/src/c++/perf_analyzer/genai-pa/genai_pa/constants.py
@@ -25,3 +25,7 @@
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 LOGGER_NAME: str = "genai-pa"
+
+OPEN_ORCA = "openorca"
+CNN_DAILY_MAIL = "cnn_dailymail"
+DEFAULT_INPUT_DATA_JSON = "llm_inputs.json"