Skip to content

Commit

Permalink
Update index.md (#19941)
Browse files Browse the repository at this point in the history
  • Loading branch information
natke authored Mar 18, 2024
1 parent e00773e commit 2e93b41
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion docs/genai/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ nav_order: 6

_Note: this API is in preview and is subject to change._

Run generative AI models with ONNX Runtime.
Run generative AI models with ONNX Runtime. Source code: https://github.com/microsoft/onnxruntime-genai

This library provides the generative AI loop for ONNX models, including inference with ONNX Runtime, logits processing, search and sampling, and KV cache management.

Users can call a high level `generate()` method, or run each iteration of the model in a loop, generating one token at a time, and optionally updating generation parameters inside the loop.

It has support for greedy/beam search and TopP, TopK sampling to generate token sequences and built-in logits processing like repetition penalties. You can also easily add custom scoring.

0 comments on commit 2e93b41

Please sign in to comment.