Skip to content

Commit

Permalink
Update README to make it clearer (#305)
Browse files Browse the repository at this point in the history
  • Loading branch information
PatriceVignola authored Apr 23, 2024
1 parent 0f7fb7a commit 486e1d7
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions examples/python/phi-3-tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,14 +49,15 @@ pip install --pre onnxruntime-genai-cuda --index-url=https://aiinfra.pkgs.visual

## Run the model

Run the model with [this script](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py).
Run the model with [model-qa.py](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py).

The script accepts a model folder and takes the generation parameters from the config in that model folder. You can also override the parameters on the command line.

This example is using the long context model running with DirectML on Windows.

```bash
python model-qa.py -m models/phi3-mini-128k-instruct-directml-int4-awq-block-128
curl https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py -o model-qa.py
python model-qa.py -m Phi-3-mini-128k-instruct-onnx/directml/directml-int4-awq-block-128
```

Once the script has loaded the model, it will ask you for input in a loop, streaming the output as it is produced the model. For example:
Expand Down

0 comments on commit 486e1d7

Please sign in to comment.