Skip to content

Commit

Permalink
Revert "Update README"
Browse files Browse the repository at this point in the history
This reverts commit 96984c5.
  • Loading branch information
PatriceVignola committed Apr 23, 2024
1 parent e73dd53 commit 9c4c08c
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions examples/python/phi-3-tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,15 +49,14 @@ pip install --pre onnxruntime-genai-cuda --index-url=https://aiinfra.pkgs.visual

## Run the model

Run the model with [model-qa.py](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py).
Run the model with [this script](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py).

The script accepts a model folder and takes the generation parameters from the config in that model folder. You can also override the parameters on the command line.

This example is using the long context model running with DirectML on Windows.

```bash
curl https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-qa.py -o model-qa.py
python model-qa.py -m Phi-3-mini-4k-instruct-onnx/directml/directml-int4-awq-block-128
python model-qa.py -m models/phi3-mini-128k-instruct-directml-int4-awq-block-128
```

Once the script has loaded the model, it will ask you for input in a loop, streaming the output as it is produced the model. For example:
Expand Down

0 comments on commit 9c4c08c

Please sign in to comment.