Skip to content

Commit

Permalink
Sweep for generate() API
Browse files Browse the repository at this point in the history
  • Loading branch information
natke committed May 21, 2024
1 parent 8e17fbe commit f1f2a89
Show file tree
Hide file tree
Showing 9 changed files with 16 additions and 16 deletions.
4 changes: 2 additions & 2 deletions docs/genai/api/c.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
title: C API
description: C API reference for ONNX Runtime GenAI
description: C API reference for ONNX Runtime generate() API
has_children: false
parent: API docs
grand_parent: Generate API (Preview)
nav_order: 3
---

# ONNX Runtime GenAI C API
# ONNX Runtime generate() C API

_Note: this API is in preview and is subject to change._

Expand Down
4 changes: 2 additions & 2 deletions docs/genai/api/csharp.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
title: C# API
description: C# API reference for ONNX Runtime GenAI
description: C# API reference for ONNX Runtime generate() API
has_children: false
parent: API docs
grand_parent: Generate API (Preview)
nav_order: 2
---

# ONNX Runtime GenAI C# API
# ONNX Runtime generate() C# API

_Note: this API is in preview and is subject to change._

Expand Down
2 changes: 1 addition & 1 deletion docs/genai/api/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: API docs
description: API documentation for ONNX Runtime GenAI
description: API documentation for ONNX Runtime generate() API
parent: Generate API (Preview)
has_children: true
nav_order: 2
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/api/python.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Python API
description: Python API reference for ONNX Runtime GenAI
description: Python API reference for ONNX Runtime generate() API
has_children: false
parent: API docs
grand_parent: Generate API (Preview)
Expand Down
4 changes: 2 additions & 2 deletions docs/genai/howto/build-model.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Build models
description: How to build models with ONNX Runtime GenAI
description: How to build models with ONNX Runtime generate() API
has_children: false
parent: How to
grand_parent: Generate API (Preview)
Expand Down Expand Up @@ -113,7 +113,7 @@ python3 builder.py -m model_name -o path_to_output_folder -p precision -e execut
To see all available options through `--extra_options`, please use the `help` commands in the `Full Usage` section above.

### Config Only
This scenario is for when you already have your optimized and/or quantized ONNX model and you need to create the config files to run with ONNX Runtime GenAI.
This scenario is for when you already have your optimized and/or quantized ONNX model and you need to create the config files to run with ONNX Runtime generate() API.
```
# From wheel:
python3 -m onnxruntime_genai.models.builder -m model_name -o path_to_output_folder -p precision -e execution_provider -c cache_dir_for_hf_files --extra_options config_only=true
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/howto/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: How to
description: How to perform specific tasks with ONNX Runtime GenAI
description: How to perform specific tasks with ONNX Runtime generate() API
parent: Generate API (Preview)
has_children: true
nav_order: 3
Expand Down
4 changes: 2 additions & 2 deletions docs/genai/howto/install.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
title: Install
description: Instructions to install ONNX Runtime GenAI on your target platform in your environment
description: Instructions to install ONNX Runtime generate() API on your target platform in your environment
has_children: false
parent: How to
grand_parent: Generate API (Preview)
nav_order: 1
---

# Install ONNX Runtime GenAI
# Install ONNX Runtime generate() API
{: .no_toc }

* TOC placeholder
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/tutorials/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Tutorials
description: Build your application with ONNX Runtime GenAI
description: Build your application with ONNX Runtime generate() API
parent: Generate API (Preview)
has_children: true
nav_order: 1
Expand Down
8 changes: 4 additions & 4 deletions docs/genai/tutorials/phi2-python.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Python phi-2 tutorial
description: Learn how to write a language generation application with ONNX Runtime GenAI in Python using the phi-2 model
description: Learn how to write a language generation application with ONNX Runtime generate() API in Python using the phi-2 model
has_children: false
parent: Tutorials
grand_parent: Generate API (Preview)
Expand All @@ -11,7 +11,7 @@ nav_order: 2

## Setup and installation

Install the ONNX Runtime GenAI Python package using the [installation instructions](../howto/install.md).
Install the ONNX Runtime generate() API Python package using the [installation instructions](../howto/install.md).

## Build phi-2 ONNX model

Expand All @@ -31,11 +31,11 @@ python -m onnxruntime_genai.models.builder -m microsoft/phi-2 -e cpu -p int4 -o
```
You can replace the name of the output folder specified with the `-o` option with a folder of your choice.

After you run the script, you will see a series of files generated in this folder. They include the HuggingFace configs for your reference, as well as the following generated files used by ONNX Runtime GenAI.
After you run the script, you will see a series of files generated in this folder. They include the HuggingFace configs for your reference, as well as the following generated files used by ONNX Runtime generate() API.

- `model.onnx`: the phi-2 ONNX model
- `model.onnx.data`: the phi-2 ONNX model weights
- `genai_config.json`: the configuration used by ONNX Runtime GenAI
- `genai_config.json`: the configuration used by ONNX Runtime generate() API

You can view and change the values in the `genai_config.json` file. The model section should not be updated unless you have brought your own model and it has different parameters.

Expand Down

0 comments on commit f1f2a89

Please sign in to comment.