Skip to content

Commit

Permalink
Python config API docs
Browse files Browse the repository at this point in the history
  • Loading branch information
RyanUnderhill authored Nov 26, 2024
1 parent 4dcc757 commit c0d16b7
Showing 1 changed file with 64 additions and 1 deletion.
65 changes: 64 additions & 1 deletion docs/genai/api/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,57 @@ pip install onnxruntime-genai
import onnxruntime_genai
```

## Config class

If the config needs to be modified at runtime to change providers/set provider options, this object can be first created and used, then a model can be created from this config object.

### Load a config

```python
onnxruntime_genai.Config(model_folder: str) -> onnxruntime_genai.Config
```
#### Parameters

- `model_folder`: Location of model and configuration on disk

#### Returns

`onnxruntime_genai.Config`

### Add a provider to the list

If the provider isn't already in the list of providers, this adds it to the end of the list.

```python
onnxruntime_genai.Config.append_provider(provider_name: str)
```

#### Parameters

- `provider_name`: (Required) The provider to set the option on (for example, "dml" or "cuda")

### Clear list of providers

Clears the list of providers. This is the only way to remove existing providers to enforce using only a specific provider.

```python
onnxruntime_genai.Config.clear_providers()
```

### Set an option for a provider

Set an option for a provider. If the provider is not already in the list, it will be added at the end (automatically calls append_provider internally).

```python
onnxruntime_genai.Config.set_provider_option(provider_name: str, option_name: str, option_value: str)
```

#### Parameters

- `provider_name`: (Required) The provider to set the option on (for example, "dml" or "cuda")
- `option_name`: (Required) Name of the option
- `option_value`: (Required) Value of the option

## Model class

### Load a model
Expand All @@ -46,6 +97,18 @@ onnxruntime_genai.Model(model_folder: str) -> onnxruntime_genai.Model

`onnxruntime_genai.Model`

### Load a model from a config

Loads the ONNX model(s) from a config object

```python
onnxruntime_genai.Model(config: config) -> onnxruntime_genai.Model
```

#### Parameters

- `config`: Config object previously created through onnxruntime_genai.Config

### Generate method

```python
Expand Down Expand Up @@ -369,4 +432,4 @@ onnxruntime_genai.Generator(adapters: Generators::Adapters, adapter: str) -> Non

#### Return value

None
None

0 comments on commit c0d16b7

Please sign in to comment.