Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add genai docs #19750

Merged
merged 45 commits into from
Mar 14, 2024
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
8b8a6fe
Add structure for GenAI
natke Feb 17, 2024
8e24b43
Start adding content for GenAI docs
natke Feb 18, 2024
7d45eff
Fill in some content for install and API
natke Feb 20, 2024
c8f6f78
Fix navigation
natke Feb 20, 2024
6a50586
Change title for GenAI nav
natke Feb 20, 2024
658ac0b
Add parents
natke Feb 20, 2024
1e74e15
Add more parents
natke Feb 20, 2024
97887ab
Add children
natke Feb 20, 2024
14e7261
Header updates
natke Feb 20, 2024
3f4ac0f
Add grand children
natke Feb 20, 2024
cce960e
Save grand children
natke Feb 20, 2024
6f9320b
Add table of contents
natke Feb 20, 2024
39d10a4
Fix link in install
natke Feb 20, 2024
61b5e89
Update Python API
natke Feb 20, 2024
ee83d2d
More edits to Python API docs
natke Feb 20, 2024
882c23d
A API docs first version
natke Feb 21, 2024
c86881c
Update to preview
natke Feb 21, 2024
27f3ae3
C# API first draft
natke Feb 21, 2024
6d1395d
Re-order APIs
natke Feb 21, 2024
b58e208
Add phi-2 Python tutorial
natke Feb 21, 2024
d30a3a4
Add config reference
natke Feb 21, 2024
cd1c989
Update Python API docs with recent changes
natke Feb 21, 2024
1011d7e
This time with the changes
natke Feb 21, 2024
30e9279
Add recent changes to the C API docs
natke Feb 21, 2024
308c58f
Update model builder doc
natke Feb 21, 2024
09c7647
Add subject to change note
natke Feb 21, 2024
b0ee1e0
Move config reference down
natke Feb 21, 2024
45981c4
Highlight json
natke Feb 21, 2024
72f5cdc
Fix broken links
natke Mar 2, 2024
46ca857
Fix list
natke Mar 11, 2024
50016b4
Update install instructions with RCs
natke Mar 11, 2024
4f8c9af
Fix Build ONNX Runtime nav
natke Mar 11, 2024
a6b5a84
Update docs/genai/tutorials/phi2-python.md
natke Mar 12, 2024
6b95851
Update docs/genai/tutorials/phi2-python.md
natke Mar 12, 2024
c3657d8
Update phi2-python.md with fixed typo
MaanavD Mar 13, 2024
42080cf
Update config and remove device type from API docs.
natke Mar 14, 2024
adec8d4
Merge branch 'add-genai-docs' of https://github.com/natke/onnxruntime…
natke Mar 14, 2024
31cd665
Updated config definitions
natke Mar 14, 2024
a74585f
Remove duplicate pad token definition
natke Mar 14, 2024
d8d4fb6
Bolden config items
natke Mar 14, 2024
debc89b
Add more definitions
natke Mar 14, 2024
354d211
Add summary section for different search and sampling
natke Mar 14, 2024
35b6400
Spell check
natke Mar 14, 2024
2219f02
num_sequences -> num_return_sequences
natke Mar 14, 2024
d3bc3d3
Not implemented -> not supported
natke Mar 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
590 changes: 590 additions & 0 deletions docs/genai/api/c.md

Large diffs are not rendered by default.

157 changes: 157 additions & 0 deletions docs/genai/api/csharp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
---
title: C# API
description: C# API reference for ONNX Runtime GenAI
has_children: false
parent: API docs
grand_parent: Generative AI (Preview)
nav_order: 2
---

# ONNX Runtime GenAI C# API

_Note: this API is in preview and is subject to change._

{: .no_toc }

* TOC placeholder
{:toc}

## Overview

## Model class

### Constructor

```csharp
public Model(string modelPath)
```

### Generate method

```csharp
public Sequences Generate(GeneratorParams generatorParams)
```

## Tokenizer class

### Constructor

```csharp
public Tokenizer(Model model)
```

### Encode method

```csharp
public Sequences Encode(string str)
```

### Encode batch method

```csharp
public Sequences EncodeBatch(string[] strings)
```

### Decode method

```csharp
public string Decode(ReadOnlySpan<int> sequence)
```

### Decode batch method

```csharp
public string[] DecodeBatch(Sequences sequences)
```

### Create stream method

```csharp
public TokenizerStream CreateStream()
```

## TokenizerStream class

### Decode method

```csharp
public string Decode(int token)
```

## GeneratorParams class

### Constructor

```csharp
public GeneratorParams(Model model)
```

### Set search option (double)

```csharp
public void SetSearchOption(string searchOption, double value)
```

### Set search option (bool) method

```csharp
public void SetSearchOption(string searchOption, bool value)
```

### Set input ids method

```csharp
public void SetInputIDs(ReadOnlySpan<int> inputIDs, ulong sequenceLength, ulong batchSize)
```

### Set input sequences method

```csharp
public void SetInputSequences(Sequences sequences)
```





## Generator class

### Constructor

```csharp
public Generator(Model model, GeneratorParams generatorParams)
```

### Is done method

```csharp
public bool IsDone()
```

### Compute logits

```csharp
public void ComputeLogits()
```

### Generate next token method

```csharp
public void GenerateNextTokenTop()
```


## Sequences class

### Num sequences member

```csharp
public ulong NumSequences { get { return _numSequences; } }
```

### [] operator

```csharp
public ReadOnlySpan<int> this[ulong sequenceIndex]
```

9 changes: 9 additions & 0 deletions docs/genai/api/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: API docs
description: API documentation for ONNX Runtime GenAI
parent: Generative AI (Preview)
has_children: true
nav_order: 2
---

_Note: this API is in preview and is subject to change._
Loading
Loading