Skip to content

Commit

Permalink
Remove hand authored config, as it now generated by the model builder
Browse files Browse the repository at this point in the history
  • Loading branch information
natke committed Feb 24, 2024
1 parent 0003e58 commit a89ef8b
Show file tree
Hide file tree
Showing 15 changed files with 15 additions and 194,450 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ test/csharp/obj/
/*.csv
.idea
cache_dir
example-models
*.onnx
*.onnx.data

Expand Down
2 changes: 1 addition & 1 deletion examples/llama/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Run the model builder script to export, optimize, and quantize the model. More d

```bash
cd examples/phi2
python -m onnxruntime_genai.models.builder -m meta-llama/Llama-2-7b-chat-hf -e cpu -p int4 -o ./model
python -m python -m onnxruntime_genai.models.builder -m meta-llama/Llama-2-7b-chat-hf -e cpu -p int4 -o ./example-models/llama2-7b-chat-int4-cpu
```

## Run Llama
Expand Down
4 changes: 3 additions & 1 deletion examples/llama/llama-loop.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import onnxruntime_genai as og

print("Loading model...")
model=og.Model("model", og.DeviceType.CPU)

# The first argument is the name of the folder containing the model files
model=og.Model("example-models/llama2-7b-chat-int4-cpu", og.DeviceType.CPU)
print("Model loaded")
tokenizer=model.create_tokenizer()
print("Tokenizer created")
Expand Down
4 changes: 3 additions & 1 deletion examples/llama/llama.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
import time

print("Loading model...")
model=og.Model("model", og.DeviceType.CPU)

# The first argument is the name of the folder containing the model files
model=og.Model("example-models/llama2-7b-chat-int4-cpu", og.DeviceType.CPU)
print("Model loaded")
tokenizer=model.create_tokenizer()
print("Tokenizer created")
Expand Down
23 changes: 0 additions & 23 deletions examples/llama/model/config.json

This file was deleted.

Loading

0 comments on commit a89ef8b

Please sign in to comment.