Skip to content

Commit

Permalink
Changes to README
Browse files Browse the repository at this point in the history
  • Loading branch information
cooljoseph1 committed Aug 28, 2024
1 parent 1baa477 commit 81fa872
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,14 @@ pip install -r requirements.txt
## Running
To do inference, run the command
```sh
python3 inference.py --prompt "<prompt>"
./infer.py --prompt "<prompt>"
```
where `<prompt>` is your text prompt. There are more options, which can be printed out using the `--help` flag.

You don't have to provide a prompt; if you don't provide a prompt, it will instead read standard input for the prompt.

## TODO
- [ ] Get inference to go longer than 1024 tokens. (GPT2 was only trained with 1024 positional embeddings. This might not be possible to do efficiently.)
- [ ] Figure out a way to do Ahead-Of-Time compilation? Might be useful for speeding things up over multiple prompts
- [ ] Add training in a train/ folder (right now it has inference in a run/ folder)
- [ ] Figure out a better way to load safetensors? Right now I'm using Pytree2Safetensors, which is a not-very-polished library I made in a few hours.

0 comments on commit 81fa872

Please sign in to comment.