diff --git a/README.md b/README.md index 376559b..eda5747 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,13 @@ Every LLM(LLama, Gemini, Mistral..) use their own Tokenizers trained on their ow this tutorial leverages the `special_tokens` usage in the Tokenizer. +Install the package + +```shell +pip install bpetokenizer +``` + + ```py from bpetokenizer import BPETokenizer