Skip to content

Commit

Permalink
Update readme (#32)
Browse files Browse the repository at this point in the history
Signed-off-by: kaixuanliu <[email protected]>
Co-authored-by: regisss <[email protected]>
  • Loading branch information
kaixuanliu and regisss authored Sep 27, 2024
1 parent 6488b73 commit 0c07d3b
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ To use [🤗 text-embeddings-inference](https://github.com/huggingface/text-embe
docker run -p 8080:80 -v $volume:/data --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e MAX_WARMUP_SEQUENCE_LENGTH=512 --cap-add=sys_nice --ipc=host ghcr.io/huggingface/tei-gaudi:latest --model-id $model --pooling cls
```
For models within the Transformers library that need remote code to run customized implementations, please set the environment variable `-e TRUST_REMOTE_CODE=TRUE` within `docker run` command line. Here is an example:
```
model="Alibaba-NLP/gte-large-en-v1.5"
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run

docker run -p 8080:80 -v $volume:/data --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e MAX_WARMUP_SEQUENCE_LENGTH=512 -e TRUST_REMOTE_CODE=TRUE --cap-add=sys_nice --ipc=host ghcr.io/huggingface/tei-gaudi:latest --model-id $model --pooling cls
3. You can then send a request:
```bash
curl 127.0.0.1:8080/embed \
Expand Down

0 comments on commit 0c07d3b

Please sign in to comment.