Skip to content

Commit

Permalink
rename repo name, update names
Browse files Browse the repository at this point in the history
  • Loading branch information
k9ele7en committed Aug 7, 2021
1 parent 6627ef5 commit 106ef21
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 11 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Thank you.
$ pip install -r requirements.txt
```
### 2. Install required environment for inference using Triton server
Check [./README_Triton.md](./README_Triton.md) for details. Install tools/packages included:
Check [./README_ENV.md](./README_ENV.md) for details. Install tools/packages included:
- TensorRT
- Docker
- nvidia-docker
Expand All @@ -48,7 +48,7 @@ IC15 | SynthText, IC15 | Eng | For IC15 only | [Click](https://drive.google.com/
LinkRefiner | CTW1500 | - | Used with the General Model | [Click](https://drive.google.com/open?id=1XSaFwBkOaFOdtk4Ane3DFyJGPRw6v5bO)

### 5. Model preparation before run Triton server:
a. Triton Inference Server inference: see details at [./README_Triton.md](./README_Triton.md)<br>
a. Triton Inference Server inference: see details at [./README_ENV.md](./README_ENV.md)<br>
Initially, you need to run a (.sh) script to prepare Model Repo, then, you just need to run Docker image when inferencing. Script get things ready for Triton server, steps covered:
- Convert downloaded pretrain into mutiple formats
- Locate converted model formats into Triton's Model Repository
Expand All @@ -66,7 +66,7 @@ $ curl -v localhost:8000/v2/health/ready
Now everythings ready, start inference by:
- Run docker image of Triton server (replace mount -v path to your full path to model_repository):
```
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/triton-server-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models
...
+------------+---------+--------+
| Model | Version | Status |
Expand Down
12 changes: 6 additions & 6 deletions README_Triton.md → README_ENV.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,9 @@ $ curl https://get.docker.com | sh \
Pull repo, image, and prepare models (Where <xx.yy> is the version of Triton that you want to use):
```
$ sudo docker pull nvcr.io/nvidia/tritonserver:<xx.yy>-py3
$ git clone https://github.com/huukim911/triton-server-CRAFT-pytorch.git
$ git clone https://github.com/huukim911/Triton-TensorRT-Inference-CRAFT-pytorch.git
Run the .sh script to convert model into target formats, prepare Model Repo and start Triton server container:
$ cd triton-server-CRAFT-pytorch
$ cd Triton-TensorRT-Inference-CRAFT-pytorch
$ sh prepare.sh
Convert source model into target formats and copy into Triton's Model Repository successfully.
```
Expand All @@ -227,9 +227,9 @@ Run server in container and client in cmd
```
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v <full_path_to/model_repository>:/models nvcr.io/nvidia/tritonserver:<xx.yy>-py3 tritonserver --model-repository=/models

For example, run on server with full path "/home/maverick911/repo/triton-server-CRAFT-pytorch
For example, run on server with full path "/home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch
/model_repository":
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/triton-server-CRAFT-pytorch
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch
/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models

+----------------------+---------+--------+
Expand All @@ -244,7 +244,7 @@ I0611 04:10:23.080860 1 http_server.c9:2906] Started Metrics Service at 0.0.0.0:
```
2. Infer by client in cmd (this repo), with method (triton), model name (<model_type>_\<format>), version (not required). For ex:
```
$ cd triton-server-CRAFT-pytorch/
$ cd Triton-TensorRT-Inference-CRAFT-pytorch/
$ python infer_triton.py -m='detec_trt' -x=1 --test_folder='./images'
Request 1, batch size 1s/sample.jpg
elapsed time : 0.9521937370300293s
Expand All @@ -258,7 +258,7 @@ elapsed time : 1.244419813156128s
Run server in container and client sdk in container:
1. Start the server side:
```
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/triton-server-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models
$ sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models

+----------------------+---------+--------+
| Model | Version | Status |
Expand Down
2 changes: 1 addition & 1 deletion prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ if [ ${failed} -ne 0 ]; then

# III. Start Triton server image in container, mount Model Repo prepared into container volume
# Update the full path to data/model_repository follow deploy server path: "-v <full_path_to>/model_repository:/models"
sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/triton-server-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models
sudo docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch/model_repository:/models nvcr.io/nvidia/tritonserver:21.05-py3 tritonserver --model-repository=/models
2 changes: 1 addition & 1 deletion test.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,4 +176,4 @@ def test_net(net, image, text_threshold, link_threshold, low_text, cuda, poly, r
print("Done, elapsed time : {}s. Check at folder result/".format(time.time() - t))

# Example cmd:
# python test.py --weight /home/maverick911/repo/triton-server-CRAFT-pytorch/weight/craft_mlt_25k.pth
# python test.py --weight /home/maverick911/repo/Triton-TensorRT-Inference-CRAFT-pytorch/weight/craft_mlt_25k.pth

0 comments on commit 106ef21

Please sign in to comment.