The models/
folder contains the files needed to run the Nvidia triton server.
models/
├── ensemble/
│ └── config.pbtxt
├── postprocess/
│ ├── 1/
│ │ └── model.py
│ └── config.pbtxt
└── yolov8/
├── 1/
│ └── model.onnx
└── config.pbtxt
To get started, simply run the below docker commands:
docker build -t yolov8-triton .
docker run -d --gpus all -p 8000:8000 -v ./models:/models yolov8-triton
The clients/
folder contains code to run inference via python & c++.
-
Install dependencies:
-
Build the project
git clone https://github.com/AbdurNawaz/yolov8-triton-ensemble cd yolov8-triton-ensemble/clients/cpp-client mkdir build cmake -B build/ cmake --build build/
-
Run inference
./build/cpp-client
-
Install the required dependencies:
pip install tritonclient[all] opencv-python numpy
-
Run infernce
git clone https://github.com/AbdurNawaz/yolov8-triton-ensemble cd yolov8-triton-ensemble/clients/ python main.py --imgpath /path/to/your/image.jpg