guides/deepstream-nvidia-jetson/

[giscus[bot]]

guides/deepstream-nvidia-jetson/

Learn how to deploy Ultralytics YOLOv8 on NVIDIA Jetson devices using TensorRT and DeepStream SDK. Explore performance benchmarks and maximize AI capabilities.

https://docs.ultralytics.com/guides/deepstream-nvidia-jetson/

[AgbajeAyomipo]

I have just done everything as I've seen, apparently i can't seem to run inference:
** ERROR: <parse_config_file:642>: parse_config_file failed
** ERROR: main:697: Failed to parse config file 'deepstream_app_config.txt'

Quitting
App Run Failed

[glenn-jocher]

@AgbajeAyomipo hi there!

Thank you for reaching out. It looks like you're encountering an issue with parsing the deepstream_app_config.txt file. This error typically indicates that there might be a syntax error or misconfiguration in your config file.

Here are a few steps to help troubleshoot and resolve the issue:

1. Verify Config File Syntax: Ensure that the deepstream_app_config.txt file is correctly formatted. Even a small typo can cause parsing errors. Double-check all paths, parameters, and ensure there are no missing or extra characters.

2. Example Config File: If possible, compare your config file with a working example. Here’s a basic structure for reference:

   [primary-gie]
   enable=1
   gpu-id=0
   config-file=config_infer_primary_yoloV8.txt
   ...
   
   [source0]
   enable=1
   type=3
   uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
   ...

3. Check File Paths: Ensure that all file paths specified in the config file are correct and accessible. This includes paths to the ONNX model, video source, and any other referenced files.

4. Dependencies and Versions: Make sure you have installed all necessary dependencies and that you are using compatible versions of DeepStream SDK and JetPack. Refer to the DeepStream Configuration for YOLOv8 for detailed instructions.

5. Reproducible Example: If the issue persists, providing a minimum reproducible example can greatly help in diagnosing the problem. You can find guidance on creating one here.

6. Latest Versions: Ensure you are using the latest versions of the packages. Sometimes, issues are resolved in newer releases.

Here’s a quick example to ensure your setup is correct:

# Clone the repository
git clone https://github.com/marcoslucianops/DeepStream-Yolo
cd DeepStream-Yolo

# Convert YOLOv8 model to ONNX
python3 utils/export_yoloV8.py -w yolov8s.pt --opset 12 --simplify

# Compile the library
make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo

# Edit config files as needed
# Run inference
deepstream-app -c deepstream_app_config.txt

If you follow these steps and still encounter issues, please share more details or any specific error messages you receive. This will help us assist you more effectively.

Best of luck, and feel free to reach out if you have any more questions! 😊

[AgbajeAyomipo]

Thanks for the reply, i fixed it already

I was not in the directory when i ran the command apparently.

[glenn-jocher]

Hi @AgbajeAyomipo,

Thank you for the update! I'm glad to hear that you were able to resolve the issue by running the command from the correct directory. It's a common oversight, and it happens to the best of us. 😊

If you have any further questions or run into any other issues, feel free to reach out. We're here to help!

[metalpole]

is the ultralytics package necessary for following this guide? would just the .pt file be sufficient for exporting with deepstream-yolo

[metalpole]

what does step 6 do?
make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo

[metalpole]

since the onnx file has already been generated in a previous step

[glenn-jocher]

Step 6 involves compiling the custom YOLO inference library for DeepStream. The make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo command first cleans any previous builds and then compiles the necessary files. This step ensures that the custom YOLO layers are correctly integrated into DeepStream for optimal performance. Even though the ONNX file is already generated, this compilation is crucial for the custom layers to function properly within the DeepStream pipeline.

[githubwalt]

@glenn-jocher, do we have to compile again each time there is a new onnx file?

[glenn-jocher]

Yes, you need to compile again each time there is a new ONNX file to ensure the custom YOLO layers are correctly integrated for inference.

[damien-pomelli]

Hello everyone, I'm trying to run the deepstream-app with an FP16 YOLOv8 model. I first followed this guide https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/YOLOv8.md but couldn't find an "fp16 option". Then, I tried to export a custom model with model.export(half=True,...), but the deepstream-app runs without displaying rectangles (I did set network-mode=2 in the config file). Has anyone successfully used an FP16 YOLO model with DeepStream?

[damien-pomelli]

@pderrenger by exporting the FP16 model you mean with model.export() or with this guide https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/YOLOv8.md ? I don't know how to do that with the DeepStream-Yolo guide so I suppose by running this :

model = YOLO('yolov8n.pt')
success = model.export(
    format="onnx",
    half=True,
    simplify=True,
    dynamic=True
)

I also tried to directly export an .engine model with the YOLO class but it doesn't seem to be compatible with DeepStream. Other question, if you want to batch 4 streams, will it work by setting batch=4 and running the deepstream-app ? My feeling is that you can only obtain a b1 model with this app.

[damien-pomelli]

@metapole, you're right, I mixed up network-mode and network-type, but I did obtain an FP32 model last week. While the model worked, it was too slow, and the output of my pipeline was very laggy (around 1 FPS). That's why I'm now trying with an FP16 model.

[metalpole]

you shouldn't be using that export if you want it to work with deepstream. the only difference is changing the config for fp16. for batching you would have to go beyond editing the inference component of the pipeline and understand how the end-to-end pipeline is set up. i personally use https://github.com/prominenceai/deepstream-services-library which is beginner friendly

[damien-pomelli]

Perfect it works now ! Yeah I thought it was necessary to import a "FP16 pre-processed" .onnx model, it's actually easier :)
It's still a bit slow in terms of fps, I'll try to optimise it. Many thanks to you both !

[pderrenger]

Great to hear it's working now! For optimizing FPS, ensure your TensorRT engine is correctly set to FP16 by using model-engine-file=model_b1_gpu0_fp16.engine and network-mode=2 in your config file. For batching streams, you may need to adjust the pipeline configuration beyond just setting batch=4. Tools like the DeepStream Services Library can simplify this process. For more detailed steps, refer to our DeepStream-NVIDIA Jetson guide.

[seifeldin33]

Hello everyone, I have just done everything as I've seen, and everything works fine. My problem is how to run RTSP streaming. I created one using VLC on my local Windows and modified these 2 arguments.

type=4
uri=rtsp://localhost:8554/

but it's giving me a black screen and showing this error.
ERROR from src_elem0: Resource not found.
Debug info: ../gst/rtsp/gstrtspsrc.c(8121): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
No valid RTSP URL was provided
** INFO: <reset_source_pipeline:1719>: Resetting source 0
ERROR from src_elem0: Resource not found.
Debug info: ../gst/rtsp/gstrtspsrc.c(8121): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
No valid RTSP URL was provided
**PERF: 0.00 (0.00)
**PERF: 0.00 (0.00)
**PERF: 0.00 (0.00)
**PERF: 0.00 (0.00)
**PERF: 0.00 (0.00)
Quitting
nvstreammux: Successfully handled EOS for source_id=0
App run successful

[glenn-jocher]

@seifeldin33 it seems like the RTSP URL might not be valid or accessible. Please ensure that the RTSP stream is correctly set up and accessible at rtsp://localhost:8554/. You can verify this by testing the stream with another RTSP client like VLC. If the issue persists, check the DeepStream logs for more detailed error information. For further guidance, refer to the DeepStream Configuration for YOLOv8.

[seifeldin33]

I have a question about integrating Kafka with Deepstream. My application is designed to send safety violations to the backend in real-time. I've heard about Kafka, and I'm wondering if it's the right approach to use Kafka for this purpose. I would also like to know how to integrate it. I want to ensure that the violations are sent securely in real-time.

[glenn-jocher]

@seifeldin33 using Kafka for real-time safety violation alerts is a solid choice due to its scalability and reliability. To integrate Kafka with DeepStream, you can use a Kafka producer to send messages from your application to a Kafka topic. Ensure your DeepStream pipeline processes the data and sends it to Kafka securely. For detailed integration steps, consider exploring Kafka's official documentation or community resources.

[seifeldin33]

and what about Rest API?

[glenn-jocher]

For integrating a REST API with DeepStream, you can use the DeepStream SDK's message broker capabilities to send data to a REST endpoint. This involves configuring the message broker to use HTTP as the protocol. For detailed steps, refer to the DeepStream documentation.

[githubwalt]

Hi,
I already installed marcoslucianops/DeepStream-Yolo which at the time was with Yolov8.
Runs fine.

I also have a separate venv with yolov8 to train on a custom dataset.
When I start training it says that there is an update available (must be yolo11 I guess).

My question is: can I just update this to yolo11 and next move my trained .pt file to the deepstream installation. Will this work?

[glenn-jocher]

@githubwalt the camera resolution is important for inference, as it should match the model's input size for optimal performance. Use --size 1280 1280 during export to align with your training size, ensuring consistency.

[githubwalt]

Yes, I understand. But why are there two options?
--size SIZE and --size HEIGHT WIDTH
Are height and width always equal?
Is --size 1280 equal to --size 1280 1280 ?

(sorry for the many questions)

[glenn-jocher]

The --size SIZE option sets both height and width to the same value, while --size HEIGHT WIDTH allows specifying different dimensions. For your camera, use --size 1280 1280 to match the training size.

[githubwalt]

Thanks for the swift replies Glenn.

[glenn-jocher]

You're welcome! For your camera resolution, use --size 1280 1280 during export to match your training size. This ensures consistency and optimal performance. If you have more questions, feel free to ask!

[SashreekMallem]

Hi,

I'm using the Jetson Orin Nano 8GB Developer Kit, and I followed the steps from the video and documentation to convert the YOLOv11 model to TensorRT. After building the model successfully, I encountered a segmentation fault (core dumped) error when trying to run the model.

[SashreekMallem]

I managed to fix the initial segmentation fault issue with the YOLOv11 model conversion to TensorRT. The model built successfully, and I was able to start it. However, now I’m encountering a new problem.

After starting the pipeline, it crashed with the following message:

**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:291>: Pipeline ready

ERROR from src_elem: Internal data stream error.
Debug info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
** INFO: <bus_callback:256>: incorrect camera parameters provided, please provide supported resolution and frame rate
I adjusted the camera resolution and frame rate as suggested, but now I’m facing a new error:

** ERROR: <parse_tiled_display:2013>: parse_tiled_display failed
** ERROR: <parse_config_file:790>: parse_config_file failed
** ERROR: main:685: Failed to parse config file 'deepstream_app_config.txt'
Quitting

(deepstream-app:16369): GStreamer-CRITICAL **: 01:09:05.775: gst_element_send_event: assertion 'GST_IS_ELEMENT (element)' failed
App run failed

[glenn-jocher]

@SashreekMallem it seems like there might be an issue with your DeepStream configuration files. Please ensure that the paths and parameters in deepstream_app_config.txt and config_infer_primary_yoloV8.txt are correctly set. For more detailed guidance, you can refer to the Ultralytics documentation on deploying YOLO models with DeepStream: https://docs.ultralytics.com/guides/deepstream-nvidia-jetson/.

[Blessing988]

Will this guide work for every yolo model including YOLOv10 or it is exclusive for YOLOV11 alone.

[glenn-jocher]

@Blessing988 the guide is primarily designed for YOLOv11, but it may also be applicable to other YOLO models with some adjustments. For specific details, please refer to the documentation or test with your model.

[dao027]

do u have an implement of the bytetrack with deepstream?

[glenn-jocher]

@dao027 currently, we don't have a specific implementation of ByteTrack with DeepStream. However, you can refer to the DeepStream-Yolo repository for integrating YOLO models with DeepStream, which may assist in developing your own implementation. If you have further questions, feel free to ask!