diff --git a/FAQ.md b/FAQ.md index c43a18d..31daf78 100644 --- a/FAQ.md +++ b/FAQ.md @@ -8,6 +8,7 @@ * [Error on setting string field](#faq5) * [Pipeline unable to perform at real time](#faq6) * [Triton container problems with multi-GPU setup](#faq7) +* [ModuleNotFoundError: No module named 'pyds'] (#faq8) ### Application fails to work with mp4 stream @@ -93,10 +94,21 @@ The application appears to be stuck without any playback activity. a) probe() callbacks are synchronous and thus holds the buffer (info.get_buffer()) from traversing the pipeline until user return. b) loops inside probe() callback could be costly in python. + ### Triton container problems with multi-GPU setup The Triton Inference Server plugin currently only supports single-GPU usage. When running the docker, please specify `--gpus device=` -e.g.: `--gpus device=0` \ No newline at end of file +e.g.: `--gpus device=0` + + + +### ModuleNotFoundError: No module named 'pyds' +The pyds extension is installed under /opt/nvidia/deepstream/deepstream/lib +The sample applications all include this path via the is_aarch_64 module. +A setup.py is also provided to install this extension into standard path. +Currently this needs to be run manually: + $ cd /opt/nvidia/deepstream/deepstream/lib + $ python3 setup.py install diff --git a/HOWTO.md b/HOWTO.md index 76d3cca..bfc7286 100644 --- a/HOWTO.md +++ b/HOWTO.md @@ -16,7 +16,7 @@ This guide provides resources for DeepStream application development in Python. ## Prerequisites * Ubuntu 18.04 -* [DeepStream SDK 5.0 Developer Preview](https://developer.nvidia.com/deepstream-download) or later +* [DeepStream SDK 5.0](https://developer.nvidia.com/deepstream-download) or later * Python 3.6 * [Gst Python](https://gstreamer.freedesktop.org/modules/gst-python.html) v1.14.5 @@ -38,13 +38,13 @@ If missing, install with the following steps: ## Running Sample Applications -Download the release package and unpack it under DS 5.0 installation: -```tar xf ds_pybind_v0.9.tbz2 -C /sources``` +Clone the deepstream_python_apps repo under /sources: +git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps This will create the following directory: -```/sources/python``` +```/sources/deepstream_python_apps``` -The Python apps and bindings are under the "python" directory. +The Python apps are under the "apps" directory. Go into each app directory and follow instructions in the README. NOTE: The app configuration files contain relative paths for models. @@ -60,23 +60,13 @@ See [sample applications](apps/) main functions for pipeline construction exampl DeepStream MetaData contains inference results and other information used in analytics. The MetaData is attached to the Gst Buffer received by each pipeline component. The metadata format is described in detail in the [SDK MetaData documentation](https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_metadata.03.1.html) and [API Guide](https://docs.nvidia.com/metropolis/deepstream/python-api/index.html). -The SDK MetaData library is developed in C/C++. Python bindings provide access to the MetaData from Python applications. The bindings are provided in a compiled module, available for x86_64 and Jetson platforms. Find them in the release package with the following layout: -``` - bindings - |- x86_64 - | |- pyds.so - |- jetson - |- pyds.so -``` +The SDK MetaData library is developed in C/C++. Python bindings provide access to the MetaData from Python applications. The bindings are provided in a compiled module, available for x86_64 and Jetson platforms. This module, pyds.so, is available as part of the DeepStream SDK installation under /lib directory. -Applications can import the module thus: -```python -import sys -sys.path.append('../') # Add path to the bindings directory -# The common/is_aarch64.py module adds the platform-specific path to pyds module -from common.is_aarch64 import is_aarch64 -import pyds -``` +The sample applications gets the import path for this module through common/utils.py. A setup.py is also included for installing the module into standard path: +cd /opt/nvidia/deepstream/deepstream/lib +python3 setup.py install + +This is currently not automatically done through the SDK installer because python usage is optional. The bindings generally follow the same API as the underlying C/C++ library, with a few exceptions detailed in sections below. diff --git a/README.md b/README.md index 00dfded..87a9cb9 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,7 @@ This repository contains Python bindings and sample applications for the [DeepStream SDK](https://developer.nvidia.com/deepstream-sdk). -The bindings and apps are currently in *Alpha* at v0.9. The API is maturing but changes are still expected. -SDK version supported: 5.0 Developer Preview +SDK version supported: 5.0 Download the latest release package complete with bindings and sample applications from the [release section](../../releases). @@ -16,7 +15,7 @@ Please report any issues or bugs on the [Deepstream SDK Forums](https://devtalk. ## Python Bindings DeepStream pipelines can be constructed using Gst Python, the GStreamer framework's Python bindings. For accessing DeepStream MetaData, -Python bindings are provided in the form of a compiled module. Download these bindings [here](https://developer.nvidia.com/deepstream-download#python_bindings). This module is generated using [Pybind11](https://github.com/pybind/pybind11). +Python bindings are provided in the form of a compiled module which is included in the DeepStream SDK. This module is generated using [Pybind11](https://github.com/pybind/pybind11).

bindings pipeline diff --git a/apps/README b/apps/README index 199370e..194be96 100644 --- a/apps/README +++ b/apps/README @@ -34,13 +34,17 @@ Package Contents -------------------------------------------------------------------------------- The DeepStream Python package includes: 1. Python bindings for DeepStream Metadata libraries - These bindings are provided as a compiled module, available for - x86_64 and Jetson platforms. Find them at: - bindings - |- x86_64 - | |- pyds.so - |- jetson - |- pyds.so + These bindings are installed as part of the SDK at: + /opt/nvidia/deepstream/deepstream/lib/pyds.so + + Sample applications that import is_aarch_64 automatically + have this path added. + + A setup.py is also provided to install this extension into standard path. + Currently this needs to be run manually: + $ cd /opt/nvidia/deepstream/deepstream/lib + $ python3 setup.py install + 2. DeepStream test apps in Python The following test apps are available: deepstream-test1 @@ -75,10 +79,11 @@ $ sudo apt install python3-gi python3-dev python3-gst-1.0 -y Running the samples -------------------------------------------------------------------------------- The apps are configured to work from inside the DeepStream SDK 5.0 installation. -Untar the contents into /sources/. -Note that the apps import the pyds module from this relative path: -../bindings/ +Clone the deepstream_python_apps repo under /sources: +$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps +This will create the following directory: +/sources/deepstream_python_apps Follow README in each app's directory to run the app. diff --git a/apps/common/is_aarch_64.py b/apps/common/is_aarch_64.py index aa0ef29..c19e2d2 100644 --- a/apps/common/is_aarch_64.py +++ b/apps/common/is_aarch_64.py @@ -27,5 +27,4 @@ def is_aarch64(): return platform.uname()[4] == 'aarch64' - -sys.path.append('../../bindings/' + ('jetson' if is_aarch64() else 'x86_64')) +sys.path.append('/opt/nvidia/deepstream/deepstream/lib') diff --git a/apps/common/utils.py b/apps/common/utils.py index 66fc815..a49740a 100644 --- a/apps/common/utils.py +++ b/apps/common/utils.py @@ -21,6 +21,8 @@ ################################################################################ import ctypes +import sys +sys.path.append('/opt/nvidia/deepstream/deepstream/lib') def long_to_int(l): value = ctypes.c_int(l & 0xffffffff).value diff --git a/apps/deepstream-imagedata-multistream/dstest_imagedata_config.txt b/apps/deepstream-imagedata-multistream/dstest_imagedata_config.txt index 18e2984..fc03f21 100644 --- a/apps/deepstream-imagedata-multistream/dstest_imagedata_config.txt +++ b/apps/deepstream-imagedata-multistream/dstest_imagedata_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -62,8 +62,10 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 process-mode=1 model-color-format=0 @@ -79,3 +81,30 @@ cluster-mode=1 threshold=0.2 eps=0.7 minBoxes=1 + +#Use the config params below for dbscan clustering mode +[class-attrs-all] +detected-min-w=4 +detected-min-h=4 +minBoxes=3 + +## Per class configurations +[class-attrs-0] +pre-cluster-threshold=0.05 +eps=0.7 +dbscan-min-score=0.95 + +[class-attrs-1] +pre-cluster-threshold=0.05 +eps=0.7 +dbscan-min-score=0.5 + +[class-attrs-2] +pre-cluster-threshold=0.1 +eps=0.6 +dbscan-min-score=0.95 + +[class-attrs-3] +pre-cluster-threshold=0.05 +eps=0.7 +dbscan-min-score=0.5 diff --git a/apps/deepstream-imagedata-multistream/imagedata-app-block-diagram.png b/apps/deepstream-imagedata-multistream/imagedata-app-block-diagram.png new file mode 100644 index 0000000..c2862cf Binary files /dev/null and b/apps/deepstream-imagedata-multistream/imagedata-app-block-diagram.png differ diff --git a/apps/deepstream-ssd-parser/README b/apps/deepstream-ssd-parser/README index 1c9d1cb..e5d54c2 100644 --- a/apps/deepstream-ssd-parser/README +++ b/apps/deepstream-ssd-parser/README @@ -70,7 +70,26 @@ This example: - Encodes OSD output and saves to MP4 file. Note that there is no visual output on screen. Known Issue: -On Jetson, if libgomp is not preloaded, this error may occur: +1. On Jetson, if libgomp is not preloaded, this error may occur: (python3:21041): GStreamer-WARNING **: 14:35:44.113: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibav.so': /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block Unable to create Encoder +2. On Jetson Nano, ssd_inception_v2 is not expected to run with GPU instance. +Switch to CPU instance when running on Nano: +update config.pbtxt files in samples/trtis_modeo_repo: +# Switch to CPU instance for Nano since memory might not be enough for +# certain Models. + +# Specify CPU instance. +instance_group { + count: 1 + kind: KIND_CPU +} + +# Specify GPU instance. +#instance_group { +# kind: KIND_GPU +# count: 1 +# gpus: 0 +#} + diff --git a/apps/deepstream-ssd-parser/dstest_ssd_nopostprocess.txt b/apps/deepstream-ssd-parser/dstest_ssd_nopostprocess.txt index 4de85e5..16d057b 100644 --- a/apps/deepstream-ssd-parser/dstest_ssd_nopostprocess.txt +++ b/apps/deepstream-ssd-parser/dstest_ssd_nopostprocess.txt @@ -9,7 +9,7 @@ infer_config { model_repo { root: "../../../../samples/trtis_model_repo" log_level: 2 - tf_gpu_memory_fraction: 0.6 + tf_gpu_memory_fraction: 0.4 tf_disable_soft_placement: 0 } } diff --git a/apps/deepstream-test1-rtsp-out/dstest1_pgie_config.txt b/apps/deepstream-test1-rtsp-out/dstest1_pgie_config.txt index 1628360..1b55866 100644 --- a/apps/deepstream-test1-rtsp-out/dstest1_pgie_config.txt +++ b/apps/deepstream-test1-rtsp-out/dstest1_pgie_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,18 +61,21 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel -model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 network-mode=1 num-detected-classes=4 interval=0 gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid +#scaling-filter=0 +#scaling-compute-hw=0 [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/apps/deepstream-test1-usbcam/README b/apps/deepstream-test1-usbcam/README index 68b4e60..a76adc9 100644 --- a/apps/deepstream-test1-usbcam/README +++ b/apps/deepstream-test1-usbcam/README @@ -33,7 +33,7 @@ Example: This document shall describe the sample deepstream-test1-usb application. It is meant for simple demonstration of how to use the various DeepStream SDK -elements in the pipeline and extract meaningful insights from a video stream. +elements in the pipeline and extract meaningful insights from a usb camera stream. This sample creates instance of "nvinfer" element. Instance of the "nvinfer" uses TensorRT API to execute inferencing on a model. Using a diff --git a/apps/deepstream-test1-usbcam/dstest1_pgie_config.txt b/apps/deepstream-test1-usbcam/dstest1_pgie_config.txt index 1628360..1b55866 100644 --- a/apps/deepstream-test1-usbcam/dstest1_pgie_config.txt +++ b/apps/deepstream-test1-usbcam/dstest1_pgie_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,18 +61,21 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel -model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 network-mode=1 num-detected-classes=4 interval=0 gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid +#scaling-filter=0 +#scaling-compute-hw=0 [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/apps/deepstream-test1/dstest1_pgie_config.txt b/apps/deepstream-test1/dstest1_pgie_config.txt index 508fe1a..1b55866 100644 --- a/apps/deepstream-test1/dstest1_pgie_config.txt +++ b/apps/deepstream-test1/dstest1_pgie_config.txt @@ -65,14 +65,17 @@ proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 network-mode=1 num-detected-classes=4 interval=0 gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid +#scaling-filter=0 +#scaling-compute-hw=0 [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/apps/deepstream-test2/dstest2_pgie_config.txt b/apps/deepstream-test2/dstest2_pgie_config.txt index 8f03b8c..32ab644 100644 --- a/apps/deepstream-test2/dstest2_pgie_config.txt +++ b/apps/deepstream-test2/dstest2_pgie_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,10 +61,11 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel -model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 network-mode=1 process-mode=1 @@ -73,8 +74,10 @@ num-detected-classes=4 interval=0 gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid +#scaling-filter=0 +#scaling-compute-hw=0 [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/apps/deepstream-test2/dstest2_sgie1_config.txt b/apps/deepstream-test2/dstest2_sgie1_config.txt index 62bdce6..1041629 100644 --- a/apps/deepstream-test2/dstest2_sgie1_config.txt +++ b/apps/deepstream-test2/dstest2_sgie1_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,11 +61,12 @@ gpu-id=0 net-scale-factor=1 model-file=../../../../samples/models/Secondary_CarColor/resnet18.caffemodel -model-engine-file=../../../../samples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine proto-file=../../../../samples/models/Secondary_CarColor/resnet18.prototxt +model-engine-file=../../../../samples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine mean-file=../../../../samples/models/Secondary_CarColor/mean.ppm labelfile-path=../../../../samples/models/Secondary_CarColor/labels.txt int8-calib-file=../../../../samples/models/Secondary_CarColor/cal_trt.bin +force-implicit-batch-dim=1 batch-size=16 # 0=FP32 and 1=INT8 mode network-mode=1 @@ -81,3 +82,6 @@ is-classifier=1 output-blob-names=predictions/Softmax classifier-async-mode=1 classifier-threshold=0.51 +process-mode=2 +#scaling-filter=0 +#scaling-compute-hw=0 diff --git a/apps/deepstream-test2/dstest2_sgie2_config.txt b/apps/deepstream-test2/dstest2_sgie2_config.txt index ddb4649..6bc6ecb 100644 --- a/apps/deepstream-test2/dstest2_sgie2_config.txt +++ b/apps/deepstream-test2/dstest2_sgie2_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,11 +61,12 @@ gpu-id=0 net-scale-factor=1 model-file=../../../../samples/models/Secondary_CarMake/resnet18.caffemodel -model-engine-file=../../../../samples/models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine proto-file=../../../../samples/models/Secondary_CarMake/resnet18.prototxt +model-engine-file=../../../../samples/models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine mean-file=../../../../samples/models/Secondary_CarMake/mean.ppm labelfile-path=../../../../samples/models/Secondary_CarMake/labels.txt int8-calib-file=../../../../samples/models/Secondary_CarMake/cal_trt.bin +force-implicit-batch-dim=1 batch-size=16 # 0=FP32 and 1=INT8 mode network-mode=1 @@ -81,3 +82,6 @@ is-classifier=1 output-blob-names=predictions/Softmax classifier-async-mode=1 classifier-threshold=0.51 +process-mode=2 +#scaling-filter=0 +#scaling-compute-hw=0 diff --git a/apps/deepstream-test2/dstest2_sgie3_config.txt b/apps/deepstream-test2/dstest2_sgie3_config.txt index c512e47..d223525 100644 --- a/apps/deepstream-test2/dstest2_sgie3_config.txt +++ b/apps/deepstream-test2/dstest2_sgie3_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,11 +61,12 @@ gpu-id=0 net-scale-factor=1 model-file=../../../../samples/models/Secondary_VehicleTypes/resnet18.caffemodel -model-engine-file=../../../../samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine proto-file=../../../../samples/models/Secondary_VehicleTypes/resnet18.prototxt +model-engine-file=../../../../samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine mean-file=../../../../samples/models/Secondary_VehicleTypes/mean.ppm labelfile-path=../../../../samples/models/Secondary_VehicleTypes/labels.txt int8-calib-file=../../../../samples/models/Secondary_VehicleTypes/cal_trt.bin +force-implicit-batch-dim=1 batch-size=16 # 0=FP32 and 1=INT8 mode network-mode=1 @@ -81,3 +82,6 @@ is-classifier=1 output-blob-names=predictions/Softmax classifier-async-mode=1 classifier-threshold=0.51 +process-mode=2 +#scaling-filter=0 +#scaling-compute-hw=0 diff --git a/apps/deepstream-test2/dstest2_tracker_config.txt b/apps/deepstream-test2/dstest2_tracker_config.txt index bcf1b29..a152649 100644 --- a/apps/deepstream-test2/dstest2_tracker_config.txt +++ b/apps/deepstream-test2/dstest2_tracker_config.txt @@ -29,7 +29,7 @@ # [tracker] tracker-width=640 -tracker-height=368 +tracker-height=384 gpu-id=0 ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so #ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so diff --git a/apps/deepstream-test2/tracker_config.yml b/apps/deepstream-test2/tracker_config.yml index ba7effd..d34b49f 100644 --- a/apps/deepstream-test2/tracker_config.yml +++ b/apps/deepstream-test2/tracker_config.yml @@ -4,7 +4,7 @@ NvDCF: useColorNames: 0 # use ColorNames feature useHog: 1 # use Histogram-of-Oriented-Gradient (HOG) feature useHighPrecisionFeature: 0 # Use high-precision in feature extraction. Default is [true] - useUniqueID: 1 # Use 64-bit long Unique ID when assignining tracker ID. Default is [true] + useUniqueID: 0 # Use 64-bit long Unique ID when assignining tracker ID. Default is [false] maxTargetsPerStream: 99 # Max number of targets to track per stream. Recommended to set >10. Note: this value should account for the targets being tracked in shadow mode as well. Max value depends on the GPU memory capacity @@ -43,6 +43,3 @@ NvDCF: trackExponentialSmoothingLr_loc: 0.5 # Learning rate for new location trackExponentialSmoothingLr_scale: 0.3 # Learning rate for new scale trackExponentialSmoothingLr_velocity: 0.05 # Learning rate for new velocity - - # Playback options - endFrameNum: 200 # stop position \ No newline at end of file diff --git a/apps/deepstream-test3/deepstream_test_3.py b/apps/deepstream-test3/deepstream_test_3.py index 73cea85..4b4fddb 100644 --- a/apps/deepstream-test3/deepstream_test_3.py +++ b/apps/deepstream-test3/deepstream_test_3.py @@ -48,11 +48,13 @@ PGIE_CLASS_ID_PERSON = 2 PGIE_CLASS_ID_ROADSIGN = 3 MUXER_OUTPUT_WIDTH=1920 -MUXER_OUTPUT_HEIGHT=80 +MUXER_OUTPUT_HEIGHT=1080 MUXER_BATCH_TIMEOUT_USEC=4000000 -TILED_OUTPUT_WIDTH=1920 -TILED_OUTPUT_HEIGHT=1080 +TILED_OUTPUT_WIDTH=1280 +TILED_OUTPUT_HEIGHT=720 GST_CAPS_FEATURES_NVMM="memory:NVMM" +OSD_PROCESS_MODE= 0 +OSD_DISPLAY_TEXT= 0 pgie_classes_str= ["Vehicle", "TwoWheeler", "Person","RoadSign"] # tiler_sink_pad_buffer_probe will extract metadata received on OSD sink pad @@ -256,6 +258,16 @@ def main(args): if not srcpad: sys.stderr.write("Unable to create src pad bin \n") srcpad.link(sinkpad) + queue1=Gst.ElementFactory.make("queue","queue1") + queue2=Gst.ElementFactory.make("queue","queue2") + queue3=Gst.ElementFactory.make("queue","queue3") + queue4=Gst.ElementFactory.make("queue","queue4") + queue5=Gst.ElementFactory.make("queue","queue5") + pipeline.add(queue1) + pipeline.add(queue2) + pipeline.add(queue3) + pipeline.add(queue4) + pipeline.add(queue5) print("Creating Pgie \n ") pgie = Gst.ElementFactory.make("nvinfer", "primary-inference") if not pgie: @@ -272,6 +284,8 @@ def main(args): nvosd = Gst.ElementFactory.make("nvdsosd", "onscreendisplay") if not nvosd: sys.stderr.write(" Unable to create nvosd \n") + nvosd.set_property('process-mode',OSD_PROCESS_MODE) + nvosd.set_property('display-text',OSD_DISPLAY_TEXT) if(is_aarch64()): print("Creating transform \n ") transform=Gst.ElementFactory.make("nvegltransform", "nvegl-transform") @@ -302,6 +316,7 @@ def main(args): tiler.set_property("columns",tiler_columns) tiler.set_property("width", TILED_OUTPUT_WIDTH) tiler.set_property("height", TILED_OUTPUT_HEIGHT) + sink.set_property("qos",0) print("Adding elements to Pipeline \n") pipeline.add(pgie) @@ -313,15 +328,21 @@ def main(args): pipeline.add(sink) print("Linking elements in the Pipeline \n") - streammux.link(pgie) - pgie.link(tiler) - tiler.link(nvvidconv) - nvvidconv.link(nvosd) + streammux.link(queue1) + queue1.link(pgie) + pgie.link(queue2) + queue2.link(tiler) + tiler.link(queue3) + queue3.link(nvvidconv) + nvvidconv.link(queue4) + queue4.link(nvosd) if is_aarch64(): - nvosd.link(transform) + nvosd.link(queue5) + queue5.link(transform) transform.link(sink) else: - nvosd.link(sink) + nvosd.link(queue5) + queue5.link(sink) # create an event loop and feed gstreamer bus mesages to it loop = GObject.MainLoop() diff --git a/apps/deepstream-test3/dstest3_pgie_config.txt b/apps/deepstream-test3/dstest3_pgie_config.txt index 40779a7..a6a797f 100644 --- a/apps/deepstream-test3/dstest3_pgie_config.txt +++ b/apps/deepstream-test3/dstest3_pgie_config.txt @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -62,8 +62,10 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 process-mode=1 model-color-format=0 @@ -74,6 +76,6 @@ gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/apps/deepstream-test4/dstest4_pgie_config.txt b/apps/deepstream-test4/dstest4_pgie_config.txt index 0199ffb..abd35e7 100644 --- a/apps/deepstream-test4/dstest4_pgie_config.txt +++ b/apps/deepstream-test4/dstest4_pgie_config.txt @@ -1,5 +1,5 @@ ################################################################################ -# Copyright (c) 2018-2019, NVIDIA CORPORATION. All rights reserved. +# Copyright (c) 2018-2020, NVIDIA CORPORATION. All rights reserved. # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -30,7 +30,7 @@ # num-detected-classes # # Optional properties for detectors: -# enable-dbscan(Default=false), interval(Primary mode only, Default=0) +# cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0) # custom-lib-path, # parse-bbox-func-name # @@ -51,7 +51,7 @@ # Other optional properties: # net-scale-factor(Default=1), network-mode(Default=0 i.e FP32), # model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path, -# mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary), +# mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary), # custom-lib-path, network-mode(Default=0 i.e FP32) # # The values in the config file are overridden by values set through GObject @@ -61,10 +61,11 @@ gpu-id=0 net-scale-factor=0.0039215697906911373 model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel -model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_int8.engine proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt +model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine labelfile-path=../../../../samples/models/Primary_Detector/labels.txt int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin +force-implicit-batch-dim=1 batch-size=1 network-mode=1 num-detected-classes=4 @@ -73,8 +74,10 @@ process-mode=1 model-color-format=0 gie-unique-id=1 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid +#scaling-filter=0 +#scaling-compute-hw=0 [class-attrs-all] -threshold=0.2 +pre-cluster-threshold=0.2 eps=0.2 group-threshold=1 diff --git a/bindings/LICENSE.pybind11 b/bindings/LICENSE.pybind11 deleted file mode 100644 index 6f15578..0000000 --- a/bindings/LICENSE.pybind11 +++ /dev/null @@ -1,29 +0,0 @@ -Copyright (c) 2016 Wenzel Jakob , All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - -1. Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. - -2. Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. - -3. Neither the name of the copyright holder nor the names of its contributors - may be used to endorse or promote products derived from this software - without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -Please also refer to the file CONTRIBUTING.md, which clarifies licensing of -external contributions to this project including patches, pull requests, etc. diff --git a/bindings/README.md b/bindings/README.md deleted file mode 100644 index 966ed71..0000000 --- a/bindings/README.md +++ /dev/null @@ -1,6 +0,0 @@ -# Python Bindings - -The DeepStream Python bindings are currently released in a compiled module. -Please see the [HOW-TO Guide](../HOWTO.md) and [FAQ](../FAQ.md) for usage information. More detailed documentation will be added as the API matures. - -Portions of these bindings are generated using [Pybind11](https://github.com/pybind/pybind11) under these [licensing terms](LICENSE.pybind11).