-
Notifications
You must be signed in to change notification settings - Fork 4
Multichannel support
Multichannel support allows for the processing of multiple buffers simultaneously by a single element.
In order to collect multiple buffers, the TIOVXMux element can be used and it then can then be spread into individual buffers by the TIOVXDemux element.
A batched pipeline would look as follows:
GStreamer Buffers Gstreamer Buffers
┌───────────┐ ┌───────────┐
│ │ │ │
│ Input ├────┐ ┌──►│ Input │
│ │ │ │ │ │
└───────────┘ │ Batched Buffers Batched Buffers │ └───────────┘
│ │
┌───────────┐ │ ┌────────────┐ ┌──────────────┐ ┌────────────┐ │ ┌───────────┐
│ │ └─────►│ │ │ │ │ ├─┘ │ │
│ Input ├──────────►│ TIOVXMux ├════►...══════►│ TIOVXElement ├═══►...═══►│ TIOVXDemux │────►│ Input │
│ │ ┌─────►│ │ │ │ │ ├─┐ │ │
└───────────┘ │ └────────────┘ └──────────────┘ └────────────┘ │ └───────────┘
. │ │ .
. │ │ .
. │ │ .
┌───────────┐ │ │ ┌───────────┐
│ │ │ │ │ │
│ Input ├────┘ └──►│ Input │
│ │ │ │
└───────────┘ └───────────┘
And it could be described by the following pipeline
gst-launch-1.0 \
videotestsrc is-live=true num-buffers=10 ! mux. \
videotestsrc is-live=true num-buffers=10 ! mux. \
videotestsrc is-live=true num-buffers=10 ! mux. \
tiovxmux name=mux ! "video/x-raw(memory:batched), width=640, height=480" ! \
# Other TIOVX elements for further batched processing go here \
tiovxdemux name=demux \
demux. ! queue ! fakesink \
demux. ! queue ! fakesink \
demux. ! queue ! fakesinkAny number of TIOVX elements can be used between the Mux and Demux elements. For SIMO elements multiple TIOVX Demuxers must be used for each of the outputs and for MISO elements multiple TIOVX Muxers can be used for each of the inputs, see the pages of the specific elements for more detailed examples.
This support doesn't limit the usage of non-batched buffers, and processing can still be performed on regular GStreamer buffers.
The presence of multichannel buffer is indicated by the (memory:batched) features in the caps, the user needs to add this feature to each of the caps they specify between a muxer and a demuxer.
The number of channels doesn't need to be specified since both muxer and demux can infer this from their amount of inputs and outputs respectively. Other TIOXV elements along the pipeline can deduce this from the caps. The number of inputs of a muxer and the outputs of a demuxer need to match otherwise processing will fail.
The usage of these features also means that no non-TIOVX elements can process between a muxer and a demuxer, this avoids undefined behavior with the larger memory used in the buffers that aren't aware of the batched memory.
-
Although the batching software support has a maximum number of channels of 16, in practice the hardware might limit this to a lower number since the needed memory will exceed its capacity.
-
The TIOVXIsp element currently only supports non-batched mode due to limitations in its behavior.