NDI multi stream stuttering

[andiempettJISC]

when trying to use multiple NDI sources in galicaster. the ndivideosrc and ndiaudiosrc will stutter.
stuttering will happen in both preview and when recording to filesink

if you play the ndi sources from gst-launch command line at the same time, they work fine with no stuttering (also recording to filesink works OK as well)

its hard to pin down the cause of the stuttering, but i dont think its compute resource as the machine running this is very powerful.

gst-launch-1.0 --verbose ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox  top=0 bottom=0 ! videorate ! xvimagesink async=true qos=true sync=true force-aspect-ratio=true

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" ! queue ! autoaudiosink

example profile:

[data]
name = NDI

[track1]
name = NDI
device = ndi
location = BIRDDOG-xxxxx (HDMI)
file = presentation.avi
flavor = presentation
audio = False

[track3]
name = NDI_Audio
device = ndi_audio
location = BIRDDOG-xxxxx (comms)
file = presenter.mp3
flavor = presenter
player = True
vumeter = True

[smarquard]

The difference in the Galicaster gstreamer pipelines is that they include a tee for both preview and filesink recording, so it's worth reproducing that pipeline with gst-launch.

We had a datapath problem once that only showed up with a tee pipeline but not a simpler pipeline.

[andiempettJISC]

@smarquard raises a good point, but sadly with more complex pipelines on the command line i still cant replicate the issue.

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" do-timestamp=true ! queue ! audiorate ! decodebin async-handling=true ! tee name=tee-aud \
tee-aud. ! queue ! audioconvert ! wavenc ! filesink location=testndi.wav async=false \
tee-aud. ! queue ! autoaudiosink

gst-launch-1.0 --verbose ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox  top=0 bottom=0 ! videorate ! tee name=gc-ndi-tee ! queue ! xvimagesink async=false \
gc-ndi-tee. ! queue ! valve drop=false ! videoconvert ! queue ! \
x264enc pass=5 quantizer=22 speed-preset=4 ! queue ! avimux ! \
queue ! filesink location=testndi.avi async=false

[Alfro]

Hi @androidwiltron

A more accurate way to test it would be to record both sources through the same pipeline. That may reproduce the issue better, since I believe Galicaster has all sources on the same process, as opposed to launching two different gst-launch-1.0 commands.

[andiempettJISC]

@Alfro
good suggestion
really i need to make a minimal python script to do this

do you think using videomixer gstreamer element with gst-launch would be sufficient?

[Alfro]

I meant something like this:

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" do-timestamp=true ! queue ! audiorate ! decodebin async-handling=true ! tee name=tee-aud
tee-aud. ! queue ! audioconvert ! wavenc ! filesink location=testndi.wav async=false
tee-aud. ! queue ! autoaudiosink ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox top=0 bottom=0 ! videorate ! tee name=gc-ndi-tee ! queue ! xvimagesink async=false \
gc-ndi-tee. ! queue ! valve drop=false ! videoconvert ! queue !
x264enc pass=5 quantizer=22 speed-preset=4 ! queue ! avimux !
queue ! filesink location=testndi.avi async=false

Let me know if it reproduces like this!

[andiempettJISC]

OK @Alfro your launch command replicates the problem. the audio stutters, the video is OK