Start recording from RTP sources at key frame (iframe boundary)

[smarquard]

Recording from an IP camera source, e.g. with the following track configuration:

Track 1 = usb audio (alsa)
Track 2 = RTP (video only), rtspt, mastroska mux
Track 3 = Presentation (Datapath v4l2), avi mux

When Galicaster starts in preview mode, the pipeline is set up and plays. When recording starts, the pipeline valves are enabled that starts sending input to the muxers.

The RTP stream consists of H264 frames that could be I frames (key frame), B or P frames. At the point when the recording starts, a number of B and/or P frames can be written to the file before the first I frame.

These frames aren't displayed by ffprobe -show_frames, but are visible using -show_pkts. Also in the -show_frames output, coded_picture_number starts at non-zero value.

ffmpeg ignores these prior frames when doing operations like re-encoding the stream. Some some ffmpeg operations on the media can cause the A/V sync to be lost. For example:

Operation that preserves a/v sync:
muxing the audio and rtp video stream

Operations that lose a/v sync:
Re-encoding the video stream (e.g. to a fixed frame rate or different quality/format)
muxing the audio and video stream

In the 2nd operation, the a/v sync will be out by the duration of the frames prior to the first I-frame.

These constraints can be worked around, but this adds complexity to the processing, and it would be simpler if Galicaster could always produce media from RTP sources that starts with an iframe, and all the other tracks at exactly the same time-point.

Two possible approaches:

1. Restart the stream when recording starts (rather than use a valve to switch recording on/off)

2. Get a signal from gstreamer when the iframe arrives on the rtp stream, and use that signal to enable the valves.

[Alfro]

Regarding option 1:

Restart the stream when recording starts (rather than use a valve to switch recording on/off)

We realized this would not solve the issue, since the pipeline does not seem to wait for an I-frame to come to start recording, even if it is restarted.

Still looking into option 2.

[vgoya]

Thank you Stephen for raising this issue. It can end in an interesting improvement to GC. We will work on it