Explicit frame drop signal from VideoEncoder

[Djuffin]

Currently the spec doesn't specify any way for VideoEncoder to signal that a frame has been dropped due to realtime mode constraints.
I think it would be useful to have such a signal.
Possible options:

1. the output callback should be called with an empty encoded chunk
2. the output callback should be called with a new flag set in EncodedVideoChunkMetadata

This came up after WPT test change that accounts for frame drops.

[Djuffin]

On the other hand, 2 existing implementations drop frames silently, so using the regular output callback as is will be a breaking change. If we ever do this we'll need a VideoEncoderConfig option to turn it on.
Let's wait and see if it actually needed by people building RTC apps.

[Djuffin]

At least Meet is interested in drop frame notifications.

[Djuffin]

Currently when a real time encoder drops a frame clients don’t get any signal from VideoEncoder. If they really want to know that a frame was dropped they need to look at the timestamps of the encoded video chunks.

This issue can be addressed by a new flag in the encoder config that would cause the encoder to call EncodedVideoChunkOutputCallback for dropped frames with an empty data buffer in the encoded chunk.

partial dictionary VideoEncoderConfig {
	boolean notifyAboutFrameDrops = false;
}

[aboba]

@Djuffin When encoding with H.264, am seeing the following messages refer to bEnableFrameSkip. Is there any way to set this to a value other than 0?

[OpenH264] this = 0x0x13c05ca6e70, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
[OpenH264] this = 0x0x13c05ca11a0, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
[OpenH264] this = 0x0x13c05ca11a0, Warning:Actual input framerate 0.008349 is different from framerate in setting 30.000000, suggest to use other rate control modes
[OpenH264] this = 0x0x13c05ca11a0, Warning:Actual input framerate 23.112173 is different from framerate in setting 30.000000, suggest to use other rate control modes

[Djuffin]

@aboba We have a task for it, but it never got any attention https://issues.chromium.org/issues/40811827
IMO this OpenH264 log message is exaggerating the problem.