From d1be7b1e98329a733dd9f8a8c5b92776adfad770 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Sat, 17 Aug 2024 05:23:37 -0700 Subject: [PATCH] Move media capabilities discussion to negotiation section --- index.html | 66 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 34 insertions(+), 32 deletions(-) diff --git a/index.html b/index.html index cb1b086..a2cfda8 100644 --- a/index.html +++ b/index.html @@ -312,16 +312,38 @@

Negotiation issues

the application will negotiate codecs to be sent and received with the conferencing server. However, since scalability modes are not negotiated in Offer/Answer, the application needs to determine - which scalability modes the conference server supports by other - means. + which scalability modes the browser and conference server support + by other means. +

+

+ The [[?Media-Capabilities]] API enables discovery of encoder and decoder + support for scalable video coding. {{VideoConfiguration//scalabilityMode}} + is used to query whether an encoder supports a + {{RTCRtpEncodingParameters/scalabilityMode}} value, indicating whether + it is "supported", "smooth" and "power efficient". +

+

+ The [[?Media-Capabilities]] API also provides information on decoder support + for spatial scalablity modes. {{VideoConfiguration/spatialScalability}} + indicates whether a decoder has the ability to support spatial prediction, + which requires the ability to use frames of a resolution different than + the current resolution as a dependency. If {{VideoConfiguration/spatialScalability}} + is set to true, the decoder can decode any + {{RTCRtpEncodingParameters/scalabilityMode}} value supported by the encoder. + If {{VideoConfiguration/spatialScalability}} is set to false + or is absent, the decoder cannot decode spatial scalability modes, but can + can decode all other {{RTCRtpEncodingParameters/scalabilityMode}} values + supported by the encoder.

- One way to handle this is for the SFM to indicate the combination - of codecs and scalability modes it can forward in the form of receiver - capabilities. After receiving the SFM's capabilities, the application - can compute the intersection of codec and - {{RTCRtpEncodingParameters/scalabilityMode}} values supported - by the browser's {{RTCRtpSender}} and the + Once an application has determined which combination of codecs and + {{RTCRtpEncodingParameters/scalabilityMode}} values it has available to use, + it needs to determine which of these combinations are supported by the SFM. + One way to handle this is for the SFM to indicate the combination of codecs + and scalability modes it can forward in the form of receiver capabilities + After receiving the SFM's capabilities, the application can compute the + intersection of codec and {{RTCRtpEncodingParameters/scalabilityMode}} values + supported by the browser's {{RTCRtpSender}} and the SFM's receiver. This can be used to determine the arguments passed to the browser's {{RTCPeerConnection/addTransceiver()}} and {{RTCRtpSender/setParameters()}} @@ -334,10 +356,10 @@

Negotiation issues

An SFM that parses codec payloads may only support the H.264/AVC codec without scalability and the VP8 codec with temporal scalability. On the other hand, the browser - may be able to encode VP8 with temporal scalability, VP9 and AV1 with temporal - and spatial scalability and H.264/AVC with temporal scalability. - In this example, an application desiring to use SVC would only - be able to encode VP8 with temporal scalability. + may be able to encode VP8 with temporal scalability, VP9 and AV1 with + temporal and spatial scalability and H.264/AVC with temporal scalability. + In this example, an application desiring to use SVC would only be able + to encode VP8 with temporal scalability.
  • An SFM may only support @@ -929,26 +951,6 @@

    Persistent information

    is decodable for the codec in use can be obtained, which provides more information on hardware capabilities.

    -

    - The [[?Media-Capabilities]] API enables discovery of encoder and decoder - support for scalable video coding. {{VideoConfiguration//scalabilityMode}} - is used to query whether an encoder supports a - {{RTCRtpEncodingParameters/scalabilityMode}} value, indicating whether - it is "supported", "smooth" and "power efficient". -

    -

    - The [[?Media-Capabilities]] API also provides information on decoder support - for spatial scalablity modes. {{VideoConfiguration/spatialScalability}} - indicates whether a decoder has the ability to support spatial prediction, - which requires the ability to use frames of a resolution different than - the current resolution as a dependency. If {{VideoConfiguration/spatialScalability}} - is set to true, the decoder can decode any - {{RTCRtpEncodingParameters/scalabilityMode}} value supported by the encoder. - If {{VideoConfiguration/spatialScalability}} is set to false - or is absent, the decoder cannot decode spatial scalability modes, but can - can decode all other {{RTCRtpEncodingParameters/scalabilityMode}} values - supported by the encoder. -

    As noted in [[?Media-Capabilities]] Section 3.1, the Media Capabilities API "will likely provide more accurate and