Skip to content

Commit

Permalink
Move media capabilities discussion to negotiation section
Browse files Browse the repository at this point in the history
  • Loading branch information
aboba authored Aug 17, 2024
1 parent 7b097ae commit d1be7b1
Showing 1 changed file with 34 additions and 32 deletions.
66 changes: 34 additions & 32 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -312,16 +312,38 @@ <h2>Negotiation issues</h2>
the application will negotiate codecs to be sent and received with
the conferencing server. However, since scalability modes are not
negotiated in Offer/Answer, the application needs to determine
which scalability modes the conference server supports by other
means.
which scalability modes the browser and conference server support
by other means.
</p>
<p>
The [[?Media-Capabilities]] API enables discovery of encoder and decoder
support for scalable video coding. {{VideoConfiguration//scalabilityMode}}
is used to query whether an encoder supports a
{{RTCRtpEncodingParameters/scalabilityMode}} value, indicating whether
it is "supported", "smooth" and "power efficient".
</p>
<p>
The [[?Media-Capabilities]] API also provides information on decoder support
for spatial scalablity modes. {{VideoConfiguration/spatialScalability}}
indicates whether a decoder has the ability to support spatial prediction,
which requires the ability to use frames of a resolution different than
the current resolution as a dependency. If {{VideoConfiguration/spatialScalability}}
is set to <code>true</code>, the decoder can decode any
{{RTCRtpEncodingParameters/scalabilityMode}} value supported by the encoder.
If {{VideoConfiguration/spatialScalability}} is set to <code>false</code>
or is absent, the decoder cannot decode spatial scalability modes, but can
can decode all other {{RTCRtpEncodingParameters/scalabilityMode}} values
supported by the encoder.
</p>
<p>
One way to handle this is for the SFM to indicate the combination
of codecs and scalability modes it can forward in the form of receiver
capabilities. After receiving the SFM's capabilities, the application
can compute the intersection of codec and
{{RTCRtpEncodingParameters/scalabilityMode}} values supported
by the browser's {{RTCRtpSender}} and the
Once an application has determined which combination of codecs and
{{RTCRtpEncodingParameters/scalabilityMode}} values it has available to use,
it needs to determine which of these combinations are supported by the SFM.
One way to handle this is for the SFM to indicate the combination of codecs
and scalability modes it can forward in the form of receiver capabilities
After receiving the SFM's capabilities, the application can compute the
intersection of codec and {{RTCRtpEncodingParameters/scalabilityMode}} values
supported by the browser's {{RTCRtpSender}} and the
<a title="Selective Forwarding Middlebox">SFM</a>'s receiver. This
can be used to determine the arguments passed to the browser's
{{RTCPeerConnection/addTransceiver()}} and {{RTCRtpSender/setParameters()}}
Expand All @@ -334,10 +356,10 @@ <h2>Negotiation issues</h2>
An <a title="Selective Forwarding Middlebox">SFM</a> that parses
codec payloads may only support the H.264/AVC codec without scalability and
the VP8 codec with temporal scalability. On the other hand, the browser
may be able to encode VP8 with temporal scalability, VP9 and AV1 with temporal
and spatial scalability and H.264/AVC with temporal scalability.
In this example, an application desiring to use SVC would only
be able to encode VP8 with temporal scalability.
may be able to encode VP8 with temporal scalability, VP9 and AV1 with
temporal and spatial scalability and H.264/AVC with temporal scalability.
In this example, an application desiring to use SVC would only be able
to encode VP8 with temporal scalability.
</li>
<li>
An <a title="Selective Forwarding Middlebox">SFM</a> may only support
Expand Down Expand Up @@ -929,26 +951,6 @@ <h2>Persistent information</h2>
is decodable for the codec in use can be obtained,
which provides more information on hardware capabilities.
</p>
<p>
The [[?Media-Capabilities]] API enables discovery of encoder and decoder
support for scalable video coding. {{VideoConfiguration//scalabilityMode}}
is used to query whether an encoder supports a
{{RTCRtpEncodingParameters/scalabilityMode}} value, indicating whether
it is "supported", "smooth" and "power efficient".
</p>
<p>
The [[?Media-Capabilities]] API also provides information on decoder support
for spatial scalablity modes. {{VideoConfiguration/spatialScalability}}
indicates whether a decoder has the ability to support spatial prediction,
which requires the ability to use frames of a resolution different than
the current resolution as a dependency. If {{VideoConfiguration/spatialScalability}}
is set to <code>true</code>, the decoder can decode any
{{RTCRtpEncodingParameters/scalabilityMode}} value supported by the encoder.
If {{VideoConfiguration/spatialScalability}} is set to <code>false</code>
or is absent, the decoder cannot decode spatial scalability modes, but can
can decode all other {{RTCRtpEncodingParameters/scalabilityMode}} values
supported by the encoder.
</p>
<p>
As noted in [[?Media-Capabilities]] Section 3.1, the Media
Capabilities API "will likely provide more accurate and
Expand Down

0 comments on commit d1be7b1

Please sign in to comment.