Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spec optional conversion to RGB in VideoFrame.copyTo() #754

Merged
merged 12 commits into from
Apr 19, 2024
88 changes: 74 additions & 14 deletions index.src.html
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@

<pre class=link-defaults>
spec:html; type:attribute; text:hidden
spec:html; type:enum-value; text:srgb
spec:infra; type:dfn; text:list
spec:infra; type:dfn; text:enqueue
</pre>
Expand All @@ -38,6 +39,8 @@
spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/;
for: ImageBitmap;
type: dfn; text: bitmap data; url: imagebitmap-and-animations.html#concept-imagebitmap-bitmap-data
for: ImageData
type: dfn; text: canvas pixel data; url: canvas.html#imagedata
for: Canvas;
type: dfn; text: Check the usability of the image argument; url: canvas.html#check-the-usability-of-the-image-argument
type: dfn; text: is not origin-clean; url: canvas.html#the-image-argument-is-not-origin-clean
Expand Down Expand Up @@ -3745,8 +3748,8 @@

: <dfn method for=VideoFrame>copyTo(|destination|, |options|)</dfn>
:: Asynchronously copies the planes of this frame into |destination| according
to |options|. The format of the data is the same as this {{VideoFrame}}'s
{{VideoFrame/format}}.
to |options|. The format of the data is specified by {{VideoFrameCopyToOptions/format}},
if present, otherwise {{VideoFrame/format}}.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved

NOTE: Promises that are returned by several calls to
{{VideoFrame/copyTo()}} are not guaranteed to resolve in the order they
Expand All @@ -3764,10 +3767,22 @@
5. If `destination.byteLength` is less than |combinedLayout|'s [=combined
buffer layout/allocationSize=], return a promise rejected with a
{{TypeError}}.
6. Let |p| be a new {{Promise}}.
7. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
8. Let |planeLayouts| be a new [=list=].
9. Enqueue the following steps to |copyStepsQueue|:
6. If |options|.{{VideoFrameCopyToOptions/format}} is equal to one of
{{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}} then:
1. Let |newOptions| be the result of running the <a>Clone Configuration</a>
algorithm with |options|.
2. Assign `undefined` to |newOptions|.{{VideoFrameCopyToOptions/format}}.
3. Let |rgbFrame| be the result of running the [=VideoFrame/Convert to RGB frame=]
algorithm with [=this=], |options|.{{VideoFrameCopyToOptions/format}},
and |options|.{{VideoFrameCopyToOptions/colorSpace}}.
4. Return the result of calling {{VideoFrame/copyTo()}} on |rgbFrame| with
|destination| and |newOptions|.
7. Otherwise, if |options|.{{VideoFrameCopyToOptions/format}} is not `undefined` or `null`
throw a {{NotSupportedError}} {{DOMException}}.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
8. Let |p| be a new {{Promise}}.
9. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
10. Let |planeLayouts| be a new [=list=].
11. Enqueue the following steps to |copyStepsQueue|:
1. Let resource be the [=media resource=] referenced by
[[resource reference]].
2. Let |numPlanes| be the number of planes as defined by
Expand Down Expand Up @@ -3804,7 +3819,7 @@
10. Increment |planeIndex| by `1`.
11. Append |layout| to |planeLayouts|.
5. [=Queue a task=] to resolve |p| with |planeLayouts|.
10. Return |p|.
12. Return |p|.

: <dfn method for=VideoFrame>clone()</dfn>
:: Creates a new {{VideoFrame}} with a reference to the same
Expand Down Expand Up @@ -4048,10 +4063,17 @@
6. Let |optLayout| be `undefined`.
7. If |options|.{{VideoFrameCopyToOptions/layout}} [=map/exists=], assign
its value to |optLayout|.
8. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
Layout and Allocation Size=] algorithm with |parsedRect|,
{{VideoFrame/[[format]]}}, and |optLayout|.
9. Return |combinedLayout|.
8. Let |format| be `undefined`.
9. If |options|.{{VideoFrameCopyToOptions/format}} is `undefined`,
assign {{VideoFrame/[[format]]}} to |format|.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
10. Otherwise, if |options|.{{VideoFrameCopyToOptions/format}} is equal to
one of {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}, then assign
|options|.{{VideoFrameCopyToOptions/format}} to |format|,
otherwise return {{NotSupportedError}}.
11. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
Layout and Allocation Size=] algorithm with |parsedRect|, |format|,
and |optLayout|.
12. Return |combinedLayout|.

: <dfn for=VideoFrame>Verify Rect Offset Alignment</dfn> (with |format| and
|rect|)
Expand Down Expand Up @@ -4183,10 +4205,34 @@
[=combined buffer layout/allocationSize=].
9. Return |combinedLayout|.

: <dfn for=VideoFrame>Convert to RGB frame</dfn> (with |frame|, |format| and |colorSpace|)
:: 1. If |format| is not equal to one {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}},
throw a {{NotSupportedError}}
1. Let |convertedFrame| be a new {{VideoFrame}}, constructed as follows:
1. Assign `false` to {{platform object/[[Detached]]}}.
2. Assign |format| to {{VideoFrame/[[format]]}}.
3. Let |width| be |frame|'s {{VideoFrame/[[visible width]]}}.
4. Let |height| be |frame|'s {{VideoFrame/[[visible height]]}}.
5. Assign |width|, |height|, 0, 0, |width|, |height|, |width|, and
|height| to {{VideoFrame/[[coded width]]}},
{{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
{{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
and {{VideoFrame/[[visible height]]}} respectively.
6. Assign |frame|'s {{VideoFrame/[[duration]]}} and |frame|'s
{{VideoFrame/[[timestamp]]}} to {{VideoFrame/[[duration]]}} and
{{VideoFrame/[[timestamp]]}} respectively.
7. Assign |colorSpace| to {{VideoFrame/[[color space]]}}.
8. Let |resource| be a new [=media resource=] containing the result of
conversion of [=media resource=] referenced by |frame|'s
{{VideoFrame/[[resource reference]]}} into a color space and pixel
format specified by |colorSpace| and |format| respectively.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
9. Assign the reference to |resource| to {{VideoFrame/[[resource reference]]}}
2. Return |convertedFrame|.

: <dfn for=VideoFrame>Copy VideoFrame metadata</dfn> (with |metadata|)
:: 1. Let |metadataCopySerialized| be [$StructuredSerialize$](|metadata|).
2. Let |metadataCopy| be [$StructuredDeserialize$](|metadataCopySerialized|, [=the current Realm=]).
3. return |metadataCopy|.
3. Return |metadataCopy|.

The goal of this algorithm is to ensure that metadata owned by a {{VideoFrame}} is immutable.

Expand Down Expand Up @@ -4239,13 +4285,15 @@

VideoFrame CopyTo() Options {#videoframe-copyto-options}
------------------------------------------------------------
Options to specify a rectangle of pixels to copy and the offset and stride of
planes in the destination buffer.
Options to specify a rectangle of pixels to copy, their format, and the offset
and stride of planes in the destination buffer.

<xmp class='idl'>
dictionary VideoFrameCopyToOptions {
DOMRectInit rect;
sequence<PlaneLayout> layout;
VideoPixelFormat format;
PredefinedColorSpace colorSpace;
};
</xmp>

Expand Down Expand Up @@ -4273,6 +4321,18 @@
to specify an offset and stride for each plane in the destination
{{BufferSource}}. If unspecified, the planes will be tightly packed. It is
invalid to specify planes that overlap.
: <dfn dict-member for=VideoFrameCopyToOptions>format</dfn>
:: A {{VideoPixelFormat}} for the pixel data in the destination
{{BufferSource}}. Potential values are: {{RGBA}}, {{RGBX}}, {{BGRA}},
{{BGRX}}. If unspecified or `null`, the the destination
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
{{BufferSource}} will match {{VideoFrame/format}} .
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
: <dfn dict-member for=VideoFrameCopyToOptions>colorSpace</dfn>
:: A {{PredefinedColorSpace}} that <em class="rfc2119">MUST</em> be used as
a target color space for the pixel data in the destination
{{BufferSource}}, but only if {{VideoFrameCopyToOptions/format}} is one of
{{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}, otherwise it is ignored.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved
If unspecified or `null`, {{srgb}} is used.
Djuffin marked this conversation as resolved.
Show resolved Hide resolved


DOMRects in VideoFrame {#videoframe-domrect}
--------------------------------------------
Expand Down
Loading